Dynamic privacy mosaic generation system based on intelligent glasses

Abstract

The invention discloses a dynamic privacy mosaic generation system based on intelligent glasses, and relates to the field of privacy protection of wearable equipment, and the dynamic privacy mosaic generation system comprises a head motion sensing module which is used for collecting inertial measurement data of the intelligent glasses; the visual field prediction module is used for calculating the head movement trend of the user based on the inertial measurement data and determining a predicted fixation area; the privacy target identification module is used for identifying a privacy target in the predicted fixation area and acquiring a privacy score; the fuzzy control module is used for calculating the fuzzy intensity of each pixel point according to the head movement trend, the angle difference between the target position and the fixation center and the privacy score; the image synthesis module is used for generating a fuzzy mask and fusing the fuzzy mask with the image frame; and the image output module is used for outputting the image frame superposed with the fuzzy mask to display equipment or a storage medium. According to the method, the problem of short-time exposure of privacy information in an image identification delay process is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of wearable device privacy protection, more particularly, to a dynamic privacy mosaic generation system based on smart glasses. BACKGROUND

[0002] With the development of smart wearable devices, smart glasses gradually popularize in scenarios such as recording the field of view, assisting navigation, information prompting, and remote collaboration. The images and video data collected by the wearer in daily activities usually contain a large amount of environmental information, which may include the facial features of other people, family photos, screen content, file materials, and other sensitive information. The recording and dissemination of these contents without authorization will pose a potential threat to personal privacy.

[0003] Current privacy protection mechanisms mainly rely on image recognition models to detect and process the collected pictures. Some systems use static region identification methods to apply mosaic effects after detecting specific objects such as human faces. However, this type of method can only start to obscure the processing when the target is completely exposed in the center of the image and is clearly captured by the recognition model. It cannot effectively deal with the privacy content that momentarily passes the edge of the field of view, the short exposure before the obscuring is completed, and the "reaction lag" caused by recognition delay.

[0004] In dynamic states such as rapid head turning, panning the line of sight, walking, and patrolling, privacy information is easily exposed to the field of view in a short time, while the image processing system is still in the target detection and judgment stage, often failing to complete the obscuring in advance. This problem is more prominent in scenarios such as recording videos, live pictures, or automatic synchronization to the cloud. How to build a system that can identify potential field of view areas based on user motion trends and implement progressive dynamic obscuring of privacy targets in these areas has become an important problem to be solved in current smart glasses privacy protection technology. SUMMARY

[0005] The technical problem solved by the present application is to provide a dynamic privacy mosaic generation system based on smart glasses to solve the problems mentioned in the background.

[0006] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows: A dynamic privacy mosaic generation system based on smart glasses, comprising: a head motion perception module for collecting inertial measurement data of the smart glasses; the inertial measurement data includes angular velocity data and linear acceleration data; a field of view prediction module for predicting the head motion trend of the user based on the inertial measurement data and determining the predicted gaze area; a privacy target identification module for identifying privacy targets in the predicted gaze area and obtaining a privacy score; a blur control module configured to calculate a blur intensity of each pixel according to a head movement trend, an angle difference between the target position and the gaze center, and a privacy score; an image synthesis module configured to generate a blur mask and fuse the blur mask with the image frame; an image output module configured to output the image frame with the superimposed blur mask to a display device or a storage medium.

[0007] In some embodiments, the inertial measurement data includes a three-axis angular velocity vector and a three-axis linear acceleration vector . The field of view prediction module jointly predicts an offset coordinate of the future gaze center in the image frame based on the three-axis angular velocity vector and the three-axis linear acceleration vector , and generates a predicted gaze region around the offset coordinate.

[0008] In some embodiments, the privacy target identification module includes an image acquisition unit and a privacy identification model; The privacy identification model, based on a deep convolutional neural network structure, performs target detection on the image acquired by the image acquisition unit and assigns a privacy score to each detected target , wherein represents a pixel position in the image.

[0009] In some embodiments, the blur control module is configured to calculate the blur intensity , which is determined by the following formula: . wherein is a synthetic motion speed module length calculated according to the angular velocity vector and the linear acceleration vector, represents an angle difference between the pixel and the predicted gaze center, is the privacy score, is a normalization function.

[0010] In some embodiments, the image synthesis module performs two-dimensional convolution processing on the blur intensity using a Gaussian blur convolution kernel to generate a blur mask, and superimposes the blur mask on the original image frame to form an output image frame.

[0011] In some embodiments, the system further includes a gaze monitoring module configured to detect a gaze time of the user in a certain region, and when the gaze time exceeds a preset threshold , the blur control module adjusts the blur intensity of the corresponding region to , wherein .

[0012] In some embodiments, the privacy target identification module is further configured to determine a visibility trend of the privacy target, and determine an occlusion early trigger condition based on a distance between a boundary of the privacy target and the predicted gaze region.

[0013] In some embodiments, the blur control module is further configured to construct a direction-aware blur propagation field based on a direction of the head movement of the user , the direction-aware blur propagation field is given by: ; wherein, is a predicted movement direction vector, is a blur propagation rate control coefficient, is a privacy score.

[0014] In some embodiments, the image synthesis module further comprises a blur boundary fade module configured to construct a blur gradient band at each edge of the occlusion region to alleviate image discontinuity at the occlusion boundary.

[0015] In some embodiments, the image output module comprises a video output sub-module configured to, after generating a complete video sequence, perform time axis analysis on the image frames, and based on the inertial measurement data, backtrack the predicted field of view range and supplementally generate an occlusion mask when an unidentified privacy target is detected.

[0016] The present application has the advantages over the prior art that the present application integrates a head movement perception module, a field of view prediction module, a privacy target identification module, a blur control module, an image synthesis module, and an image output module to construct a dynamic privacy mosaic generation system for smart glasses. The system can analyze the movement trend based on the inertial measurement data of the user's head in real time, and predict the position of the target region that may enter the gaze region in the future image frames, so that the related region can be pre-occluded before explicit identification is completed, thereby solving the problem of privacy information leakage caused by identification delay or short-time exposure.

[0017] The system predicts the gaze region and identifies the privacy target based on the convolutional neural network structure, and automatically assigns a privacy sensitive score to the target region in the image. On this basis, the blur control module calculates the pixel-level blur intensity by fusing the head movement speed, angle deviation, and target score, and further generates a blur mask using the Gaussian convolution method to realize dynamic occlusion with variable region range and progressive blur degree.

[0018] In order to ensure the normal use experience and viewing needs of the user, the system further integrates a gaze monitoring module, when the user gazes at a certain area for a long time, the corresponding blur degree will automatically decay, thereby providing a flexible and strong adaptability privacy protection strategy. In addition, the application also proposes to construct a blur propulsion field based on a predicted direction, edge transition softening, and a video frame backtracking compensation mechanism, comprehensively improving the forward-looking, coherence and integrity of privacy protection. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is the overall schematic diagram of the system of the application; Figure 2 is the workflow diagram of the head movement perception module of the system of the application; Figure 3 is the workflow diagram of the field of view prediction module of the system of the application; Figure 4 is the workflow diagram of the privacy target identification module of the system of the application. DETAILED DESCRIPTION

[0020] The specific embodiments of the application will be described below in conjunction with the accompanying drawings.

[0021] As shown in Figure 1 , the application comprises the following modules: a head movement perception module, for collecting inertial measurement data of the intelligent glasses; the inertial measurement data comprises angular velocity data and linear acceleration data; a field of view prediction module, for calculating the head movement trend of the user and determining the predicted gaze area based on the inertial measurement data; a privacy target identification module, for identifying the privacy target in the predicted gaze area and obtaining a privacy score; a blur control module, for calculating the blur intensity of each pixel point according to the angle difference between the head movement trend, the target position and the gaze center, and the privacy score; an image synthesis module, for generating a blur mask and fusing it with the image frame; an image output module, for outputting the image frame superimposed with the blur mask to a display device or a storage medium.

[0022] More specifically: As shown in Fig. 2, the head movement perception module is responsible for collecting the head movement data of the wearer of the intelligent glasses, laying a foundation for subsequent field of view prediction and privacy protection. This module mainly relies on an inertial measurement unit (IMU), a small sensor device integrated in the intelligent glasses, which can measure the motion state of the head in real time. The data collected by the IMU includes a three-axis angular velocity vector and a three-axis linear acceleration vector Angular velocity vectors represent the rotational speed of the head around three spatial axes, typically in units of radians per second (rad / s); linear acceleration vectors reflect the acceleration of the head in three directions, usually in units of meters per squared second (m / s²). These data are sampled at high frequencies (e.g., 100 Hz or higher) to ensure that subtle head movements are captured.

[0023] In practical applications, IMU data can be affected by noise, temperature drift, or sensor bias. To improve data quality, the system usually preprocesses the raw data. For example, a Kalman filter can be used to smooth angular velocity and acceleration data, reducing noise interference. In addition, to correct long-term drift, a magnetometer can be combined (if supported by the IMU) for attitude calibration. The processed data will be passed to the field of view prediction module in the form of a time series to predict the trend of head movement.

[0024] The accuracy and sampling frequency of the IMU directly affect the real-time performance and prediction accuracy of the system. In high-performance application scenarios, industrial-grade six-axis or nine-axis IMUs can be selected to achieve higher measurement accuracy.

[0025] As shown in Figure 3, the field of view prediction module uses the data provided by the head movement perception module to predict the trend of the user's head movement and determine the predicted gaze region. The process of achieving this function can be divided into several steps.

[0026] First, based on the three-axis angular velocity vector , the rotation angle of the head is calculated by time integration, and then the current head attitude is determined. The attitude is usually represented by a quaternion to avoid the gimbal lock problem of Euler angles. For example, if the initial attitude is , the current attitude can be updated by the following differential equation: , where represents quaternion multiplication.

[0027] Next, combined with the linear acceleration vector , the motion trajectory of the head in the future short time (e.g., 0.1 seconds) is predicted. A simplified method is to assume that the head movement is approximately uniform acceleration, and the displacement can be calculated by quadratic integration: ; However, due to the large noise of acceleration data, prediction models (such as linear regression or kinematic filtering) are often used to estimate future positions in actual applications.

[0028] Based on the head attitude and motion trajectory, the system calculates the offset coordinates of the future gaze center in the image frame Assuming the camera of the smart glasses is aligned with the user's line of sight, the gaze center can be approximated as the geometric center of the image frame. By mapping the head rotation angle to the image coordinate system, the offset coordinates can be determined by the following relationship: ; where and are the focal length parameters of the camera, and are the rotation angles in horizontal and vertical directions.

[0029] Finally, a predicted gaze region is generated around the offset coordinates . This region can be a fixed-size rectangle (e.g., 200x200 pixels) or dynamically adjusted according to the head movement speed. For example, if the movement speed is high, the predicted region can be appropriately enlarged to cover a larger potential gaze range. The generated predicted gaze region is passed to the privacy target recognition module as the key region for target detection.

[0030] As shown in FIG. 4, the task of the privacy target recognition module is to identify privacy targets in the predicted gaze region and assign a privacy score to each target. This module combines image acquisition and deep learning techniques to quickly detect sensitive objects such as faces, license plates, and ID cards.

[0031] The module includes two core parts. The image acquisition unit acquires image frames in real time through the camera on the smart glasses, with the resolution and frame rate (e.g., 1080p, 30fps) configured according to application requirements. The acquired image frames are first cropped to the predicted gaze region to reduce the computational load of subsequent processing.

[0032] The privacy recognition model is based on deep convolutional neural networks, such as YOLOv5 or Faster R-CNN. These models are pre-trained to recognize multiple privacy target categories. The detection process includes the following steps: First, the model performs feature extraction on the image frame to generate the bounding box of the target; then, through the classifier, the object category (such as "face" or "license plate") in each bounding box is determined; finally, the privacy score of each target is calculated .

[0033] The privacy score reflects the privacy sensitivity of the target, and its calculation can integrate multiple factors. For example: The score of a face is higher than that of a license plate, and the base value can be set to 1.0 for a face and 0.7 for a license plate.

[0034] Larger targets (such as occupying more than 10% of the image area) may be more sensitive, and the score is increased in proportion.

[0035] If the target is blurred or partially occluded, the score is reduced appropriately.

[0036] For each pixel in the image , if it is within the bounding box of a certain target, then: ; Otherwise, .

[0037] is used to dynamically adjust the privacy score according to the size and clarity of the privacy target that the pixel is in. For example, if the target is large in the image (e.g. a big face) or clear (e.g. the features can be seen), the privacy sensitivity will increase, and the value of will become larger. If the target is small or blurred (e.g. a face in the distance), the sensitivity is lower, and the value of will become smaller. This function makes the privacy score more flexible and can adjust according to the actual situation of the target. The specific value can be determined by custom setting or experiment.

[0038] In addition, this module can also judge the visibility trend of the privacy target. By tracking the motion trajectory of the target (based on multi-frame analysis), the system predicts whether the target will soon enter or leave the predicted gaze area. If the distance between the target boundary and the predicted gaze area is less than a certain threshold (such as 50 pixels), the system will trigger the occlusion condition in advance to ensure the timeliness of privacy protection.

[0039] The blur control module is responsible for calculating the blur intensity of each pixel point according to the head motion trend, the angle difference between the target position and the gaze center, and the privacy score. The design goal of this module is to achieve dynamic and adaptive privacy protection.

[0040] The calculation formula of blur intensity is: ; Where the meanings of the parameters are as follows. The synthetic motion speed is approximated by the angular velocity and acceleration data: ; Where is the acceleration weight coefficient (e.g. 0.1) to balance the influence of rotation and linear motion.

[0041] The angle difference between the pixel point and the predicted gaze center is calculated by the following formula: ; The privacy score Provided by the previous module, is a sigmoid function that normalizes the result to the interval [0, 1].

[0042] The dynamic nature of blur intensity is reflected in the following aspects: when the head movement speed is fast, the blur intensity increases; when the pixel is far from the gaze center, the blur intensity also increases; when the privacy score is high, the blur effect is more pronounced. This design ensures the adaptability of the system in different scenarios.

[0043] In addition, the blur control module also constructs a direction-aware blur propagation field: ; where is the head movement direction vector (derived from angular velocity), is the propagation rate coefficient (such as 0.5). This propagation field makes the blur effect spread ahead along the motion direction, further enhancing the protection effect.

[0044] The image synthesis module generates a blur mask according to the blur intensity and fuses it with the original image frame to form the final output image. This process is divided into two steps.

[0045] First, use a Gaussian blur convolution kernel to process the blur intensity to generate a blur mask. The radius and standard deviation of the Gaussian kernel vary with , for example: where is the proportionality coefficient (such as 10). For each pixel, apply a two-dimensional convolution to calculate the blur value, forming a smooth mask.

[0046] Next, fuse the blur mask with the original image. Fusion can use alpha blending technology.

[0047] To alleviate the abruptness of the occlusion boundary, the system introduces a blur boundary gradual transition module. At the edge of each occlusion region, a gradient band with a width of 20 pixels is constructed, and the blur intensity is smoothly transitioned through linear interpolation, making the image more natural.

[0048] The image output module outputs the processed image frame to a display device (such as the screen of smart glasses) or storage medium (such as an SD card or cloud). The output image frame has superimposed blur masks and can be directly viewed or saved by the user.

[0049] The module also includes a video output sub-module for video sequence processing. After generating a complete video, the sub-module performs time axis analysis on the image frames. If an unobstructed privacy target is detected, the system backtracks and predicts the field of view range based on IMU data, and supplements the generation of an obstruction mask. For example, if an undetected target is found in frame 50, the system can analyze the motion data of the previous frames, recalculate the blur mask, and update the video.

[0050] In another embodiment, the system also integrates a gaze monitoring module for optimizing user experience. This module detects the gaze time of the user in a certain area through eye tracking or head pose analysis. If the gaze time exceeds a threshold (such as 2 seconds), the blur intensity is adjusted to , where to improve clarity.

[0051] The workflow of the system includes data acquisition, field of view prediction, target identification, blur calculation, image synthesis, and output. Each module works closely to ensure real-time performance and protection effect. In social scenarios, the system can blur the faces of passersby; in videos, it can protect sensitive information; and in AR applications, it can dynamically obscure private content.

[0052] The above description is only a preferred embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can make equivalent replacements or changes to the technical solution and the inventive concept of the present application within the scope of the disclosed technology, which should be covered within the protection scope of the present application.

Claims

1. A smart glasses based dynamic privacy mosaic generation system, characterized in that, The method comprises the following steps: a head motion sensing module is configured to collect inertial measurement data of the smart glasses, wherein the inertial measurement data comprises angular velocity data and linear acceleration data; a field of view prediction module is configured to calculate a head motion trend of a user and determine a predicted gaze area based on the inertial measurement data; a privacy target identification module is configured to identify a privacy target in the predicted gaze area and obtain a privacy score; a blur control module is configured to calculate a blur intensity of each pixel point according to the head motion trend, an angle difference between a target position and a gaze center, and the privacy score; an image synthesis module is configured to generate a blur mask and fuse the blur mask with an image frame; an image output module is configured to output the image frame with the superimposed blur mask to a display device or a storage medium.

2. The system of claim 1, wherein, The inertial measurement data includes a three-axis angular velocity vector and a three-axis linear acceleration vector ; The field of view prediction module jointly predicts the offset coordinates of the future gaze center in the image frame based on a three-axis angular velocity vector and a three-axis linear acceleration vector and generates a predicted gaze region around the offset coordinates.

3. The system of claim 1, wherein, The privacy target identification module comprises an image acquisition unit and a privacy identification model. The privacy recognition model is based on a deep convolutional neural network structure, performs target detection on an image collected by an image collection unit, and gives each detected target a privacy score wherein denotes a pixel position in the image.

4. The system of claim 3, wherein, The blur control module is configured to calculate a blur strength The blur strength is determined by the following equation: ； wherein, is a resultant motion velocity module calculated from the angular velocity vector and the linear acceleration vector, represents a pixel point is an angle difference between the predicted gaze center and the predicted gaze center, is a privacy score, is a normalization function.

5. The system of claim 1, wherein, The image synthesis module performs two-dimensional convolution processing on the blur intensity using a Gaussian blur convolution kernel to generate a blur mask, and superimposes the blur mask on an original image frame to form an output image frame.

6. The system of claim 4, wherein, The system further comprises a gaze monitoring module for detecting a gaze time of the user at a certain region, and when the gaze time exceeds a preset threshold , the blur control module adjusts the blur intensity of the corresponding region to , wherein .

7. The system of claim 1, wherein, The privacy target identification module is further configured to judge a visible trend of the privacy target, and determine a occlusion early trigger condition according to a distance between a boundary of the privacy target and the predicted gaze area.

8. The system of claim 1, wherein, The fuzzy control module further constructs a direction-aware fuzzy propulsion field based on the user head motion direction The direction-aware fuzzy propulsion field is given by: ； wherein, is a predicted motion direction vector, is a blur push rate control coefficient, is a privacy score.

9. The system of claim 1, wherein, The image synthesis module further comprises a blur boundary gradual change module, which constructs a blur gradient zone at the edge of each occlusion area to alleviate the image discontinuity phenomenon at the occlusion boundary.

10. The system of claim 1, wherein, The image output module comprises a video output sub-module, which performs time axis analysis on the image frames after generating a complete video sequence, and backtracks the predicted field of view range based on the inertial measurement data and supplements an occlusion mask when an unidentified occluded privacy target is detected.

Images