Screen peep prevention method, system, device and medium based on multispectral imaging

Abstract

This application relates to the technical field of screen privacy protection, and more particularly to screen privacy protection methods, systems, devices, and media based on multispectral imaging. This application first collects user image information, extracts iris features for identity authentication using deep learning algorithms, and analyzes the gaze direction based on reflected light signals from the human eye. When the system detects unauthorized user gaze behavior, it intelligently triggers protection mechanisms according to the threat level, including locally blurring sensitive areas or limiting the viewing angle, causing brightness reduction of the displayed content at specific angles. Multispectral perception improves recognition accuracy, iris authentication ensures the reliability of identity verification, and dynamic adjustment strategies maximize display quality while protecting information security.

Description

Technical Field

[0001] This application relates to the technical field of screen privacy protection, and in particular to screen privacy protection methods, systems, devices and media based on multispectral imaging. Background Technology

[0002] With the widespread use of mobile work and electronic devices in public places, screen information security is becoming increasingly important. Especially in open environments such as public transportation and cafes, users are easily spied on when processing sensitive information, posing a risk of information leakage. Protecting the privacy and security of screen display content has become a pressing technical challenge that needs to be addressed.

[0003] Currently, the main protective measures on the market are physical barriers such as privacy screen protectors, which limit the viewing angle of the display screen to prevent others from peeping. Additionally, there is camera-based eye-tracking technology that analyzes the gaze behavior of people in the surrounding area to determine if there is a spying threat.

[0004] However, privacy screen protectors reduce the display effect from a normal viewing angle and cannot dynamically adjust according to actual threats; while single eye-tracking technology is easily affected by ambient light and has a high false alarm rate, resulting in unsatisfactory protection; these issues need further improvement. Summary of the Invention

[0005] To address the problems of existing anti-spyware methods being unable to dynamically adjust to actual threats and having high false alarm rates, resulting in unsatisfactory protection effects, this application provides a screen anti-spyware method, system, device, and medium based on multispectral imaging, employing the following technical solution: In a first aspect, this application provides a screen privacy protection method based on multispectral imaging, comprising the following steps: User image information is acquired by a multispectral camera located on the bezel of the display screen. The multispectral camera includes a visible light sensor and an infrared sensor. Based on the image information, extract the user's iris features and the reflected light signals from the surrounding human eyes; The iris feature is used for identity authentication to determine whether the user is an authorized user; Based on the reflected light signal from the human eye, the direction of gaze is analyzed to determine whether voyeurism exists. When unauthorized user spying is detected, a specific area of ​​the display screen is dynamically adjusted, including partial blurring and / or limiting the viewing angle.

[0006] By adopting the above technical solutions, with the continuous improvement of informatization levels in fields such as finance, military, and healthcare, the information protection problem of important display terminals with high security requirements in multi-person collaborative environments is becoming increasingly prominent. In order to deal with the risk of information leakage caused by unauthorized personnel visually peeping, major institutions usually use traditional protection methods such as physical isolation and video surveillance. However, existing privacy films will reduce the display effect, physical barriers will affect the collaboration efficiency, and single video surveillance is difficult to prevent peeping behavior in real time. This application first collects user image information, extracts iris features for identity authentication through deep learning algorithms, and analyzes the gaze direction based on the reflected light signal of the human eye. When the system detects the gaze behavior of unauthorized users, it intelligently triggers protection mechanisms according to the threat level, including local blurring of sensitive areas or limiting the viewing angle, so that the brightness of the displayed content is reduced at a specific angle. Multispectral perception improves the recognition accuracy, iris authentication ensures the reliability of identity verification, and dynamic adjustment strategy preserves the display effect to the greatest extent while protecting information security.

[0007] Optionally, the user's iris features and surrounding eye reflected light signals are extracted based on the image information, specifically including the following steps: The visible light sensor acquires an image of the user's eye facing the display screen, and the iris region is located in the eye image; The iris region is divided into multiple concentric ring regions, and the texture features of each ring region are extracted to construct the user's iris feature vector; The infrared sensor scans and collects infrared reflection signals from the area surrounding the display screen within a preset angle range; Spatial filtering is performed on the infrared reflection signal to identify reflection points with characteristics of the human cornea; Calculate the spatial position and reflection intensity of the reflection point relative to the display screen to construct the characteristics of the reflected light signal from the surrounding human eyes.

[0008] By adopting the above technical solution, the visible light sensor is responsible for capturing the user's frontal eye image. The system uses an improved Hough transform algorithm to accurately locate the iris region and innovatively introduces a concentric ring partitioning strategy to divide the iris region into multiple equally wide rings, extracting texture features within each ring to construct a multi-dimensional feature vector. Simultaneously, the infrared sensor scans around the display screen, using a spatial filtering algorithm to identify reflection points with corneal features. Combining geometric optics principles, the spatial coordinates and reflection intensity of the reflection points are calculated, enabling accurate capture of peripheral gaze behavior. The introduction of the concentric ring partitioning strategy significantly improves the expressive power of iris features, and the reflection point analysis method combining spatial filtering and geometric optics provides a reliable basis for accurately judging voyeuristic behavior.

[0009] Optionally, based on the reflected light signal from the human eye, a line-of-sight direction analysis is performed to determine whether voyeurism exists, specifically including the following steps: Acquire multiple sets of sampled data of the characteristics of the reflected light signal from the surrounding human eyes within a preset time window; Based on the spatial location of the reflection point in each set of sampled data, a line-of-sight vector is calculated, which points from the reflection point to the center of the display screen. The gaze vector is assigned a weight value based on the reflection intensity; the greater the reflection intensity, the higher the degree of fixation, and the greater the weight value. Determine whether the line-of-sight vector falls within a preset danger angle range, and whether the weight value corresponding to the line-of-sight vector exceeds a preset threshold; When the number of weighted line-of-sight vectors within the dangerous field of view exceeds a preset percentage within the time window, it is determined that there is spying behavior. Based on the spatial distribution of the weighted line-of-sight vector, the position area of ​​the voyeur relative to the display screen is determined.

[0010] By adopting the above technical solution, the system first continuously collects reflected light signals from surrounding human eyes within a preset time window, and calculates the gaze vector pointing towards the center of the display screen based on the spatial coordinates of the reflection point; it introduces reflection intensity as a quantitative indicator of the degree of gaze, and distinguishes different degrees of gaze behavior by setting weight values; the system compares the gaze vector with a predefined dangerous viewing angle range, and examines whether the weight value exceeds a threshold; when the proportion of weighted gaze vectors located in the dangerous area exceeds a preset value, the system determines it as voyeurism; finally, by analyzing the spatial distribution characteristics of the weighted gaze vectors, the system accurately locates the area where the voyeur is located, providing a basis for subsequent protection strategies.

[0011] Optionally, a specific area of ​​the display screen can be partially blurred, which includes the following steps: Determine the visual area of ​​the display screen that the voyeur might observe based on the voyeur's location area; Sensitivity analysis is performed on the displayed content within the visual area, and the displayed content is classified into sensitivity levels; Different blurring parameters are set according to different sensitivity levels, and adaptive blurring processing is performed on the corresponding sensitivity level regions according to the blurring parameters; The system tracks changes in the location of the spy in real time and dynamically updates the visual area and the blurred area.

[0012] By adopting the above technical solution, the system first calculates the possible screen area that the voyeur can observe based on the acquired location information of the voyeur using a visual geometric model; it then intelligently analyzes the displayed content within this area, assesses the sensitivity of the content through techniques such as feature matching and semantic understanding, and classifies it into different levels; the system configures corresponding blurring parameters for each sensitivity level, and implements differentiated blurring processing for different areas; simultaneously, the system continuously tracks the voyeur's position changes and adjusts the protected area and blurring parameters in real time to ensure the dynamic nature and accuracy of the protection; thus significantly improving the practicality and reliability of the anti-voyeurism solution.

[0013] Optionally, a sensitivity analysis is performed on the displayed content within the visual area, and the displayed content is classified into sensitivity levels, specifically including the following steps: Based on a pre-defined sensitive word library and sensitive image feature library, feature matching is performed on the displayed content; Optical character recognition technology is used to identify the text in the displayed content, and semantic analysis is performed on the identified text to extract keywords and contextual information; Based on the feature matching results and semantic analysis results, calculate the sensitivity score of the displayed content; Based on the sensitivity score, the displayed content is divided into three sensitivity levels: high, medium, and low, and the corresponding areas are marked.

[0014] By adopting the above technical solution, the system first establishes a dual feature library containing a sensitive word library and a sensitive image feature library to perform feature matching on the displayed content. Simultaneously, optical character recognition technology is introduced to accurately extract text from various display formats. Natural language processing technology is then combined to perform deep semantic analysis on the text content, recognizing not only keywords but also considering the context. The system weightedly fuses the results of feature matching and semantic analysis to calculate a comprehensive sensitivity score, and automatically classifies the content into high, medium, and low levels accordingly, achieving precise region labeling. This effectively solves the problems of low efficiency and insufficient accuracy in traditional grading methods.

[0015] Optionally, a specific viewing angle can be limited to a particular area of ​​the display screen, which includes the following steps: Calculate the viewing angle range from a specific area of ​​the display screen to the voyeur, based on the voyeur's location area; Divide the display screen into sub-pixel units, and determine the set of sub-pixel units that need to be restricted by the viewing angle based on the viewing angle range; Adjust the luminous intensity distribution curve of the sub-pixel unit set to cause brightness attenuation of the displayed content outside a specific viewing angle range; The degree of brightness attenuation is dynamically adjusted based on the weight values ​​of the weighted gaze vector; the larger the weight value, the more obvious the brightness attenuation.

[0016] By adopting the above technical solution, the system first uses spatial geometry algorithms to accurately calculate the viewing angle range from each area of ​​the display screen to the viewing position based on the real-time acquired location information of the voyeur; it then uses a sub-pixel partitioning strategy to subdivide the display screen into the smallest control unit and determines the set of sub-pixels that need to be restricted based on the calculated viewing angle range; by modulating the luminous intensity distribution curve of these sub-pixel units, it achieves directional light intensity attenuation outside a specific viewing angle range; the system also dynamically adjusts the degree of brightness attenuation based on the previously acquired degree of gaze, imposing stronger restrictions on viewing angles with higher threat levels; thus solving the problem of difficulty in balancing protection effect and display quality in existing technologies.

[0017] Optionally, based on the stated viewing angle range, a set of sub-pixel units requiring viewing angle restriction is determined, specifically including the following steps: Establish a display screen coordinate system and map the location area of ​​the voyeur to the coordinate system; Based on the aforementioned viewing angle range, calculate the angle between each sub-pixel unit of the display screen and the position of the voyeur; Based on the normal direction of the display screen, the included angle is divided into a safe viewing area and a dangerous viewing area; Cluster analysis is performed on sub-pixel units that fall into the dangerous viewing angle area to form a continuous viewing angle restriction region; Based on the clustering results, determine the set of sub-pixel units that require viewpoint restriction, and calculate the restriction parameters for each unit.

[0018] By adopting the above technical solution, this application first establishes a three-dimensional coordinate system with the center of the display screen as the origin, and accurately maps the detected voyeur position information into this coordinate system; the system calculates the spatial angle formed by each sub-pixel unit of the display screen to the voyeur position one by one, and establishes an accurate angle quantification standard by comparing it with the normal direction of the display screen; a dual-region division mechanism is introduced to divide the viewing angle space into a safe zone and a dangerous zone, and cluster analysis is performed on the sub-pixel units falling into the dangerous viewing angle zone to form a complete restricted area through spatial continuity constraints; finally, the final set of restricted units is determined according to the clustering results, and personalized restriction parameters are calculated for each unit; thus, fine-grained control of the viewing angle is achieved.

[0019] Secondly, this application provides a screen privacy protection system based on multispectral imaging, comprising: The user image information acquisition module is used to acquire user image information through a multispectral camera set on the bezel of the display screen. The multispectral camera includes a visible light sensor and an infrared sensor. The information extraction module is used to extract the user's iris features and the reflected light signals from the surrounding human eyes based on the image information; The user authorization determination module is used to authenticate the identity of the iris feature and determine whether it is an authorized user; The gaze direction analysis module is used to analyze the gaze direction based on the reflected light signal from the human eye to determine whether there is voyeurism. The display screen dynamic adjustment module is used to dynamically adjust a specific area of ​​the display screen when unauthorized user peeping behavior is detected. The dynamic adjustment includes local blurring display and / or limiting viewing angle display.

[0020] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described screen privacy protection method based on multispectral imaging.

[0021] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described screen privacy protection method based on multispectral imaging.

[0022] In summary, this application includes at least one of the following beneficial technical effects: This application first collects user image information, extracts iris features through deep learning algorithms for identity authentication, and analyzes the gaze direction based on the reflected light signals of the human eye. When the system detects the gaze behavior of an unauthorized user, it intelligently triggers protection mechanisms according to the threat level, including local blurring of sensitive areas or limiting the viewing angle, causing brightness attenuation of the displayed content at a specific angle. Multispectral perception improves recognition accuracy, iris authentication ensures the reliability of identity verification, and dynamic adjustment strategies preserve the display effect to the greatest extent while protecting information security. The visible light sensor captures images of the user's eyes from the front. The system uses an improved Hough transform algorithm to accurately locate the iris region and innovatively introduces a concentric ring partitioning strategy to divide the iris region into multiple equally wide rings, extracting texture features from each ring to construct a multi-dimensional feature vector. Simultaneously, the infrared sensor scans the periphery of the display screen, using a spatial filtering algorithm to identify reflection points with corneal features. Combining geometric optics principles, the spatial coordinates and reflection intensity of these reflection points are calculated, enabling precise capture of peripheral gaze behavior. The introduction of the concentric ring partitioning strategy significantly enhances the expressive power of iris features, and the reflection point analysis method combining spatial filtering and geometric optics provides a reliable basis for accurately judging voyeuristic behavior. The system first continuously collects reflected light signals from surrounding human eyes within a preset time window, and calculates the gaze vector pointing towards the center of the display screen based on the spatial coordinates of the reflection point. Reflection intensity is introduced as a quantitative indicator of the degree of gaze, and different levels of gaze behavior are distinguished by setting weight values. The system compares the gaze vector with a predefined dangerous viewing angle range, while also checking whether the weight values ​​exceed a threshold. When the proportion of weighted gaze vectors located within the dangerous area exceeds a preset value, the system determines it as voyeurism. Finally, by analyzing the spatial distribution characteristics of the weighted gaze vectors, the system accurately locates the area where the voyeur is located, providing a basis for subsequent protection strategies. Attached Figure Description

[0023] Figure 1 This is a schematic flowchart of a screen privacy protection method based on multispectral imaging, according to an embodiment of this application. Figure 2 This is a flowchart illustrating step S120 in the screen privacy protection method based on multispectral imaging according to an embodiment of this application. Figure 3 This is a flowchart illustrating step S140 in the screen privacy protection method based on multispectral imaging according to an embodiment of this application. Figure 4 This is a schematic diagram of the blurring display process in the screen privacy protection method based on multispectral imaging in the embodiments of this application; Figure 5 This is a flowchart illustrating step S420 in the screen privacy protection method based on multispectral imaging in an embodiment of this application. Figure 6 This is a schematic diagram of the process of limiting the viewing angle display in the screen privacy protection method based on multispectral imaging in the embodiments of this application; Figure 7 This is a flowchart illustrating step S620 in the screen privacy protection method based on multispectral imaging in an embodiment of this application. Figure 8 This is a schematic diagram of the screen privacy system based on multispectral imaging according to an embodiment of this application; Figure 9 This is an internal structural diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0024] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0025] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0026] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0027] Firstly, this application provides a screen privacy protection method based on multispectral imaging, referring to... Figure 1 It includes the following steps: S110: Acquires user image information via a multispectral camera located on the bezel of the display screen. The multispectral camera includes a visible light sensor and an infrared sensor.

[0028] In this embodiment, a multispectral camera refers to an image acquisition device that simultaneously possesses visible light and infrared imaging capabilities. The visible light sensor is primarily used to acquire high-definition images of the user's facial features and iris texture, while the infrared sensor is specifically designed to capture the reflected signals of infrared light from the human eye. Multiple cameras are positioned around the display screen bezel to ensure coverage from different angles.

[0029] Specifically, the system installs miniature multispectral cameras at the four corners of the display screen. The visible light sensor uses a high-definition RGB imaging chip, and the infrared sensor uses the near-infrared band, paired with an infrared supplementary light. The main control chip synchronizes the clock and registers the two sensors to achieve real-time acquisition and preprocessing of dual-spectral images.

[0030] S120: Extract user iris features and reflected light signals from surrounding human eyes based on image information.

[0031] In this embodiment, iris features refer to the texture features of the iris surface, including unique patterns such as ridges, depressions, stripes, and spots; human eye reflected light signals refer to the position, size, and intensity information of the reflected light spots generated when infrared light irradiates the cornea.

[0032] Specifically, the system first uses a pre-trained facial feature localization model to detect the eye region, then extracts the iris boundary using an edge detection algorithm, and normalizes the iris region. Wavelet transform is used to extract iris texture features and generate feature vectors. For infrared images, the system extracts the reflected light spots from the eyes through adaptive threshold segmentation, recording the spatial coordinates of the reflection points and the reflection intensity parameters.

[0033] S130. Authenticate identity based on iris features to determine whether the user is an authorized user.

[0034] In this embodiment, identity authentication refers to comparing the extracted iris features with the pre-stored authorized user features to determine whether the current user has the permission to view the displayed content.

[0035] Specifically, the system maintains a lightweight authorized user feature index table and employs a hierarchical retrieval strategy to improve authentication efficiency. During authentication, the iris feature vector extracted in real time is compared with the features in the index table to calculate similarity, and the user's identity is determined based on a preset authentication threshold.

[0036] S140. Analyze the direction of gaze based on the reflected light signal from the human eye to determine whether there is any peeping behavior.

[0037] In this embodiment, gaze direction analysis refers to determining the direction of the observer's gaze based on the light signal reflected from the human eye, which is used to identify whether there is any behavior of peeping at the displayed content.

[0038] Specifically, the system pre-establishes a correspondence table between reflected light characteristics and line-of-sight directions. Based on the reflection point location, a line-of-sight vector is calculated, and the front direction of the display screen is set as the reference to establish a viewing angle reference system. The system predefines a dangerous viewing angle range and sets basic judgment rules based on reflection intensity. When a line of sight is detected to be continuously within the dangerous range, it is combined with other parameters to comprehensively determine whether it constitutes peeping behavior.

[0039] S150. When unauthorized user spying behavior is detected, a specific area of ​​the display screen is dynamically adjusted. The dynamic adjustment includes partial blurring and / or limiting the viewing angle.

[0040] In this embodiment, dynamic adjustment refers to adaptive protection processing of the displayed content based on the detection results of the peeping behavior, including selectively reducing the visibility or viewing angle of some areas.

[0041] Specifically, the system divides the display screen into protected areas based on the spying detection results and uses basic image processing methods to adjust the display effect of the corresponding areas. It supports multiple levels of protection strategies, allowing users to select the appropriate processing method based on the specific needs of the scenario. The system manages the protection strategies for different scenarios through a parameter configuration table, ensuring a balance between protection effectiveness and display experience.

[0042] In one embodiment, refer to Figure 2 In step S120, the user's iris features and the reflected light signals from the surrounding human eyes are extracted based on the image information, specifically including the following steps: S121. Acquire an image of the user's eye facing the display screen using a visible light sensor, and locate the iris region in the eye image.

[0043] In this embodiment, the eye image refers to a high-definition image containing the user's eye region acquired by a visible light sensor; iris region localization refers to accurately identifying the boundary range of the iris in the eye image.

[0044] Specifically, the system uses integral projection to quickly segment the eye image, locating the approximate eye region by accumulating gray values ​​in the horizontal and vertical directions. Then, a circular edge detection algorithm is used to determine the inner and outer boundaries of the iris. This algorithm, based on image gradient information, finds the best-matching circular boundary through iterative optimization.

[0045] S122. Divide the iris region into multiple concentric ring regions, extract the texture features within each ring region, and construct the user's iris feature vector.

[0046] In this embodiment, the concentric ring region refers to the ring-shaped region divided at certain radii with the center of the pupil as the center; texture features refer to the image grayscale change features within each ring region.

[0047] Specifically, the system divides the located iris region into eight concentric rings of equal width. Each ring is then transformed using polar coordinates, unfolding into a rectangular image. A directional filter bank is used to extract local texture features, including directional and periodicity parameters. Finally, the features from each ring are sequentially concatenated to form a fixed-dimensional feature vector.

[0048] S123. The infrared sensor scans and collects the infrared reflection signal of the area surrounding the display screen within a preset angle range.

[0049] In this embodiment, the preset angle range refers to the scanning range of the infrared sensor; the infrared reflection signal refers to the intensity of reflection of incident infrared light by the object's surface.

[0050] Specifically, the system uses an array of infrared transmitters and receivers to perform a fan-shaped scan in both horizontal and vertical directions. The scanning range covers an area of ​​±45 degrees on the front of the display screen, and time-division multiplexing is used to sequentially acquire reflected signals from different angles. The system records the correspondence between scanning angles and signal strengths using a lookup table.

[0051] S124. Spatial filtering is performed on the infrared reflection signal to identify reflection points with characteristics of the human cornea.

[0052] In this embodiment, spatial filtering refers to noise suppression in the spatial domain of the acquired reflected signal; corneal features refer to the characteristic reflection pattern of infrared light by the human cornea.

[0053] Specifically, the system first performs median filtering on the original reflection signal to remove discrete noise points. Then, it matches the signal against a pre-defined corneal reflection feature template, which includes characteristic parameters such as corneal curvature and reflection intensity distribution. The system uses relevant matching methods to select reflection points that match the corneal characteristics.

[0054] S125. Calculate the spatial position and reflection intensity of the reflection point relative to the display screen, and construct the characteristics of the reflected light signal from the surrounding human eyes.

[0055] In this embodiment, spatial position refers to the coordinates of the reflection point relative to the display screen in three-dimensional space; reflection intensity refers to the energy of the reflected light signal.

[0056] Specifically, the system establishes a spatial coordinate system with the center of the display screen as the origin. Based on the installation position and scanning angle of the infrared sensor, it calculates the spatial coordinates of the reflection point using the principle of triangulation. Simultaneously, it records the signal intensity value of the reflection point and performs normalization processing. The system combines the spatial location and reflection intensity information to construct a feature descriptor for the reflected light signal.

[0057] In one embodiment, refer to Figure 3 In step S140, the direction of gaze is analyzed based on the reflected light signal from the human eye to determine whether voyeurism exists. This includes the following steps: S141. Acquire multiple sets of sampled data of the characteristics of reflected light signals from surrounding human eyes within a preset time window.

[0058] In this embodiment, the preset time window refers to the time range within which the system performs line-of-sight analysis; the sampling data refers to multiple sets of reflected light signal feature records acquired within this time range.

[0059] Specifically, the system continuously acquires reflected light signal characteristics within a sliding time window. The sampled data includes the spatial coordinates of the reflection point and the reflection intensity information.

[0060] S142. Based on the spatial location of the reflection point in each set of sampled data, calculate the line-of-sight vector, which points from the reflection point to the center of the display screen.

[0061] In this embodiment, the gaze vector refers to the direction vector from the reflection point to the center of the display screen, which is used to characterize the observer's gaze direction.

[0062] Specifically, the system establishes a spatial vector calculation model, using the center of the display screen as the origin to create a coordinate system. For each reflection point, the unit vector of the line connecting it to the center of the display screen is calculated. The system maintains a vector cache table to temporarily store the calculated line-of-sight vectors.

[0063] S143. Assign weight values ​​to the gaze vector based on the reflection intensity. The greater the reflection intensity, the higher the degree of fixation, and the greater the weight value.

[0064] In this embodiment, the weight value refers to the importance of the line of sight calculated based on the reflection intensity, which is used to distinguish the threat level of different line of sight vectors.

[0065] Specifically, the system pre-establishes a mapping table between reflection intensity and weight values. After normalizing the reflection intensity, the corresponding weight value is quickly obtained by looking up the table. The larger the weight value, the higher the degree of fixation in that line of sight.

[0066] S144. Determine whether the line-of-sight vector falls within the preset danger angle range and whether the weight value corresponding to the line-of-sight vector exceeds the preset threshold.

[0067] In this embodiment, the dangerous viewing angle range refers to the range of line of sight that may lead to information leakage; the preset threshold refers to the weight value standard for determining the threat level.

[0068] Specifically, the system determines whether the line-of-sight vector falls within a danger zone through angle calculation. Using the display screen's normal direction as a reference, a predefined danger angle range is established. Simultaneously, the weight value corresponding to the line-of-sight vector is checked and compared with a preset threat threshold.

[0069] S145. When the number of weighted line-of-sight vectors within the dangerous field of view exceeds a preset percentage within a time window, it is determined that there is spying behavior.

[0070] In this embodiment, the preset proportion refers to the threshold of the proportion of dangerous line-of-sight vectors within the time window, which is used to comprehensively judge voyeuristic behavior.

[0071] Specifically, the system counts the number of line-of-sight vectors that meet the dangerous conditions within a statistical time window and calculates their proportion of the total number of samples. By comparing this proportion with a preset threshold, it determines whether there is continuous spying behavior.

[0072] S146. Determine the position area of ​​the voyeur relative to the display screen based on the spatial distribution of the weighted line-of-sight vector.

[0073] In this embodiment, the location area refers to the approximate spatial range around the display screen where the spy is located, which is used for subsequent adjustments to the protection strategy.

[0074] Specifically, the system performs spatial distribution analysis on the starting points of detected dangerous line-of-sight vectors. Using a region division method, the starting points of the line-of-sight vectors are mapped to predefined spatial regions. The density distribution of points within these regions determines the range of locations most likely occupied by the voyeur.

[0075] In one embodiment, refer to Figure 4 To partially blur a specific area of ​​the display screen, the following steps are involved: S410. Determine the visual area of ​​the display screen that the voyeur may observe, based on the voyeur's location area.

[0076] In this embodiment, the visual area refers to the range of the display screen that can be observed from the location of the voyeur.

[0077] Specifically, the system calculates the range of display areas that the voyeur can observe based on a pre-established visual accessibility model and the voyeur's spatial location.

[0078] S420. Perform sensitivity analysis on the displayed content within the visual area and classify the displayed content into sensitivity levels.

[0079] In this embodiment, sensitivity analysis refers to assessing the sensitivity of the displayed content; sensitivity levels are used to distinguish the protection priorities of different content.

[0080] Specifically, the system employs a rule-based rapid classification method, pre-defining a sensitive content feature library, including keyword and specific formatting rules. It performs feature matching on the displayed content within the visual area and classifies the content into three sensitivity levels: high, medium, and low, based on the matching results.

[0081] S430. Set different blurring parameters according to different sensitivity levels, and perform adaptive blurring processing on the corresponding sensitivity level regions according to the blurring parameters.

[0082] In this embodiment, the blurring parameters include processing parameters such as blur radius and intensity; adaptive blurring processing refers to dynamically adjusting the blurring effect according to the sensitivity level.

[0083] Specifically, the system establishes a correspondence table between sensitivity levels and blurring parameters. High-sensitivity areas use larger blurring parameters to ensure the content is completely unrecognizable; medium-sensitivity areas use a moderate blurring effect; and low-sensitivity areas undergo only slight processing. The system uses basic image blurring algorithms, adjusting parameters to achieve different degrees of blurring effects.

[0084] S440: Real-time tracking of changes in the location area of ​​the voyeur, dynamically updating the visual area range and the blurred area.

[0085] In this embodiment, real-time tracking refers to continuously monitoring changes in the location of the spy; dynamic updating refers to adjusting the protection strategy in a timely manner based on changes in location.

[0086] Specifically, the system employs a lightweight position tracking algorithm that calculates position offset based on changes in infrared reflection signals. When a significant position change is detected, the visual region is recalculated, and the blurring region is updated.

[0087] In one embodiment, refer to Figure 5In step S420, a sensitivity analysis is performed on the displayed content within the visual area, and the displayed content is classified into sensitivity levels. This specifically includes the following steps: S421. Based on the preset sensitive word library and sensitive image feature library, perform feature matching on the displayed content.

[0088] In this embodiment, the sensitive word library refers to a predefined set of sensitive information keywords; the sensitive image feature library contains feature descriptions of image content that needs to be protected.

[0089] Specifically, the system maintains a hierarchical sensitive feature index table, containing both text and image components. The text component uses a trie structure to store sensitive words, supporting fast retrieval and fuzzy matching. The image component stores basic feature templates, such as feature descriptors like ID photo formats and table structures.

[0090] S422. Optical character recognition technology is used to recognize the text in the displayed content, and semantic analysis is performed on the recognized text content to extract keywords and contextual information.

[0091] In this embodiment, optical character recognition technology is used to extract text information from the displayed content; semantic analysis refers to understanding the meaning and relevance of the text content.

[0092] Specifically, the system uses a lightweight OCR engine for text recognition, primarily focusing on text regions with specific formats and layouts. For the recognized text, a rule-based fast semantic analysis method is employed to extract key information.

[0093] S423. Calculate the sensitivity score of the displayed content based on the feature matching results and semantic analysis results.

[0094] In this embodiment, the sensitivity score refers to a numerical indicator that quantifies the sensitivity of the displayed content, taking into account both feature matching and semantic analysis results.

[0095] Specifically, the system establishes a scoring rule table to assign base scores to different types of sensitive features. The number and importance of feature matches will increase the score, and contextual relevance will also affect the final score.

[0096] S424. Based on the sensitivity score, the displayed content is divided into three sensitivity levels: high, medium, and low, and the corresponding areas are marked.

[0097] In this embodiment, the sensitivity level classification refers to classifying the displayed content according to the sensitivity score, which facilitates subsequent differentiated protection.

[0098] Specifically, the system presets two scoring thresholds to classify displayed content into three sensitivity levels: high, medium, and low. For content spanning multiple regions, the highest sensitivity level is applied. The system uses bitmap markers to record the sensitivity information for different regions, supporting quick querying and updates.

[0099] In one embodiment, refer to Figure 6 To limit the viewing angle of a specific area of ​​the display screen, the following steps are included: S610. Calculate the viewing angle range from a specific area of ​​the display screen to the voyeur based on the location area of ​​the voyeur.

[0100] In this embodiment, the viewing angle range refers to the angle from the observer's position to a specific area of ​​the display screen; the specific area refers to the display area where the viewing angle needs to be limited.

[0101] Specifically, the system establishes a spatial geometric model, using the display screen plane as the reference plane. Based on the observer's spatial coordinates, it calculates the viewing angle formed by the observer and various boundary points of the display area. The system maintains an angle lookup table to quickly obtain the viewing angle range from different positions.

[0102] S620: Divide the display screen into sub-pixel units, and determine the set of sub-pixel units that need to be restricted by viewing angle based on the viewing angle range.

[0103] In this embodiment, a sub-pixel unit refers to the smallest controllable light-emitting unit of the display screen; a sub-pixel unit set refers to a group of pixels that require viewing angle limitation.

[0104] Specifically, the system divides the display screen into sub-pixel grids according to its physical structure and establishes a position index table. Based on the pre-calculated viewing angle range, the affected sub-pixel units are determined using ray projection. The system employs a region growing algorithm to expand the selection range, ensuring the continuity of the viewing angle limiting effect.

[0105] S630: Adjust the luminous intensity distribution curve of the sub-pixel unit set to cause brightness attenuation of the displayed content outside a specific viewing angle range.

[0106] In this embodiment, the luminous intensity distribution curve describes the brightness variation characteristics of the sub-pixel unit under different viewing angles; brightness attenuation refers to the viewing angle limitation effect achieved by adjusting the luminous characteristics.

[0107] Specifically, the system presets a basic luminous intensity template, including brightness distribution parameters for both normal and limited viewing angles. Target brightness values ​​for sub-pixel units at different viewing angles are obtained through table lookups. The system uses interpolation algorithms to achieve smooth brightness transitions, avoiding abrupt changes in display effects. Corresponding parameter configuration tables are maintained for different types of display panels.

[0108] S640. The degree of brightness attenuation is dynamically adjusted according to the weight value of the weighted gaze vector. The larger the weight value, the more obvious the brightness attenuation.

[0109] In this embodiment, dynamic adjustment refers to changing the display effect in real time according to the severity of the spying behavior; the weight value reflects the threat level of the spying behavior.

[0110] Specifically, the system establishes a mapping relationship between weight values ​​and brightness attenuation parameters. When the detected gaze vector weight value is large, the intensity of brightness attenuation is increased; when the weight value is small, a milder limiting effect is adopted.

[0111] In one embodiment, refer to Figure 7 In step S620, based on the viewing angle range, the set of sub-pixel units that need to be restricted by the viewing angle is determined, specifically including the following steps: S621. Establish a coordinate system for the display screen and map the location area of ​​the voyeur to the coordinate system.

[0112] In this embodiment, the display screen coordinate system refers to a three-dimensional spatial coordinate system established with the display screen as a reference; position mapping refers to transforming the position of the voyeur into this coordinate system.

[0113] Specifically, the system establishes a right-handed coordinate system with the center of the display screen as the origin and the display screen plane as the XY plane. Through simple coordinate transformations, the spatial position of the voyeur is converted to this coordinate system.

[0114] S622. Based on the viewing angle range, calculate the angle between each sub-pixel unit of the display screen and the position of the voyeur.

[0115] In this embodiment, the included angle calculation refers to determining the angle difference between the light emission direction of the sub-pixel unit and the viewing direction; the sub-pixel unit is the smallest controllable light emission unit of the display screen.

[0116] Specifically, the system employs a vector calculation method, using the display screen normal as the reference direction. For each sub-pixel unit, it calculates the direction vector from its position to the viewer's position and determines the angle between the vector vector and the normal direction.

[0117] S623. Based on the normal direction of the display screen, the included angle is divided into a safe viewing area and a dangerous viewing area.

[0118] In this embodiment, the safe viewing area refers to the normal viewing angle range; the dangerous viewing area refers to the viewing angle range that needs to be restricted in display.

[0119] Specifically, the system predefines a safe viewing angle threshold, typically set within ±30 degrees of the display screen's normal direction. The calculated angle is compared to this threshold to delineate safe and dangerous zones. The system uses bitmap markers to record the region attributes of each sub-pixel unit.

[0120] S624. Perform cluster analysis on sub-pixel units that fall into the dangerous viewing area to form a continuous viewing area restriction region.

[0121] In this embodiment, cluster analysis refers to combining adjacent dangerous viewpoint sub-pixel units into continuous regions; continuous regions facilitate unified control and parameter settings.

[0122] Specifically, the system uses a region growing algorithm for clustering. Starting from any pixel in the danger zone, it gradually expands to adjacent pixels in the danger zone until it can no longer be expanded.

[0123] S625. Based on the clustering results, determine the set of sub-pixel units that need to be restricted by the viewpoint, and calculate the restriction parameters for each unit.

[0124] In this embodiment, the limiting parameter refers to the specific parameter value that controls the light emission characteristics of the sub-pixel unit; the parameter calculation needs to take into account the viewing angle and position factors.

[0125] Specifically, the system calculates the center position and average viewing angle for each cluster region. Based on a preset parameter template, and combining the viewing angle and spatial location, it calculates the constraint parameters for each sub-pixel unit.

[0126] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0127] Secondly, this application provides a screen privacy protection system based on multispectral imaging. The screen privacy protection system based on multispectral imaging of this application will be described below in conjunction with the above-mentioned screen privacy protection method based on multispectral imaging.

[0128] Reference Figure 8 A screen privacy system based on multispectral imaging, comprising: The user image information acquisition module is used to acquire user image information through a multispectral camera set on the bezel of the display screen. The multispectral camera includes a visible light sensor and an infrared sensor. The information extraction module is used to extract the user's iris features and the reflected light signals from the surrounding human eyes based on image information; The user authorization determination module is used to authenticate the identity of an iris feature and determine whether the user is an authorized user. The line-of-sight analysis module is used to analyze the line-of-sight direction based on the light signals reflected from the human eye to determine whether there is any peeping behavior. The display screen dynamic adjustment module is used to dynamically adjust a specific area of ​​the display screen when unauthorized user peeping behavior is detected. The dynamic adjustment includes partial blurring and / or limiting the viewing angle.

[0129] In one embodiment, this application provides an electronic device, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a screen privacy protection method based on multispectral imaging.

[0130] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0131] In one embodiment, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0132] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0133] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A screen anti-peeping method based on multi-spectrum imaging, characterized in that, Includes the following steps: User image information is acquired by a multispectral camera located on the bezel of the display screen. The multispectral camera includes a visible light sensor and an infrared sensor. Based on the image information, extract the user's iris features and the reflected light signals from the surrounding human eyes; The iris feature is used for identity authentication to determine whether the user is an authorized user; Based on the reflected light signal from the human eye, the direction of gaze is analyzed to determine whether voyeurism exists. When unauthorized user spying is detected, a specific area of ​​the display screen is dynamically adjusted, including partial blurring and / or limiting the viewing angle.

2. The multi-spectral imaging based screen peeking prevention method of claim 1, wherein, Extracting user iris features and surrounding eye reflected light signals based on the image information includes the following steps: The visible light sensor acquires an image of the user's eye facing the display screen, and the iris region is located in the eye image; The iris region is divided into multiple concentric ring regions, and the texture features of each ring region are extracted to construct the user's iris feature vector; The infrared sensor scans and collects infrared reflection signals from the area surrounding the display screen within a preset angle range; Spatial filtering is performed on the infrared reflection signal to identify reflection points with characteristics of the human cornea; Calculate the spatial position and reflection intensity of the reflection point relative to the display screen to construct the characteristics of the reflected light signal from the surrounding human eyes.

3. The multi-spectral imaging based screen peeking prevention method of claim 2, wherein, Based on the reflected light signal from the human eye, the direction of gaze is analyzed to determine whether voyeurism has occurred. This includes the following steps: Acquire multiple sets of sampled data of the characteristics of the reflected light signal from the surrounding human eyes within a preset time window; Based on the spatial location of the reflection point in each set of sampled data, a line-of-sight vector is calculated, which points from the reflection point to the center of the display screen. The gaze vector is assigned a weight value based on the reflection intensity; the greater the reflection intensity, the higher the degree of fixation, and the greater the weight value. Determine whether the line-of-sight vector falls within a preset danger angle range, and whether the weight value corresponding to the line-of-sight vector exceeds a preset threshold; When the number of weighted line-of-sight vectors within the dangerous field of view exceeds a preset percentage within the time window, it is determined that there is spying behavior. Based on the spatial distribution of the weighted line-of-sight vector, the position area of ​​the voyeur relative to the display screen is determined.

4. The screen privacy protection method based on multispectral imaging according to claim 3, characterized in that, To partially blur a specific area of ​​the display screen, the following steps are involved: Determine the visual area of ​​the display screen that the voyeur might observe based on the voyeur's location area; Sensitivity analysis is performed on the displayed content within the visual area, and the displayed content is classified into sensitivity levels; Different blurring parameters are set according to different sensitivity levels, and adaptive blurring processing is performed on the corresponding sensitivity level regions according to the blurring parameters; The system tracks changes in the location of the spy in real time and dynamically updates the visual area and the blurred area.

5. The screen privacy protection method based on multispectral imaging according to claim 4, characterized in that, Sensitivity analysis is performed on the displayed content within the visual area, and the displayed content is classified into sensitivity levels. The specific steps include the following: Based on a pre-defined sensitive word library and sensitive image feature library, feature matching is performed on the displayed content; Optical character recognition technology is used to identify the text in the displayed content, and semantic analysis is performed on the identified text to extract keywords and contextual information; Based on the feature matching results and semantic analysis results, calculate the sensitivity score of the displayed content; Based on the sensitivity score, the displayed content is divided into three sensitivity levels: high, medium, and low, and the corresponding areas are marked.

6. The screen privacy protection method based on multispectral imaging according to claim 3, characterized in that, To limit the viewing angle of a specific area of ​​the display screen, the following steps are included: Calculate the viewing angle range from a specific area of ​​the display screen to the voyeur, based on the voyeur's location area; Divide the display screen into sub-pixel units, and determine the set of sub-pixel units that need to be restricted by the viewing angle based on the viewing angle range; Adjust the luminous intensity distribution curve of the sub-pixel unit set to cause brightness attenuation of the displayed content outside a specific viewing angle range; The degree of brightness attenuation is dynamically adjusted based on the weight values ​​of the weighted gaze vector; the larger the weight value, the more obvious the brightness attenuation.

7. The screen privacy protection method based on multispectral imaging according to claim 6, characterized in that, Based on the aforementioned viewing angle range, the set of sub-pixel units that require viewing angle restriction is determined, specifically including the following steps: Establish a display screen coordinate system and map the location area of ​​the voyeur to the coordinate system; Based on the aforementioned viewing angle range, calculate the angle between each sub-pixel unit of the display screen and the position of the voyeur; Based on the normal direction of the display screen, the included angle is divided into a safe viewing area and a dangerous viewing area; Cluster analysis is performed on sub-pixel units that fall into the dangerous viewing angle area to form a continuous viewing angle restriction region; Based on the clustering results, determine the set of sub-pixel units that require viewpoint restriction, and calculate the restriction parameters for each unit.

8. A screen privacy protection system based on multispectral imaging, characterized in that, include: The user image information acquisition module is used to acquire user image information through a multispectral camera set on the bezel of the display screen. The multispectral camera includes a visible light sensor and an infrared sensor. The information extraction module is used to extract the user's iris features and the reflected light signals from the surrounding human eyes based on the image information; The user authorization determination module is used to authenticate the identity of the iris feature and determine whether it is an authorized user; The gaze direction analysis module is used to analyze the gaze direction based on the reflected light signal from the human eye to determine whether there is voyeurism. The display screen dynamic adjustment module is used to dynamically adjust a specific area of ​​the display screen when unauthorized user peeping behavior is detected. The dynamic adjustment includes local blurring display and / or limiting viewing angle display.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the screen privacy protection method based on multispectral imaging as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the screen privacy protection method based on multispectral imaging as described in any one of claims 1-7.

Images