Secret-related file compact shelf monitoring management method and device, electronic equipment and medium

Abstract

The invention relates to the technical field of image processing, and provides a secret-related file compact shelf monitoring management method and device, electronic equipment and a medium, and the method comprises the steps: processing a monitoring image, and obtaining image data; inputting the image data into a material identification model to obtain material information; performing fuzzy processing on an area of which the material information is a preset archive material in the monitoring image by adopting a dynamic Gaussian blur algorithm; and utilizing the monitoring image and the motion prediction model to obtain a predicted archive trajectory of at least one future moment, and performing fuzzy processing on an area, of which the material information is the preset archive material, in the monitoring image of at least one future moment according to the predicted archive trajectory. Through the processes of file material identification, motion trail prediction and real-time fuzzification processing, manual intervention links are reduced, the operation complexity is reduced, meanwhile, the high efficiency and consistency of the processing process are ensured, omissions or errors possibly caused by manual operation are avoided, and the management safety and supervision efficiency of confidential files are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method, device, electronic device and medium for monitoring and managing a classified file compact shelf. Background Art

[0002] The safe storage of classified files is an important part of national security and information confidentiality, and the storage process needs to strictly prevent information leakage. Traditional storage of classified files usually relies on manual supervision or basic monitoring equipment, but there are significant defects. Relying on manual supervision, the efficiency is low and it is easy to miss something, and it is difficult to track the process of file access in real time; while relying on basic monitoring equipment, although it can record the behavior of file access and storage, the camera may capture key information such as file numbers and titles on the surface of the file box, and these information themselves are classified content, which is easy to cause secondary leakage of classified information and form new security risks.

[0003] At present, through video occlusion technology, sensitive areas can be prevented from being recorded by video monitoring, but this type of technology requires pre-set fixed occlusion areas, and the positions of files in the archive room will change dynamically due to daily access, resulting in the inability to match the pre-set occlusion areas with the actual sensitive positions, and it is difficult to continuously and effectively protect information. Or use a combination of OCR technology and privacy coding to perform blurring processing after text recognition. However, this method is mainly designed for plane text, and the text to be recognized needs to be in a horizontal state. However, in the actual archive room environment, the file box is often in an inclined state, or the acquired image is in an inclined state due to the installation angle of the monitor, resulting in the OCR algorithm being prone to text detection failure, and it is easy to miss sensitive information due to the inability to detect text in complex scenarios, and the problem of information leakage has not been fundamentally solved. Summary of the Invention

[0004] The problem solved by the present invention is how to improve the security and supervision efficiency of file management.

[0005] To solve the above problems, the present invention provides a method, device, electronic device and medium for monitoring and managing a classified file compact shelf.

[0006] In a first aspect, the present invention provides a method for monitoring and managing a classified file compact shelf, including: Processing the monitoring image obtained by photographing the classified file compact shelf to obtain image data; Inputting the image data into a material recognition model to obtain material information, wherein the material recognition model is trained by performing material marking on the historical image data of the classified file compact shelf and according to the marking result and the historical image data; Performing blurring processing on the area of the monitoring image where the material information is a preset file material by using a dynamic Gaussian blurring algorithm; Using the monitoring image and the motion prediction model to obtain the predicted file trajectories at at least one future moment, and blurring the areas in the monitoring images at at least one future moment where the material information is the preset file material according to the predicted file trajectories, wherein the motion prediction model is trained according to the historical file motion information.

[0007] Optionally, the material recognition model includes: A feature extraction module for extracting features from the image data; An attention mechanism module for determining the key areas in the image data according to the extracted features; A feature fusion module for fusing the extracted features of the key areas to obtain fused features; A classifier for classifying the material according to the fused features to obtain the material information of the key areas.

[0008] Optionally, the processing of the monitoring image obtained by photographing the classified file rack to obtain the image data includes: Obtaining RGB image data and multispectral imaging data according to the monitoring image; Performing grayscale processing, median filtering operation and histogram equalization operation on the RGB image data and the multispectral imaging data respectively to obtain the image data.

[0009] Optionally, the using the dynamic Gaussian blur algorithm to blur the areas in the monitoring image where the material information is the preset file material includes: Obtaining environmental parameters according to the image data; Adjusting the blur radius of the areas where the material information is the preset file material according to the environmental parameters, and adjusting the blur intensity according to the blur radius.

[0010] Optionally, the environmental parameters include light intensity data; the adjusting the blur intensity according to the blur radius includes: When the blur radius is less than or equal to the preset radius, the blur intensity is the initial intensity; When the blur radius is greater than the preset radius, increasing the blur intensity based on the initial intensity, and the blur intensity has a positive correlation with the blur radius; Or, when the blur radius is greater than the preset radius, obtaining an intensity base value based on a preset radius factor, the initial intensity and the blur radius, determining an intensity adjustment value according to a preset light intensity factor and the light intensity data, and obtaining the sum of the intensity base value and the intensity adjustment value as the blur intensity.

[0011] Optionally, the motion prediction model includes: An optical flow method module, configured to obtain the optical flow information of the area in the monitoring image where the material information is the preset file material, and obtain the predicted motion trend of the area where the material information is the preset file material at at least one future moment according to the optical flow information; A Kalman filtering module, configured to obtain the file motion state of the area in the monitoring image where the material information is the preset file material, and obtain the predicted file trajectory of the area where the material information is the preset file material at at least one future moment according to the file motion state and the predicted motion trend.

[0012] Optionally, the material recognition model further includes an environment fusion module, and the output of the environment fusion module is connected to the input of the feature extraction module; the environment fusion module is configured to fuse the environment parameters obtained according to the image data in advance and the image data.

[0013] In a second aspect, the present invention provides a monitoring and management device for a classified file rack, including: A processing unit, configured to process the monitoring image obtained by photographing the classified file rack to obtain image data; An identification unit, configured to input the image data into a material recognition model to obtain material information, where the material recognition model is obtained by performing material marking on the historical image data of the classified file rack and training according to the marking result and the historical image data; A blurring unit, configured to perform blurring processing on the area in the monitoring image where the material information is the preset file material by using a dynamic Gaussian blurring algorithm; A prediction unit, configured to use the monitoring image and a motion prediction model to obtain a predicted file trajectory at at least one future moment, and perform blurring processing on the area in the monitoring image at at least one future moment where the material information is the preset file material according to the predicted file trajectory, where the motion prediction model is obtained by training according to historical file motion information.

[0014] In a third aspect, the present invention provides an electronic device, including a memory and a processor; The memory is configured to store a computer program; The processor is configured to, when executing the computer program, implement the monitoring and management method for a classified file rack as described in the first aspect.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the monitoring and management method for a classified file rack as described in the first aspect is implemented.

[0016] The beneficial effects of the monitoring and management method for the classified file compact storage rack of the present invention are as follows: The image data obtained from the monitoring images of the classified file compact storage rack is input into a pre-trained material recognition model to obtain material information. Then, the dynamic Gaussian blur algorithm is used to blur the areas in the monitoring images where the material information is the preset file material. Compared with the related technology that first recognizes text and then performs privacy encoding on the text part, the protection object is upgraded from the text level to the material level. Blurring is performed immediately after the preset file material is recognized, so that only the non-classified file access behavior images are retained in the monitoring records, ensuring that classified content (such as the text information after text recognition) is not retained in the monitoring link, eliminating potential safety hazards at the source. At the same time, it can ensure that the closed-loop operation from detection to protection is completed in a short time, increasing the security of file management and the supervision efficiency. The predicted file trajectories at at least one future moment are obtained by using the monitoring images and the motion prediction model, and the areas in the monitoring images at at least one future moment where the material information is the preset file material are blurred according to the predicted file trajectories. To address the dynamic change problem caused by the movement of file boxes, the motion prediction model is used to predict the movement trajectory of the file in advance, and accurate trajectory prediction is achieved in a high-frame-rate video stream to blur the areas corresponding to the preset file material, improving the intelligence and automation level of classified file management. Through the processes of file material recognition, motion trajectory prediction, and real-time blurring in this embodiment, the manual intervention link is reduced, the operation complexity is lowered, and at the same time, the efficiency and consistency of the processing process are ensured, avoiding omissions or errors that may be caused by manual operations, and significantly improving the security of classified file management and the supervision efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic flowchart of the monitoring and management method for the classified file compact storage rack according to an embodiment of the present invention; Figure 2 is a schematic structural diagram of the material recognition model according to an embodiment of the present invention; Figure 3 is a schematic structural diagram of the motion prediction model according to an embodiment of the present invention; Figure 4 is a schematic structural diagram of the monitoring and management device for the classified file compact storage rack according to an embodiment of the present invention; Figure 5 is a schematic structural diagram of the electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] To make the above objects, features, and advantages of the present invention more apparent and understandable, the following provides a detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments described herein. Instead, these embodiments are provided to more thoroughly and completely understand the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not used to limit the protection scope of the present invention.

[0019] It should be understood that the steps recorded in the method embodiments of the present invention can be executed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this regard.

[0020] The term "including" and its variants used herein are open-ended, that is, "including but not limited to"; the term "based on" is "at least partially based on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts such as "first" and "second" mentioned in the present invention are only used to distinguish different devices, modules, or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules, or units.

[0021] It should be noted that the modifications of "one" and "multiple" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless clearly stated otherwise in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] In view of the problems existing in the above related technologies, this embodiment provides a method, device, electronic device, and medium for monitoring and managing a classified file compact shelf.

[0024] As Figure 1 shown, a method for monitoring and managing a classified file compact shelf provided by an embodiment of the present invention includes: Step S1: Process the monitoring images obtained by photographing the classified file compact shelf to obtain image data.

[0025] Specifically, monitor images are obtained based on the monitor video of the classified archive rack. The monitor images can be each frame image of the monitor video, and then the monitor images are preprocessed. For example, bilinear interpolation, bicubic interpolation, etc. are used to process the image resolution, and Gaussian filtering, bilateral filtering, etc. are used for denoising processing, etc., to improve the quality of image data acquisition of the monitor images and make them more suitable for subsequent analysis and processing. The image data can be RGB image data and / or multispectral imaging data.

[0026] Step S2: Input the image data into the material recognition model to obtain material information, where the material recognition model is trained by marking the material of the historical image data of the classified archive rack and according to the marking results and the historical image data.

[0027] Specifically, the materials at various positions in the monitor video of the classified archive rack are recognized, the materials of the file boxes and the corresponding positions are recognized, and subsequent precise processing is carried out for the positions of the file boxes. The historical image data of the classified archive rack is retrieved in advance, and the material information in the historical image data is marked for materials to obtain the marking results corresponding to the historical image data. The historical image data and the marking results are used as the training set and the test set to train the preset model constructed in advance to obtain a trained material recognition model. The input of the material recognition model is image data, and the output is the material information of each region in the image data. The preset model can be constructed by using, for example, Convolutional Neural Networks (CNN). Finally, the obtained image data is input into the trained material recognition model to obtain the material information.

[0028] Step S3: Use the dynamic Gaussian blur algorithm to blur the region in the monitor image where the material information is the preset archive material.

[0029] Specifically, according to the material information obtained in step S1, regions corresponding to the same or similar preset archive materials with different material information are obtained, and the dynamic Gaussian blur algorithm is used to blur this region. Among them, algorithms such as Euclidean distance, cosine similarity, KL divergence, JS divergence, etc. can be used to calculate the similarity between different material information and the preset archive material, and then the regions with similarity greater than the preset threshold are determined as the target archive regions, that is, the regions that need to be subject to privacy processing, effectively increasing the accuracy of archive privacy processing.

[0030] Step S4: Use the monitor image and the motion prediction model to obtain at least one predicted archive trajectory at a future moment, and blur the region in the monitor image at at least one future moment where the material information is the preset archive material according to the predicted archive trajectory, where the motion prediction model is trained according to the historical archive motion information.

[0031] Specifically, since the monitoring and management are based on the real-time monitoring of the archives room, when, for example, staff members retrieve archives, the positions of the archives will change in real time. Therefore, a motion prediction model is adopted to predict the motion positions of the archives at at least one future moment, and the areas with the material information of the preset archive material in the monitoring images at at least one future moment are blurred in real time according to the obtained predicted archive trajectories, thereby enhancing the security of archive management. First, the historical archive motion information in the historical monitoring video is retrieved to train the preset prediction model, and a trained motion prediction model is obtained. Among them, the input of the motion prediction model is the monitoring image, and the output is the predicted archive trajectory, that is, the predicted archive motion trajectory. Then, the obtained monitoring image is input into the trained motion prediction model to obtain the predicted archive trajectory, and the positions of the areas with the material information of the preset archive material in the monitoring images at at least one future moment are determined according to the predicted archive trajectory, and a dynamic Gaussian blur algorithm is used for blurring. For example, the monitoring image is obtained in real time. When the monitoring image at a future moment is obtained, the positions of the areas with the material information of the preset archive material determined by the predicted archive trajectory in the monitoring image at this moment are blurred.

[0032] In addition, during the above process, the time and operations of each process are recorded, and the data is transmitted to the backend server in a wired manner through the camera communication unit using the Ethernet interface. After receiving the data, the backend server stores it in a secure database using encryption technology, and at the same time, the stored data can be managed and analyzed, such as querying and retrieving according to conditions such as time and archive number, providing support for archive management, facilitating subsequent archive management investigations and other operations, helping to achieve the organic unity of the accuracy of classified archive supervision and privacy protection. While blurring the classified information, the key behavioral data such as the time, personnel, and location of archive access and storage are completely recorded, which not only meets the tracing requirements for the archive transfer process in security management but also avoids the privacy risks caused by excessive collection of sensitive information.

[0033] Additionally, in terms of hardware design, an intelligent camera with an open interface (such as the SDC6582-I of Huawei) is selected. Through the computing power and interfaces provided by the camera, the monitoring and management method for classified file compact shelves is embedded. The hardware architecture of the camera mainly consists of an image acquisition unit, an algorithm processing unit, a storage unit, and a communication unit. The image acquisition unit is composed of a high-definition lens and an image sensor. The high-definition lens has high resolution, wide viewing angle, and good optical performance, and can clearly capture the surface information of the file box under different lighting conditions. The high-performance chip built into the camera has multi-core processing capabilities and powerful floating-point operation capabilities, providing computing resources for the operation of the monitoring and management method for classified file compact shelves. A high-speed storage medium is used to temporarily store the acquired image data and intermediate results during the processing, ensuring the data read and write speed and avoiding processing delays. The communication unit supports multiple wired communication interfaces, such as Ethernet interfaces, and can stably and quickly transmit the processed privacy-encoded data to the backend server.

[0034] In this embodiment, the image data obtained from the monitoring images of the classified file compact shelves is input into a pre-trained material recognition model to obtain material information. Then, the dynamic Gaussian blur algorithm is used to blur the areas in the monitoring images where the material information is the preset file material. Compared with the related technology that first recognizes text and then performs privacy encoding on the text part, the protection object is upgraded from the text level to the material level. After recognizing the preset file material, blurring processing is performed immediately, so that only the non-classified file access behavior images are retained in the monitoring records, ensuring that classified content (such as the text information after text recognition) is not retained during the monitoring process, eliminating potential safety hazards at the source, and at the same time ensuring that the closed-loop operation from detection to protection can be completed in a short time, increasing the security and supervision efficiency of file management. The predicted file trajectories at at least one future moment are obtained using the monitoring images and the motion prediction model, and the areas in the monitoring images at at least one future moment where the material information is the preset file material are blurred according to the predicted file trajectories. To address the dynamic change problem caused by the movement of the file box, the motion prediction model is used to predict the movement trajectory of the file in advance, and accurate trajectory prediction is achieved in the high-frame-rate video stream, so as to blur the areas corresponding to the preset file material, improving the intelligence and automation level of classified file management. Through the processes of file material recognition, motion trajectory prediction, and real-time blurring processing in this embodiment, the manual intervention link is reduced, the operation complexity is lowered, and at the same time, the efficiency and consistency of the processing process are ensured, avoiding omissions or errors that may be caused by manual operations, and significantly improving the security and supervision efficiency of classified file management.

[0035] Optionally, the material recognition model includes; A feature extraction module for extracting features from the image data; An attention mechanism module for determining a key region in the image data based on the extracted features; A feature fusion module for fusing the extracted features of the key region to obtain a fused feature; A classifier for performing material classification based on the fused feature to obtain the material information of the key region.

[0036] Specifically, as Figure 2 shown, the material recognition model includes an adjusted feature extraction module, an attention mechanism module, a feature fusion module, and a classifier. The feature extraction module may include a convolutional neural network, such as ResNet, VGG, etc. Taking ResNet as an example, it consists of multiple convolutional layers, residual blocks, and pooling layers. These layers can gradually extract low-level features (such as edges and corners) and high-level features (such as textures and shapes) in the image. If the image data is multi-spectral imaging data, it can be adjusted according to the characteristics of the multi-spectral data. For example, the number of input channels should match the number of channels of the multi-spectral data to extract more accurate features. The attention mechanism module may include an SE (Squeeze-and-Excitation) module or a CBAM (Convolutional Block Attention Module). Taking the SE module as an example, first, global average pooling is performed on the features extracted from the image data to compress the features into a one-dimensional vector. Secondly, through two fully connected layers, the first fully connected layer reduces the vector dimension, and the second fully connected layer restores the dimension to the original number of channels, and the Sigmoid function is used to map the output value to the interval [0, 1] to obtain the attention weight of each channel. Finally, the attention weight is multiplied by the corresponding feature map channel by channel to enhance the feature representation of the important channels. The feature fusion module fuses the image features after being processed by the attention mechanism. A simple splicing method can be adopted to splice multiple features together in the channel dimension to obtain a fused feature map. The classifier performs global average pooling on the fused feature map to convert it into a one-dimensional vector, and passes through one or more fully connected layers, and finally connects a Softmax layer to output the probability distribution corresponding to each material type, that is, the confidence.

[0037] Optionally, the processing of the monitoring image obtained by photographing the classified file compact shelf to obtain the image data includes: Obtaining RGB image data and multi-spectral imaging data according to the monitoring image; Performing grayscale processing, median filtering operation, and histogram equalization operation on the RGB image data and the multi-spectral imaging data respectively to obtain the image data.

[0038] Specifically, in the archive room, the positions of the surveillance cameras are usually fixed. Due to the placement of the archives and changes in lighting, the captured images may have problems such as uneven lighting and noise interference. Therefore, it is necessary to preprocess the captured surveillance images to improve the accuracy of image data acquisition. In this application, RGB image data and multispectral imaging data are obtained from the surveillance images. Then, grayscale processing is performed on the two types of data respectively to convert the color images into grayscale images to reduce the data volume and highlight the material features. Next, filtering operations such as median filtering are used to remove noise. Finally, histogram equalization is performed to enhance the image contrast, making the paper fiber texture on the surface of the archive box clearer and providing good input for material identification.

[0039] Optionally, the step of blurring the area of the surveillance image where the material information is the preset archive material by using the dynamic Gaussian blur algorithm includes: Obtaining environmental parameters based on the image data; Adjusting the blur radius of the area where the material information is the preset archive material according to the environmental parameters, and adjusting the blur intensity according to the blur radius.

[0040] Specifically, the environmental parameters may include light intensity, backlight detection, and light tilt angle. The original data of the environmental light intensity can be obtained by using the photosensitive sensor built in the surveillance camera, and its range is 0 - 2000 Lux. The larger the value, the stronger the light. Since the sensor usually outputs an analog signal and the computer can only process digital signals, analog-to-digital conversion is required to convert the analog light intensity data into digital form, and the converted digital data is mapped to the [0, 1] interval to obtain the light intensity factor I. When I = 0, it means the environment is completely dark, and when I = 1, it means the environmental light intensity reaches the strong light of 2000 Lux. Normalization processing helps the unified calculation of subsequent algorithms. For backlight scene detection, the gradient of the RGB image data is calculated by using the image analysis module. The gradient can reflect the change rate of pixel values in the image and can help identify information such as edges and textures in the image. Generally, the RGB image is first converted into a grayscale image, and then methods such as the Sobel operator are used to calculate the gradient. Screening conditions are set, such as the gradient amplitude of pixel points being greater than 50 and the brightness value exceeding 200 gray levels. The number of pixel points that meet these conditions is counted, and its proportion in the total number of pixels in the entire image is calculated. When the proportion of pixels that meet the conditions exceeds 15%, the area is determined as a backlight area, and a binary marker δ is generated. 逆光, where 1 represents backlight and 0 represents normal. In the calculation of the tilt angle, the optical flow method is used to track the movement trajectories of at least 10 feature corner points on the file box. Using the three-dimensional perspective transformation matrix, the spatial attitude of the edge contour of the file box is fitted by the least squares method. The three-dimensional perspective transformation matrix can describe the projection relationship of an object in three-dimensional space, and the least squares method is used to find the optimal fitting parameters. Finally, the tilt angle θ around the optical axis of the camera is calculated. Here, 0° represents the frontal plane and 90° represents the vertical side. The environmental parameters are passed as inputs to the dynamic Gaussian blur algorithm to obtain the blur radius. The blur radius R adopts a base value of R0 = 8 pixels and is dynamically adjusted in combination with the illumination compensation term and the tilt enhancement term. The specific formula is R = R0×(1 + 0.3I + 0.5δ 逆光 + 0.01θ), and the blur effect can be adaptively adjusted according to different environmental parameters.

[0041] It should be noted that the blur intensity can also be adjusted according to the blur radius.

[0042] Optionally, the environmental parameters include light intensity data; the adjustment of the blur intensity according to the blur radius includes: When the blur radius is less than or equal to the preset radius, the blur intensity is the initial intensity; When the blur radius is greater than the preset radius, the blur intensity is increased based on the initial intensity. The blur intensity has a positive correlation with the blur radius. For example, when the blur radius R increases, the blur intensity increases accordingly. A functional relationship can be set, such as the blur intensity S = kRS 初 , where k represents the preset radius factor, and a suitable value is determined through experiments.

[0043] It should be noted that from the perspective of human visual perception, a larger blur radius will make the details and edges in the image become more blurred and unclear, which is consistent with our intuitive understanding of the blur intensity. In practical applications, such as when blurring privacy areas, we hope to enhance the blur intensity by increasing the blur radius to better hide sensitive information. If the blur radius increases but the blur intensity does not increase accordingly, the expected blur effect cannot be achieved, and the purpose of privacy protection or other blurring requirements cannot be effectively realized. Therefore, the blur intensity is set to have a positive correlation with the blur radius. And to avoid the blur intensity being too small due to a small blur radius and the leakage of file information, when the blur radius is less than or equal to the preset radius, the blur intensity is set to the initial intensity. The preset radius can be set according to the size of the file text.

[0044] Alternatively, when the blur radius is greater than the preset radius, a base intensity value is obtained based on a preset radius factor, the initial intensity, and the blur radius. An intensity adjustment value is determined according to a preset light intensity factor and the light intensity data. The sum of the base intensity value and the intensity adjustment value is obtained to get the blur intensity.

[0045] Specifically, in addition to the influence of the radius, the light intensity can also be used to make additional adjustments to the blur intensity. In a strong light environment (where the light intensity factor I is large), the blur intensity can be appropriately increased to better handle problems such as possible reflections and overexposure. In a weak light environment, the blur intensity is appropriately reduced to avoid the image becoming too blurred and losing details. Since there is no change in the blur intensity when the blur radius is less than or equal to the preset radius, this method is only carried out on the basis that the blur radius is greater than the preset radius. First, based on the positive correlation between the blur intensity and the blur radius, a base intensity value is obtained. Then, according to the preset light intensity factor and the light intensity data, an intensity adjustment value for the light intensity is determined. The sum of the base intensity value and the intensity adjustment value is taken to get the blur intensity, such as the blur intensity S = kR S 初 + dI S 初 , where d represents the preset light intensity factor.

[0046] In this embodiment, the blur intensity is dynamically adjusted by the blur radius and the light intensity. In scenarios involving privacy protection, such as when blurring sensitive information such as faces and license plates in surveillance videos, a larger blur radius means a wider coverage area, and more surrounding areas can be included in the blurred range. At this time, increasing the blur intensity accordingly can more effectively make the details of the sensitive area completely unidentifiable, thereby better protecting personal privacy and sensitive information and preventing information leakage. Different application scenarios have different requirements for the degree of blurring. By adjusting the blur intensity according to the blur radius, various scenario requirements can be flexibly met. And in actual applications, the content and characteristics of images are different. Adjusting the blur intensity according to the blur radius gives the dynamic Gaussian blur algorithm better adaptability. For example, for areas with complex textures, a larger blur radius and a corresponding higher blur intensity may be required to achieve an ideal blur effect. For areas with simple textures, a smaller blur radius and a lower blur intensity can be used, which can automatically adjust the blur intensity according to the specific situation of the image without manually frequently adjusting multiple parameters, improving the versatility and practicality of the algorithm.

[0047] Optionally, as Figure 3 shown, the motion prediction model includes an optical flow method module and a Kalman filter module; The optical flow method module is used to obtain the optical flow information of the area in the monitoring image where the material information is the preset file material, and obtain the predicted motion trend of the area where the material information is the preset file material at at least one future moment according to the optical flow information; A Kalman filter module is used to obtain the archival motion state of the area in the monitored image where the material information is the preset archival material, and obtain the predicted archival trajectory of the area where the material information is the preset archival material at at least one future moment according to the archival motion state and the predicted motion trend.

[0048] Specifically, the optical flow method module first extracts feature points from consecutive frame monitored images in the monitored video. Commonly used feature point extraction algorithms include SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), or ORB (Oriented FAST and Rotated BRIEF), etc. Feature points can represent key positions in the image and have a certain degree of stability and uniqueness. Based on the extracted feature points, an optical flow algorithm (such as the Lucas-Kanade optical flow algorithm, the Farneback optical flow algorithm, etc.) is used to calculate the motion vectors of the feature points between adjacent frames. Optical flow represents the motion direction and speed of objects in the image between consecutive frames, and the preliminary motion information of the object can be obtained through optical flow calculation. The Kalman filter module is used to predict the archival motion trajectory. First, a state transition matrix, a process noise covariance matrix, a measurement matrix, a measurement noise covariance matrix, etc. are constructed, and then the Kalman gain K is calculated through the Kalman filter formula to balance the weights of the predicted value and the measured value. The current state is predicted using the state transition matrix to obtain the state prediction value at at least one future moment. Combining the motion vectors calculated by the optical flow method as the measured values (converting the optical flow vectors into position information), the predicted value is updated through the Kalman gain to obtain a more accurate state estimate value. This process is repeated to predict the state of the archival box at at least one future moment, thereby obtaining the pre-movement offset (Δx, Δy), that is, the predicted archival trajectory at at least one future moment.

[0049] Further, after obtaining the predicted archival trajectory at at least one future moment, a dynamic Gaussian blur algorithm or a dynamic occlusion mask generation module is used to blur the area in the monitored image at at least one future moment where the material information is the preset archival material. Among them, the dynamic occlusion mask generation module is used to form a blurred area synchronized with the movement of the archival box. Based on the binary region mask output by the material recognition module, a morphological dilation operation is used. A structural element such as a 5x5 (such as a square structural element) is selected to perform a 5-pixel dilation on the mask to compensate for possible boundary errors in material recognition and ensure that the archival box area is completely covered. According to the pre-movement offset (Δx, Δy) obtained by the Kalman filter module, the dilated mask is translated in real time on the image plane to synchronize the position of the mask with the predicted movement trajectory of the archival box, thereby forming a blurred area synchronized with the movement of the archival box.

[0050] Optionally, the material recognition model further includes an environment fusion module, and the output of the environment fusion module is connected to the input of the feature extraction module; the environment fusion module is used to fuse the environment parameters obtained in advance according to the image data and the image data.

[0051] Specifically, the environment fusion module is arranged before the feature extraction module. The environment fusion module first obtains and processes the environment parameters. The environment parameters are obtained through various sensors or devices. For example, a photosensitive sensor is used to obtain the light intensity, and a temperature sensor is used to obtain the ambient temperature, etc. For some indirect environment parameters, such as the parameters of the shooting device, they can be obtained from the metadata of the image or the above-mentioned method for obtaining environment parameters can be referred to. The obtained environment parameters are preprocessed, including data cleaning (removing outliers, handling missing values, etc.), normalization (mapping parameter values in different ranges to a unified interval, such as [0, 1]), and format conversion (converting the parameters into a format suitable for fusing with the image data), etc., to ensure that the environment parameters can accurately participate in the subsequent fusion process. Secondly, the input image data (RGB image data or hyperspectral imaging data) is subjected to conventional preprocessing operations, such as adjusting the image size to meet the requirements of the feature extraction module, normalizing the image pixel values (scaling the pixel value range from [0, 255] to [0, 1]), etc. Then, the environment parameters and the image data are fused. The preprocessed environment parameters are concatenated with the image data as additional channels or dimensions. For example, if the environment parameters are a three-dimensional vector containing light intensity, temperature, and humidity, and the RGB image data has three channels (red, green, blue), the environment parameter vector can be concatenated with the RGB image data in the channel dimension to form a new data with six channels; according to the influence degree of the environment parameters on the image features, a weight is assigned to each environment parameter, and then the environment parameters and the image data are weighted and summed. For example, in the case where the light intensity has a greater influence on the image features, a higher weight can be assigned to the light intensity, so that the information of the light intensity can more significantly affect the image data during the fusion process; the environment parameters are encoded as feature vectors, and then these feature vectors are embedded into the feature space of the image data. For example, a neural network can be used to map the environment parameters into a low-dimensional feature vector, and then the feature vector is element-wise added or other forms of fusion operations are performed with the feature map obtained by the convolution operation of the image data. Then, the fused environment parameters and image data are output to the feature extraction module as the input data of the feature extraction module. The feature extraction module will perform subsequent feature extraction operations based on these fused data, enabling the model to learn the relationship between the environment parameters and the image features, thereby improving the accuracy and robustness of material recognition.

[0052] In this embodiment, the light intensity information may affect the extraction direction and focus of image features in subsequent convolution operations. An environment fusion module is set before the feature extraction module to enable the interaction between the light intensity information and the image data at the early stage of model processing. If the light intensity is high, some image details may be lost due to overexposure. After fusing the light intensity information at an early stage, the model can adaptively adjust the parameters of the convolution kernel or the way of convolution operations according to the light intensity, making the extracted image features more in line with the actual situation under the current light intensity conditions, reducing the risk of information loss, making the model's processing process closer to the real situation, and thus being more conducive to the subsequent accurate judgment of the object material.

[0053] As Figure 4 shown, a monitoring and management device 400 for a classified file compact shelf provided by an embodiment of the present invention includes: A processing unit 410, configured to process the monitoring image obtained by photographing the classified file compact shelf to obtain image data; An identification unit 420, configured to input the image data into a material identification model to obtain material information, where the material identification model is obtained by performing material marking on the historical image data of the classified file compact shelf and training according to the marking result and the historical image data; A blurring unit 430, configured to perform blurring processing on the area of the monitoring image where the material information is a preset file material by using a dynamic Gaussian blurring algorithm; A prediction unit 440, configured to use the monitoring image and a motion prediction model to obtain at least one predicted file trajectory at a future moment, and perform blurring processing on the area of the monitoring image where the material information is a preset file material at at least one future moment according to the predicted file trajectory, where the motion prediction model is trained according to historical file motion information.

[0054] As Figure 5 shown, an electronic device 500 provided by an embodiment of the present invention includes a memory 510 and a processor 520; the memory 510 is used to store a computer program; the processor 520 is configured to implement the above-mentioned method for monitoring and managing a classified file compact shelf when executing the computer program.

[0055] Or, an electronic device 500 includes a memory 510 and a processor 520 coupled to the memory 510; the memory 510 is configured to store a computer program; the processor 520 is configured to perform the following operations when executing the computer program: Process the monitoring image obtained by photographing the classified file compact shelf to obtain image data; Input the image data into a material recognition model to obtain material information, where the material recognition model is trained by marking the historical image data of the classified document compact shelf and based on the marking results and the historical image data. Use a dynamic Gaussian blur algorithm to blur the area in the surveillance image where the material information is the preset document material. Use the surveillance image and a motion prediction model to obtain at least one predicted document trajectory at a future time, and blur the area in the surveillance image at at least one future time where the material information is the preset document material according to the predicted document trajectory, where the motion prediction model is trained based on historical document motion information.

[0056] A computer-readable storage medium provided by an embodiment of the present invention, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned classified document compact shelf monitoring and management method is implemented.

[0057] Or, a non-volatile computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor performs the following operations: Process the surveillance image obtained by photographing the classified document compact shelf to obtain image data; Input the image data into a material recognition model to obtain material information, where the material recognition model is trained by marking the historical image data of the classified document compact shelf and based on the marking results and the historical image data. Use a dynamic Gaussian blur algorithm to blur the area in the surveillance image where the material information is the preset document material. Use the surveillance image and a motion prediction model to obtain at least one predicted document trajectory at a future time, and blur the area in the surveillance image at at least one future time where the material information is the preset document material according to the predicted document trajectory, where the motion prediction model is trained based on historical document motion information.

[0058] Now, an electronic device 500 that can be a server or a client of the present invention will be described. It is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device 500 is intended to represent various forms of digital electronic computer devices, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 500 can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0059] The electronic device 500 includes a computing unit that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The computing unit, the ROM, and the RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0060] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc. In the present application, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present invention. In addition, in each embodiment of the present invention, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0061] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will all fall within the protection scope of the present invention.

Claims

1. A monitoring and management method for a classified file compact storage rack, characterized in that, Including: Processing the monitoring images obtained by photographing the classified file compact shelves to obtain image data; Inputting the image data into a material recognition model to obtain material information, where the material recognition model is trained by marking the materials of the historical image data of the classified file compact shelves and according to the marking results and the historical image data; Performing blurring processing on the area in the monitoring image where the material information is the preset file material by using a dynamic Gaussian blurring algorithm; Using the monitoring image and a motion prediction model to obtain at least one predicted file trajectory at future moments, and performing blurring processing on the area in the monitoring image at at least one future moment where the material information is the preset file material according to the predicted file trajectory, where the motion prediction model is trained according to historical file motion information.

2. The classified file compact storage rack monitoring and management method according to claim 1, wherein The material recognition model includes: A feature extraction module for extracting features from the image data; An attention mechanism module for determining key areas in the image data according to the extracted features; A feature fusion module for fusing the extracted features of the key areas to obtain fused features; A classifier for classifying materials according to the fused features to obtain the material information of the key areas.

3. The classified file compact storage rack monitoring and management method according to claim 1, characterized in that The processing the monitoring images obtained by photographing the classified file compact shelves to obtain image data includes: Obtaining RGB image data and hyperspectral imaging data according to the monitoring images; Performing grayscale processing, median filtering operation and histogram equalization operation on the RGB image data and the hyperspectral imaging data respectively to obtain the image data.

4. The classified file compact storage rack monitoring and management method according to claim 1, characterized in that The performing blurring processing on the area in the monitoring image where the material information is the preset file material by using a dynamic Gaussian blurring algorithm includes: Obtaining environmental parameters according to the image data; Adjusting the blurring radius of the area in the monitoring image where the material information is the preset file material according to the environmental parameters, and adjusting the blurring intensity according to the blurring radius.

5. The classified file compact storage rack monitoring and management method according to claim 4, wherein, The environmental parameters include light intensity data; the adjusting the blurring intensity according to the blurring radius includes: When the blurring radius is less than or equal to the preset radius, the blurring intensity is the initial intensity; When the blurring radius is greater than the preset radius, increasing the blurring intensity based on the initial intensity, and the blurring intensity has a positive correlation with the blurring radius; Or, when the blurring radius is greater than the preset radius, obtaining an intensity base value based on a preset radius factor, the initial intensity and the blurring radius, determining an intensity adjustment value according to a preset light intensity factor and the light intensity data, and obtaining the sum of the intensity base value and the intensity adjustment value to obtain the blurring intensity.

6. The classified file compact rack monitoring and management method according to claim 1, characterized in that The motion prediction model includes: An optical flow method module for obtaining the optical flow information of the area in the monitoring image where the material information is the preset file material, and obtaining the predicted motion trend of the area in the monitoring image where the material information is the preset file material at at least one future moment according to the optical flow information. A Kalman filter module, configured to obtain the archival motion state of the area in the monitored image where the material information is the preset archival material, and obtain the predicted archival trajectory of the area where the material information is the preset archival material at at least one future moment according to the archival motion state and the predicted motion trend.

7. The classified file compact storage rack monitoring and management method according to claim 1, characterized in that The material recognition model further includes an environment fusion module, and the output of the environment fusion module is connected to the input of the feature extraction module; the environment fusion module is configured to fuse the environment parameters obtained according to the image data in advance and the image data.

8. A monitoring and management device for a classified file compact rack, characterized in that, Comprising: A processing unit, configured to process the monitored image obtained by photographing a classified archival rack to obtain image data; An identification unit, configured to input the image data into a material recognition model to obtain material information, wherein the material recognition model is obtained by performing material marking on the historical image data of the classified archival rack and training according to the marking result and the historical image data; A blurring unit, configured to perform blurring processing on the area in the monitored image where the material information is the preset archival material by using a dynamic Gaussian blurring algorithm; A prediction unit, configured to obtain a predicted archival trajectory at at least one future moment by using the monitored image and a motion prediction model, and perform blurring processing on the area in the monitored image at at least one future moment where the material information is the preset archival material according to the predicted archival trajectory, wherein the motion prediction model is obtained by training according to historical archival motion information.

9. An electronic device, characterized in that, Comprising a memory and a processor; The memory is configured to store a computer program; The processor is configured to, when executing the computer program, implement the classified archival rack monitoring and management method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the storage medium, and when the computer program is executed by the processor, the classified archival rack monitoring and management method according to any one of claims 1 to 7 is implemented.

Images