Video monitoring face information acquisition method and system in medical environment and storage medium

Abstract

The invention discloses a video monitoring face information acquisition method and system in a medical environment and a storage medium, and the method comprises the following steps: (1) obtaining image data, carrying out the bit plane segmentation of the image data, retaining a bit plane forming the main features of an image, and recombining the bit plane into simplified image data; (2) improving the contrast ratio and the brightness of the simplified image data, denoising, and storing the processed simplified image data as a numpy array; (3) detecting the numpy array by using a convolutional neural network to obtain four coordinate values and feature vectors of each face region; the device can improve the face information acquisition efficiency and ensure the acquisition accuracy at the same time.

Description

Technical Field

[0001] The present invention relates to face collection, and in particular to a method, system and storage medium for collecting face information via video surveillance in a medical environment. Background Art

[0002] The medical environment is a highly complex and dynamically changing place, and its background is often accompanied by significant noisy characteristics. There are many medical staff and patients' families, and they are active frequently, which results in a large number of people in the surveillance video images of the hospital environment. If you want to collect face information from the video images for subsequent face recognition processing, the data volume is large and the processing efficiency is low.

[0003] The Chinese invention patent with publication number CN113378764B discloses a method, device, equipment and medium for video face acquisition based on a clustering algorithm, wherein the method includes obtaining the video to be acquired, parsing and identifying the video to be acquired to obtain the face image information in the video to be acquired; for each frame of the face image, calculating and processing the preset key points in the face to obtain the target face image; then extracting the features of the target face image to obtain the feature information corresponding to the target face image, and then using a density clustering algorithm to cluster the feature information to obtain a feature bundle, and arranging them according to the frame order corresponding to the feature bundle to obtain a frame index time sequence, and then segmenting and merging the frame index time sequence to obtain a video clip of the target person. This application is conducive to improving the accuracy of video face acquisition by screening and clustering the face images in the video.

[0004] The above method directly processes each frame of the image. Although it can accurately collect facial information in the video image, the amount of data processed in this way is large and the efficiency is low. When processing images from video surveillance in a medical environment, it is impossible to quickly collect facial information while ensuring the accuracy of face collection. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a method, system and storage medium for collecting face information via video surveillance in a medical environment, which improves the efficiency of face collection while ensuring the accuracy of collection.

[0006] Technical solution: The method for collecting face information of a video surveillance user in a medical environment of the present invention comprises the following steps:

[0007] (1) acquiring image data and dividing it into bit planes, retaining the bit planes constituting the main features of the image and recombining them into simplified image data;

[0008] (2) improving the contrast and brightness of the simplified image data and removing noise, and saving the processed simplified image data as a numpy array;

[0009] (3) Use a convolutional neural network to detect the numpy array and obtain the four coordinate values ​​and feature vectors of each face area.

[0010] Based on the above technical solution, by first performing bit plane segmentation on the acquired image data, retaining some bit planes and reorganizing them into simplified image data for subsequent processing, the image features are retained while the amount of data processing is greatly reduced; improving the contrast and brightness of the simplified image data and denoising can make the faces in the image more prominent, thereby improving the accuracy of face collection; after the above processing, the image data is stored as a numpy array for subsequent processing, and then the coordinate values ​​and feature vectors of the face area are collected by numpy through a convolutional neural network. Compared with directly collecting and processing each frame of original image data, the amount of data is much smaller, which improves the collection efficiency and ensures the accuracy of face collection.

[0011] Preferably, the step (2) comprises the following sub-steps:

[0012] (2.1) Improve image contrast through histogram equalization;

[0013] (2.2) Improve the brightness of the image through gamma correction;

[0014] (2.3) Increase image contrast by contrast stretching;

[0015] (2.4) Reduce the noise in the image by Gaussian filtering;

[0016] (2.5) Store the image data obtained through the above four steps as a numpy array.

[0017] Histogram equalization, gamma correction, contrast stretching and Gaussian filtering are performed in sequence to enhance image contrast and brightness and remove image noise, thereby improving image quality, making faces more prominent and improving the accuracy of subsequent face acquisition.

[0018] Preferably, the formula used for gamma correction in step (2.2) is:

[0019] I out =A*I in γ

[0020] Among them, I in Represents the pixel value of the input image, I out Represents the pixel value of the output image, A represents the gain parameter of gamma correction, and γ is the gamma value;

[0021] The formula used for contrast stretching in step (2.3) is

[0022] I out =m*I in +b

[0023] Among them, m represents the slope and b represents the intercept;

[0024] The formula used for Gaussian filtering in step (2.4) is

[0025]

[0026] Among them, I in (x, y) represents the pixel value of the input image at (x, y), I out (x,y) represents the pixel value of the output image at (x,y), W(i,j) represents the weight of the Gaussian kernel, and k represents the size of the Gaussian kernel.

[0027] Preferably, the step (3) further comprises storing the coordinate values ​​and feature vectors of all face regions in a dictionary form as a total feature vector of the image.

[0028] The coordinate values ​​and feature vectors of all face regions are stored in a dictionary form as the total feature vector of the image. A face region in the image can be quickly and accurately found based on the coordinate values, and the feature vector of the face region can be directly given, facilitating subsequent rapid face recognition.

[0029] Preferably, the key of each element in the dictionary is four coordinate values ​​of any face region, and the value corresponding to the key is a feature vector of the face region.

[0030] Preferably, the dictionary is converted into a byte string for storage.

[0031] The dictionary is converted into a byte string so that it is easier for computers to recognize and use, which is convenient for subsequent face recognition.

[0032] Preferably, the image data acquired in step (1) is transmitted by the image acquisition device to the RTMP server via RTSP protocol control and sent in real time.

[0033] Video image data can be transmitted in real time with high quality through the RTMP server, thereby ensuring that subsequent face collection can be carried out efficiently.

[0034] Preferably, the video image data transmission is based on a multicast protocol, and independent multicast addresses are set for the image acquisition device and the image receiving device.

[0035] Using multicast protocol to transmit images can save server resources and network backbone bandwidth resources.

[0036] The video surveillance face information collection system in a medical environment of the present invention comprises:

[0037] Image preprocessing module: used to obtain image data and segment it into bit planes, retain the bit planes constituting the main features of the image and reassemble them into simplified image data;

[0038] Image optimization module: used to improve the contrast and brightness of simplified image data and remove noise, and save the processed simplified image data as a numpy array;

[0039] Face information collection module: By using a convolutional neural network to detect the numpy array, the four coordinate values ​​and feature vectors of each face area are obtained.

[0040] The computer-readable storage medium storing one or more programs according to the present invention includes instructions, and when the instructions are executed by a computing device, the computing device executes any one of the above methods.

[0041] Beneficial effects: Compared with the prior art, the present invention has the following significant effects: by retaining part of the bit planes through bit plane segmentation and reorganizing them into a new image, the main features of the image are retained, the amount of image data is reduced, and the subsequent processing speed is improved; optimizing the reorganized image ensures the accuracy of face collection, so the present invention can both improve the efficiency of face collection and ensure the accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is the image before bit plane segmentation;

[0043] Figure 2 Simplify the image after reorganizing the bit planes;

[0044] Figure 3 Simplified images before and after processing to improve contrast and remove noise, where Figure 3 a is the simplified image before processing, Figure 3 b is the simplified image after processing;

[0045] Figure 4 It is a schematic diagram of the overall process of the present invention. DETAILED DESCRIPTION

[0046] As shown in the figure, the method for collecting face information of a video surveillance in a medical environment of the present invention comprises the following steps:

[0047] (1) acquiring image data and dividing it into bit planes, retaining the bit planes constituting the main features of the image and recombining them into simplified image data;

[0048] (2) improving the contrast and brightness of the simplified image data and removing noise, and saving the processed simplified image data as a numpy array;

[0049] (3) Use a convolutional neural network to detect the numpy array and obtain the four coordinate values ​​and feature vectors of each face area.

[0050] The image data acquired in step (1) is collected by an image acquisition device and then transmitted. The image acquisition device includes multiple high-resolution cameras. Multiple cameras are set at each location in the hospital to ensure that images of each location are collected without blind spots; the image acquisition device also includes a mini-program acquisition module, which allows users to upload image information independently through a mobile phone mini-program. The mini-program acquisition module serves as an auxiliary image acquisition device. The image data of this application is mainly for image data collected by the camera.

[0051] The image data collected by the image acquisition device is transmitted through high-definition video transmission technology, multi-view installation and video stitching technology, and low-latency communication technology. These technologies are used to improve the transmission efficiency and quality of video data.

[0052] A one-to-many multicast technology is used between the image acquisition device and the image receiving device. A special multicast address is used to send video data from the camera to multiple receiving ends at the same time, saving bandwidth resources and reducing transmission delays. The special multicast address is an IP address range reserved specifically for multicast communications. These addresses will not be assigned to any single device, but are used to identify a group of receivers who wish to receive the same data stream. This can effectively improve data transmission efficiency, reduce bandwidth consumption and transmission delays, and provide strong support for real-time, high-definition video transmission. This technology ensures real-time, smooth video transmission even in medical environments with large traffic and complex communication requirements, thereby supporting high-definition face capture without blind spots.

[0053] Image super-resolution technology and intelligent noise reduction algorithms have been introduced to address the problem of blurred images caused by the coming and going of people in medical environments. Intelligent noise reduction algorithms can use artificial intelligence technology to process noise in images or signals to improve the quality of images or signals. Image super-resolution technology can significantly improve image clarity and detail expression through algorithm optimization without increasing hardware costs, making originally blurred facial images clearly recognizable. At the same time, combined with intelligent noise reduction algorithms, it can effectively filter out environmental noise and interference, further improving the accuracy of face recognition.

[0054] For newly purchased cameras, they need to be initialized. According to the camera's official website, follow the corresponding steps to reset the IP and password, and set up the corresponding RTSP streaming service; use Python's cv2 library to implement the real-time video playback function. If the corresponding video can be displayed, you can proceed to the next step.

[0055] Preprocess the surveillance camera, use FFmpeg to convert the RTSP stream, and clearly specify TCP as the transport layer protocol through the RTSP protocol to ensure the reliability and stability of the video stream. Use the libx264 encoder to perform H.264 advanced video encoding on the video stream, add the ACC encoder to encode the audio, output the video in FLV format, and finally send the video stream to the specified RTMP server; you can encode the video without encoding the audio and send it to the specified RTMP server.

[0056] In terms of camera interaction, we use a video stream transmission method based on network protocols (such as RTSP) to ensure seamless connection between the camera and the back-end processing system, that is, the image receiving device. By configuring a specific IP address and port number, the camera can stably send video data to the specified server, and the back-end processing system will perform grayscale value calculation, facial feature extraction and other processing; the above processing can improve the efficiency and quality of image data transmission from the image acquisition device to the image receiving device, so that the image receiving device can receive high-quality image data in real time to collect facial information.

[0057] The bit plane segmentation in step (1) means: in a digital image, the grayscale value of a pixel is usually represented by an 8-bit binary number ranging from 0 to 255; each bit of the binary number has a different contribution to the pixel value, the least significant bit (LSB) has the least effect on the pixel value, and the most significant bit (MSB) has the greatest effect; bit plane segmentation means dividing the 8-bit grayscale image into 8 different binary images, each image representing a bit level, for example, the 0th bit plane (LSB) only contains the least significant bit of each pixel, and the 7th bit plane (MSB) only contains the most significant bit; the high-bit planes contain most of the important visual information of the image, such as edges and contours, so when these planes change, the visual effect of the image will change significantly; the low-bit planes contain more subtle details and have less visual impact on the image, and changing these bits usually only causes a slight change in the grayscale of the image.

[0058] In the step (1), the bit planes constituting the main features of the image are retained and reorganized into simplified image data, that is, only the high-bit planes that have a great impact on the main features of the image are retained and these high-bit planes are reorganized into simplified image data. This will not affect the main features of the image and can reduce the amount of image data.

[0059] The step (2) comprises the following sub-steps:

[0060] (2.1) Improve image contrast through histogram equalization;

[0061] (2.2) Improve the brightness of the image through gamma correction;

[0062] The formula used for gamma correction is

[0063] I out =A*I in γ

[0064] Among them, I in Represents the pixel value of the input image, I out Represents the pixel value of the output image, A represents the gain parameter of gamma correction, and γ is the gamma value;

[0065] (2.3) Increase image contrast by contrast stretching;

[0066] The formula used for contrast stretching is

[0067] I out =m*I in +b

[0068] Among them, m represents the slope and b represents the intercept;

[0069] (2.4) Reduce the noise in the image by Gaussian filtering;

[0070] The formula used for Gaussian filtering is

[0071]

[0072] Among them, I in (x, y) represents the pixel value of the input image at (x, y), I out (x,y) represents the pixel value of the output image at (x,y), W(i,j) represents the weight of the Gaussian kernel, and k represents the size of the Gaussian kernel.

[0073] (2.5) Store the image data obtained through the above four steps as a numpy array.

[0074] The step (3) of using a convolutional neural network to obtain the position coordinates and feature vectors of the face area is a conventional prior art and will not be repeated here. Step (3) also includes storing the coordinate values ​​and feature vectors of all face areas in the form of a dictionary as the total feature vector of the image.

[0075] The key of each element in the dictionary is the coordinate value of any face region, and the value corresponding to the key is the feature vector of the face region.

[0076] For example, in an image, there are three face regions, and the four coordinate values ​​of each region are A i , B i , C i and Di , where i = 1, 2, 3, the value of i corresponds to the three face regions; the feature vector of each region is E i , then the face information collected by the image is the total feature vector of the image, which is stored in the following dictionary

[0077] {(A 1 ,B 1 ,C 1 ,D 1 ):E 1 , (A 2 ,B 2 ,C 2 ,D 2 ):E 2 , (A 3 ,B 3 ,C 3 ,D 3 ):E 3}

[0078] Since feature vectors are usually numpy arrays, the above list can also be converted into a byte string as the main features of the face in the image and stored in the corresponding database for easy use in face recognition.

[0079] The video surveillance face information collection system in a medical environment of the present invention comprises:

[0080] Image preprocessing module: used to obtain image data and segment it into bit planes, retain the bit planes constituting the main features of the image and reassemble them into simplified image data;

[0081] Image optimization module: used to improve the contrast and brightness of simplified image data and remove noise, and save the processed simplified image data as a numpy array;

[0082] Face information collection module: By using a convolutional neural network to detect the numpy array, the four coordinate values ​​and feature vectors of each face area are obtained.

[0083] The computer-readable storage medium storing one or more programs described in the present invention is characterized in that: the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device executes the above method.

Claims

1. A method for collecting face information from video surveillance in a medical environment, characterized in that: The following steps are involved: (1) acquiring image data and dividing it into bit planes, retaining the bit planes constituting the main features of the image and recombining them into simplified image data; (2) improving the contrast and brightness of the simplified image data and removing noise, and saving the processed simplified image data as a numpy array; (3) Use a convolutional neural network to detect the numpy array and obtain the four coordinate values ​​and feature vectors of each face area.

2. The method for collecting face information via video surveillance in a medical environment according to claim 1, characterized in that: The step (2) comprises the following sub-steps: (2.1) Improve image contrast through histogram equalization; (2.2) Improve the brightness of the image through gamma correction; (2.3) Increase image contrast by contrast stretching; (2.4) Reduce the noise in the image by Gaussian filtering; (2.5) Store the image data obtained through the above four steps as a numpy array.

3. The method for collecting face information via video surveillance in a medical environment according to claim 2, characterized in that: The formula used for gamma correction in step (2.2) is I out =A*I in γ Among them, I in Represents the pixel value of the input image, I out Represents the pixel value of the output image, A represents the gain parameter of gamma correction, and γ is the gamma value; The formula used for contrast stretching in step (2.3) is I out =m*I in +b Among them, m represents the slope and b represents the intercept; The formula used for Gaussian filtering in step (2.4) is Among them, I in (x, y) represents the pixel value of the input image at (x, y), I out (x,y) represents the pixel value of the output image at (x,y), W(i,j) represents the weight of the Gaussian kernel, and k represents the size of the Gaussian kernel.

4. The method for collecting face information via video surveillance in a medical environment according to claim 1, characterized in that: The step (3) also includes storing the coordinate values ​​and feature vectors of all face regions in a dictionary form as a total feature vector of the image.

5. The method for collecting face information via video surveillance in a medical environment according to claim 4, characterized in that: The key of each element in the dictionary is the four coordinate values ​​of any face region, and the value corresponding to the key is the feature vector of the face region.

6. The method for collecting face information via video surveillance in a medical environment according to claim 4, characterized in that: The dictionary is converted to a byte string for storage.

7. The method for collecting face information via video surveillance in a medical environment according to claim 1, characterized in that: The image data acquired in the step (1) is transmitted by the image acquisition device to the RTMP server via the RTSP protocol control and sent in real time.

8. The method for collecting face information via video surveillance in a medical environment according to claim 7, characterized in that: The image data transmission is based on a multicast protocol, and independent multicast addresses are set for the image acquisition device and the image receiving device.

9. A video surveillance face information collection system in a medical environment, characterized in that: The system comprises: Image preprocessing module: used to obtain image data and segment it into bit planes, retain the bit planes constituting the main features of the image and reassemble them into simplified image data; Image optimization module: used to improve the contrast and brightness of simplified image data and remove noise, and save the processed simplified image data as a numpy array; Face information collection module: By using a convolutional neural network to detect the numpy array, the four coordinate values ​​and feature vectors of each face area are obtained.

10. A computer-readable storage medium storing one or more programs, characterized in that : The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any of the methods according to claims 1 to 8.

Images