Monitoring video frame identification method based on image segmentation technology

Abstract

The invention relates to the technical field of monitoring video processing, in particular to a monitoring video frame identification method based on an image segmentation technology, which comprises the following steps: S1, acquiring a video file: downloading past monitoring videos through a network, and uniformly modifying the monitoring videos into the same format; s2, performing optimization processing on the video: selecting a monitoring video, inputting the monitoring video into optimization software for optimization processing, and improving the definition of an image in the monitoring video; according to the method, a model program can be constructed, the model program performs deep processing on the image through an open source data set in the fields of deep learning and image processing, efficient feature representation can be realized, the accuracy and the fine degree of image semantic segmentation are improved to a great extent, the number of image segmentation is increased, the error segmentation phenomenon is reduced, and the image segmentation efficiency is improved. The phenomena of image edge blurring, wrong segmentation and instable segmentation of a segmentation result generated by a traditional segmentation method are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring video processing, and in particular to a monitoring video frame recognition method based on image segmentation technology. Background Art

[0002] Surveillance video is a technology and system that uses cameras and related equipment to monitor and record activities in a specific area or place in real time. It is usually used in security monitoring, anti-theft, traffic supervision, employee supervision and public safety. Surveillance video can improve the safety of the monitored location. Therefore, in order to ensure the maximum utilization of surveillance video, the identification of surveillance video has become the focus.

[0003] For example, a method and device for identifying video key frames with authorization announcement number CN202211715764.X and publication date 20240702, the method for identifying video key frames includes: obtaining the video category of the video to be identified; determining the corresponding target traversal step based on the mapping relationship between the video category of the video to be identified and the preset category step; obtaining the reference frame image and the target frame image in the video to be identified based on the target traversal step, wherein the time interval between the reference frame image and the target frame image is the target traversal step, and the reference frame image is located before the target frame image; calculating the image similarity between the reference frame image and the target frame image; if the image similarity is less than the preset similarity threshold, determining the target frame image as a key frame; the present application performs targeted key frame identification on the corresponding video through the target traversal step corresponding to the video, which can take into account both recognition efficiency and recognition accuracy.

[0004] Although the above-mentioned video frame recognition methods and those in the prior art can perform targeted key frame recognition on surveillance videos, key frame recognition is sensitive to noise and lighting changes. When the scene changes slowly, key frame recognition is unstable, and the extracted key frames may not accurately reflect the main content of the video. Moreover, in complex scenes, the key frame extraction algorithm may not be able to accurately capture all key frames, resulting in information loss or misjudgment. In view of this, it is urgent to design a surveillance video frame recognition method based on image segmentation technology to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a monitoring video frame recognition method based on image segmentation technology to solve the above-mentioned deficiencies in the prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solutions: The monitoring video frame recognition method based on image segmentation technology includes the following steps: S1. Obtain video files: Download past surveillance videos through the Internet and modify them into MP4 format; S2. Optimize the video: Select the surveillance video and input it into the optimization software for optimization to improve the clarity of the image in the surveillance video. The operation steps are as follows: (1) Import video files; (2) Select a suitable restoration mode and choose the image quality enhancement function; (3) Set the output parameters, click OK, start processing and wait for completion; (4) View and save the output video; S3. Extract feature data: traverse the video, identify and extract feature data in the video. The process selects appropriate traversal methods according to different surveillance targets of the surveillance video. The traversal methods are composed of three methods: traversing video frames using the OpenCV library, feature extraction and support vector machine (SVM) optimization algorithm, and feature extraction and decision tree fast encoding method. The feature data is extracted using a convolutional neural network deep learning model; S4. Construct a segmentation model: confirm the feature data type, then select the corresponding image segmentation technology to construct an image segmentation model, and edit it into a model program through a programming language. The feature data type is established based on the monitoring target extracted by the convolutional neural network deep learning model. The image segmentation technology in step S4 selects spatial single-frame segmentation and temporal frame segmentation. The model program in step S4 is divided into a spatial segmentation function module, a temporal segmentation function module, a video category judgment module, and an image training module. The spatial segmentation function module and the temporal segmentation function module are used to perform image segmentation processing on the monitoring video. The video category judgment module has two functions of identification and calling. The identification function is used to identify and judge the monitoring target in the video. The calling function calls the spatial segmentation function module and the temporal segmentation function module according to the result of the identification function to perform video segmentation processing. The image training module uses open source data sets in the field of deep learning and image processing, and the open source data sets include target detection, face recognition, text recognition, image classification, image segmentation, image defogging, key point detection, and action recognition. S5. Video frame capture and recognition: Install the model program on the computer device connected to the monitoring device, open the model program, select the video of the corresponding period, segment and extract the video, and confirm the monitoring video frame. The video of the corresponding period is the period from when the target appears in the monitoring range of the monitoring device to when the target leaves the monitoring range of the monitoring device. After the target monitoring video frame is identified and extracted in step S5, it will be displayed through a picture; S6. Real-time analysis and feedback: Analyze and judge the target surveillance video frames, and give corresponding processing methods based on the analysis and judgment results to solve the problem.

[0007] The surveillance video frame recognition method based on image segmentation technology provided by the present invention can construct a model program. The model program performs in-depth processing of images through deep learning and open source data sets in the field of image processing, and can achieve efficient feature representation, which greatly improves the accuracy and refinement of image semantic segmentation, expands the number of image segmentations, reduces mis-segmentation, and avoids the phenomenon of blurred image edges, mis-segmentation, and unstable segmentation of segmentation results produced by traditional segmentation methods.

[0008] Furthermore, in the step S3, a suitable traversal method is selected according to different surveillance targets of the surveillance video, and the traversal method consists of three methods: traversing video frames using the OpenCV library, feature extraction and support vector machine (SVM) optimization algorithm, and feature extraction and decision tree fast encoding method. The feature data extraction in the step S3 adopts a convolutional neural network deep learning model.

[0009] It should be noted that OpenCV iterates over the video frames: Read video file: video = cv2.VideoCapture('video.mp4').

[0010] Traverse the video frames: whileTrue:ret,frame=video.read();ifnotret:break;#Process the frames.

[0011] Release resources: video.release();cv2.destroyAllWindows()‌1.

[0012] ‌Feature extraction and SVM optimization algorithm‌: Get the CU features during the original encoding process.

[0013] Use SVM to learn CU features to obtain the prediction model.

[0014] Extract the same type of features for the current coding block, input them into the SVM prediction model, and perform fast coding‌2.

[0015] ‌Feature extraction and decision tree optimization method‌: In the original process, the feature information of the coding block is extracted and the decision tree model is trained.

[0016] Calculate the prediction accuracy of each node of the decision tree model.

[0017] Use decision tree prediction with node accuracy to encode block partitioning and mode selection, skipping unnecessary computation steps ‌2.

[0018] Furthermore, in the step S4, the feature data type is established based on the monitoring target extracted by the convolutional neural network deep learning model. The image segmentation technology in the step S4 selects spatial single frame segmentation and temporal frame segmentation. The model program in the step S4 is divided into a spatial segmentation function module, a temporal segmentation function module, a video category judgment module, and an image training module. The spatial segmentation function module and the temporal segmentation function module are used to perform image segmentation processing on the monitoring video. The video category judgment module has two functions of recognition and calling. The recognition function is used to recognize and judge the monitoring target in the video. The calling function calls the spatial segmentation function module and the temporal segmentation function module according to the result of the recognition function to perform video segmentation processing.

[0019] It should be noted that the video segmentation based on the spatiotemporal joint segmentation method greatly improves the accuracy of surveillance video segmentation, can adapt to complex scenes, effectively eliminate background noise and redundant information, improve the segmentation effect, and can also repair the surveillance matching area, improving the effect of gait recognition and image restoration; The convolutional neural network deep learning model is a neural network model that extracts image features through convolutional layers, pooling layers, and fully connected layers. The convolutional layer performs convolution operations on the image through convolution kernels to extract features such as edges and colors. The pooling layer downsamples the feature map through average pooling or maximum pooling to reduce the size of the feature map. The fully connected layer classifies the features through fully connected neurons to obtain the labels in the image.

[0020] Furthermore, the image training module adopts open source data sets in the field of deep learning and image processing, and the open source data sets include eight types: target detection, face recognition, text recognition, image classification, image segmentation, image dehazing, key point detection, and action recognition.

[0021] It should be noted that the open source data sets are as follows: 1. Object Detection T-LESS dataset, pedestrian interaction detection dataset, SpotGarbage garbage recognition dataset, Labelme image dataset, vehicle dataset, SUN09 scene understanding, Unsplash image retrieval dataset, HICO-DET character interaction detection dataset, life, flame and smoke image dataset, pothole detection dataset, Linkopings traffic sign dataset, dense pedestrian detection dataset, traffic camera detection dataset.

[0022] 2. Face Face image dataset, public figure face database, face recognition dataset 3. Text Recognition Geolocation information dataset 4. Image Classification Traffic sign dataset, vehicle dataset, car image data, street scene data 5. Image segmentation of high-density crowds and moving object video datasets, DAVIS video segmentation dataset 6. Image dehazing D-HAZY dataset, RESIDE dataset, MiddleburyStereo binocular stereo, NH-HAZE real dataset, DENSE-HAZE corresponding haze-free image dataset.

[0023] 7. Key point detection Hand pose key point detection dataset, human pose dataset, human foot key point, crowd pose dataset.

[0024] 8. Action Recognition HMDB human action video dataset, UCF50 action recognition dataset, SBUKinect interaction dataset, FineDiving diving dataset.

[0025] Furthermore, the corresponding time period video in step S5 is the time period from when the target appears in the monitoring range of the monitoring device to when the target leaves the monitoring range of the monitoring device. After the target monitoring video frame in step S5 is identified and extracted, it will be displayed through a picture.

[0026] In the above technical solution, the monitoring video frame recognition method based on image segmentation technology provided by the present invention has the following beneficial effects: The surveillance video frame recognition method designed by the present invention can construct a model program. The model program performs in-depth processing of images through open source data sets in the field of deep learning and image processing, and can achieve efficient feature representation, which greatly improves the accuracy and refinement of image semantic segmentation, expands the number of image segmentations, reduces mis-segmentation, and avoids the phenomenon of blurred image edges, mis-segmentation, and unstable segmentation in segmentation results produced by traditional segmentation methods.

[0027] The surveillance video frame recognition method designed by the present invention can segment the video based on the spatiotemporal joint segmentation method, which greatly improves the accuracy of surveillance video segmentation, can adapt to complex scenes, effectively eliminate background noise and redundant information, improve the segmentation effect, and can also repair the surveillance matching area, thereby improving the effects of gait recognition and image restoration. The monitoring video frame recognition method designed by the present invention can be widely used in computers in various monitoring areas by constructing a model program, thereby increasing the application scope of the monitoring video frame recognition method. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0029] Figure 1 A method flow chart of an embodiment of a surveillance video frame recognition method based on image segmentation technology provided by the present invention.

[0030] Figure 2 A schematic diagram of the model program structure provided for an embodiment of the monitoring video frame recognition method based on image segmentation technology of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0032] like Figure 1-2 As shown, the monitoring video frame recognition method based on image segmentation technology provided by the embodiment of the present invention includes the following steps: S1. Obtain video files: Download past surveillance videos through the Internet and modify the surveillance videos into the same format; S2. Optimize the video: select the surveillance video, input the surveillance video into the optimization software for optimization, and improve the clarity of the image in the surveillance video; S3. Extract feature data: traverse the video, identify and extract feature data in the video; S4. Constructing a segmentation model: confirming the feature data type, then selecting the corresponding image segmentation technology to construct an image segmentation model, and editing it into a model program through a programming language; S5. Video frame capture and recognition: Install the model program on the computer device connected to the monitoring device, open the model program, select the video of the corresponding period, segment and extract the video, and confirm the monitoring video frame; S6. Real-time analysis and feedback: Analyze and judge the target surveillance video frames, and give corresponding processing methods based on the analysis and judgment results to solve the problem.

[0033] Specifically, in this embodiment, the following steps are included: S1. Obtain video files: Download past surveillance videos through the Internet and modify them into MP4 format; S2. Optimize the video: Select the surveillance video and input it into the optimization software for optimization to improve the clarity of the image in the surveillance video. The operation steps are as follows: (1) Import video files; (2) Select a suitable restoration mode and choose the image quality enhancement function; (3) Set the output parameters, click OK, start processing and wait for completion; (4) View and save the output video; S3. Extract feature data: traverse the video, identify and extract feature data in the video. The process selects appropriate traversal methods according to different surveillance targets of the surveillance video. The traversal methods are composed of three methods: traversing video frames using the OpenCV library, feature extraction and support vector machine (SVM) optimization algorithm, and feature extraction and decision tree fast encoding method. The feature data is extracted using a convolutional neural network deep learning model; S4. Construct a segmentation model: confirm the feature data type, then select the corresponding image segmentation technology to construct an image segmentation model, and edit it into a model program through a programming language. The feature data type is established based on the monitoring target extracted by the convolutional neural network deep learning model. The image segmentation technology in step S4 selects spatial single-frame segmentation and temporal frame segmentation. The model program in step S4 is divided into a spatial segmentation function module, a temporal segmentation function module, a video category judgment module, and an image training module. The spatial segmentation function module and the temporal segmentation function module are used to perform image segmentation processing on the monitoring video. The video category judgment module has two functions of recognition and call. The recognition function is used to recognize and judge the monitoring target in the video. The call function calls the spatial segmentation function module and the temporal segmentation function module according to the result of the recognition function to perform video segmentation processing. The image training module uses open source data sets in the field of deep learning and image processing, and the open source data sets include target detection, face recognition, text recognition, image classification, image segmentation, image defogging, key point detection, and action recognition. S5. Video frame capture and recognition: Install the model program on the computer device connected to the monitoring device, open the model program, select the video of the corresponding period, segment and extract the video, and confirm the monitoring video frame. The video of the corresponding period is the period from when the target appears in the monitoring range of the monitoring device to when the target leaves the monitoring range of the monitoring device. After the target monitoring video frame is identified and extracted in step S5, it will be displayed through a picture; S6. Real-time analysis and feedback: Analyze and judge the target surveillance video frames, and give corresponding processing methods based on the analysis and judgment results to solve the problem.

[0034] The surveillance video frame recognition method based on image segmentation technology provided by the present invention can construct a model program. The model program performs in-depth processing of images through deep learning and open source data sets in the field of image processing, and can achieve efficient feature representation, which greatly improves the accuracy and refinement of image semantic segmentation, expands the number of image segmentations, reduces mis-segmentation, and avoids the phenomenon of blurred image edges, mis-segmentation, and unstable segmentation of segmentation results produced by traditional segmentation methods.

[0035] Furthermore, in step S3, a suitable traversal method is selected according to different surveillance targets of the surveillance video, and the traversal method consists of three methods: traversing video frames using the OpenCV library, feature extraction and support vector machine (SVM) optimization algorithm, and feature extraction and decision tree fast encoding method. The feature data extraction in step S3 adopts a convolutional neural network deep learning model.

[0036] It should be noted that OpenCV iterates over the video frames: Read video file: video = cv2.VideoCapture('video.mp4').

[0037] Traverse the video frames: whileTrue:ret,frame=video.read();ifnotret:break;#Process the frames.

[0038] Release resources: video.release();cv2.destroyAllWindows()‌1.

[0039] ‌Feature extraction and SVM optimization algorithm‌: Get the CU features during the original encoding process.

[0040] Use SVM to learn CU features to obtain the prediction model.

[0041] Extract the same type of features for the current coding block, input them into the SVM prediction model, and perform fast coding‌2.

[0042] ‌Feature extraction and decision tree optimization method‌: In the original process, the feature information of the coding block is extracted and the decision tree model is trained.

[0043] Calculate the prediction accuracy of each node of the decision tree model.

[0044] Use decision tree prediction with node accuracy to encode block partitioning and mode selection, skipping unnecessary computation steps ‌2.

[0045] Furthermore, in step S4, the feature data type is established based on the monitoring target extracted by the convolutional neural network deep learning model. The image segmentation technology in step S4 selects spatial domain single frame segmentation and temporal domain frame segmentation. The model program in step S4 is divided into a spatial domain segmentation function module, a temporal domain segmentation function module, a video category judgment module, and an image training module. The spatial domain segmentation function module and the temporal domain segmentation function module are used to perform image segmentation processing on the monitoring video. The video category judgment module has two functions of recognition and calling. The recognition function is used to recognize and judge the monitoring target in the video. The calling function calls the spatial domain segmentation function module and the temporal domain segmentation function module according to the result of the recognition function to perform video segmentation processing.

[0046] It should be noted that the video segmentation based on the spatiotemporal joint segmentation method greatly improves the accuracy of surveillance video segmentation, can adapt to complex scenes, effectively eliminate background noise and redundant information, improve the segmentation effect, and can also repair the surveillance matching area, improving the effect of gait recognition and image restoration; The convolutional neural network deep learning model is a neural network model that extracts image features through convolutional layers, pooling layers, and fully connected layers. The convolutional layer performs convolution operations on the image through convolution kernels to extract features such as edges and colors. The pooling layer downsamples the feature map through average pooling or maximum pooling to reduce the size of the feature map. The fully connected layer classifies the features through fully connected neurons to obtain the labels in the image.

[0047] Furthermore, the image training module adopts open source data sets in the field of deep learning and image processing, and the open source data sets include eight types: target detection, face recognition, text recognition, image classification, image segmentation, image dehazing, key point detection, and action recognition.

[0048] It should be noted that the open source data sets are as follows: 1. Object Detection T-LESS dataset, pedestrian interaction detection dataset, SpotGarbage garbage recognition dataset, Labelme image dataset, vehicle dataset, SUN09 scene understanding, Unsplash image retrieval dataset, HICO-DET character interaction detection dataset, life, flame and smoke image dataset, pothole detection dataset, Linkopings traffic sign dataset, dense pedestrian detection dataset, traffic camera detection dataset.

[0049] 2. Face Face image dataset, public figure face database, face recognition dataset 3. Text Recognition Geolocation information dataset 4. Image Classification Traffic sign dataset, vehicle dataset, car image data, street scene data 5. Image segmentation of high-density crowds and moving object video datasets, DAVIS video segmentation dataset 6. Image dehazing D-HAZY dataset, RESIDE dataset, MiddleburyStereo binocular stereo, NH-HAZE real dataset, DENSE-HAZE corresponding haze-free image dataset.

[0050] 7. Key point detection Hand pose key point detection dataset, human pose dataset, human foot key point, crowd pose dataset.

[0051] 8. Action Recognition HMDB human action video dataset, UCF50 action recognition dataset, SBUKinect interaction dataset, FineDiving diving dataset.

[0052] Furthermore, the corresponding time period video in step S5 is the time period from when the target appears in the monitoring range of the monitoring device to when the target leaves the monitoring range of the monitoring device. After the target monitoring video frame is identified and extracted in step S5, it will be displayed through a picture.

[0053] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A surveillance video frame recognition method based on image segmentation technology, characterized in that: The following steps are involved: S1. Obtain video files: Download past surveillance videos through the Internet and modify the surveillance videos into the same format; S2. Optimize the video: select the surveillance video, input the surveillance video into the optimization software for optimization, and improve the clarity of the image in the surveillance video; S3. Extract feature data: traverse the video, identify and extract feature data in the video; S4. Constructing a segmentation model: confirming the feature data type, then selecting the corresponding image segmentation technology to construct an image segmentation model, and editing it into a model program through a programming language; S5. Video frame capture and recognition: Install the model program on the computer device connected to the monitoring device, open the model program, select the video of the corresponding period, segment and extract the video, and confirm the monitoring video frame; S6. Real-time analysis and feedback: Analyze and judge the target surveillance video frames, and give corresponding processing methods based on the analysis and judgment results to solve the problem.

2. The monitoring video frame recognition method based on image segmentation technology according to claim 1 is characterized in that: The past surveillance videos in step S1 are previous data shot by the surveillance equipment. In step S1, the selected surveillance videos will be uniformly modified into MP4 format.

3. The monitoring video frame recognition method based on image segmentation technology according to claim 1 is characterized in that: The step S2 optimization software uses a video repair tool, and the operation steps are as follows: (1) Import video files; (2) Select a suitable restoration mode and choose the image quality enhancement function; (3) Set the output parameters, click OK, start processing and wait for completion; (4) View and save the output video.

4. The monitoring video frame recognition method based on image segmentation technology according to claim 1 is characterized in that: In step S3, a suitable traversal method is selected according to different surveillance targets of the surveillance video, and the traversal method includes three methods: traversing video frames using the OpenCV library, feature extraction and support vector machine (SVM) optimization algorithm, and feature extraction and decision tree fast encoding method.

5. The monitoring video frame recognition method based on image segmentation technology according to claim 1 is characterized in that: The feature data in step S3 is extracted using a convolutional neural network deep learning model.

6. The monitoring video frame recognition method based on image segmentation technology according to claim 1 is characterized in that: The feature data type in step S4 is established based on the monitoring target extracted by the convolutional neural network deep learning model, and the image segmentation technology in step S4 uses spatial domain single frame segmentation and temporal domain frame segmentation.

7. The monitoring video frame recognition method based on image segmentation technology according to claim 1 is characterized in that: In the step S4, the model program is divided into a spatial domain segmentation function module, a temporal domain segmentation function module, a video category judgment module, and an image training module. The spatial domain segmentation function module and the temporal domain segmentation function module are used to perform image segmentation processing on the surveillance video.

8. The monitoring video frame recognition method based on image segmentation technology according to claim 7 is characterized in that: The video category judgment module has two functions of identification and calling. The identification function is used to identify and judge the monitoring target in the video, and the calling function calls the spatial domain segmentation function module and the temporal domain segmentation function module according to the result of the identification function to segment the video.

9. The monitoring video frame recognition method based on image segmentation technology according to claim 7 is characterized in that: The image training module uses open source data sets in the field of deep learning and image processing, and the open source data sets include nine types: target detection, face recognition, text recognition, image classification, image segmentation, image defogging, key point detection, and action recognition.

10. The monitoring video frame recognition method based on image segmentation technology according to claim 1, characterized in that: The corresponding time period video in step S5 is the time period from when the target appears in the monitoring range of the monitoring device to when the target leaves the monitoring range of the monitoring device. After the target monitoring video frame in step S5 is identified and extracted, it will be displayed through a picture.

Images