Intelligent monitoring system based on computer vision technology

Abstract

The present invention discloses an intelligent monitoring system based on computer vision technology. The intelligent monitoring system based on computer vision technology relates to the technical field of monitoring image processing, including a video acquisition module, a transmission module, an image cloud processing module, a storage module and a terminal module; the video acquisition module detects the ambient light intensity index, determines the shooting mode of the monitoring camera based on the threshold judgment, and obtains the video acquisition data by recording; the image cloud processing module calculates the grayscale image data, identifies and classifies it through a first machine learning model and completes the box selection, classifies the body shape of the human target image through a second machine learning model, and sends a notification or alarm to the user; the terminal module provides the user with a login interface and a viewing interface and provides the user with a two-way call function, and sends an audible and visual alarm when the monitoring camera executes the first special alarm and the second special alarm; the storage module stores the video acquisition data through the monitoring camera, and stores the log data through the cloud server.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring image processing, and in particular to an intelligent monitoring system based on computer vision technology. Background Art

[0002] In recent years, with the rapid development of electronic information technology, image processing technology and computer vision technology, the functions of surveillance cameras have been continuously improved and improved, and great progress has been made in the degree of high definition, wirelessness and miniaturization. Surveillance cameras are not only widely used in traditional scenes such as urban roads, transportation hubs, supermarkets and banks, factory warehouses and hospital campuses, but also because of their ability to perform security functions while taking care of family members, their application popularity in the personal field has also grown rapidly. Therefore, the development of a surveillance system with a higher degree of integration and intelligence meets the needs of the market.

[0003] At present, the Chinese invention patent application with application number CN201610174003.6 discloses an intelligent video surveillance method and system based on computer vision analysis technology. The application includes: video acquisition step: acquiring surveillance video of a fixed scene through a camera in a fixed position; video analysis step: analyzing the surveillance video to obtain analysis results; monitoring management step: displaying the analysis results to personnel or issuing an alarm. The application uses computer vision technology to automatically analyze surveillance videos, further extract specific information from the video, and automatically generate alarm information and record it in the database when a specific event occurs, thereby improving the intelligence of video surveillance and greatly reducing the labor intensity of monitoring personnel. However, the application extracts pedestrian height features, edge features, and texture features of images through traditional methods, and the speed of calculating real-time analysis results is slow, and the applicability is not sufficient. Summary of the invention

[0004] The technical problem solved by the present invention is that the existing application uses traditional methods to extract pedestrian height features, edge features and texture features of images, but the speed of calculating and obtaining real-time analysis results is slow and the applicability is not sufficient.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] Intelligent monitoring system based on computer vision technology, including video acquisition module, transmission module, image cloud processing module, storage module and terminal module;

[0007] The video acquisition module includes a light detection unit, a threshold judgment unit and a shooting unit, which are used to detect the ambient light intensity index, determine the shooting mode of the surveillance camera through threshold judgment, and record the surveillance area to obtain video acquisition data;

[0008] The image cloud processing module includes a preprocessing unit, a target detection unit and a posture recognition unit, which are used to extract grayscale image data frame by frame from the video acquisition data and calculate the grayscale image data, identify and classify the grayscale image data through a first machine learning model and complete the box selection to obtain a box selection atlas, send a regular notification or a first special alarm to the user based on the identification and classification result, select a human target image from the box selection atlas, classify the human target image into posture through a second machine learning model to obtain a posture classification result, and send a second special alarm to the user based on the posture classification result;

[0009] The terminal module includes an application unit and an audio-visual unit, which are used to provide a login interface and a viewing interface for the user, display a real-time monitoring screen to the user through a cloud server and receive real-time reception notifications, provide the user with a two-way communication function between the application terminal and the monitoring camera, and send an audio-visual alarm when the monitoring camera executes a first special alarm and a second special alarm;

[0010] The storage module includes a camera storage unit and a cloud storage unit, which are used to store video acquisition data through the SD memory card of the monitoring camera and store log data through the cloud server.

[0011] As a preferred solution of the intelligent monitoring system based on computer vision technology described in the present invention, the light detection unit is used to detect the ambient light intensity index, and the ambient light intensity of the monitoring camera is obtained by detecting the light sensor array, and T is calculated based on the ambient light intensity. 1 The rate at which ambient light intensity decreases over time;

[0012] The threshold judgment unit is used to monitor the camera shooting mode based on the ambient light intensity and T 1 The rate at which the ambient light intensity decreases within a certain period of time is used to perform a threshold judgment, and the logic of the threshold judgment includes:

[0013] When the ambient light intensity is greater than the first light threshold, determining the natural light shooting mode, starting the natural light camera and turning off the infrared camera;

[0014] When the ambient light intensity is less than a second light threshold, determining the infrared shooting mode, starting the infrared camera and turning off the natural light camera;

[0015] When the rate of decrease of the ambient light intensity is greater than a third light threshold, determining the infrared shooting mode, starting the infrared camera and turning off the natural light camera;

[0016] The shooting is used to record the monitored area to obtain video acquisition data. When the monitoring camera shooting mode is determined to be a natural light shooting mode, the natural light video data is obtained by collecting through the natural light camera;

[0017] When the surveillance camera shooting mode is determined to be the infrared shooting mode, infrared video data is acquired through the infrared camera;

[0018] The ambient light intensity index includes the ambient light intensity and the rate at which the ambient light intensity decreases within T time;

[0019] The surveillance camera shooting modes include natural light shooting mode and infrared shooting mode;

[0020] The video acquisition data includes natural light video data and infrared video data.

[0021] As a preferred solution of the intelligent monitoring system based on computer vision technology described in the present invention, wherein: the preprocessing unit is used to traverse the video frames of the video acquisition data, obtain natural light image data and infrared image data by extracting frame by frame, grayscale the natural light image data to obtain natural light grayscale image data, perform edge detection on the infrared image data through the Canny operator to obtain infrared image edge point pixels, perform nonlinear transformation on the infrared image edge point pixels through gamma correction to obtain infrared grayscale image data, and merge the natural light grayscale image data with the infrared grayscale image data to obtain grayscale image data.

[0022] As a preferred solution of the intelligent monitoring system based on computer vision technology described in the present invention, wherein: the target detection unit is used to identify and classify objects in the monitoring area, identify and classify the grayscale image data through the first machine learning model to obtain a recognition and classification result, and select the identified and classified targets through a bounding box to obtain a frame selection atlas, and send a regular notification or a first special alarm to the user based on the recognition and classification result;

[0023] The identification and classification results include: human body, pet, vehicle, smoke and flame;

[0024] The logic for sending notifications or alerts to users includes: sending regular notifications to customers when the recognition and classification results are human, pet, and vehicle;

[0025] When the identification and classification result is smoke and the duration is less than or equal to T 2 When the identification and classification result is smoke and the duration is greater than T, a regular notification is sent to the customer. 2 When the time comes, the first special alert is sent to the customer;

[0026] When the identification classification result is flame and the duration is less than or equal to T 3 When the identification result is flame and the duration is greater than T, a regular notification is sent to the customer. 3 At the time, the first special alert is sent to the customer.

[0027] As a preferred solution of the intelligent monitoring system based on computer vision technology described in the present invention, the logic of recognition and classification includes: performing recognition and classification tasks on grayscale image data through a first machine learning model, the first machine learning model is an improved YOLOv7 model, and performing mosaic data enhancement and initial bounding box calculation on the grayscale image data through the output layer in the first machine learning model to obtain grayscale image enhancement data and initial bounding box parameters;

[0028] The backbone network introduces deep separable convolution and channel self-attention mechanism. The first feature map is obtained by extracting the features of grayscale image enhancement data through deep separable convolution. The first feature map is decomposed based on the channel through the ELAN module integrated with the first channel self-attention mechanism to obtain the first feature sub-map. The first feature sub-map is upsampled by the path aggregation network PAFPN and the second feature map is obtained by element-by-element addition. The second channel self-attention mechanism is used to complete the weighted calculation to obtain the deep feature map.

[0029] A step feature fusion module is introduced in the detection head. The fused features are obtained by fusing the deep feature maps layer by layer. The fused features are mapped into the category space through the fully connected layer and the Softmax function to obtain the category probability distribution. The recognition and classification results are obtained by sorting based on the distribution of category probabilities. The bounding box regression prediction calculation and non-maximum suppression are performed based on the recognition and classification results and the initial bounding box parameters to obtain the center point coordinates, width and height of the bounding box corresponding to the recognition and classification results. The box selection atlas is generated based on the center point coordinates, width and height of the bounding box corresponding to the recognition and classification results.

[0030] As a preferred solution of the intelligent monitoring system based on computer vision technology described in the present invention, wherein: the posture recognition unit is used to perform posture recognition on a human target, extract images whose recognition and classification results are human bodies from the frame selection image set to obtain a human target frame selection image, perform image segmentation based on a bounding box in the human target frame selection image to obtain a human target image, perform posture classification on the human target image through a second machine learning model to obtain a posture classification result, and send a second special alarm to the user based on the posture classification result;

[0031] The body posture classification results include: standing, walking, squatting, sitting, lying down and twisting;

[0032] The logic of sending the second special alarm to the user includes: when the body posture classification result is twisted and the duration is greater than T 4 When a second special alert is sent to the customer;

[0033] When the posture classification result is lying down and the duration is greater than T 5 When a second special alert is sent to the customer;

[0034] When the body posture classification result is squatting and the duration is greater than T 6 When a second special alert is sent to the customer;

[0035] When the body posture classification result is sitting and the duration is greater than T 7 A second special alert is sent to the customer.

[0036] As a preferred solution of the intelligent monitoring system based on computer vision technology described in the present invention, wherein: a posture classification task is performed on a human target image by a second machine learning model, the second machine learning model is an improved convolutional neural network model, and the logic of the posture classification includes: convolving the human target image by introducing a convolution layer of a ResNet module, and obtaining a third feature map by regularization calculation, tanh activation function processing and average pooling calculation;

[0037] The fully connected layer of the Transformer module is introduced to perform one-dimensional expansion of the third feature map according to the channel dimension of the third feature map to obtain a one-dimensional vector, and the one-dimensional vector is sequentially sorted according to the channel order to obtain a one-dimensional feature sequence. The one-dimensional feature sequence is mapped by linear transformation to obtain a position coding data set, and the one-dimensional feature sequence is segmented by a multi-head probabilistic sparse self-attention mechanism to calculate the attention result, and the attention result is calculated by a linear transformation layer and a ReLU activation function layer to obtain an updated feature sequence. The dimension is restored based on the updated feature sequence and the corresponding position coding data set to obtain a fourth feature map, and the fourth feature map is calculated by a fully connected layer by performing neuron weighted multiplication and bias addition to obtain a posture feature vector set, and the posture feature vector set is classified by a support vector machine to obtain a posture key point classification result;

[0038] Based on the posture key point classification result, the posture key points are encoded to obtain a posture key point index, and a posture classification vector is generated based on the coordinate data, modulus data, direction data, and Euclidean distance posture key point index corresponding to each vector in the posture feature vector set, and the posture classification vector is secondary classified by a second support vector machine to obtain a posture classification result;

[0039] The body key point classification results include left hand, left shoulder, left arm joint, left leg joint, left foot, head, right hand, right shoulder, right arm joint, right leg joint, and right foot.

[0040] Among them, the posture recognition unit uses an improved convolutional neural network model and introduces the ResNet module, Transformer module and support vector machine for multi-level classification, which is conducive to accurately and efficiently completing the classification of human posture, and can provide intelligent warnings, which is conducive to timely detection of dangerous situations such as fainting, convulsions and falls. At the same time, the time threshold is introduced to effectively reduce the probability of false alarms.

[0041] As a preferred solution of the intelligent monitoring system based on computer vision technology described in the present invention, wherein: the terminal module includes an application unit and an audio-visual unit, the application unit is used to provide a login interface and a viewing interface for the user, and display the real-time monitoring screen, recognition and classification results, body classification results, frame selection atlas data, regular notifications, first special alarm and second special alarm information to the user through the cloud server, and provide the user with a two-way call function between the application terminal and the monitoring camera;

[0042] The sound and light unit is used to provide a two-way communication function for the user, and to issue a prompt by making a sound through a microphone and by flashing a warning light when the monitoring camera executes the first special alarm and the second special alarm.

[0043] As a preferred solution of the intelligent monitoring system based on computer vision technology described in the present invention, wherein: the storage module includes a camera storage unit and a cloud storage unit, and the camera storage unit is used to store the video acquisition data obtained by the monitoring camera through an SD memory card;

[0044] The cloud storage unit is used to store log data through a cloud server, and the log data includes user login data, user operation records, system error information, ambient light intensity index, natural light image data, infrared image data, recognition and classification results, posture classification results, frame selection atlas, general notifications, first special alarm and second special alarm information.

[0045] The beneficial effects of the present invention are as follows: grayscale image data is obtained by graying natural light image data and edge enhancement of infrared image data, which unifies the image data standards collected by different cameras and helps reduce computational complexity. Recognition and classification, bounding box generation and posture classification are performed on the cloud platform through the multi-level classification modules in the first machine learning model and the second machine learning model. Compared with the traditional method, the accuracy and efficiency of completing the classification task are significantly improved, and the terminal can be linked to realize intelligent early warning, which is conducive to timely detection of dangerous situations such as fire, human intrusion, human syncope, human convulsion and human fall. At the same time, the time threshold is introduced to effectively reduce the probability of false alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A basic flow chart of an intelligent monitoring system based on computer vision technology provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0048] Example 1, reference Figure 1 , is an embodiment of the present invention, and provides an intelligent monitoring system based on computer vision technology, including: a video acquisition module, a transmission module, an image cloud processing module, a storage module and a terminal module;

[0049] The video acquisition module includes a light detection unit, a threshold judgment unit and a shooting unit, which are used to detect the ambient light intensity index, determine the shooting mode of the surveillance camera through threshold judgment, and record the surveillance area to obtain video acquisition data;

[0050] The image cloud processing module includes a preprocessing unit, a target detection unit and a posture recognition unit, which are used to extract grayscale image data frame by frame from the video acquisition data and calculate the grayscale image data, identify and classify the grayscale image data through a first machine learning model and complete the box selection to obtain a box selection atlas, send a regular notification or a first special alarm to the user based on the identification and classification result, select a human target image from the box selection atlas, classify the human target image into posture through a second machine learning model to obtain a posture classification result, and send a second special alarm to the user based on the posture classification result;

[0051] The terminal module includes an application unit and an audio-visual unit, which are used to provide a login interface and a viewing interface for the user, display a real-time monitoring screen to the user through a cloud server and receive real-time reception notifications, provide the user with a two-way communication function between the application terminal and the monitoring camera, and send an audio-visual alarm when the monitoring camera executes a first special alarm and a second special alarm;

[0052] The storage module includes a camera storage unit and a cloud storage unit, which are used to store video acquisition data through the SD memory card of the monitoring camera and store log data through the cloud server.

[0053] In this embodiment, the light detection unit is used to detect the ambient light intensity index. The ambient light intensity of the monitoring camera is detected by the photosensitive sensor array, and T is calculated based on the ambient light intensity. 1 The rate at which ambient light intensity decreases over time;

[0054] The threshold judgment unit is used to monitor the camera shooting mode based on the ambient light intensity and T 1 The rate at which the ambient light intensity decreases within a certain period of time is used to perform a threshold judgment, and the logic of the threshold judgment includes:

[0055] When the ambient light intensity is greater than the first light threshold, determining the natural light shooting mode, starting the natural light camera and turning off the infrared camera;

[0056] When the ambient light intensity is less than a second light threshold, determining the infrared shooting mode, starting the infrared camera and turning off the natural light camera;

[0057] When the rate of decrease of the ambient light intensity is greater than a third light threshold, determining the infrared shooting mode, starting the infrared camera and turning off the natural light camera;

[0058] The shooting is used to record the monitored area to obtain video acquisition data. When the monitoring camera shooting mode is determined to be a natural light shooting mode, the natural light video data is obtained by collecting through the natural light camera;

[0059] When the surveillance camera shooting mode is determined to be the infrared shooting mode, infrared video data is acquired through the infrared camera;

[0060] The ambient light intensity index includes the ambient light intensity and the rate at which the ambient light intensity decreases within T time;

[0061] The surveillance camera shooting modes include natural light shooting mode and infrared shooting mode;

[0062] The video acquisition data includes natural light video data and infrared video data.

[0063] In this embodiment, the preprocessing unit is used to perform video frame traversal on the video acquisition data, obtain natural light image data and infrared image data by extracting frame by frame, grayscale the natural light image data to obtain natural light grayscale image data, perform edge detection on the infrared image data by using the Canny operator to obtain infrared image edge point pixels, perform nonlinear transformation on the infrared image edge point pixels by gamma correction to obtain infrared grayscale image data, and merge the natural light grayscale image data with the infrared grayscale image data to obtain grayscale image data.

[0064] Among them, the calculation expression for grayscale processing of natural light image data is:

[0065] NGV=αR+βG+γB

[0066] Among them, NGR represents the grayscale value obtained by converting the pixels of the natural light image, R represents the red channel value, α represents the weight of the red channel value, G represents the green channel value, β represents the weight of the green channel value, B represents the blue channel value, and γ represents the weight of the blue channel value.

[0067] In this embodiment, the target detection unit is used to identify and classify objects in the monitoring area, identify and classify the grayscale image data through the first machine learning model to obtain an identification and classification result, and select the identified and classified targets through a bounding box to obtain a frame selection atlas, and send a regular notification or a first special alert to the user based on the identification and classification result;

[0068] The identification and classification results include: human body, pet, vehicle, smoke and flame;

[0069] The logic for sending notifications or alerts to users includes: sending regular notifications to customers when the recognition and classification results are human, pet, and vehicle;

[0070] When the identification and classification result is smoke and the duration is less than or equal to T 2 When the identification and classification result is smoke and the duration is greater than T, a regular notification is sent to the customer. 2 When the time comes, the first special alert is sent to the customer;

[0071] When the identification classification result is flame and the duration is less than or equal to T 3 When the identification result is flame and the duration is greater than T, a regular notification is sent to the customer. 3 At the time, the first special alert is sent to the customer.

[0072] In this embodiment, the recognition and classification logic includes: performing recognition and classification tasks on grayscale image data through a first machine learning model, where the first machine learning model is an improved YOLOv7 model, and performing mosaic data enhancement and initial bounding box calculation on the grayscale image data through an output layer in the first machine learning model to obtain grayscale image enhancement data and initial bounding box parameters;

[0073] The backbone network introduces deep separable convolution and channel self-attention mechanism. The first feature map is obtained by extracting the features of grayscale image enhancement data through deep separable convolution. The first feature map is decomposed based on the channel through the ELAN module integrated with the first channel self-attention mechanism to obtain the first feature sub-map. The first feature sub-map is upsampled by the path aggregation network PAFPN and the second feature map is obtained by element-by-element addition. The second channel self-attention mechanism is used to complete the weighted calculation to obtain the deep feature map.

[0074] A step feature fusion module is introduced in the detection head. The fused features are obtained by fusing the deep feature maps layer by layer. The fused features are mapped into the category space through the fully connected layer and the Softmax function to obtain the category probability distribution. The recognition and classification results are obtained by sorting based on the distribution of category probabilities. The bounding box regression prediction calculation and non-maximum suppression are performed based on the recognition and classification results and the initial bounding box parameters to obtain the center point coordinates, width and height of the bounding box corresponding to the recognition and classification results. The box selection atlas is generated based on the center point coordinates, width and height of the bounding box corresponding to the recognition and classification results.

[0075] Among them, the first machine learning model is the improved YOLOv7 model that has been trained. Its training logic is: divide the image data set with labeled classification recognition results and bounding boxes into training set, validation set and test set according to 7:2:1, calculate the classification loss and bounding box regression loss through the WIOU loss function of the dynamic non-monotonic focusing mechanism, and update the model parameters based on the classification loss and bounding box regression loss. Iterative calculation, measure the performance of the model by calculating the precision, recall rate, average precision mean and detection speed, obtain the optimal model parameters, and complete the training.

[0076] Among them, the depthwise separable convolution can effectively reduce the number of model parameters and computational complexity when extracting image features, and improve the model's running speed compared to traditional convolution. The ELAN module that integrates the channel self-attention mechanism enables the model to automatically focus on channel features that are important for target detection. By performing channel-based decomposition on the first feature map, it is helpful to capture detailed information in the image and dynamically adjust the weights of each channel, highlighting key features and suppressing irrelevant information. The step feature fusion module is conducive to fully integrating feature information at different levels and improving the accuracy of the model in detecting targets of different scales.

[0077] In this embodiment, the posture recognition unit is used to perform posture recognition on a human target, extract images whose recognition and classification results are human bodies from the frame selection image set to obtain a human target frame selection image, perform image segmentation based on a bounding box in the human target frame selection image to obtain a human target image, perform posture classification on the human target image through a second machine learning model to obtain a posture classification result, and send a second special alert to the user based on the posture classification result;

[0078] The body posture classification results include: standing, walking, squatting, sitting, lying down and twisting;

[0079] The logic of sending the second special alarm to the user includes: when the body posture classification result is twisted and the duration is greater than T 4 When a second special alert is sent to the customer;

[0080] When the posture classification result is lying down and the duration is greater than T 5 When a second special alert is sent to the customer;

[0081] When the body posture classification result is squatting and the duration is greater than T 6 When a second special alert is sent to the customer;

[0082] When the body posture classification result is sitting and the duration is greater than T 7 A second special alert is sent to the customer.

[0083] In this embodiment, a posture classification task is performed on a human target image by a second machine learning model, wherein the second machine learning model is an improved convolutional neural network model, and the logic of the posture classification includes: convolving the human target image by introducing a convolutional layer of a ResNet module, and obtaining a third feature map by regularization calculation, tanh activation function processing, and average pooling calculation;

[0084] The fully connected layer of the Transformer module is introduced to perform one-dimensional expansion of the third feature map according to the channel dimension of the third feature map to obtain a one-dimensional vector, and the one-dimensional vector is sequentially sorted according to the channel order to obtain a one-dimensional feature sequence. The one-dimensional feature sequence is mapped by linear transformation to obtain a position coding data set, and the one-dimensional feature sequence is segmented by a multi-head probabilistic sparse self-attention mechanism to calculate the attention result, and the attention result is calculated by a linear transformation layer and a ReLU activation function layer to obtain an updated feature sequence. The dimension is restored based on the updated feature sequence and the corresponding position coding data set to obtain a fourth feature map, and the fourth feature map is calculated by a fully connected layer by performing neuron weighted multiplication and bias addition to obtain a posture feature vector set, and the posture feature vector set is classified by a support vector machine to obtain a posture key point classification result;

[0085] Based on the posture key point classification result, the posture key points are encoded to obtain a posture key point index, and a posture classification vector is generated based on the coordinate data, modulus data, direction data, and Euclidean distance posture key point index corresponding to each vector in the posture feature vector set, and the posture classification vector is secondary classified by a second support vector machine to obtain a posture classification result;

[0086] The body key point classification results include left hand, left shoulder, left arm joint, left leg joint, left foot, head, right hand, right shoulder, right arm joint, right leg joint, and right foot.

[0087] Among them, the second machine learning model is a trained improved convolutional neural network model, and its training logic is: divide the image data set with labeled body classification results into training set, validation set and test set according to 7:2:1, load the pre-trained ResNet module and Transformer module weights to speed up the convergence of the model. The accuracy of the model prediction is evaluated by the cross entropy loss function, and the back propagation update is performed by the stochastic gradient descent algorithm. By calculating the gradient of the cross entropy loss function to the model parameters, the parameters are updated and iteratively calculated along the direction of gradient descent, and the performance of the model is measured by calculating the precision, recall rate, total number of parameters, model size and number of floating-point operations, the optimal model hyperparameters corresponding to the second machine learning model are obtained, and the training is completed.

[0088] In this embodiment, the terminal module includes an application unit and an audio-visual unit. The application unit is used to provide a login interface and a viewing interface for the user, and to display the real-time monitoring screen, recognition and classification results, body classification results, frame selection atlas data, regular notifications, first special alarm and second special alarm information to the user through the cloud server, and to provide the user with a two-way call function between the application terminal and the surveillance camera;

[0089] The sound and light unit is used to provide a two-way communication function for the user, and to issue a prompt by making a sound through a microphone and by flashing a warning light when the monitoring camera executes the first special alarm and the second special alarm.

[0090] In this embodiment, the storage module includes a camera storage unit and a cloud storage unit, and the camera storage unit is used to store the video acquisition data acquired by the surveillance camera through an SD memory card;

[0091] The cloud storage unit is used to store log data through a cloud server, and the log data includes user login data, user operation records, system error information, ambient light intensity index, natural light image data, infrared image data, recognition and classification results, posture classification results, frame selection atlas, general notifications, first special alarm and second special alarm information.

[0092] It should be understood by those skilled in the art that embodiments of the present invention may provide methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. Among them, the storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0093] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. Intelligent monitoring system based on computer vision technology, characterized by: include: Video acquisition module, transmission module, image cloud processing module, storage module and terminal module; The video acquisition module includes a light detection unit, a threshold judgment unit and a shooting unit, which are used to detect the ambient light intensity index, determine the shooting mode of the surveillance camera through threshold judgment, and record the surveillance area to obtain video acquisition data; The image cloud processing module includes a preprocessing unit, a target detection unit and a posture recognition unit, which are used to extract grayscale image data frame by frame from the video acquisition data and calculate the grayscale image data, identify and classify the grayscale image data through a first machine learning model to obtain an identification and classification result, complete box selection to obtain a box selection atlas, send a regular notification or a first special alarm to the user based on the identification and classification result, select a human target image from the box selection atlas, classify the human target image through a second machine learning model to obtain a posture classification result, and send a second special alarm to the user based on the posture classification result; The terminal module includes an application unit and an audio-visual unit, which are used to provide a login interface and a viewing interface for the user, display a real-time monitoring screen to the user through a cloud server and receive real-time reception notifications, provide the user with a two-way communication function between the application terminal and the monitoring camera, and send an audio-visual alarm when the monitoring camera executes a first special alarm and a second special alarm; The storage module includes a camera storage unit and a cloud storage unit, which are used to store video acquisition data through the SD memory card of the monitoring camera and store log data through the cloud server.

2. The intelligent monitoring system based on computer vision technology as claimed in claim 1, characterized in that: The light detection unit is used to detect the ambient light intensity index, obtain the ambient light intensity of the monitoring camera through the photosensitive sensor array detection, and calculate the ambient light intensity reduction rate within T1 time based on the ambient light intensity; The threshold judgment unit is used to monitor the camera shooting mode, and performs threshold judgment based on the ambient light intensity and the rate of decrease of the ambient light intensity within the T1 time. The logic of the threshold judgment includes: When the ambient light intensity is greater than the first light threshold, determining the natural light shooting mode, starting the natural light camera and turning off the infrared camera; When the ambient light intensity is less than a second light threshold, determining the infrared shooting mode, starting the infrared camera and turning off the natural light camera; When the rate of decrease of the ambient light intensity is greater than a third light threshold, determining the infrared shooting mode, starting the infrared camera and turning off the natural light camera; The shooting is used to record the monitored area to obtain video acquisition data. When the monitoring camera shooting mode is determined to be a natural light shooting mode, the natural light video data is obtained by collecting through the natural light camera; When the surveillance camera shooting mode is determined to be the infrared shooting mode, infrared video data is acquired through the infrared camera; The ambient light intensity index includes the ambient light intensity and the rate at which the ambient light intensity decreases within T time; The surveillance camera shooting modes include natural light shooting mode and infrared shooting mode; The video acquisition data includes natural light video data and infrared video data.

3. The intelligent monitoring system based on computer vision technology as claimed in claim 1, characterized in that: The preprocessing unit is used to perform video frame traversal on the video acquisition data, obtain natural light image data and infrared image data by extracting frame by frame, grayscale the natural light image data to obtain natural light grayscale image data, perform edge detection on the infrared image data by the Canny operator to obtain infrared image edge point pixels, perform nonlinear transformation on the infrared image edge point pixels by gamma correction to obtain infrared grayscale image data, and merge the natural light grayscale image data with the infrared grayscale image data to obtain grayscale image data.

4. The intelligent monitoring system based on computer vision technology as claimed in claim 1, characterized in that: The target detection unit is used to identify and classify objects in the monitoring area, identify and classify the grayscale image data through the first machine learning model to obtain an identification and classification result, and select the identified and classified targets through a bounding box to obtain a frame selection atlas, and send a general notification or a first special alert to the user based on the identification and classification result; The identification and classification results include: human body, pet, vehicle, smoke and flame; The logic for sending notifications or alerts to users includes: sending regular notifications to customers when the recognition and classification results are human, pet, and vehicle; When the identification and classification result is smoke and the duration is less than or equal to T2 time, a regular notification is sent to the customer; when the identification and classification result is smoke and the duration is greater than T2 time, a first special alarm is sent to the customer; When the identification and classification result is flame and the duration is less than or equal to T3 time, a regular notification is sent to the customer. When the identification and classification result is flame and the duration is greater than T3 time, a first special alarm is sent to the customer.

5. The intelligent monitoring system based on computer vision technology as claimed in claim 4, characterized in that: The recognition and classification logic includes: performing recognition and classification tasks on grayscale image data through a first machine learning model, wherein the first machine learning model is an improved YOLOv7 model, and performing mosaic data enhancement and initial bounding box calculation on the grayscale image data through an output layer in the first machine learning model to obtain grayscale image enhancement data and initial bounding box parameters; The backbone network introduces deep separable convolution and channel self-attention mechanism. The first feature map is obtained by extracting the features of grayscale image enhancement data through deep separable convolution. The first feature map is decomposed based on the channel through the ELAN module integrated with the first channel self-attention mechanism to obtain the first feature sub-map. The first feature sub-map is upsampled by the path aggregation network PAFPN and the second feature map is obtained by element-by-element addition. The second channel self-attention mechanism is used to complete the weighted calculation to obtain the deep feature map. A step feature fusion module is introduced in the detection head. The fused features are obtained by fusing the deep feature maps layer by layer. The fused features are mapped into the category space through the fully connected layer and the Softmax function to obtain the category probability distribution. The recognition and classification results are obtained by sorting based on the distribution of category probabilities. The bounding box regression prediction calculation and non-maximum suppression are performed based on the recognition and classification results and the initial bounding box parameters to obtain the center point coordinates, width and height of the bounding box corresponding to the recognition and classification results. The box selection atlas is generated based on the center point coordinates, width and height of the bounding box corresponding to the recognition and classification results.

6. The intelligent monitoring system based on computer vision technology as claimed in claim 4, characterized in that: The posture recognition unit is used to perform posture recognition on a human target, extract images whose recognition and classification results are human bodies from the frame selection image set to obtain a human target frame selection image, perform image segmentation based on a bounding box in the human target frame selection image to obtain a human target image, perform posture classification on the human target image through a second machine learning model to obtain a posture classification result, and send a second special alert to the user based on the posture classification result; The body posture classification results include: standing, walking, squatting, sitting, lying down and twisting; The logic of sending the second special alert to the user includes: when the body posture classification result is distortion and the duration is greater than T4, sending the second special alert to the customer; When the body posture classification result is lying down and the duration is greater than T5, a second special alarm is sent to the customer; When the body posture classification result is squatting and the duration is greater than T6, a second special alarm is sent to the customer; When the posture classification result is sitting still and the duration is greater than T7, a second special alert is sent to the client.

7. The intelligent monitoring system based on computer vision technology as claimed in claim 6, characterized in that: Performing a posture classification task on the human target image through a second machine learning model, wherein the second machine learning model is an improved convolutional neural network model, and the logic of the posture classification includes: convolving the human target image through a convolution layer of a ResNet module, and obtaining a third feature map through regularization calculation, tanh activation function processing and average pooling calculation; The fully connected layer of the Transformer module is introduced to perform one-dimensional expansion of the third feature map according to the channel dimension of the third feature map to obtain a one-dimensional vector, and the one-dimensional vector is sequentially sorted according to the channel order to obtain a one-dimensional feature sequence. The one-dimensional feature sequence is mapped by linear transformation to obtain a position coding data set, and the one-dimensional feature sequence is segmented by a multi-head probabilistic sparse self-attention mechanism to calculate the attention result, and the attention result is calculated by a linear transformation layer and a ReLU activation function layer to obtain an updated feature sequence. The dimension is restored based on the updated feature sequence and the corresponding position coding data set to obtain a fourth feature map, and the fourth feature map is calculated by a fully connected layer by performing neuron weighted multiplication and bias addition to obtain a posture feature vector set, and the posture feature vector set is classified by a support vector machine to obtain a posture key point classification result; Based on the posture key point classification result, the posture key points are encoded to obtain a posture key point index, and a posture classification vector is generated based on the coordinate data, modulus data, direction data, and Euclidean distance posture key point index corresponding to each vector in the posture feature vector set, and the posture classification vector is secondary classified by a second support vector machine to obtain a posture classification result; The body key point classification results include left hand, left shoulder, left arm joint, left leg joint, left foot, head, right hand, right shoulder, right arm joint, right leg joint, and right foot.

8. The intelligent monitoring system based on computer vision technology as claimed in claim 1, characterized in that: The terminal module includes an application unit and an audio-visual unit, wherein the application unit is used to provide a login interface and a viewing interface for the user, and to display the real-time monitoring screen, recognition and classification results, body classification results, frame selection atlas data, regular notifications, first special alarm and second special alarm information to the user through the cloud server, and to provide the user with a two-way communication function between the application terminal and the monitoring camera; The sound and light unit is used to provide a two-way communication function for the user, and to issue a prompt by making a sound through a microphone and by flashing a warning light when the monitoring camera executes the first special alarm and the second special alarm.

9. The intelligent monitoring system based on computer vision technology as claimed in claim 1, characterized in that: The storage module includes a camera storage unit and a cloud storage unit, wherein the camera storage unit is used to store video acquisition data acquired by the surveillance camera through an SD memory card; The cloud storage unit is used to store log data through a cloud server, and the log data includes user login data, user operation records, system error information, ambient light intensity index, natural light image data, infrared image data, recognition and classification results, posture classification results, frame selection atlas, general notifications, first special alarm and second special alarm information.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the intelligent monitoring system based on computer vision technology described in any one of claims 1 to 9 is implemented.

Images