Image target detection method and system based on deep reinforcement learning
By adopting a deep reinforcement learning-based image object detection method in the video surveillance system, using the Actor-Critic network and optimization algorithm, the problem of inaccurate detection of small objects and fast moving objects in the prior art is solved, and higher detection accuracy and processing speed are achieved.
Patent Information
- Application Number
- CN202411757019.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-03
- Publication Date
- 2025-03-14
AI Technical Summary
When processing video images, existing video surveillance systems have problems such as inaccurate detection of small objects and fast moving targets and slow processing speed, which cannot meet the needs of intelligent analysis.
The image object detection method based on deep reinforcement learning is adopted, and the Actor-Critic network is trained, and the network parameters are optimized using simulated annealing algorithm and taboo search algorithm to improve the accuracy and processing speed of object detection.
It significantly improves the accuracy of detecting human features and object features in surveillance videos, and accelerates processing speed, which can better assist in intelligent analysis and processing.
Smart Images
Figure CN119625639A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of video image processing, and in particular relates to an image target detection method and system based on deep reinforcement learning. Background Art
[0002] At present, video surveillance systems have been increasingly deployed in communities, schools, commercial places, and public places. One of the main purposes of installing video surveillance systems is for safety. However, current video surveillance systems cannot respond in real time, especially for some special situations that endanger safety, such as someone breaking into a door, picking a lock, climbing over a wall, etc. It will be too late to search for the video afterwards. Therefore, it is necessary to be able to detect abnormal scenes in real time so as to respond quickly. This puts forward the demand for intelligent analysis and processing of the video of the video surveillance system.
[0003] Intelligent analysis of the video of the video surveillance system is mainly completed by analyzing and processing the video frame images, among which image target detection is a very important step. In the prior art, YOLOv5 is often used to complete the target detection function. However, YOLOv5 also has some shortcomings, such as the inaccurate detection of small objects in the video image, the poor detection effect of complex human movements or fast-moving targets in the video, and the slow processing speed; therefore, the detection results cannot really meet the needs of intelligent analysis. Summary of the invention
[0004] The present invention proposes an image target detection method and system based on deep reinforcement learning, which can improve the accuracy of image target detection and meet the needs of intelligent analysis of surveillance videos.
[0005] To achieve the above object, the technical solution of the present invention is achieved as follows:
[0006] An image target detection method based on deep reinforcement learning, comprising:
[0007] S1. Obtain monitoring video streams of the same monitoring device at different time periods and decode them into sequential frame images respectively;
[0008] S2, preprocessing the sequence frame images, and using the preprocessed images as image data sets;
[0009] S3, the image data in the image dataset is input into the Actor-Critic network for training;
[0010] S4. Optimize the parameters of the Actor neural network in the Actor-Critic network based on the simulated annealing algorithm;
[0011] S5. Optimize the parameters of the Critic neural network in the Actor-Critic network based on the taboo search algorithm;
[0012] S6. Train an image object detection model for object detection in video frame images.
[0013] Further, step S3 includes:
[0014] S301, annotating images in the image data set, annotating human features and object features in the images;
[0015] S302, inputting the labeled image data into the Actor neural network in the Actor-Critic network; the Actor neural network is set to have two fully connected layers, the first fully connected layer outputs human features, and the second fully connected layer outputs object features;
[0016] S303, inputting the human body features and object features output by the Actor neural network into the Critic neural network for evaluation, wherein the Critic neural network includes an input layer, a hidden layer, and an output layer.
[0017] Furthermore, in step S4, the parameters of the Actor neural network are optimized by using a simulated annealing algorithm and a cross entropy loss function with the goal of minimizing the loss function; the parameters of the Actor neural network include convolution kernel size, number of convolution kernel channels, pooling matrix size, and initial learning rate.
[0018] Furthermore, in step S5, the weights and thresholds of the hidden layer and the output layer are adjusted by the taboo search algorithm to minimize the error function.
[0019] On the other hand, the present invention also proposes an image target detection system based on deep reinforcement learning, comprising:
[0020] Acquisition module: acquires the surveillance video streams of the same surveillance device at different time periods and decodes them into sequence frame images respectively;
[0021] Preprocessing module: preprocess the sequence frame images, and the preprocessed images are used as image data sets;
[0022] Training module: The image data in the image dataset is input into the Actor-Critic network for training;
[0023] The first optimization module: optimizes the parameters of the Actor neural network in the Actor-Critic network based on the simulated annealing algorithm;
[0024] The second optimization module: optimizes the parameters of the Critic neural network in the Actor-Critic network based on the taboo search algorithm;
[0025] Model module: Train an image object detection model for object detection in video frame images.
[0026] Furthermore, the training modules include:
[0027] Annotation unit: annotate the images in the image data set, and annotate the human features and object features in the images;
[0028] Actor neural network unit: inputting the labeled image data into the Actor neural network in the Actor-Critic network; the Actor neural network is set to have two fully connected layers, the first fully connected layer outputs human features, and the second fully connected layer outputs object features;
[0029] Critic neural network unit: The human body features and object features output by the Actor neural network are input into the Critic neural network for evaluation. The Critic neural network includes an input layer, a hidden layer, and an output layer.
[0030] Furthermore, in the first optimization module, the parameters of the Actor neural network are optimized by using a simulated annealing algorithm and a cross entropy loss function with the goal of minimizing the loss function; the parameters of the Actor neural network include convolution kernel size, number of convolution kernel channels, pooling matrix size, and initial learning rate.
[0031] Furthermore, in the second optimization module, the weights and thresholds of the hidden layer and the output layer are adjusted through the taboo search algorithm to minimize the error function.
[0032] The present invention also provides a computer-readable storage medium, which stores a computer program, and the computer program is used to execute the above-mentioned image target detection method based on deep reinforcement learning.
[0033] The present invention also provides a computer program product, including a computer program, which implements the above-mentioned image target detection method based on deep reinforcement learning when executed by a processor.
[0034] Compared with the prior art, the method and system for automatically updating planning vector data in the field of urban planning proposed by the present invention have the following beneficial effects:
[0035] The present invention uses an Actor-Critic network to detect image targets, and uses a simulated annealing algorithm to optimize the parameters of the Actor neural network, and uses a taboo search algorithm to optimize the parameters of the Critic neural network, which can improve the accuracy of target detection such as human features and object features in surveillance videos, and increase the processing speed relatively quickly, so it can better assist the next step of intelligent analysis and processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 It is a schematic diagram of the overall process of Example 1 of the present invention;
[0037] Figure 2 Schematic diagram of the Actor-Critic network training steps of Example 1 of the present invention;
[0038] Figure 3 It is a schematic diagram of the optimization process of the taboo search algorithm of Example 1 of the present invention. DETAILED DESCRIPTION
[0039] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.
[0040] In order to make the purpose and features of the present invention more obvious and understandable, the specific implementation methods of the present invention are further described below in conjunction with the accompanying drawings.
[0041] The present invention mainly performs image target detection on monitored video images and accurately detects image features of human bodies and objects in video images.
[0042] The design idea of the present invention is mainly as follows: using Actor-Critic network for image target detection, using simulated annealing algorithm to optimize the parameters of Actor network, and using taboo search algorithm to optimize the parameters of Critic network, in order to obtain more accurate image target detection results.
[0043] Embodiment 1:
[0044] Based on the above design ideas, the image target detection method based on deep reinforcement learning implemented in this embodiment is as follows Figure 1 As shown, including:
[0045] 1. Obtain video image data;
[0046] The purpose of this step is to collect image data for intelligent model training. The method is as follows:
[0047] The monitoring video streams of the same monitoring device at different time periods are obtained and decoded into sequence frame images respectively; that is, the historical monitoring video of the monitoring device is obtained, multiple video streams of different time periods are obtained, each video stream is decoded to obtain a sequence frame image.
[0048] 2. Preprocess and construct image dataset;
[0049] 2.1 Preprocessing the sequence frame images;
[0050] In this step, the preprocessing refers to processing the video frame image into an image standard suitable for target detection according to the characteristics of the video frame image, and can adopt processing methods including image scaling, color space conversion and normalization of the frame image, etc.
[0051] Image scaling refers to adjusting the size of the frame image by adding or removing pixels to change the size of the frame image, thereby making a trade-off between processing efficiency and the smoothness and clarity of the result. When the frame image size increases, the visibility of the pixels increases; when the frame image size decreases, the image will be smoother and clearer.
[0052] Color space conversion refers to converting a frame image from a non-RGB color space to an RGB color space. Videos shot by surveillance equipment usually have different color spaces, such as YUV, YV12, YCbCr, RGB, etc. The RGB color space is based on the three primary colors of red, green, and blue, and various colors are generated by adjusting the mixing ratio of these three colors. Compared with other color spaces, the RGB color space is more intuitive and easier to understand, so it is more suitable for image target detection algorithms.
[0053] Normalization helps reduce noise in the frame image data, makes the model easier to process, and optimizes the performance of the model. In this embodiment, the normalization method uses Z-score standardization, calculates the mean and standard deviation of the frame image, and linearly transforms each pixel value of the frame image so that its mean is 0 and its standard deviation is 1.
[0054] 2.2 Build image dataset;
[0055] All preprocessed frame images constitute an image dataset, and the image dataset is divided into a training set, a validation set, and a test set in a ratio of 7:2:1.
[0056] 3. The image data in the image dataset is input into the Actor-Critic network for training. The steps are as follows Figure 2 As shown, including;
[0057] 301. Image annotation: Annotate the images in the training set, validation set, and test set, and annotate the human features and object features in the images;
[0058] 302. Input the labeled image data into the Actor neural network in the Actor-Critic network;
[0059] In this step, the Actor neural network sets up convolutional layers and pooling layers, uses Sigmoid function as activation function, extracts features through convolutional layers and pooling layers, performs feature matching convolution calculations on each position of the original image through feature vectors, and then reduces the feature image through pooling to reduce the amount of calculation, and then adds nonlinear elements through activation functions. Two fully connected layers are set up afterwards, the first fully connected layer outputs human features, and the second fully connected layer outputs object features; the loss function adopts the cross entropy loss function;
[0060] 303. The human body features and object features output by the Actor neural network are input into the Critic neural network for evaluation.
[0061] In this step, the Critic neural network includes an input layer, a hidden layer, and an output layer. The hidden layer has a relu activation function, and the output layer has a softmax activation function.
[0062] 4. Optimize the parameters of the Actor neural network based on the simulated annealing algorithm;
[0063] The simulated annealing algorithm optimizes the parameters of the Actor neural network with the goal of minimizing the loss function; the parameters of the Actor neural network include the convolution kernel size, the number of convolution kernel channels, the pooling matrix size, and the initial learning rate.
[0064] The optimization process includes: the simulated annealing algorithm initializes the above parameters and randomly generates a set of initial parameters; then sets the initial temperature and termination temperature for the simulated annealing algorithm; decides whether to accept the neighborhood solution based on the objective function value of the neighborhood solution and the objective function value of the current solution; if the objective function value of the neighborhood solution is better, the neighborhood solution is accepted; otherwise, the difference solution is accepted with a certain probability; when the termination temperature or other termination conditions are reached, the algorithm is terminated and the currently found optimal parameters are returned.
[0065] 5. Optimize the parameters of the Critic neural network based on the taboo search algorithm;
[0066] The taboo search algorithm mainly adjusts the weights and thresholds of the hidden layer and the output layer to minimize the error function. In this embodiment, the mean absolute error MAE is used as the error function, and the optimization process of the taboo search algorithm is as follows: Figure 3 As shown, including:
[0067] (1) Initialization: assign a small random number to each parameter as the initialization value;
[0068] (2) The current solution of each parameter is taken as the optimal solution and stored in the taboo table;
[0069] (3) Take a neighborhood solution of the initialization value and calculate the error based on the neighborhood solution and the error based on the optimal solution;
[0070] (4) If the neighborhood solution does not change for multiple consecutive times, the algorithm is terminated and the result is output, and the optimization ends. Otherwise, execute step (5);
[0071] (5) If the error based on the neighborhood solution is less than the error based on the optimal solution, the neighborhood solution is stored in the taboo table as the optimal solution, the taboo table is updated, and the original stored content in the taboo table is moved backward;
[0072] If the error based on the neighborhood solution is greater than or equal to the error based on the optimal solution, determine whether the neighborhood solution is taboo. If the neighborhood solution is in a given neighborhood of a storage point in the taboo table, it means that the neighborhood solution is taboo, and a new neighborhood solution is generated; if it is not taboo, the neighborhood solution is assigned to the current solution and the taboo table is updated at the same time;
[0073] (6) Generate a neighborhood solution for the current solution, and then execute step (4).
[0074] At the end of the optimization, the weights and thresholds of the trained hidden layer and output layer are obtained.
[0075] 6. Train the image target detection model for target detection in video frame images.
[0076] The method described in this embodiment can improve the accuracy of target detection such as human features and object features in surveillance videos, and increase the processing speed relatively quickly.
[0077] Embodiment 2:
[0078] In Example 2, an image target detection system based on deep reinforcement learning is proposed, including:
[0079] Acquisition module: acquires the surveillance video streams of the same surveillance device at different time periods and decodes them into sequence frame images respectively;
[0080] Preprocessing module: preprocess the sequence frame images, and the preprocessed images are used as image data sets;
[0081] Training module: The image data in the image dataset is input into the Actor-Critic network for training;
[0082] The first optimization module: optimizes the parameters of the Actor neural network in the Actor-Critic network based on the simulated annealing algorithm;
[0083] The second optimization module: optimizes the parameters of the Critic neural network in the Actor-Critic network based on the taboo search algorithm;
[0084] Model module: Train an image object detection model for object detection in video frame images.
[0085] The training modules include:
[0086] Annotation unit: annotate the images in the image data set, and annotate the human features and object features in the images;
[0087] Actor neural network unit: inputting the labeled image data into the Actor neural network in the Actor-Critic network; the Actor neural network is set to have two fully connected layers, the first fully connected layer outputs human features, and the second fully connected layer outputs object features;
[0088] Critic neural network unit: The human body features and object features output by the Actor neural network are input into the Critic neural network for evaluation. The Critic neural network includes an input layer, a hidden layer, and an output layer.
[0089] In the first optimization module, the parameters of the Actor neural network are optimized by using a simulated annealing algorithm and a cross entropy loss function with the goal of minimizing the loss function; the parameters of the Actor neural network include convolution kernel size, number of convolution kernel channels, pooling matrix size, and initial learning rate.
[0090] In the second optimization module, the weights and thresholds of the hidden layer and the output layer are adjusted through the taboo search algorithm to minimize the error function.
[0091] The image target detection system based on deep reinforcement learning described in this embodiment can implement the image target detection method based on deep reinforcement learning described in Example 1, and the technical effect achieved is the same as that of Example 1.
[0092] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for image target detection based on deep reinforcement learning, characterized in that: include: S1. Obtain monitoring video streams of the same monitoring device at different time periods and decode them into sequential frame images respectively; S2, preprocessing the sequence frame images, and using the preprocessed images as image data sets; S3, the image data in the image dataset is input into the Actor-Critic network for training; S4. Optimize the parameters of the Actor neural network in the Actor-Critic network based on the simulated annealing algorithm; S5. Optimize the parameters of the Critic neural network in the Actor-Critic network based on the taboo search algorithm; S6. Train an image object detection model for object detection in video frame images.
2. The image target detection method based on deep reinforcement learning according to claim 1, characterized in that: Step S3 includes: S301, annotating images in the image data set, annotating human features and object features in the images; S302, inputting the labeled image data into the Actor neural network in the Actor-Critic network; the Actor neural network is set to have two fully connected layers, the first fully connected layer outputs human features, and the second fully connected layer outputs object features; S303, inputting the human body features and object features output by the Actor neural network into the Critic neural network for evaluation, wherein the Critic neural network includes an input layer, a hidden layer, and an output layer.
3. The image target detection method based on deep reinforcement learning according to claim 2 is characterized in that: In step S4, the parameters of the Actor neural network are optimized by using a simulated annealing algorithm and a cross entropy loss function with the goal of minimizing the loss function; the parameters of the Actor neural network include convolution kernel size, number of convolution kernel channels, pooling matrix size, and initial learning rate.
4. The image target detection method based on deep reinforcement learning according to claim 2, characterized in that: In step S5, the weights and thresholds of the hidden layer and the output layer are adjusted by the taboo search algorithm to minimize the error function.
5. An image target detection system based on deep reinforcement learning, characterized in that: include: Acquisition module: acquires the surveillance video streams of the same surveillance device at different time periods and decodes them into sequence frame images respectively; Preprocessing module: preprocess the sequence frame images, and the preprocessed images are used as image data sets; Training module: The image data in the image dataset is input into the Actor-Critic network for training; The first optimization module: optimizes the parameters of the Actor neural network in the Actor-Critic network based on the simulated annealing algorithm; The second optimization module: optimizes the parameters of the Critic neural network in the Actor-Critic network based on the taboo search algorithm; Model module: Train an image object detection model for object detection in video frame images.
6. The image target detection system based on deep reinforcement learning according to claim 5, characterized in that: The training modules include: Annotation unit: annotate the images in the image data set, and annotate the human features and object features in the images; Actor neural network unit: inputting the labeled image data into the Actor neural network in the Actor-Critic network; the Actor neural network is set to have two fully connected layers, the first fully connected layer outputs human features, and the second fully connected layer outputs object features; Critic neural network unit: The human body features and object features output by the Actor neural network are input into the Critic neural network for evaluation. The Critic neural network includes an input layer, a hidden layer, and an output layer.
7. The image target detection system based on deep reinforcement learning according to claim 6, characterized in that: In the first optimization module, the parameters of the Actor neural network are optimized by using a simulated annealing algorithm and a cross entropy loss function with the goal of minimizing the loss function; the parameters of the Actor neural network include convolution kernel size, number of convolution kernel channels, pooling matrix size, and initial learning rate.
8. The image target detection system based on deep reinforcement learning according to claim 6, characterized in that: In the second optimization module, the weights and thresholds of the hidden layer and the output layer are adjusted through the taboo search algorithm to minimize the error function.
9. A computer-readable storage medium storing a computer program, characterized in that: The computer program is used to execute the image target detection method based on deep reinforcement learning as described in any one of claims 1 to 4.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image target detection method based on deep reinforcement learning as described in any one of claims 1 to 4 is implemented.