Deep neural network object detection method, device and product based on reinforcement learning

Abstract

This application discloses a method, device, and product for object detection based on reinforcement learning and deep neural networks, relating to the field of computer vision. The method includes performing object detection on an input image using a deep neural network; and dynamically optimizing the object detection process using a reinforcement learning agent coupled to the deep neural network. The dynamic optimization includes: the reinforcement learning agent taking at least one intermediate feature of the deep neural network as a state input; the reinforcement learning agent responding to the state input to output an action policy, the action policy being used to dynamically adjust at least one operating parameter of the deep neural network; and the deep neural network outputting the detection result after applying the action policy. This improves prediction accuracy and generalization ability to unknown scenes.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a method, device and product for object detection based on deep neural networks of reinforcement learning. Background Technology

[0002] Object detection is a key task in the field of computer vision, playing a central role in scenarios such as autonomous driving, intelligent security, and robot navigation. Existing object detection technologies are mainly divided into two categories: One approach is traditional image processing methods, such as those based on manually designed features like edges, colors, and textures. These methods are effective in simple or specific scenarios, but they rely on fixed feature extraction rules and are not robust to dynamically changing environments such as lighting changes, object occlusion, and complex backgrounds, making them unsuitable for the needs of complex real-world scenarios.

[0003] Second, there are deep learning-based detection methods, with convolutional neural networks at their core, such as the YOLO series and Faster R-CNN. These methods, through end-to-end training on large-scale labeled datasets, can automatically learn deep features in images, achieving significant improvements in detection accuracy and speed compared to traditional methods. However, these methods also have inherent drawbacks: First, their performance is highly dependent on massive amounts of high-quality labeled data, which is costly to obtain; second, once the model is trained, its network structure and parameters are relatively fixed, and its generalization ability and detection performance will significantly decrease for unknown types of targets or sudden scenes not present in the training dataset; finally, in pursuit of high accuracy, the models are usually designed to be quite complex, resulting in huge consumption of computational resources and posing challenges for deployment on resource-constrained edge devices.

[0004] Therefore, there is an urgent need for a target detection method that can simultaneously improve prediction accuracy and generalize to unknown scenarios. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, and product for object detection based on reinforcement learning deep neural networks, so as to improve prediction accuracy and generalization ability to unknown scenarios.

[0006] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a deep neural network-based object detection method based on reinforcement learning, comprising: Target detection of input images using deep neural networks; and The target detection process is dynamically optimized by a reinforcement learning agent coupled to the deep neural network. The dynamic optimization includes: The reinforcement learning agent takes at least one intermediate feature of the deep neural network as a state input; The reinforcement learning agent responds to the state input to output an action policy, the action policy being used to dynamically adjust at least one operating parameter of the deep neural network; The deep neural network outputs the detection results after applying the action strategy.

[0007] In a second aspect, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the deep neural network object detection method based on reinforcement learning as described above.

[0008] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the deep neural network object detection method based on reinforcement learning described above.

[0009] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the deep neural network object detection method based on reinforcement learning described above.

[0010] According to the specific embodiments provided in this application, the following technical effects are disclosed: 1. By introducing a reinforcement learning agent module coupled to the deep neural network object detection module, and utilizing the intermediate features of the deep neural network as state input, the reinforcement learning agent can perceive the internal state of the detection process and generate action strategies to dynamically adjust the operating parameters of the deep neural network. This closed-loop dynamic optimization mechanism enables the object detection model to adapt to changes in the input image in real time, thereby improving the accuracy and robustness of detection in complex and variable scenarios (such as changes in object scale, dense distribution, etc.).

[0011] 2. A comprehensive reward mechanism was constructed by using prediction loss, the average intersection-union ratio (IU) of predicted and ground truth boxes, and the forward inference latency as target metrics for the reinforcement learning agent. This guides the agent to pursue not only high detection accuracy but also computational efficiency and real-time performance when learning optimization strategies, achieving an effective balance between detection performance and resource consumption, making it more suitable for applications with high real-time requirements.

[0012] 3. By specifically defining the action strategy as adjusting the adaptive attention weight parameters for feature fusion at different scales and the filtering threshold for non-maximum suppression, precise control over key aspects such as feature fusion and candidate box post-processing is achieved. This targeted adjustment is more efficient than adjusting the entire network and can quickly optimize the model's performance in specific scenarios. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart illustrating a deep neural network-based target detection method using reinforcement learning in one embodiment of this application. Figure 2 This is a schematic diagram of the architecture of a deep neural network and a reinforcement learning agent in one embodiment of this application; Figure 3 This is a schematic diagram of the architecture of module C3 in one embodiment of this application; Figure 4 This is a schematic diagram of the architecture of the SPPF module in one embodiment of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] The deep neural network object detection method based on reinforcement learning provided in this embodiment is executed by a computer device, which can be a terminal computing device or a server. The terminal computing device can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, and IoT devices. The server can be a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0018] To integrate reinforcement learning into deep learning neural network models, this application provides a reinforcement learning-based deep neural network object detection method. By dynamically adjusting the detection strategy and optimizing the feature extraction and candidate box generation processes, the efficiency and accuracy of object detection are improved. This addresses the limitations of deep learning neural network models in detecting unknown objects and enhances the generalization and accuracy of deep learning object detection models.

[0019] In an exemplary embodiment, a deep neural network-based target detection method based on reinforcement learning is provided, which specifically includes steps S110 and S120.

[0020] Step S110: Target detection is performed on the input image using a deep neural network.

[0021] Deep neural networks are responsible for extracting hierarchical feature information from images and generating initial detection results. Existing technologies typically stop here; the model's parameters are fixed after training, making it difficult to adapt to the changing environments encountered during testing.

[0022] To address the aforementioned issues, this embodiment introduces a reinforcement learning agent coupled to a deep neural network.

[0023] Step S120: The target detection process is dynamically optimized by a reinforcement learning agent coupled to a deep neural network.

[0024] Dynamic optimization includes steps S121 and S122.

[0025] Step S121: Provide at least one intermediate feature of the deep neural network as a state input to the reinforcement learning agent.

[0026] The design principle of this step is that intermediate features contain richer and more abstract semantic information than the original image, and can more accurately reflect the difficulty of the current detection task and the characteristics of the scene, such as whether there are small targets, whether the targets are dense, and what the lighting conditions are.

[0027] Step S122: The reinforcement learning intelligent body responds to the state input to output an action policy, which is used to dynamically adjust at least one operating parameter of the deep neural network.

[0028] The policy network inside a reinforcement learning agent responds to the current state input and outputs an action policy. This action policy does not directly affect the final detection result, but rather dynamically adjusts at least one operating parameter of the deep neural network. In this way, the reinforcement learning agent can act as an external "meta-controller," fine-tuning the deep neural network itself based on real-time feedback to adapt it to the specific needs of the current frame. This design avoids the drawback of traditional methods where static parameters cannot cope with dynamic changes, achieving online and fine-grained control of the detection process.

[0029] Step S123: The deep neural network outputs the detection results after applying the action strategy.

[0030] After applying the action strategy output by the reinforcement learning agent (i.e., adjusting its own operating parameters), the deep neural network continues to complete the subsequent detection process and outputs the final detection result. The entire process forms a closed loop of "perception-decision-execution-feedback", enabling the detection system to have the ability to learn and adapt.

[0031] In this embodiment, the deep neural network includes a backbone network, a neck network, and a prediction head stacked sequentially; at least one intermediate feature is a feature extracted by the neck network.

[0032] The backbone network is responsible for extracting basic features from the input image; the neck network fuses the features extracted by the backbone network at different levels to obtain richer semantic information and spatial details; the prediction head performs the final target classification and bounding box regression based on the fused features output by the neck network.

[0033] In this structure, at least one intermediate feature serving as the state input is specifically defined as a feature extracted by the neck network. The principle behind this design is that the output features of the neck network are the result of multi-scale fusion, simultaneously containing high-level semantic information from deep networks and fine-grained texture information from shallow networks, making them an ideal carrier for describing the current image content and the state of the target to be detected. Using these features as state inputs to the reinforcement learning agent provides high-quality decision-making basis, enabling it to more accurately assess the complexity of the current scene and thus formulate more reasonable action strategies. By utilizing the features of the neck network, this embodiment achieves efficient information interaction between the deep detection network and the reinforcement learning agent, laying a solid foundation for subsequent dynamic parameter adjustments.

[0034] Specifically, the layers in the backbone network, from shallowest to deepest, are convolutional modules, convolutional modules, C3 modules, convolutional modules, C3 modules, convolutional modules, C3 modules, convolutional modules, C3 modules, and SPPF modules. This structure draws inspiration from the design of modern, efficient object detection networks (such as the YOLO series). By alternating between convolutional and C3 modules, it effectively controls the computational load of the model while ensuring powerful feature extraction capabilities.

[0035] The neck network comprises a first path and a second path. In the first path, the network layers, from shallowest to deepest, are a convolutional module, an upsampling module, a stitching module, a C3 module, another convolutional module, another upsampling module, and a stitching module. In the second path, the network layers, from shallowest to deepest, are a C3 module, a convolutional module, a stitching module, a C3 module, another convolutional module, another stitching module, and a C3 module. The outputs of the second C3 module, the third C3 module, and the SPPF module in the backbone network are respectively provided to the second stitching module, the first stitching module, and the first convolutional module in the first path. The outputs of the first convolutional module, the second convolutional module, and the second stitching module in the first path are respectively provided to the second stitching module, the first stitching module, and the first C3 module in the second path. At least one intermediate feature includes the outputs of the three C3 modules in the second path, and at least one intermediate feature also serves as input to the prediction head.

[0036] This ingenious path aggregation network structure enables full interaction of features at different levels, generating a feature pyramid that is robust to changes in the target scale.

[0037] Building upon this, at least one intermediate feature serving as the state input is further defined as the output of the three C3 modules in the second path. The outputs of these three modules also serve as input to the prediction head. The technical advantage of this design is that these three feature maps represent the network's final understanding of the target at three different scales. Using them as state input to the reinforcement learning agent is equivalent to allowing the agent to directly observe the features that the detection head will use for prediction, thereby enabling it to make the most direct and relevant parameter adjustment decisions.

[0038] Furthermore, to clarify the specific implementation of module C3, such as Figure 3 As shown, the C3 module adopts a CSP architecture. The input feature map is first divided into two branches by two parallel convolutional modules. One branch performs deep feature extraction through n Bottleneck modules, while the other branch directly retains some of the original features. Finally, the outputs of the two branches are concatenated along the channel dimension by a concatenation module, and then integrated by a convolutional module. This design solves the gradient vanishing problem in deep networks by splitting the gradient flow, while reducing computational bottlenecks and improving learning efficiency and inference speed.

[0039] Furthermore, to clarify the specific implementation of the SPPF module, refer to... Figure 4 The input feature map first passes through a convolutional module and then through three max pooling branches, which include 1, 2 and 3 max pooling layers respectively. The outputs of the three max pooling branches and the output of the convolutional module are then fed into the concatenation module, and finally output through the convolutional module, realizing the rapid fusion of multi-scale features.

[0040] In summary, deep neural networks have the following characteristics: The backbone network extracts feature information from the input image and passes it to the neck network for feature fusion. This network mainly consists of three modules: Conv (convolutional), C3, and SPPF. For example, the Conv module typically comprises convolutional layers, batch normalization layers, and activation functions. It first performs a convolution operation on the input feature map, then the batch normalization layer normalizes the feature map and channel feature information extracted by the convolutional layers, and finally, the activation function performs a non-linear transformation on the output features, thereby enhancing the network model's ability to express the target features of the image.

[0041] The C3 module adopts a CSP architecture, which specifically divides the input feature map of the convolutional layer into two branches for processing. The first branch consists of n stacked Bottlenecks and one Conv, while the second branch consists of only one Conv. Finally, the feature maps from the two outputs are concatenated to achieve feature information integration. For example, for each Bottleneck in the C3 module, it borrows from the ResNet residual structure, first using a 1×1 convolution to halve the number of channels for dimensionality reduction, then using a 3×3 convolution to double the number of channels for dimensionality increase, and finally using a shortcut for residual connection to add the feature maps from the two outputs, achieving feature fusion. The specific structure is as follows: Figure 2 As shown.

[0042] The SPPF module is an improved version of the SPP module. In older versions of YOLOv5, SPP used parallel 5×5, 9×9, and 13×13 convolutional kernels to perform max pooling on feature maps, thereby obtaining feature maps with different receptive fields. These maps were then concatenated using a concat operation to achieve the fusion of global and local features, improving the network model's feature representation capability. The SPPF module replaces the single large convolutional kernel in the original parallel structure of SPP with multiple serially cascaded small-sized convolutional kernels, achieving the same multi-scale feature fusion function while improving speed. The specific structure is as follows: Figure 3 As shown.

[0043] The neck network fuses shallow graphical features and deep semantic features output from the backbone network. Its structure employs a path aggregation network (PANet) improved from Feature Pyramid Networks (FPN). PANet relies on two paths, top-down and bottom-up, for feature fusion. This structure helps to obtain feature maps with rich details and sufficient semantics, improving the accuracy of target localization and classification.

[0044] The Head network uses feature maps of different scales output by the Neck network to achieve object localization and classification. First, it generates several candidate boxes on the image, then uses non-maximum suppression to filter out candidate boxes that do not meet a threshold. Finally, it outputs the type, confidence score, and bounding box location of the detected object, and annotates them on the original image. During the network model training phase, the object detection results from the Head network, along with the ground truth bounding box localization, classification, and confidence scores, are used to calculate the loss function.

[0045] For example, the loss function of a deep neural network is: ; in, , and These are classification loss, localization loss, and confidence loss, respectively. These correspond to the weighting coefficients for the three types of losses, respectively.

[0046] Both classification loss and confidence loss are calculated using the binary cross-entropy function, while localization loss is calculated using CIOU, an improved method based on IOU and DIOU. IOU measures the overlap between predicted and ground truth bounding boxes during object detection.

[0047] CIOU, based on DIOU, further considers the aspect ratio of the bounding box and incorporates it into the IOU penalty term, resulting in the CIOU loss. The calculation formula is as follows: ; ; ; in, These are the weighting coefficients; This parameter measures the consistency of the aspect ratio between the predicted bounding box and the ground truth bounding box. and These represent the width and height of the actual bounding box, respectively. w and h These represent the width and height of the prediction box, respectively.

[0048] In the above technical solution, the reinforcement learning agent includes a state representation module and a policy network; the state representation module transforms at least one intermediate feature into a state input for the policy network, and the policy network responds to the state input to output an action policy.

[0049] The deep neural network and the reinforcement learning agent are jointly trained. During the joint training process, the policy network calculates action rewards based on the target metric to update its own policy parameters. The target metric includes the prediction loss of the deep neural network.

[0050] The principle is to directly use the performance of the deep neural network (represented by prediction loss) as feedback for reinforcement learning. When the agent's actions reduce the loss of the detection network, a positive reward is given; otherwise, a negative reward is given. In this way, the update direction of the policy network is completely aligned with the optimization objective of the detection network, achieving end-to-end co-evolution of the two modules.

[0051] Specifically, the state representation module is used to define the feature statistics of the image feature map (such as color, texture, etc.) as input to the policy network. The policy network takes the feature statistics of the image feature map (such as color, texture, etc.) as input and adaptively adjusts the region candidate parameters (including at least one running parameter) for different scenarios (such as dense small targets, large-scale targets), and optimizes the feature extraction and selection of the deep neural network, thereby achieving the purpose of dynamically adjusting parameters and optimizing the feature extraction network in real time.

[0052] Preferably, the target metrics also include the average intersection-union ratio (IUU) of the predicted bounding boxes and the ground truth bounding boxes of the deep neural network, as well as the latency of forward inference in the current frame.

[0053] To achieve a balance between detection accuracy and real-time performance, the target metrics, in addition to the prediction loss, further include the average intersection-over-union (IoU) between the predicted and ground truth bounding boxes of the deep neural network, as well as the latency of forward inference in the current frame. IoU is a commonly used metric for measuring localization accuracy; directly incorporating it into the reward function can more directly guide the agent to learn strategies to improve localization accuracy. Introducing latency as a penalty term effectively constrains the agent's behavior, preventing it from choosing parameter configurations that lead to a dramatic increase in computation in pursuit of minor accuracy improvements. Through this multi-objective reward function design, this embodiment can find an ideal balance between accuracy and speed according to actual application requirements.

[0054] Specifically, prediction loss includes classification loss, localization loss, and confidence loss. These three losses respectively measure the performance of the detection model on the three core tasks of object detection: "what" (classification), "where" (localization), and "how certain" (confidence). Using them together as the basis for reward calculation ensures that the optimization behavior of the reinforcement learning agent is comprehensive, improving the performance of the detection model in all aspects simultaneously, rather than unilaterally pursuing one metric at the expense of others.

[0055] Specifically, at least one operating parameter includes an adaptive attention weight parameter for feature fusion at different scales and a non-maximum suppression (NMS) filtering threshold. The adaptive attention weight parameter operates on the feature fusion stage of the neck network, allowing the agent to dynamically enhance or suppress feature maps from different scales, thus enabling the network to focus more on the features most important to the current scene. The NMS filtering threshold is used in the post-processing stage of the prediction head. Dynamically adjusting this threshold can effectively address different situations such as dense or sparse targets, improving the problem of missed or false detections. By giving the agent control over these two key parameters, this embodiment achieves dynamic and intelligent control of the two core links in the detection process: feature processing and result selection.

[0056] The following example illustrates reinforcement learning agents in detail.

[0057] First, there's the state representation. The input to the state representation module in the reinforcement learning agent comes from the output features of the C3 module in the deep neural network. This includes the current frame feature map, historical action sequences, and environmental complexity metrics (such as illumination intensity and motion blur). The state representation module transforms the feature vectors in the deep neural network into input vectors in the reinforcement learning agent, i.e., the state inputs. This is achieved through matrix transformations, converting the output feature vectors in the deep neural network into the state input vectors in the reinforcement learning agent. . A , B , C This refers to the feature statistics corresponding to the feature map in a deep neural network, such as brightness, color, texture, etc.

[0058] Next is policy execution. The execution policy adaptively outputs action instructions to "adjust region candidate parameters" based on the complexity and multi-scale features of the current frame image, thereby dynamically optimizing the feature extraction and candidate box generation process of the deep neural network. Specifically, it includes four core steps: action definition, reward mechanism, state evaluation, and parameter update.

[0059] (1) Action space and strategy output.

[0060] The policy network obtains the current state vector output by the "state representation module".S t Then, action vectors are generated and executed. A t In this example framework, actions A t Directly applied to the "Neck" feature fusion network, it is defined as: . This represents the adaptive attention weight parameters applied to the fusion of features at different scales in the Neck network (such as before the Concat operation); This indicates that the filtering threshold of nonmaximum suppression (NMS) is dynamically adjusted to adapt to different scenarios such as dense small targets or large-scale targets.

[0061] Policy networks (Actors) use parameterized probability distribution functions. ( (For policy network parameters) to map states to action selection probabilities: .

[0062] (2) Dynamic adaptive reward mechanism.

[0063] This example breaks away from the traditional single-reward setting and constructs a multi-objective dynamic reward function based on detection accuracy, real-time performance, and model loss. ; in, This is the average intersection-union ratio (IU) between the predicted bounding boxes and the ground truth bounding boxes output by the deep neural network in the current step. , and These are the localization loss, classification loss, and confidence loss calculated by the deep neural network, respectively. This is the delay time for forward inference in the current frame, used to constrain the computational resource consumption and real-time performance of the model; , and To dynamically adjust the weighting factors and maximize the reward This guides the evolution of policy networks towards higher precision and lower latency.

[0064] (3) Value function and state assessment.

[0065] The state value function is denoted as ( (These are the value network parameters), used to evaluate the input vector of an agent in a given state. S At that time, it represents the expected value of long-term cumulative rewards that can be obtained. The calculation formula is: .

[0066] Based on the action value function (Q function), further evaluation can be performed in the state. st Execute specific actions a t Expected returns afterward: .

[0067] (4) RL agent parameter update mechanism.

[0068] This example uses a policy gradient algorithm based on the advantage function to perform real-time updates of the policy network parameters and value evaluation parameters. First, the time difference error is calculated as the advantage function. Measure the current action a t The extent to which it outperforms the average strategy. .

[0069] Finally, after each round of detection iteration, the deep neural network calculates the advantage function. The policy network parameters are updated simultaneously through backpropagation. .

[0070] This forms a closed-loop control, enabling the policy network to dynamically learn the optimal region candidate adjustment parameters and feature extraction optimization strategies through continuous interaction with the target detection environment.

[0071] In summary, this embodiment has the following technical effects: 1. By introducing a reinforcement learning agent module coupled to the deep neural network object detection module, and utilizing the intermediate features of the deep neural network as state input, the reinforcement learning agent can perceive the internal state of the detection process and generate action strategies to dynamically adjust the operating parameters of the deep neural network. This closed-loop dynamic optimization mechanism enables the object detection model to adapt to changes in the input image in real time, thereby improving the accuracy and robustness of detection in complex and variable scenarios (such as changes in object scale, dense distribution, etc.).

[0072] 2. A comprehensive reward mechanism was constructed by using prediction loss, the average intersection-union ratio (IU) of predicted and ground truth boxes, and the forward inference latency as target metrics for the reinforcement learning agent. This guides the agent to pursue not only high detection accuracy but also computational efficiency and real-time performance when learning optimization strategies, achieving an effective balance between detection performance and resource consumption, making it more suitable for applications with high real-time requirements.

[0073] 3. By specifically defining the action strategy as adjusting the adaptive attention weight parameters for feature fusion at different scales and the filtering threshold for non-maximum suppression, precise control over key aspects such as feature fusion and candidate box post-processing is achieved. This targeted adjustment is more efficient than adjusting the entire network and can quickly optimize the model's performance in specific scenarios.

[0074] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0075] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0076] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0077] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0078] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0079] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0080] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0081] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A deep neural network-based target detection method based on reinforcement learning, characterized in that, include: Target detection is performed on the input image using a deep neural network; as well as The target detection process is dynamically optimized by a reinforcement learning agent coupled to the deep neural network. The dynamic optimization includes: The reinforcement learning agent takes at least one intermediate feature of the deep neural network as a state input; The reinforcement learning agent responds to the state input to output an action policy, the action policy being used to dynamically adjust at least one operating parameter of the deep neural network; The deep neural network outputs the detection results after applying the action strategy.

2. The deep neural network target detection method based on reinforcement learning according to claim 1, characterized in that, The reinforcement learning agent includes a state representation module and a policy network; The state representation module transforms the at least one intermediate feature into a state input for the policy network, and the policy network responds to the state input to output the action policy. The deep neural network and the reinforcement learning agent are jointly trained. During the joint training process: The policy network calculates action rewards based on a target metric to update its own policy parameters; wherein the target metric includes the prediction loss of the deep neural network.

3. The deep neural network target detection method based on reinforcement learning according to claim 2, characterized in that, The prediction loss includes classification loss, localization loss, and confidence loss.

4. The deep neural network target detection method based on reinforcement learning according to claim 2, characterized in that, The target metrics also include the average intersection-over-union ratio (IoU) of the predicted bounding boxes and the ground truth bounding boxes of the deep neural network, as well as the delay time of forward inference in the current frame.

5. The deep neural network target detection method based on reinforcement learning according to claim 2, characterized in that, The at least one operating parameter includes an adaptive attention weight parameter for feature fusion at different scales and a filtering threshold for nonmaximum suppression.

6. The deep neural network target detection method based on reinforcement learning according to claim 1, characterized in that, The deep neural network comprises a backbone network, a neck network, and a prediction head stacked sequentially. The at least one intermediate feature is a feature extracted from the neck network.

7. The deep neural network target detection method based on reinforcement learning according to claim 6, characterized in that, The backbone network consists of layers from shallowest to deepest as follows: convolution module, convolution module, C3 module, convolution module, C3 module, convolution module, C3 module, convolution module, C3 module, and SPPF module. The neck network includes a first path and a second path. In the first path, the network layers, from shallowest to deepest, are a convolutional module, an upsampling module, a stitching module, a C3 module, another convolutional module, another upsampling module, and a stitching module. In the second path, the network layers, from shallowest to deepest, are a C3 module, a convolutional module, a stitching module, a C3 module, another convolutional module, another stitching module, and a C3 module. The outputs of the second C3 module, the third C3 module, and the SPPF module in the backbone network are respectively provided to the second stitching module, the first stitching module, and the first convolutional module in the first path. The outputs of the first convolutional module, the second convolutional module, and the second stitching module in the first path are respectively provided to the second stitching module, the first stitching module, and the first C3 module in the second path. The at least one intermediate feature includes the outputs of the three C3 modules in the second path, and the at least one intermediate feature also serves as the input to the prediction head.

8. The deep neural network target detection method based on reinforcement learning according to claim 7, characterized in that, The C3 module adopts the CSP architecture.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the deep neural network object detection method based on reinforcement learning as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the deep neural network object detection method based on reinforcement learning as described in any one of claims 1-7.

Images