Adaptive deep reinforcement learning method for binocular vision modeling in complex environment
By constructing an adaptive deep reinforcement learning network and combining binocular vision and self-attention mechanisms, a dynamic reinforcement learning framework for complex environments is designed, which solves the problem of insufficient generalization and robustness of existing models in complex scenarios and achieves high-accuracy object detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NORTHWESTERN POLYTECHNICAL UNIV
- Filing Date
- 2023-12-20
- Publication Date
- 2026-05-29
AI Technical Summary
Existing models have low generalization ability in complex scenarios, cannot meet the requirements of adaptation, and rely on training for specific scenarios, resulting in insufficient robustness.
An adaptive deep reinforcement learning network is constructed, which calculates the depth information of objects through binocular vision. Combining a self-attention mechanism and a group normalization module, a positive feedback reward mechanism is designed, a dynamic reinforcement learning network framework is constructed, common scene prior knowledge is extracted, the human brain's visual attention system is simulated, and feature channels and space are weighted.
It improves the model's generalization ability and robustness in complex scenarios, effectively distinguishes similar non-standard targets, improves detection accuracy, reduces dependence on specific scenarios, and enhances the model's adaptability.
Smart Images
Figure CN117853788B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of dynamic reinforcement learning technology, and more particularly to an adaptive deep reinforcement learning method for binocular vision modeling in complex environments. Background Technology
[0002] With the development of deep learning technology, neural networks have been used in a wide variety of scenarios to achieve specific tasks, such as classification, segmentation, and detection. Among related technologies,
[0003] While the proposed object detection model can effectively mine prior knowledge and guide more accurate detection for specific scenarios, it lacks the necessary generalization ability when facing complex scenarios. Furthermore, although the proposed semantic segmentation method can learn deep common features from different domains to guide the model to adapt to different data domains, it still requires labeled data from different domains. Existing technologies have only explored different aspects of mining scene information; however, existing models are usually dependent on specific scenarios, resulting in low generalization ability and low robustness when transferred to other scenarios, failing to meet the adaptation requirements of complex scenarios. Summary of the Invention
[0004] To avoid the shortcomings of existing technologies, this application provides an adaptive deep reinforcement learning method for binocular vision modeling in complex environments. This method addresses the problem that existing models are typically dependent on specific scenarios, resulting in low generalization ability of the trained models and low robustness when transferred to other scenarios, thus failing to meet the adaptive requirements in complex scenarios.
[0005] According to embodiments of this disclosure, an adaptive deep reinforcement learning method for binocular vision modeling in complex environments is provided, the method comprising:
[0006] An adaptive deep reinforcement learning network is constructed, and the state of the deep reinforcement learning network and the initial position of the agent are initialized; wherein, the adaptive deep reinforcement learning network consists of 7 convolutional layers and 3 fully connected layers connected in series;
[0007] The depth information of the objects in the original image is calculated based on the left and right views of the original image, and the depth information and the original image are input into the feature extractor in the deep reinforcement learning network to extract the first feature and the second feature, respectively.
[0008] The first feature and the second feature are concatenated along the channel dimension to obtain a fused feature that incorporates the depth information;
[0009] The fused features are extended into multi-layer cascaded channel number features using VGGNet-16 with an embedded group normalization module. The channels of the channel number features are reweighted through a self-attention mechanism to obtain self-attention features. The size similarity of the detected objects and the reward corresponding to the detected objects are calculated to update the agent state.
[0010] A feature pyramid structure is constructed from the self-attention features, and the final bird's-eye view detection result is obtained through the detection head. The position of the agent is iteratively updated until the termination condition is met.
[0011] Furthermore, the step of calculating the depth information of objects in the original image based on the left and right views of the original image, and inputting the depth information and the original image into the feature extractor in the deep reinforcement learning network to extract the first feature and the second feature, respectively, includes:
[0012] The left and right focal lengths of the binocular cameras used to acquire the left and right views of the original image are both f, and the distance between the binocular cameras is d. c The parallax is d x Then the formula for calculating the depth of spatial point P from the camera is:
[0013] Depth = f * d c / d x (1)
[0014] In the formula, Depth represents the depth information of spatial point P from the camera;
[0015] The calculated depth information and the original image (Image) are simultaneously input into the feature extractor to extract the first feature F. d and the second feature F i ;in c, h, w represent the first feature F respectively. d and the second feature F i The number of channels, height, and width.
[0016] Furthermore, the step of concatenating the first feature and the second feature along the channel dimension to obtain a fused feature incorporating the depth information includes:
[0017] The final output fusion features can be calculated using the following formula:
[0018] F = concat(F) c ,F i (2)
[0019] Here, the function concat represents the concatenation operation, and F... d and F i The features are spliced together along the channel dimension to obtain the fused feature F that incorporates depth information.
[0020] Furthermore, the step of extending the fused features into multi-layered cascaded channel number features using VGGNet-16 with an embedded group normalization module, and reweighting the channels of the channel number features through a self-attention mechanism to obtain self-attention features includes:
[0021] The input to the multi-layer feature is F out The query, index weight key, and query value for self-attention are calculated using the following formula:
[0022] f q ,f k ,f v =f conv (F out ,n) (3)
[0023] Wherein, function f conv This represents a convolution operation, where n represents the number of output channels, and f... q f k and f v They represent F respectively out The obtained query, index weight key, and query value;
[0024] The self-attention feature is then calculated using the following formula:
[0025]
[0026] Where, d k f k The number of channels.
[0027] Furthermore, the termination condition for the agent's state is the reward obtained, i.e., the Q value; the Q value is calculated using the following formula:
[0028] Q = reward + γ·arg max(q) val (5)
[0029] Wherein, variable q val This represents the output of the neural network, which has a length of three. Each element represents the weight value of the corresponding action classification. The larger the weight value, the higher the probability of being classified as that action. γ represents the greedy coefficient for reward calculation, and reward represents the action reward corresponding to different actions.
[0030] The loss of the entire network is L qThe calculation formula is as follows:
[0031] L q =(q pred -q val ) 2 (6)
[0032] Wherein, variable q pred and variable q val Let L represent the predicted value and the true value, respectively, of the samples randomly sampled from the memory pool as input. The loss L of the network described here... q It is used to promote the overall convergence of the network.
[0033] Furthermore, the reward mechanism is responsible for scoring the actions of the agent and outputting the action reward; wherein, each action reward of the agent is based on prior knowledge of the scene, that is, the degree of similarity between similar objects;
[0034] Cosine similarity is used as an indicator to judge the degree of similarity between similar objects. The calculation formula is as follows:
[0035]
[0036] Where S represents the final similarity, and g and d represent the size vector information of the objects contained in the original input image and the same number of standardized unit vectors, respectively, i.e., g = {1,1,1,...,1}, d = {l1,l2,l3,...,l n};
[0037] Add a new criterion to the action reward, namely the number of detected objects, and calculate it using the following formula:
[0038]
[0039] Among them, R i This represents the proportion of detected objects corresponding to the agent in a single state, with variables n and N representing the number of detected objects and the total number of objects, respectively.
[0040] Furthermore, the average intersection-union ratio (mIOU) is used to evaluate the detection performance of the proposed network structure, and the mean absolute error (MAE) is used to evaluate the difference between the agent's final position and the ideal position. The calculation formulas for the average intersection-union ratio (mIOU) and the mean absolute error (MAE) are as follows:
[0041]
[0042] MAE = |p gt -p pred | (10)
[0043] Wherein, variable p ii Represents the first region consistent with the truth value, variable p ij and variable p ji These represent the second and third regions, respectively, that are inconsistent with the truth value; furthermore, p gt and p pred These represent the final predicted position and the true position of the agent, respectively.
[0044] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:
[0045] In the embodiments of this disclosure, an adaptive deep reinforcement learning method for binocular vision modeling in complex environments is employed. Based on binocular vision, depth information of objects in images is calculated using images from different left and right views, and this information is fed back to the neural network, enabling the neural network to accurately extract target features. To address the model adaptation problem in complex scenes, a dynamic reinforcement learning network framework based on common scene priors is constructed. This framework provides unified modeling for different environments and proposes a lightweight and effective agent action classification network. This network effectively classifies actions correctly for different environments and states, significantly mitigating the decline in model generalization caused by scene changes. Furthermore, to address the issue of diverse prior knowledge in complex scenes, an adaptive positive feedback reward mechanism is designed. This mechanism fully exploits common prior knowledge in complex scenes and establishes a size similarity evaluation system for similar individuals in different scenes, incentivizing the agent to continuously make correct action choices. To address the issue of heterogeneity among similar individuals in complex environments, during the feature extraction stage, a model simulating the human brain's visual attention system is used to construct visual attention features at different levels. Adaptive weighting of the feature channels and spaces effectively extracts representative features among similar individuals in complex environments, enabling effective differentiation of heterogeneous targets and improving detection accuracy. Attached Figure Description
[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.
[0047] Figure 1 The diagram illustrates the steps of an adaptive deep reinforcement learning method for binocular vision modeling in a complex environment according to an exemplary embodiment of the present disclosure.
[0048] Figure 2 A graph showing the comparison of detection performance curves during the training and testing phases of the method of this application in an exemplary embodiment of this disclosure is provided.
[0049] Figure 3 The diagram shows a comparison of detection results between the method of this application and other methods under complex conditions in an exemplary embodiment of this disclosure;
[0050] Figure 4 A table comparing experimental results of the method of this application with other methods in exemplary embodiments of this disclosure is shown;
[0051] Figure 5 A flowchart illustrating an adaptive deep reinforcement learning method for binocular vision modeling in complex environments is shown in an exemplary embodiment of this disclosure. Detailed Implementation
[0052] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0053] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.
[0054] This example implementation provides an adaptive deep reinforcement learning method for binocular vision modeling in complex environments. (Reference) Figure 1 As shown, the adaptive deep reinforcement learning method for binocular vision modeling in this complex environment may include steps S101 to S105.
[0055] Step S101: Construct an adaptive deep reinforcement learning network and initialize the state of the deep reinforcement learning network and the initial position of the agent;
[0056] Step S102: Calculate the depth information of the objects in the original image based on the left and right views of the original image, and input the depth information and the original image into the feature extractor in the deep reinforcement learning network to extract the first feature and the second feature respectively;
[0057] Step S103: Concatenate the first feature and the second feature along the channel dimension to obtain a fused feature that incorporates the depth information;
[0058] Step S104: Using VGGNet-16 with an embedded group normalization module, the fused features are extended into multi-layer cascaded channel number features. The channels of the channel number features are reweighted through a self-attention mechanism to obtain self-attention features. The size similarity of the detected objects and the reward corresponding to the detected objects are calculated, and the agent state is updated.
[0059] Step S105: Construct a feature pyramid structure from the self-attention features, and obtain the final bird's-eye view detection result through the detection head. Iteratively update the position of the agent until the termination condition is met.
[0060] This invention employs an adaptive deep reinforcement learning method for binocular vision modeling in complex environments. By combining this method with a theoretical model of brain neural perception, common prior knowledge across various complex environments is extracted, and a positive feedback incentive mechanism is designed to address the challenges of insufficient model generalization and versatility caused by environmental changes. Based on binocular vision, this method calculates the depth information of objects in images from different left and right views and aligns this information with the neural network, enabling the network to accurately extract target features. To address the problem of model adaptation in complex scenes, this invention constructs a dynamic reinforcement learning network framework based on common scene priors. This framework provides unified modeling for different environments and proposes a lightweight and effective agent action classification network. This network effectively classifies actions correctly for different environments and states, significantly mitigating the decline in model generalization caused by scene changes. Furthermore, to address the issue of diverse prior knowledge in complex scenes, this invention designs an adaptive positive feedback reward mechanism. This mechanism fully exploits common prior knowledge in complex scenes and establishes a size similarity evaluation system for similar individuals in different scenes, incentivizing the agent to continuously make correct action choices. To address the issue of different shapes (color, form) among similar individuals in complex environments, this invention models and simulates the human brain's visual attention system during the feature extraction stage. It constructs visual attention features at different levels and adaptively assigns weights to the channels and spaces of the features. This effectively extracts representative features among similar individuals in complex environments, enabling the effective differentiation of similar different-shaped targets and improving detection accuracy.
[0061] Below, we will refer to Figures 1 to 5 The steps of the adaptive deep reinforcement learning method for binocular vision modeling in complex environments described in this example implementation will be explained in more detail.
[0062] In steps S101 and S102, the reinforcement learning method in this application can be viewed as a game of finding the optimal position. The agent's actions are classified into three categories: up, down, and terminate. Based on this, the neural network (DRL) of the Dynamic Reinforcement Learning network framework is essentially an action classification network. The specific network structure consists of 7 convolutional layers and 3 fully connected layers connected in series, and its input is the self-attention feature F obtained in step two. sa Its fully connected layer has an output length of 3, which respectively represent the three action states mentioned above.
[0063] Assume the left and right focal lengths of the stereo cameras are equal, both being f, and the distance between the stereo cameras is d. c The parallax is d x The depth of spatial point P from the camera can be calculated using the following formula:
[0064] Depth = f * d c / d x (1)
[0065] In the formula, Depth represents the depth information of spatial point P from the camera; the calculated depth information and the original image Image are simultaneously input into the feature extractor to extract the first feature F. d and the second feature F i ;in c, h, w represent the first feature F respectively. d and the second feature F i The number of channels, height, and width are given. The final output fused features can then be calculated using the following formula:
[0066] F = concat(F) c ,F i (2)
[0067] Here, the function concat represents the concatenation operation, and F... d and F i The features are spliced together along the channel dimension to obtain the fused feature F that incorporates depth information.
[0068] In step S103, the extracted feature F is expanded into a multi-layered cascaded channel number feature using VGGNet-16 with an embedded Batch Normalization module, and the channels of the feature are reweighted using a self-attention mechanism. Assume the input to the multi-layer feature is F. out Then, the query, index weight key, and query value for self-attention can be calculated using the following formula:
[0069] fq ,f k ,f v =f conv (F out ,n) (3)
[0070] Wherein, function f conv This represents a convolution operation, where n represents the number of output channels, and f... q f k f v They represent F respectively out The obtained query, index weight key, and query value are then used to calculate the self-attention feature using the following formula:
[0071]
[0072] Where d k f k The number of channels. This step yields the self-attention feature F. sa .
[0073] In steps S104 and S105, during the initial training of the neural network DRL, each action judgment by the network provides a training example to the memory pool. Each new action of the agent generates a new state and simultaneously obtains the corresponding reward. Subsequently, the new state is input into the network to continue providing new actions for the agent. The termination condition for the entire agent state is to obtain as many rewards as possible, i.e., the Q-value. The Q-value can be calculated using the following formula:
[0074] Q = reward + γ·arg max(q) val (5)
[0075] Wherein, variable q val This represents the output of the DRL neural network, with a length of three elements. Each element represents the weight value for the corresponding action classification; the larger the weight value, the higher the probability of being classified as that action. γ represents the greedy coefficient for reward calculation. reward represents the reward corresponding to different actions. The loss of the entire network is L. q The calculation formula is as follows:
[0076] L q =(q pred -q val ) 2 (6)
[0077] Wherein, variable q pred and variable q val These represent the predicted and true values, respectively, of the samples randomly sampled from the memory pool as input. This step calculates the overall loss L of the network. qIt is used to promote the overall convergence of the network.
[0078] In this application, the reward mechanism is responsible for scoring the agent's actions and outputting a reward. The reward for each action is based on prior knowledge of the scene, specifically the degree of similarity between similar objects. This application uses cosine similarity as the metric for evaluating the similarity between similar objects, and its calculation formula is as follows:
[0079]
[0080] Where S represents the final similarity, and g and d represent the size vector information of the objects contained in the input image and the same number of standardized unit vectors, respectively, i.e., g = {1,1,1,...,1}, d = {l1,l2,l3,...,l n Since the number of objects detected by the network varies depending on the agent's location, and ideally the network should detect as many objects as possible, this application adds a criterion for the reward design: the number of detected objects. The calculation formula is as follows:
[0081]
[0082] Among them, R obtained in this step i This represents the proportion of detected objects corresponding to the agent in a single state, with variables n and N representing the number of detected objects and the total number of objects, respectively.
[0083] This application uses a detection head to regress the target bounding box position from a bird's-eye view and the corresponding heatmap. The heatmap and bounding box regression output network correspond to single-channel and dual-channel convolutional layers, respectively, each containing single-layer convolutions at three scales. The final detection result is obtained by fusing three layers of features. The network's generalization and robustness are evaluated through visualization and quantitative analysis. This application selects the mean intersection-over-union ratio (mIOU) to evaluate the detection performance of the proposed network structure and the mean absolute error (MAE) to evaluate the difference between the agent's final position and the ideal position. The formulas for calculating mIOU and MAE are as follows:
[0084]
[0085] MAE = |p gt -p pred | (10)
[0086] Wherein, variable p ii The variable p represents the region consistent with the truth value. ij and variable p jiThese represent regions that do not conform to the true value. Furthermore, p gt and p pred These represent the final predicted position and the true position of the agent, respectively. This step yields the network's performance evaluation results.
[0087] The effectiveness of this application can be further illustrated by the following simulation experiments.
[0088] 1. Simulation conditions.
[0089] The method described in this application is a simulation performed using the deep learning framework PyTorch on a CPU consisting of an Intel(R) Xeon(R) CPU E5-2680 v4 2.40GHz, 128GB of memory, and a Linux operating system.
[0090] The data used in the simulation consisted of publicly available datasets and datasets collected by ourselves.
[0091] 2. Simulation content.
[0092] first, Figure 2 This is a comparison chart of the detection performance curves during the training phase of the method in this application. Figure 3 This is a comparison of detection performance curves during the testing phase. Furthermore, the ablation comparison of different modules with respect to the final results is also visualized using curves.
[0093] Figure 4 This is a comparison chart of the detection results of the method in this application with other methods under complex conditions. (Through...) Figure 4 It can be seen that the method proposed in this application can effectively detect similar irregular targets in complex environments. This effect is a qualitative analysis and can be obtained by comparing with the visualization results of other algorithms.
[0094] Table 1 compares the experimental results of the method in this application with those of other methods. The detection accuracy of the algorithm in this application reaches over 84.3%.
[0095] Table 1 Comparison of Detection Results
[0096]
[0097] like Figure 5As shown, this method constructs an adaptive deep reinforcement learning network framework and designs a reward mechanism based on common prior knowledge in complex scenarios. This greatly improves the robustness and generalization ability of the network. Furthermore, the attention mechanism based on human brain visual mechanisms effectively filters channels with smaller weights in the feature map, allowing the network to focus its attention on the channels that truly influence the final classification action of the agent. Long-distance supervision reduces excessive reliance on data and improves the model's generalization ability in complex scenarios. The lightweight and concise reinforcement learning network structure reduces the overall number of parameters compared to traditional neural networks, increasing the model's computational speed. Extraction of common prior knowledge in complex scenarios provides positive feedback for network training, promoting network convergence and gradually reducing network loss. This application treats the network's prediction results as a game through reinforcement learning, constructing a complete and efficient deep reinforcement learning framework through iterative optimization of the agent and common prior knowledge in complex scenarios, thus eliminating dependence on data under cross-scenario conditions. Competitive results are achieved even without specific scene-labeled data, with a detection accuracy exceeding 84.3%.
[0098] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.
[0099] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.
[0100] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.
Claims
1. An adaptive deep reinforcement learning method for binocular vision modeling in complex environments, characterized in that, The method includes: An adaptive deep reinforcement learning network is constructed, and the state of the deep reinforcement learning network and the initial position of the agent are initialized; wherein, the adaptive deep reinforcement learning network consists of 7 convolutional layers and 3 fully connected layers connected in series; The depth information of the objects in the original image is calculated based on the left and right views of the original image, and the depth information and the original image are input into the feature extractor in the deep reinforcement learning network to extract the first feature and the second feature, respectively. The first feature and the second feature are concatenated along the channel dimension to obtain a fused feature that incorporates the depth information; The fused features are expanded into multi-layered cascaded channel number features using VGGNet-16 with an embedded group normalization module. The channels of these channel number features are then reweighted using a self-attention mechanism to obtain self-attention features. Finally, the size similarity of the detected objects and the corresponding action rewards are calculated. reward Update the state of the intelligent agent; A feature pyramid structure is constructed from the self-attention features, and the final bird's-eye view detection result is obtained through a detection head. The position of the agent is iteratively updated until a termination condition is met; wherein, The reward mechanism is responsible for scoring the actions of the agent and outputting the reward for those actions. reward ; wherein, the agent receives a reward for each action. reward All of these are based on prior knowledge of the scene, namely the degree of similarity between objects of the same kind; use cosine Similarity, as an indicator for judging the degree of similarity between similar objects, is calculated using the following formula: （7） Among them, variables S The variable represents the final similarity. g and variables d These represent the size vector information of the objects contained in the original input image and the normalized number of unit vectors, respectively. , ; Reward the action reward Add a new evaluation criterion: the number of objects detected. The calculation formula is as follows: （8） in, The variable represents the proportion of detected objects corresponding to the agent in a single state. n and N These represent the number of detected objects and the total number of objects, respectively.
2. The adaptive deep reinforcement learning method for binocular vision modeling in complex environments according to claim 1, characterized in that, The step of calculating the depth information of objects in the original image based on the left and right views of the original image, and inputting the depth information and the original image into the feature extractor in the deep reinforcement learning network to extract the first feature and the second feature, respectively, includes: The left and right focal lengths of the binocular camera used to acquire the left and right views of the original image are both . f The distance between the binocular cameras is Parallax is Then the formula for calculating the depth of spatial point P from the camera is: （1） In the formula, depth This represents the depth information of spatial point P from the camera; Input the calculated depth information and the original image. Image Simultaneously input into the feature extractor, the first feature is extracted respectively. and the second feature ;in , , c , h , w Representing the first feature respectively and the second feature The number of channels, height, and width.
3. The adaptive deep reinforcement learning method for binocular vision modeling in complex environments according to claim 2, characterized in that, The step of concatenating the first feature and the second feature along the channel dimension to obtain a fused feature incorporating the depth information includes: The final output fusion features can be calculated using the following formula: （2） Among them, the function This indicates a splicing operation, here. and By concatenating the data along the channel dimension, a fused feature combining depth information is obtained. .
4. The adaptive deep reinforcement learning method for binocular vision modeling in complex environments according to claim 3, characterized in that, The step of extending the fused features into multi-layered cascaded channel number features using VGGNet-16 with an embedded group normalization module, and reweighting the channels of the channel number features through a self-attention mechanism to obtain self-attention features includes: The input of multi-layer features is The query, index weight key, and query value for self-attention are calculated using the following formula: （3） Among them, the function This represents the convolution operation. n Indicates the output channel. , and They represent respectively by The obtained query, index weight key, and query value; The self-attention feature is then calculated using the following formula: （4） in, express The number of channels.
5. The adaptive deep reinforcement learning method for binocular vision modeling in complex environments according to claim 4, characterized in that, The termination condition for the agent's state is the reward obtained, i.e., the Q value; the Q value is calculated using the following formula: （5） Among them, variables This represents the output of the neural network, which is three elements long. Each element represents the weight value for the corresponding action classification. The larger the weight value, the higher the probability of being classified as that action. This represents the greed coefficient used in reward calculation. reward This indicates the action reward corresponding to different actions; The loss of the entire network is The calculation formula is as follows: （6） Among them, variables and variables Let $\mathbf$ represent the predicted value and the true value, respectively, of the samples randomly sampled from the memory pool as input. The loss of the network described here is... It is used to promote the overall convergence of the network.
6. The adaptive deep reinforcement learning method for binocular vision modeling in complex environments according to claim 5, characterized in that, Take the average crossover ratio mIOU To evaluate the detection performance and mean absolute error of the proposed network structure. MAE The difference between the agent's final position and its ideal position is evaluated; the average intersection-union ratio is... mIOU and the mean absolute error MAE The calculation formulas are as follows: （9） （10） Among them, variables The first region that corresponds to the truth value, the variable and variables These represent the second and third regions, respectively, that are inconsistent with the truth value; furthermore... and These represent the final predicted position and the true position of the agent, respectively.