Underwater sonar small target detection method based on markov decision process

Abstract

The application provides a small target detection method for underwater sonar based on a Markov decision process, and the method comprises the following steps: constructing a multi-scale target detection model based on deep learning, setting a dynamic fusion weight coefficient for a feature pyramid network, and initializing a weight coefficient for each scale in a loss function; defining a state space S, a state transition probability P and a reward value R of the target detection model; defining an action set A and a strategy pi of dynamic feature fusion; defining an action set A' and a strategy pi' of compensation scale training; starting training, calculating a state and a reward at each time; updating a weight value at a current time according to the obtained probability; calculating an overall loss to update model parameters through back propagation until the training is completed, and generating a target detection model; and the application achieves good detection accuracy in the small target detection task of underwater sonar.

Description

Technical Field

[0001] This invention belongs to the field of sonar image processing and relates to an underwater sonar small target detection method based on Markov decision process. Background Technology

[0002] Synthetic Aperture Sonar (SAS), as a novel underwater acoustic imaging device, boasts significant advantages in its high spatial resolution and mapping efficiency. SAS offers a much higher underwater target recognition rate than traditional sonar, reducing the number of deployments and the time required for detection. Furthermore, SAS can operate at lower frequencies, providing high efficiency for wide-area scanning and enabling the detection of buried objects that are difficult to detect with traditional side-scan sonar. Therefore, many civilian and military vessel systems are widely using SAS for underwater target detection. With the development of big data and artificial intelligence technologies, target detection algorithms have achieved remarkable results in visible light images. However, the imaging mechanism of SAS differs greatly from that of color cameras, and the detected target occupies a small area relative to the entire image, increasing the difficulty of target detection.

[0003] Traditional sonar image target detection methods primarily rely on manual feature extraction. Palomeras et al. proposed a template matching-based method, utilizing manually designed target template features for target localization and classification. Dura et al. proposed a foreground-shadow region matching method, which effectively utilizes the texture features of foreground targets and background regions in sonar images by designing region matching pairs, enabling effective detection of foreground targets in sonar images with low speckle noise. Other methods include edge feature detection, HSV color distribution histogram features, gradient histogram features, and local binary pattern features for sonar image feature extraction. However, traditional methods that use manually designed features as the target identification standard cannot utilize the deep semantic features of sonar images for recognition. Furthermore, traditional sonar image target detection methods have low robustness, are easily affected by speckle noise, have high requirements for image quality and target size, lack generalization ability, and are difficult to apply to underwater sonar small target detection tasks.

[0004] In recent years, deep learning-based target detection algorithms have developed rapidly, and the problem of small target detection in visible light images has been solved to some extent. Deep neural networks extract hierarchical features, where shallow spatial features, due to their smaller receptive fields, are more suitable for target localization; deep semantic features, due to their larger receptive fields and deeper semantic information, are more conducive to target classification. Each level of features is significant for target detection tasks. Therefore, Lin et al. designed the Feature Pyramid Network (FPN), which progressively fuses the semantic information of high-level features into low-level features, constructing a pyramid feature structure for multi-scale target detection. FPN has improved the detection accuracy of models for small-scale targets and has become a model paradigm for multi-scale target detection. However, directly applying multi-scale target detection models to the task of small target detection in underwater synthetic aperture sonar images is difficult to achieve good results. Therefore, there is an urgent need to propose a method to improve the accuracy of deep learning models in underwater sonar small target detection tasks. Summary of the Invention

[0005] In view of this, the present invention provides an underwater sonar small target detection method based on Markov decision process, which can directly extract relevant features of sonar images for autonomous target detection using deep neural networks. It has low requirements for target background contrast and image quality, which helps improve detection accuracy while saving manpower and resources. The method includes:

[0006] Step 1: Construct a deep learning-based object detection model, load the pre-trained model and sonar image dataset, and initialize the initial parameters for model training;

[0007] Step 2: Define the state space S, state transition probability P, and reward value R of the target detection model;

[0008] Step 3: Define the action set A and policy π for dynamic feature fusion. Policy π determines which action to perform in the current state to update the parameters of feature fusion based on probability.

[0009] Step 4: Define the action set A′ and policy π′ for compensated scale training. Policy π′ determines which action to perform in the current state based on probability to update the weights of each scale in the loss function.

[0010] Step 5: Begin model training. Process the training set images input for each iteration within each iteration interval σ; calculate the multi-scale loss value, as defined in Step 2 for each time step's state S. t and rewards Assignment;

[0011] Step 6: Calculate strategy π and strategy π′ according to the formulas in Steps 3 and 4 based on the reward value R, and select the corresponding action a based on the obtained probabilities. k and a′ k Update the feature fusion weights and multi-scale loss weights at the current moment;

[0012] Step 7: Calculate the overall loss, perform backpropagation to update the model parameters, and continue training until the training reaches the required number of iterations.

[0013] Specifically, step 1 includes: the deep learning-based object detection model is constructed using the ATSS model; wherein the Feature Pyramid Network (FPN) has 5 layers to generate pyramid features. The process is as follows:

[0014]

[0015] in The expression for 1×1 convolution at each feature level is shown below, where ↑2× represents upsampling by a factor of 2 for nearest neighbor interpolation. The expression for feature fusion after introducing a weighting factor is as follows:

[0016]

[0017] in The weight value during feature fusion is a real number that changes with time t. Represents the weight value vector:

[0018] Compensated scale training weights the multi-scale loss, with each scale having a weight of w. i，t ;w t Represents the weight value vector: [w 2，t w 3，t w 4，t w 5，t w 6，t The overall loss value after introducing weights is as follows:

[0019]

[0020] Specifically, step 2 includes: during detector training, as training progresses, the sum of the loss values ​​generated within a fixed iteration interval forms a time-ordered sequence, as shown below:

[0021]

[0022] Where n is the current iteration, T is the total number of iterations, and i represents the index of each scale level of the pyramid; to avoid outliers in the loss sequence, an iteration interval hyperparameter σ is introduced to average the loss values ​​within a fixed interval; furthermore, the loss values ​​at each scale are standardized using linear standardization, as shown below:

[0023]

[0024] State space S = {s1, s2, s3, ..., s} t′}, where t′ is the time when training ends. The state at each time step reflects the state of the normalized loss value of the object detection model during training, i.e. The detector's value for each state is unknown at the start of training; values ​​are assigned to each state as training progresses. And state s... t The next moment must be s. t+1 transition probability That is, the actions performed by the agent do not affect the state transition, but only change the value of the state;

[0025] The reinforcement learning agent executes actions based on the maximum reward value at each moment. Representing the intelligent agent in S t-1 At that time, action A was performed. t-1 Afterwards, in S t The reward value provided in real time is calculated using the following formula:

[0026]

[0027] The greater the percentage decrease in the overall loss between two adjacent time points, the greater the reward; conversely, the reward is 0 if the overall loss increases.

[0028] Specifically, step 3 includes: a dynamic feature fusion action set A = {a1, a2, a3}, where each action element a k These all represent a way to update the feature fusion weights; where a1 represents replacing the weights used in the previous time step with the newly generated weights; a2 represents continuing to use the weight coefficients from the previous time step; a3 represents updating the current weights with the historically optimal weights; γ(t) represents the decay coefficient, and the formula for generating the new weights is as follows:

[0029]

[0030]

[0031] brelu(x) = min(max(x, β) min ), β max )

[0032] The Brelu function limits the range of values ​​to prevent abnormal weight values, and is simply set to [0.1, 10] here; the iteration interval σ is 50 to make the multi-scale loss values ​​more stable and differentiated; γ(t) is the decay coefficient, with a range of [0, 1].

[0033] During the update of feature fusion weights, a1: execute Assignment operation; a2: Execute Assignment operation; a3: Execute Assignment operation; strategy π is for each action a k Assignment probability p k The reward value is updated at each time step t.

[0034] In particular, to prevent a certain group Throughout the entire training cycle, the forgetting probability is introduced to forget existing data. That is, adopt renew The process is as follows:

[0035]

[0036] Where p d The value is 10%, and it is only applied when the condition is met. The strategy for determining whether an update is triggered is timed, where π represents each action a. k Assignment probability p k At each time t, the reward value is updated, and they satisfy p1+p2+p3=1; at each time t, the probability value is updated based on the reward value, and the calculation formula is as follows: p1=0.5γ(t);

[0037] Specifically, step 4 includes: calculating the proportion of each scale's loss in the total loss according to the following formula:

[0038]

[0039] The two scales with the smallest percentage of loss are weighted upwards, with larger weights assigned to scales with smaller losses, while scales with larger losses remain unchanged at 1. The calculation method is shown below:

[0040]

[0041] The set of actions for compensated scale training is A′={a′1,a′2,a′3,a′4}; each action element a′ kBoth represent a method for updating multi-scale loss, where the action set A′={a′1, a′2, a′3, a′4} is used for compensated scale training. Each action element a′ k Both represent a way to update the multi-scale loss, specifically defined as follows: a′1: execute w i，t :=w′ i，t Assignment operation; a′2: execute w t :=w t-1 Assignment operation; a′3: execute w t :=w best Assignment operation; a′4: execute w t := 1 assignment operation; strategy π′ is for each action a′ k Assignment probability p′ k .

[0042] Specifically, strategy π′ is for each action a′ k Assignment probability p′ k Specifically, this includes: strategy π′ for each action a′ k Assignment probability p′ k The condition p′1 + p′2 + p′3 + p′4 = 1 is satisfied. At each time t, the probability value is updated based on the reward value, calculated using the following formula: p′1 = 0.5γ(t); p′4=0.5(1-γ(t)).

[0043] Specifically, step 7 includes: performing backpropagation to update the model parameters. If the set number of iterations has not been reached, return to step 5 to continue training; if the set number of iterations has been reached, training ends, and the generated target detection model can be directly used for underwater sonar small target detection tasks.

[0044] In particular, the deep learning-based object detection model can also be constructed using YOLO, FCOS, RetinaNet, or Faster R-CNN models.

[0045] Beneficial effects:

[0046] 1) This invention does not require manual design of image features. Instead, it uses deep neural networks to directly extract relevant features from sonar images for autonomous target detection. It does not have high requirements for the contrast of the target background and image quality, which helps to improve detection accuracy and save manpower and resources.

[0047] 2) This invention compensates for the cross-domain accuracy loss when applying multi-scale target detection algorithms suitable for visible light images to small target detection tasks in sonar images, thereby improving the detection accuracy of small underwater targets.

[0048] 3) The end-to-end model generated by this invention can be directly deployed on a computer system, thereby providing good technical support for unmanned vessels to perform anti-submarine warfare, mine sweeping, information warfare, reconnaissance and surveillance, search and rescue support and other tasks.

[0049] 4) In this invention, the forgetting probability is introduced during the model training process, which makes the model training faster and more accurate.

[0050] 5) The compensation scale training in this invention defines what action to perform in the current state to update the weights of each scale in the loss function, so that the loss estimation is more accurate.

[0051] 6) The model in this invention can be ATSS, YOLO, FCOS, RetinaNet, Faster R-CNN, etc. All deep learning multi-scale object detection models of all feature network pyramids can be used in this invention, which has a wide range of applications. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating the underwater sonar small target detection method based on Markov decision process in this invention;

[0053] Figure 2 This is a framework diagram of the multi-scale underwater sonar small target detection model in this invention. Detailed Implementation

[0054] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0055] This invention provides an underwater sonar small target detection method based on Markov decision process.

[0056] like Figure 1 As shown, the specific implementation steps of the underwater sonar small target detection method of the Markov decision process of the present invention are as follows:

[0057] Step 1: Construct a deep learning-based object detection model, load the pre-trained model and sonar image dataset, initialize all initial parameters for model training, initialize the weight coefficients of each layer of features for FPN feature fusion, initialize the weight coefficients of the loss values ​​at each scale in the loss function, and set a certain number of iterations.

[0058] Theoretically, all deep learning multi-scale object detection models based on feature network pyramids can be used in this invention. This embodiment uses the ATSS model for construction, such as... Figure 2 As shown. The FPN has 5 layers, generating pyramid features. The process is as follows:

[0059]

[0060] in ↑2× represents a 1×1 convolution at each feature level, and ↑2× represents upsampling by a factor of 2 for nearest neighbor interpolation. The expression after introducing a weight factor for feature fusion is as follows:

[0061]

[0062] in The weight value during feature fusion is a real number that changes with time t. Represents the weight value vector:

[0063] Compensated scale training weights the multi-scale loss, with each scale having a weight of w. i，t w t Represents the weight value vector: [w 2，t w 3，t w 4，t w 5，t w 6，t The overall loss after introducing weights is as follows:

[0064]

[0065] Step 2: Define the state space S, state transition probability P, and reward value R of the target detection model.

[0066] During detector training, the total number of iterations is fixed and can be considered as a time dimension. As training progresses, the sum of the loss values ​​generated within the fixed iteration interval forms a time-ordered sequence, as shown below:

[0067]

[0068] Where n is the current iteration, T is the total number of iterations, and i represents the index of each scale level of the pyramid. To avoid outliers in the loss sequence, an iteration interval hyperparameter σ is introduced to average the loss values ​​within a fixed interval. Furthermore, the loss values ​​at each scale are standardized using linear standardization, as shown below:

[0069]

[0070] State space S = {s1, s2, s3, ..., s} t′}, where t′ is the time when training ends. The state at each time step reflects the state of the normalized loss value of the object detection model during training, i.e. The detector's value for each state is unknown at the start of training; values ​​are assigned to each state as training progresses. And state s... t The next moment must be s. t-1transition probability That is, the actions performed by the agent do not affect the state transition, but only change the value of the state.

[0071] Based on expected returns Design reward values ​​using a greedy strategy This is equivalent to executing actions of the reinforcement learning agent based on the maximum reward value at each moment. Representing the intelligent agent in S t-1 At that time, action A was performed. t-1 Afterwards, in S t The reward value provided in real time is calculated using the following formula:

[0072]

[0073] The greater the percentage decrease in the overall loss between two adjacent time points, the greater the reward; conversely, the reward is 0 if the overall loss increases.

[0074] Step 3: Define the action set A and policy π for dynamic feature fusion. Policy π determines which action to perform in the current state to update the parameters of feature fusion based on probability.

[0075] The formula for generating new weights through dynamic feature fusion is as follows:

[0076]

[0077]

[0078] brelu(x) = min(max(x, β) min ), β max )

[0079] The Brelu function has a defined range of values ​​to prevent abnormal weight values; here, it is simply set to [0.1, 10]. The iteration interval σ is set to 50 to make the multi-scale loss values ​​more stable and differentiated. γ(t) is the decay coefficient, which linearly changes from 1 to 0 as training progresses, because excessive changes to the weights may have a negative impact when the model is close to convergence.

[0080] The action set A = {a1, a2, a2} for dynamic feature fusion, where each action element a k Both represent a way to update the feature fusion weights. Where a1: updates the weights according to equation (4.7), performing... Assignment operation.

[0081] Based on reward value Determine at the current moment whether the previous weight update was reasonable. And if... This indicates that the weights used in the previous iteration may have negatively impacted the training results. If... Therefore, continuing to use the weight values ​​from the previous time step is also a good approach. This leads to a2: Execution. Assignment operation.

[0082] If a weight used in a particular instance yields the network the maximum reward, then that weight value will be used multiple times thereafter. Therefore, a set of global variables is introduced to record: the maximum reward value and the corresponding weight value, i.e. They were initialized to 0 and 1 respectively. When Execute R directly best :=R best and Update operation. Therefore, we get a3: Execution Assignment operation.

[0083] Because the selection of feature importance changes at each epoch during model training, and the total loss value decreases rapidly at the beginning of training, to prevent a particular set of features from being prioritized... Throughout the entire training cycle, the forgetting probability is introduced to forget existing data. That is, adopt renew The process is as follows:

[0084]

[0085] Where p d The value is 10%, and it is only applied when the condition is met. The determination of whether an update has been triggered is made only at that time.

[0086] Strategy π is for each action a k Assignment probability p k They satisfy p1 + p2 + p3 = 1. At each time t, the probability value is updated based on the reward value, calculated using the following formula:

[0087] p1 = 0.5γ(t)

[0088]

[0089]

[0090] Step 4: Define the action set A′ and policy π′ for compensated scale training. Policy π′ determines, based on probability, which action to perform in the current state to update the weights of each scale in the loss function.

[0091] First, calculate the proportion of loss at each scale in the total loss using the following formula:

[0092]

[0093] Higher-scale levels typically have smaller loss values ​​and are not adequately trained. Therefore, they are compensated for by weighting the two scales with the smallest loss values ​​upwards. Scales with smaller losses receive larger weights, while scales with larger losses retain a weight of 1. The calculation method is as follows:

[0094]

[0095] The set of actions for compensated scale training is A′={a′1, a′2, a′3, a′4}. Each action element a′ k Both represent a method for updating multi-scale loss, similar to step 3, and also involve determining the forgetting probability. The specific definitions are as follows:

[0096] a′1: Execute w i，t =w′ i，t Assignment operation.

[0097] a′2: Execute w t :=w t-1 Assignment operation.

[0098] a′3: Execute w t :=w best Assignment operation.

[0099] a′4: Execute w t The assignment operation is :=1.

[0100] Strategy π′ is for each action a′ k Assignment probability p′ k The probability value satisfies p′1 + p′2 + p′3 + p′4 = 1. At each time t, the probability value is updated based on the reward value, calculated using the following formula:

[0101] p'1=0.5γ(t)

[0102]

[0103]

[0104] p′4=0.5(1-γ(t))

[0105] Step 5: Begin model training. Process the training set images input for each iteration within the interval σ of each iteration.

[0106] Calculate the multi-scale loss value, and use the formula in step 2 to calculate the state S at each time step. t and rewards Assign a value.

[0107] Step 6: Calculate p based on the reward value using the formulas from Steps 3 and 4.k and p′ k And select the corresponding 'a' based on the obtained probability. k and a′ k The feature fusion weights and multi-scale loss weights at the current moment are updated.

[0108] Step 7: Calculate the overall loss and perform backpropagation to update the model parameters. If the set number of iterations has not been reached, return to step 5 to continue training; if the set number of iterations has been reached, training ends, and the generated target detection model can be directly used for underwater sonar small target detection tasks, realizing an end-to-end operation mode of "input → model → result output", and can be directly deployed on computer equipment.

[0109] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention, such as replacing ATSS with YOLO, FCOS, RetinaNet, Faster R-CNN, etc., should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0110] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0111] It will be apparent to those skilled in the art that the embodiments of the present invention are not limited to the details of the exemplary embodiments described above, and that the embodiments of the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the embodiments of the present invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the embodiments of the present invention is defined by the appended claims rather than the foregoing description. Therefore, all variations falling within the meaning and scope of equivalents of the claims are intended to be encompassed within the embodiments of the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units, modules, or devices recited in the system, apparatus, or terminal claims may also be implemented by the same unit, module, or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention and are not intended to limit them. Although the embodiments of the present invention have been described in detail with reference to the above preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the embodiments of the present invention should not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting small underwater sonar targets based on Markov decision processes, characterized in that, include: Step 1: Construct a deep learning-based object detection model, load the pre-trained model and sonar image dataset, and initialize the initial parameters for model training; Step 2: Define the state space of the target detection model. State transition probability Reward Value ; Step 3: Define the action set for dynamic feature fusion and strategy ,Strategy The appropriate action to perform in the current state to update the parameters of the feature fusion is determined based on probability. Step 4: Define the set of movements for compensatory metric training and strategy ,Strategy Based on probability, determine which action to perform in the current state to update the weights of each scale in the loss function; Step 5: Begin model training, processing the training set images input for each iteration, within each iteration interval. Internally; statistically analyze the multi-scale loss value, and define the state at each time step according to the definition in step 2. and rewards Assignment; Step 6: Based on the reward value Calculation strategy according to the formulas in steps 3 and 4 and strategy And select the corresponding action based on the obtained probability. and Update the feature fusion weights and multi-scale loss weights at the current moment; Step 7: Calculate the overall loss, perform backpropagation to update the model parameters, and continue training until the training reaches the required number of iterations.

2. The underwater sonar small target detection method based on Markov decision process as described in claim 1, characterized in that, Step 1 includes: the deep learning-based target detection model is constructed using the ATSS model; wherein the Feature Pyramid Network (FPN) has 5 layers to generate pyramid features. The process is as follows: in This represents a 1×1 convolution at each feature level. This indicates that the nearest neighbor interpolation is upsampled by a factor of 2; the expression after introducing a weighting factor for feature fusion is as follows: in The weight values ​​during feature fusion are a function that varies with time. The changing real number, Represents the weight value vector: ; Compensated scale training weights the multi-scale loss, with the weights for each scale being... ; Represents the weight value vector: The overall loss value after introducing weights is as follows: 。 3. The underwater sonar small target detection method based on Markov decision process as described in claim 2, characterized in that, Step 2 includes: during detector training, as training progresses, the sum of the loss values ​​generated within a fixed iteration interval forms a time-ordered sequence, as shown below: ； in For the current iteration, This represents the total number of iterations. The indexes represent the different scale levels of the pyramid; to avoid outliers in the loss sequence, an iterative interval hyperparameter is introduced. The loss values ​​within a fixed interval are averaged; furthermore, the loss values ​​at each scale are standardized using linear standardization, as shown below: state space ,in This is the moment when training ends; the state at each moment reflects the state of the normalized loss value of the object detection model during training, i.e. The detector's value for each state is unknown at the start of training; values ​​are assigned to each state as training progresses. It will definitely be in the next moment. transition probability That is, the actions performed by the agent do not affect the state transition, but only change the value of the state; The reinforcement learning agent executes actions based on the maximum reward value at each moment. Representing the intelligent agent in At that time, he made a move. Afterwards, The reward value provided in real time is calculated using the following formula: The greater the percentage decrease in the overall loss between two adjacent time points, the greater the reward; conversely, the reward is 0 if the overall loss increases.

4. The underwater sonar small target detection method based on Markov decision process as described in claim 3, characterized in that, Step 3 includes: a set of actions for dynamic feature fusion. Each action element Both represent a way to update the feature fusion weights; among them This means replacing the weights used in the previous time step with the newly generated weights; This indicates that the weighting coefficients from the previous time step will continue to be used; This means updating the current weights with the historically optimal weights; The formula for generating new weights, representing the attenuation coefficient, is as follows: in The function limits the range of values ​​to prevent abnormal weight values; here it is simply set to... Iteration interval The value is set to 50 to make the multi-scale loss values ​​more stable and differentiated. It is the attenuation coefficient, and its value range is... ; During the update of feature fusion weights, :implement Assignment operation; :implement Assignment operation; :implement Assignment operation, where The weight corresponding to the highest historical reward value; strategy For each action Assignment probability At every moment Updated based on reward value.

5. The underwater sonar small target detection method based on Markov decision process as described in claim 4, characterized in that, To prevent a certain group Throughout the entire training cycle, the forgetting probability is introduced to forget existing data. That is, adopt renew The process is as follows: ； in The value is 10%, and it only applies if the condition is met. A strategy for determining whether to trigger an update is implemented only at certain times. For each action Assignment probability At every moment Updated based on reward values, they satisfy ; The highest reward value in history, with a corresponding weight of At every moment The probability value is updated based on the reward value, and the calculation formula is as follows: ; ; ;in To prevent the denominator from being 0.

6. The underwater sonar small target detection method based on Markov decision process as described in claim 5, characterized in that, Step 4 includes: calculating the proportion of loss at each scale in the total loss according to the following formula: The two scales with the smallest percentage of loss are weighted upwards, with larger weights assigned to scales with smaller losses, while scales with larger losses are weighted at 1. The calculation method is shown below: Compensation scale training movement set Each action element Both represent a way to update multi-scale loss, specifically defined as follows: :implement Assignment operation; :implement Assignment operation; :implement Assignment operation; :implement Assignment operation; strategy For each action Assignment probability .

7. The underwater sonar small target detection method based on Markov decision process as described in claim 6, characterized in that, Strategy For each action Assignment probability Specifically, this includes: strategies For each action Assignment probability ,satisfy At every moment The probability value is updated based on the reward value, and the calculation formula is as follows: ; ; ; .

8. The underwater sonar small target detection method based on Markov decision process as described in claim 1, characterized in that, Step 7 specifically includes: performing backpropagation to update model parameters; if the set number of iterations has not been reached, returning to step 5 to continue training; if the set number of iterations has been reached, training ends, and the generated target detection model can be directly used for underwater sonar small target detection tasks.

9. The underwater sonar small target detection method based on Markov decision process as described in any one of claims 2-8, characterized in that, The deep learning-based object detection model can also be constructed using YOLO, FCOS, RetinaNet, or Faster R-CNN models.

Images