Diffusion model-based small target detection model training method, small target detection method and system

Abstract

The invention discloses a small target detection model training method, a small target detection method and a small target detection system based on a diffusion model, and relates to the field of computer vision, the small target detection model training method comprises the following steps: training a small target detection model by using a high-resolution image and a low-resolution image, the training comprising forward propagation and back propagation; the small target detection model comprises a backbone network, a super-resolution branch, a target detection branch and a dense condition module; performing back propagation on the small target detection model according to the loss value so as to optimize the small target detection model; according to the method, the super-resolution branch and the target detection branch are jointly optimized in the training process, the target detection branch provides accurate target position information for the super-resolution branch, and the super-resolution branch pertinently enhances the features of the target area, so that the accuracy of small target detection is improved.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a method for training a small target detection model based on a diffusion model, a small target detection method, and a system. Background Technology

[0002] Super-resolution techniques can directly improve the resolution of small targets and restore their detailed information, providing a more promising solution for improving small target detection performance. However, super-resolution methods based on convolutional neural networks often result in overly smoothed images and a lack of high-frequency details; while methods based on generative adversarial networks (GANs), although capable of generating sharper textures, suffer from training instability and artifact problems. In contrast, diffusion models start with random noise and reconstruct images through gradual denoising, combining the advantages of concise mathematical expression and stable training, and are expected to overcome the limitations of existing methods.

[0003] Diffusion models are one of the mainstream methods in the field of deep generative models. Depending on the generation conditions, diffusion models can be divided into unconditional generation and conditional generation. The former focuses on improving the diversity of generated samples, while the latter introduces conditional information to control the output results, ensuring they meet the expected goals. Currently, diffusion models have been successfully applied in multiple areas of computer vision. However, due to their high computational cost and slow inference speed, diffusion models are difficult to directly integrate into super-resolution tasks to support real-time detection. Furthermore, when using super-resolution techniques to assist in small object detection, image distortion and mode collapse occur, limiting the improvement in object detection accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a small target detection model training method, a small target detection method and system based on a diffusion model, which can improve the accuracy of small target detection.

[0005] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for training a small target detection model based on a diffusion model, the method comprising: The small object detection model is trained using high-resolution and low-resolution images from visible light images. The training process includes forward propagation and backward propagation, where: The forward propagation includes inputting the high-resolution image and the low-resolution image into a small target detection model to obtain the detection results of the small targets to be detected in the visible light image; the small target detection model includes a backbone network, a super-resolution branch, a target detection branch, and a dense conditional module; wherein: A backbone network is used to extract features from the low-resolution image to obtain multi-scale features; the multi-scale features serve as inputs to the target detection branch and the dense conditional module. The target detection branch, consisting of a feature pyramid and a path aggregation network, is used to process the multi-scale features to obtain predicted confidence scores, classification labels, and bounding box coordinates. The predicted confidence scores, classification labels, and bounding box coordinates are used as detection results to define the small targets to be detected in the visible light image. The super-resolution branch is used to add noise to the high-resolution image, transforming the high-resolution image into a pure noise image, and using a neural network model to extract features from the pure noise image to obtain downsampling features and upsampling features. The dense condition module is used to fuse the multi-scale features, the downsampling features, and the upsampling features to obtain a super-resolution image; The backpropagation includes calculating the loss value of the small object detection model based on the predicted confidence, classification label, bounding box coordinates, the super-resolution image, and the super-resolution ground truth; and performing backpropagation on the small object detection model based on the loss value to optimize the small object detection model. When the number of training iterations reaches a preset value, the trained small target detection model is obtained.

[0006] Secondly, this application provides a small target detection method, the small target detection method comprising: Acquire high-resolution and low-resolution images of the target image; The high-resolution and low-resolution images of the target image are input into a small target detection model based on a diffusion model to output the small targets to be detected in the target image. The small target detection model based on the diffusion model is trained according to any of the above-mentioned small target detection model based on the diffusion model training method.

[0007] Thirdly, this application provides a small target detection system, the small target detection system comprising: An image input unit is used to acquire high-resolution and low-resolution images of the target image; The small target detection unit is used to input the high-resolution and low-resolution images of the target image into the diffusion model-based small target detection model, and output the small targets to be detected in the target image. The small target detection model based on the diffusion model is trained according to any of the above-mentioned small target detection model based on the diffusion model training method.

[0008] Fourthly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the small target detection method described in any one of the above.

[0009] Fifthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the small target detection method described in any one of the above descriptions.

[0010] Sixthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the small target detection method described in any one of the above descriptions.

[0011] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a small target detection model training method, a small target detection method, and a system based on a diffusion model. The small target detection model training method based on a diffusion model utilizes a super-resolution branch based on a diffusion model to gradually add noise to the real image, transforming the data into a pure noise distribution, and then reversing the noise addition process to generate a super-resolution image. This iterative optimization method for generating images has excellent performance in terms of image detail restoration and texture enhancement. Furthermore, it utilizes a dense condition module to fuse multi-scale features, downsampling features, and upsampling features, thereby combining high-level semantic features with low-level details in low-resolution images to generate multi-scale high-resolution features, which assists the super-resolution branch in generating super-resolution images more accurately.

[0012] The training method for the small target detection model based on the diffusion model jointly optimizes the super-resolution branch and the target detection branch during the training process. The target detection branch provides the super-resolution branch with accurate target location information, and the super-resolution branch specifically enhances the features of the target region, thereby improving the accuracy of small target detection. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart illustrating a small target detection model training method based on a diffusion model in one embodiment of this application; Figure 2 A schematic diagram of the framework of a small target detection model provided in an embodiment of this application; Figure 3 A schematic diagram of the framework of a dense condition module provided in an embodiment of this application; Figure 4 To provide another embodiment of this application, low-resolution images and high-resolution images are provided; Figure 5 This is a super-resolution image ground truth map provided in an embodiment of this application; Figure 6 This is a comparison chart of the detection results of the small target detection model in the second embodiment of this application with the baseline networks YOLOv5 and SuperYOLO. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] First Embodiment Please see Figures 1 to 5 This application provides a method for training a small target detection model based on a diffusion model, including the following steps S101 to S104. Wherein: Step S101: Train the small target detection model using high-resolution and low-resolution images of visible light images. The training process includes forward propagation and backward propagation.

[0018] Step S102: The forward propagation includes inputting the high-resolution image and the low-resolution image into the small target detection model to obtain the detection result of the small target to be detected in the visible light image. The low-resolution visible light image used in this application is... The resolution of the visible light image is ;in, Indicates the height of the image. Indicates the width of the image, such as Figure 4 As shown.

[0019] As one embodiment of this application, such as Figure 2 as well as Figure 3 The small target detection model includes a backbone network, a super-resolution branch, a target detection branch, and a dense conditional module; wherein: A backbone network is used to extract features from the low-resolution image to obtain multi-scale features; these multi-scale features serve as inputs to the target detection branch and the dense conditional module. This invention uses the backbone network of the YOLOv5s model as the backbone network of a small target detection model based on a diffusion model. YOLOv5 includes a Focus module, which partitions the image at intervals and then reconstructs it to obtain smaller complementary images. However, since the size of the reconstructed image decreases with the increase in the number of channels, this leads to reduced resolution and loss of spatial information of small targets, which is detrimental to small target detection. Therefore, this application removes the Focus module from the YOLOv5s model and directly uses the low-resolution image... The low-resolution image is fed into the backbone network of the YOLOv5s model for feature extraction. Specifically, in this embodiment, the feature extraction of the low-resolution image to obtain multi-scale features includes: The low-resolution image is downsampled in multiple stages using the backbone network of the YOLOv5s model to obtain multi-scale features; the multi-scale features include multi-layer downsampling features. , of which: The expression for the layer downsampling feature is: ; In the formula: This indicates that the feature extraction of the low-resolution image by the backbone network of the YOLOv5s model is the first... Layer downsampling features; This represents the backbone network of the YOLOv5s model; This represents a low-resolution image. In this application, the backbone network of the YOLOv5s model performs five stages of downsampling operations on the low-resolution image sequentially, resulting in downsampled features. , .

[0020] The multi-scale features extracted by the backbone network of the YOLOv5s model are used as inputs to the target detection branch to classify and locate small targets to be detected in visible light images, and also as inputs to the dense conditional module to generate super-resolution images.

[0021] The target detection branch, consisting of a dual-tower structure of a Feature Pyramid Network (FPN) and a Path Aggregation Network (PANet), processes the multi-scale features to obtain predicted confidence scores, classification labels, and bounding box coordinates. The predicted confidence scores, classification labels, and bounding box coordinates are used as detection results to define the small targets to be detected in the visible light image.

[0022] In this embodiment of the application, the processing of the multi-scale features to obtain the predicted confidence score, classification label, and bounding box coordinates includes the following steps S201 to S203. Wherein: Step S201: The target detection branch, composed of a feature pyramid and a path aggregation network, processes the last three layers of features in the multi-layer downsampling features to obtain the processing result. The target detection branch, through the feature pyramid and path aggregation network, transmits high-order semantics from top to bottom to the lower layers of the last three layers of features in the five-layer downsampling features, and then sends detailed information back to the higher layers from bottom to top, completing multi-scale fusion of "high-level semantics + low-level details" to improve the target localization capability. Specifically, the operation is as follows: .

[0023] Step S202: Input the processing result into the detection head to obtain the target confidence, class probability, and bounding box offset; that is... Figure 2 middle Classification (Category Probability) Regression (bounding box offset) and : Confidence (target confidence level).

[0024] Step S203: Non-maximum suppression is used to deduplicate the target confidence, class probability, and bounding box offset, thus obtaining the predicted confidence, classification label, and bounding box coordinates. The predicted confidence, classification label, and bounding box coordinates are: ; In the formula: These represent the prediction confidence level, classification label, and bounding box coordinates, respectively. This indicates the detection head.

[0025] The super-resolution branch is used to add noise to the high-resolution image, transforming it into a pure noise image. A neural network model is then used to extract features from the pure noise image, obtaining downsampled and upsampled features. The super-resolution branch uses a diffusion model to process the high-resolution image to obtain a super-resolution image. The diffusion model is a probability-based generative model that generates data through the diffusion of data distribution and a denoising process. Specifically, in this embodiment, the super-resolution branch's process of processing the high-resolution image using the diffusion model includes a noise addition process and a denoising process, wherein: The noise-adding process, as described in this embodiment, involves adding noise to the high-resolution image to transform it into a purely noisy image. Specifically, this includes steps S301 to S305. Wherein: Step S301: Add noise conforming to a standard normal distribution to the high-resolution image to obtain a noisy image; Step S302, for the time step From 1 to ,definition Noise image at time step and Noise image at time step Satisfying the relation: ; In the formula: It is a fixed constant. With time step The increase becomes larger; Indicates noise; Step S303, Definition The relation is transformed into: ; Step S304: Derive the transformed relational expression using mathematical induction: ; In the formula: This represents a high-resolution image at the initial moment.

[0026] Step S305: Add a time step , making time step Tend to , With time step The increase becomes larger. Approaching zero Tendency towards noise This converts the high-resolution image into a purely noisy image.

[0027] This application performs reverse denoising on the generated pure noise image, which constitutes the denoising process; specifically, during the denoising process, this application utilizes the image from the previous time step. To predict the image of the next moment During the noise reduction process, this application utilizes the image at the current moment. To predict the previous moment ( (Image) ;in, Images of moments The probability distribution is as follows: ; In the formula: Indicates that in the known Images of moments Under the conditions, speculate Images of moments The probability distribution; Indicates known Images of moments Under the conditions, speculate Images of moments The probability distribution; express Images of moments The marginal probability; express Images of moments The marginal probability.

[0028] Given the original image Under the conditions, the above Images of moments The probability distribution is transformed into: ; In the formula: Indicates that in the known Images of moments And the original image Under the conditions, speculate Images of moments The probability distribution; Indicates known Images of moments And the original image Under the conditions, speculate Images of moments The probability distribution; This indicates that the original image is known. Under the conditions, speculate Images of moments The probability distribution; , indicating that the original image is known. Under the conditions, speculate Images of moments The probability distribution; The probability distribution formula after the above transformation is rewritten based on the forward noise addition process, and finally obtained: Images of moments The probability distribution follows a normal distribution: ; In the formula: Indicates a normal distribution. This represents the mean of a normal distribution; This represents the variance in a normal distribution; The value can be obtained through a neural network model. As can be seen from the above normal distribution, the image after adding noise... The data is input into a neural network model for feature extraction and prediction at different times. Added noise It can be predicted Images of moments The probability of [the event / determined]. Considering the large computational cost of diffusion models, this application directly predicts the super-resolution image rather than noise during model training. Specifically, [the application will]... Rewritten as: ; As can be seen from the rewritten formula above, predicting super-resolution images is equivalent to predicting noise, thereby significantly reducing the computational cost of the diffusion model.

[0029] In this embodiment of the application, the neural network model is a diffusion U-Net model, that is... Figure 2 The DiffusionU-Net in this application implements the denoising process using the diffusion U-Net model, i.e., predicting the super-resolution image of a purely noisy image. To provide richer control over the diffusion generation process, this application employs dense conditional modules to process multi-scale features and uses them as conditionally guided diffusion models to generate super-resolution images, wherein: The Dense Condition Module (DCM) is used to fuse the multi-scale features, the downsampled features, and the upsampled features to obtain a super-resolution image.

[0030] In this embodiment of the application, the fusion of the multi-scale features, the downsampling features, and the upsampling features to obtain a super-resolution image includes the following steps S401 to S405. Wherein: Step S401: Obtain multi-scale features. These multi-scale features are multi-layer downsampling features obtained by the backbone network through multi-stage feature extraction of the low-resolution image. The multi-layer downsampling features include high-level semantic features obtained from the final stage of feature extraction and multiple other layer features obtained from other stages. In this embodiment, the multi-scale features include five layers of downsampling features, wherein the high-level semantic features are... The features of each of the other layers are as follows: , , , .

[0031] Step S402: The high-level semantic features are fused with the features of other layers to obtain enhanced features, and the enhanced feature with the largest size is upsampled; wherein, the generation method of each enhanced feature is as follows: ; In the formula: Indicates the first One enhanced feature; These represent the weights and biases of the convolution, respectively. This indicates an upsampling operation.

[0032] Performing step S402 yields five enhanced features. Since both the backbone network and the diffuse U-Net model undergo five downsampling operations, the former's output feature size is only half that of the latter. Considering that directly performing upsampling to increase feature size can easily lead to the loss of details and semantic information, this application therefore incorporates high-level semantic information... Other layer features The enhanced features are obtained through fusion.

[0033] Step S403: Obtain downsampling features and upsampling features, wherein the downsampling features and upsampling features are obtained by a neural network model performing feature extraction on the pure noise image; Step S404: Concatenate upsampled features and downsampled features of the same scale, and then use enhanced features to model the concatenated features to obtain latent features; wherein, the latent features are generated in the following way: ; In the formula: Indicates potential characteristics; Indicates the first One enhanced feature; Indicates the first One downsampled feature; Indicates the first Upsampled features It should be noted that the diffuse U-Net network model obtains downsampled features after convolutional encoding and decoding. and upsampling features To effectively perform image super-resolution, the upsampled feature map is first concatenated with the feature map of the corresponding scale in the diffuse U-Net network model, and then enhanced features are used. Instruct the network to represent latent features Modeling is performed.

[0034] Step S405: Process the latent features using implicit feature representation to obtain the super-resolution image. Example: Step S405, in each latent feature... Implicit Feature Representation (INR) is then added. The current discrete feature map is treated as a two-dimensional coordinate system. Feature vectors of latent codes with uniform distribution And add coordinate position encoding This data is then fed into a multilayer perceptron (MLP) with two hidden layers of 256 dimensions, resulting in continuous high-resolution feature representations. The implicit neural functions in INR... It can be represented as: In the formula: The features represented by super-resolution are called super-resolution images; Implicit Feature Representation (INR) enables the model to capture more detailed information in the model, which helps to more accurately locate and identify small targets.

[0035] Step S103: The backpropagation includes calculating the loss value of the small object detection model based on the predicted confidence, classification label, bounding box coordinates, the super-resolution image, and the super-resolution ground truth; and performing backpropagation on the small object detection model based on the loss value to optimize the small object detection model.

[0036] During model training, this application jointly trains the super-resolution branch and the object detection branch. This not only reduces computation time but also allows the super-resolution branch and the object detection branch to optimize each other. The diffusion-based super-resolution model in the super-resolution branch enables the object detection network to learn high-resolution features of small objects, while the accurate detection of the object detection branch, in turn, prompts the super-resolution branch to better reconstruct the high-resolution details of the target region.

[0037] As one embodiment of this application, step S103, which involves calculating the loss value of the small object detection model based on the predicted confidence level, classification label, bounding box coordinates, the super-resolution image, and the super-resolution ground truth, includes steps S501 to S503. Wherein: Step S501: Calculate the loss value of the target detection branch based on the predicted confidence level, classification label, and bounding box coordinates. The formula for calculating the loss value of the target detection branch is as follows: ; In the formula: This represents the loss value of the object detection branch; These represent the confidence levels of the prediction and the actual result, respectively. These represent the predicted and actual classification labels, respectively. These represent the predicted and actual bounding box coordinates, respectively. Represents the cross-entropy loss function. Describes the L1 loss function. This represents the weighting coefficient.

[0038] Considering that small targets typically occupy only a few dozen pixels in an image, applying uniform super-resolution processing to the entire image, while enhancing the features of small targets to some extent, also enhances the information of the background region. This approach cannot fully leverage the advantages of super-resolution technology in small target detection, limiting its ability to improve small target detection performance. To enhance small target features and reduce interference from background features, this application proposes a differential super-resolution loss method, which allows the model to focus on the small target region during the super-resolution process, as shown in step S502: Step S502: Calculate the loss value of the super-resolution branch based on the super-resolution image and the super-resolution ground truth, specifically including the following steps S601 to S603. Wherein: Step S601: Create a binary mask for the visible light image. The binary mask is used to set the pixel values ​​in the target area to 1 and the pixel values ​​in the background area to 0. The target area is the area of ​​the small target to be detected in the visible light image. Step S602: Multiply the binary mask by the high-resolution image to obtain the true value GT of the super-resolution image, i.e., as shown below. Figure 2 as well as Figure 5 As shown.

[0039] Step S603: Calculate the loss value of the super-resolution branch based on the ground truth value of the super-resolution image and the super-resolution image. The formula for calculating the loss value of the super-resolution branch is as follows: ; In the formula: Indicates a super-resolution image; Represents a binary mask; Represents a high-resolution image; express Norm.

[0040] Step S503: Calculate the loss value of the small object detection model using the loss value of the object detection branch and the loss value of the super-resolution branch. The formula for calculating the loss value of the small object detection model is as follows: ; In the formula: This represents the loss value of the small object detection model; This represents the loss value of the object detection branch; This represents the loss value of the super-resolution branch; This represents the adjustment coefficient.

[0041] Step S104: When the number of times the training process is executed reaches the preset value, the trained small target detection model is obtained.

[0042] In the training process of the small object detection model based on the diffusion model, this invention uses stochastic gradient descent (SGD) to train the small object detection model with an initial learning rate of 0.01, a momentum of 0.937, a weight decay of 0.0005 for Nesterov acceleration gradients, and a batch size of 2. After training, the optimal weight information is saved, and then the super-resolution branch is removed during the testing phase. The weights are then directly used to detect the input image to obtain the object classification and bounding box location.

[0043] To improve the training speed of the small object detection model, the image input to the network is first adjusted to 256×256, and then changed to 512×512 through the super-resolution branch; during testing, the image size is 256×256. The diffusion step count is set to 1500 steps.

[0044] By implementing steps S101 to S104 above, this small target detection model training method based on the diffusion model utilizes the super-resolution branch based on the diffusion model to gradually add noise to the real image, transforming the data into a pure noise distribution, and then reversing the noise addition process to generate a super-resolution image. This iterative optimization method for generating images has excellent performance in terms of image detail restoration and texture enhancement. Furthermore, it utilizes a dense condition module to fuse multi-scale features, the downsampling features, and the upsampling features, thereby combining high-level semantic features with low-level details in the low-resolution image to generate multi-scale high-resolution features, which assists the super-resolution branch in generating super-resolution images more accurately.

[0045] The training method for the small target detection model based on the diffusion model jointly optimizes the super-resolution branch and the target detection branch during the training process. The target detection branch provides the super-resolution branch with accurate target location information, and the super-resolution branch specifically enhances the features of the target region, thereby improving the accuracy of small target detection.

[0046] Second Embodiment Based on the same inventive concept, this application also provides a small target detection method, which includes the following steps S701 to S702. Wherein: Step S701: Obtain the high-resolution image and the low-resolution image of the target image; Step S702: Input the high-resolution and low-resolution images of the target image into the small target detection model based on the diffusion model to output the small targets to be detected in the target image; The small target detection model based on the diffusion model is trained according to the small target detection model training method based on the diffusion model in the first embodiment.

[0047] In one test scenario, the small object detection method described in the second embodiment is used to assist in small object detection. The small object detection model trained in the first embodiment is selected as the example, and the baseline network YOLOv5 and SuperYOLO (with an added CNN-based super-resolution branch) are selected as comparison examples. The comparison of detection results is shown in the figure below. Figure 6 As shown.

[0048] To quantitatively evaluate the performance of the above method, this invention selected the average accuracy mean. The average accuracy when IoU is 0.5 Average recall rate and the number of image frames processed per second by the model These four indicators, the higher the value of each indicator, the more accurate the target detection. The detection results are shown in Table 1: Table 1 Comparison of Test Results

[0049] As can be seen from the comparison of detection results and Table 1, the diffusion model-assisted small target detection method proposed in this invention outperforms the baseline network YOLOv5 and SuperYOLO (which adds a CNN-based super-resolution branch) in all indicators.

[0050] Third Embodiment Based on the same inventive concept, this application also provides a small target detection system, the small target detection system comprising: An image input unit is used to acquire high-resolution and low-resolution images of the target image; The small target detection unit is used to input the high-resolution and low-resolution images of the target image into the diffusion model-based small target detection model, and output the small targets to be detected in the target image. The small target detection model based on the diffusion model is trained according to the small target detection model training method based on the diffusion model in the first embodiment.

[0051] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0052] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0053] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0054] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0055] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0056] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0057] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0058] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A training method for a small target detection model based on a diffusion model, characterized in that, The training method for the small target detection model based on the diffusion model includes: The small object detection model is trained using high-resolution and low-resolution images from visible light images. The training process includes forward propagation and backward propagation, where: The forward propagation includes inputting the high-resolution image and the low-resolution image into a small target detection model to obtain the detection results of the small targets to be detected in the visible light image; the small target detection model includes a backbone network, a super-resolution branch, a target detection branch, and a dense conditional module; wherein: A backbone network is used to extract features from the low-resolution image to obtain multi-scale features; the multi-scale features serve as inputs to the target detection branch and the dense conditional module. The target detection branch, consisting of a feature pyramid and a path aggregation network, is used to process the multi-scale features to obtain predicted confidence scores, classification labels, and bounding box coordinates. The predicted confidence scores, classification labels, and bounding box coordinates are used as detection results to define the small targets to be detected in the visible light image. The super-resolution branch is used to add noise to the high-resolution image, transforming the high-resolution image into a pure noise image, and using a neural network model to extract features from the pure noise image to obtain downsampling features and upsampling features. The dense condition module is used to fuse the multi-scale features, the downsampling features, and the upsampling features to obtain a super-resolution image; The backpropagation includes calculating the loss value of the small object detection model based on the predicted confidence, classification label, bounding box coordinates, the super-resolution image, and the super-resolution ground truth; and performing backpropagation on the small object detection model based on the loss value to optimize the small object detection model. When the number of training iterations reaches a preset value, the trained small target detection model is obtained.

2. The method for training a small target detection model based on a diffusion model according to claim 1, characterized in that, The step of extracting features from the low-resolution image to obtain multi-scale features includes: The low-resolution image is downsampled in multiple stages using the backbone network of the YOLOv5s model to obtain multi-scale features; the multi-scale features include multi-layer downsampling features. , of which: The expression for the layer downsampling feature is: ； In the formula: This indicates that the feature extraction of the low-resolution image by the backbone network of the YOLOv5s model is the first... Layer downsampling features; This represents the backbone network of the YOLOv5s model; This indicates a low-resolution image.

3. The method for training a small target detection model based on a diffusion model according to claim 2, characterized in that, The process of processing the multi-scale features to obtain the predicted confidence score, classification label, and bounding box coordinates includes: The target detection branch, composed of feature pyramid and path aggregation network, is used to process the last three layers of features in the multi-layer downsampling features to obtain the processing result; The processing results are input into the detection head to obtain the target confidence, class probability, and bounding box offset. The target confidence, class probability, and bounding box offset are processed by nonmaximum suppression deduplication to obtain the predicted confidence, classification label, and bounding box coordinates.

4. The method for training a small target detection model based on a diffusion model according to claim 1, characterized in that, Adding noise to the high-resolution image to transform it into a purely noisy image includes: Adding noise that conforms to a standard normal distribution to a high-resolution image yields a noisy image; For time step From 1 to ,definition Noise image at time and Noise image at time Satisfying the relation: ； In the formula: It is a fixed constant. With time step The increase becomes larger; Indicates noise; definition The relation is transformed into: ； The transformed relation is derived using mathematical induction: ； In the formula: Represents a high-resolution image at the initial moment; Increase time steps , making time step Tend to This is to convert the high-resolution image into a purely noisy image.

5. The method for training a small target detection model based on a diffusion model according to claim 1, characterized in that, The process of fusing the multi-scale features, the downsampling features, and the upsampling features to obtain a super-resolution image includes: Multi-scale features are obtained by the backbone network performing multi-stage feature extraction on low-resolution images. These multi-scale features include high-level semantic features obtained from the last stage feature extraction and multiple other layer features obtained from other stage feature extraction. The high-level semantic features are fused with features from other layers to obtain enhanced features, and the enhanced feature with the largest size is upsampled. The generation method for each enhanced feature is as follows: ； In the formula: Indicates the first One enhanced feature; These represent the weights and biases of the convolution, respectively. Indicates an upsampling operation; The downsampling features and upsampling features are obtained by feature extraction of the pure noise image by a neural network model; Upsampled and downsampled features of the same scale are concatenated, and then enhanced features are used to model the concatenated features to obtain latent features; the latent features are generated as follows: ； In the formula: Indicates potential characteristics; Indicates the first One enhanced feature; Indicates the first One downsampled feature; Indicates the first Upsampled features ; The latent features are processed using implicit feature representation to obtain the super-resolution image.

6. The method for training a small target detection model based on a diffusion model according to claim 1, characterized in that, The step of calculating the loss value of the small object detection model based on the predicted confidence level, classification label, bounding box coordinates, and the super-resolution image and super-resolution ground truth includes: The loss value of the target detection branch is calculated based on the predicted confidence level, classification label, and bounding box coordinates; The loss value of the super-resolution branch is calculated based on the super-resolution image and the super-resolution ground truth. The loss value of the small object detection model is calculated using the loss value of the object detection branch and the loss value of the super-resolution branch. The formula for calculating the loss value of the small object detection model is as follows: ； In the formula: This represents the loss value of the small object detection model; This represents the loss value of the object detection branch; This represents the loss value of the super-resolution branch; This represents the adjustment coefficient.

7. The method for training a small target detection model based on a diffusion model according to claim 6, characterized in that, The formula for calculating the loss value of the target detection branch based on the predicted confidence level, classification label, and bounding box coordinates is as follows: ； In the formula: This represents the loss value of the object detection branch; These represent the confidence levels for the prediction and the actual result, respectively. These represent the predicted and actual classification labels, respectively. These represent the predicted and actual bounding box coordinates, respectively. Represents the cross-entropy loss function. Describes the L1 loss function. This represents the weighting coefficient.

8. The method for training a small target detection model based on a diffusion model according to claim 7, characterized in that, The step of calculating the loss value of the super-resolution branch based on the super-resolution image and the super-resolution ground truth includes: A binary mask is created for the visible light image, which sets the pixel values ​​in the target region to 1 and the pixel values ​​in the background region to 0; the target region is the region of the small target to be detected in the visible light image. The binary mask is multiplied by the high-resolution image to obtain the true value of the super-resolution image; The loss value of the super-resolution branch is calculated based on the ground truth value of the super-resolution image and the super-resolution image itself. The formula for calculating the loss value of the super-resolution branch is as follows: ； In the formula: Indicates a super-resolution image; Represents a binary mask; Represents a high-resolution image; express Norm.

9. A method for detecting small targets, characterized in that, The small target detection method includes: Acquire high-resolution and low-resolution images of the target image; The high-resolution and low-resolution images of the target image are input into a small target detection model based on a diffusion model to output the small targets to be detected in the target image. The small target detection model based on the diffusion model is trained according to any one of the small target detection model training methods based on the diffusion model as described in claims 1-8.

10. A small target detection system, characterized in that, The small target detection system includes: An image input unit is used to acquire high-resolution and low-resolution images of the target image; The small target detection unit is used to input the high-resolution and low-resolution images of the target image into the diffusion model-based small target detection model, and output the small targets to be detected in the target image. The small target detection model based on the diffusion model is trained according to any one of the small target detection model training methods based on the diffusion model as described in claims 1-8.

Images