Model training method and device, target detection method and device and storage medium

Abstract

The invention provides a model training method, a target detection method, equipment and a storage medium, and the method comprises the steps: inputting a first sample image into a first network model, and inputting a second sample image into a second network model, the second sample image being generated by adding a degradation parameter to the first sample image; inputting a first feature image output by the first network model and a second feature image output by the second network model into a degradation parameter prediction module; obtaining a degradation parameter prediction result output by a degradation parameter prediction module; calculating a first loss function value based on the degradation parameter and the degradation parameter prediction result; and adjusting model parameters of the second network model based on the first loss function value, continuing training until a preset training completion condition is met, and obtaining a trained second network model. According to the embodiment of the invention, the degradation parameter loss function is calculated, and the network parameters are optimized based on the loss function, so that discriminative features under different degradation degrees can be identified, and the detection precision is improved.

Description

Technical Field

[0001] The present application belongs to the field of model training technology, and specifically relates to a model training, target detection method, device and storage medium. Background Art

[0002] Object detection is an important task in the field of computer vision. Its goal is to detect and identify the locations and categories of multiple objects in images or videos. Since object detection not only expands the capabilities of image processing, but also enables computers to understand and process visual information more intelligently, object monitoring has a wide range of applications in many fields.

[0003] However, if the input image is blurry during the target detection process, the target in the image may not be recognized or the recognition accuracy may be low. Summary of the invention

[0004] The present application proposes a model training method, a target detection method, a device and a storage medium, which can solve the technical problems that in the current target detection process, the input image is blurred, or the target in the image cannot be recognized or the recognition accuracy is low.

[0005] The first embodiment of the present application proposes a model training method, including:

[0006] Inputting a first sample image into a first network model and inputting a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image, and the degradation parameters include: motion blur parameters and noise information;

[0007] Inputting a first feature image output by the first network model and a second feature image output by the second network model into a degradation parameter prediction module;

[0008] Obtaining a degradation parameter prediction result output by the degradation parameter prediction module;

[0009] Calculating a first loss function value based on the degradation parameter and the degradation parameter prediction result;

[0010] The model parameters of the second network model are adjusted based on the first loss function value, and the training is continued until a preset training completion condition is met to obtain a trained second network model.

[0011] The embodiment of the second aspect of the present application provides a model training device, including:

[0012] An input module, used for inputting a first sample image into a first network model and a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image, and the degradation parameters include: motion blur parameters and noise information;

[0013] The input module is used to input the first feature image output by the first network model and the second feature image output by the second network model into the degradation parameter prediction module;

[0014] An acquisition module, used for acquiring the degradation parameter prediction result output by the degradation parameter prediction model;

[0015] A calculation module, used for calculating a first loss function value based on the degradation parameter and the degradation parameter prediction result;

[0016] An adjustment module is used to adjust the model parameters of the second network model based on the first loss function value, and continue training until a preset training completion condition is met to obtain a trained second network model.

[0017] The third aspect of the present application provides a target detection method, including:

[0018] Inputting the image to be recognized into a second network model, wherein the second network model is obtained by the training method described in the first aspect;

[0019] Determine, by using the second network model, an equivariant representation feature image of the feature image corresponding to the image to be identified before degradation;

[0020] The target object in the image to be identified is identified in the equivariant representation feature image using the second network model.

[0021] The fourth aspect of the present application provides a target detection device, including:

[0022] An input module, used for inputting the image to be recognized into a second network model, wherein the second network model is obtained by the training method described in the first aspect;

[0023] A determination module, configured to determine, by using the second network model, an equivariant representation feature image of the feature image corresponding to the image to be identified before degradation;

[0024] A recognition module is used to recognize a target object in the image to be recognized in the equivariant representation feature image using the second network model.

[0025] An embodiment of the fifth aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect or the third aspect above.

[0026] The embodiment of the sixth aspect of the present application provides a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to implement the method described in the first aspect or the third aspect above.

[0027] The technical solution provided in the embodiments of the present application has at least the following technical effects or advantages:

[0028] The present application proposes a model training method, device and storage medium, including: The present application proposes a model training method, target detection method, device and storage medium, including: inputting a first sample image into a first network model and inputting a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image; inputting a first feature image output by the first network model and a second feature image output by the second network model into a degradation parameter prediction model; obtaining the degradation parameter prediction result output by the degradation parameter prediction model; calculating a first loss function value based on the degradation parameter and the degradation parameter prediction result; adjusting the model parameters of the second network model based on the first loss function value, and continuing training until the preset training completion conditions are met to obtain a trained second network model. The embodiment of the present application calculates the degradation parameter loss function and optimizes the network parameters based on the loss function, so that it can recognize the discriminative features under different degradation degrees and improve the detection accuracy.

[0029] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0031] Figure 1 A flow chart of a model training method provided by an embodiment of the present application is shown;

[0032] Figure 2 A schematic diagram of the structure of a target network model provided by an embodiment of the present application is shown; Figure 3-Figure 5 An example diagram of image degradation provided by an embodiment of the present application is shown;

[0033] Figure 6 A schematic diagram of the structure of a degradation parameter prediction model provided by an embodiment of the present application is shown;

[0034] Figure 7 A flow chart of a target detection method provided by an embodiment of the present application is shown;

[0035] Figure 8 A schematic diagram of the structure of a model training device provided in one embodiment of the present application is shown;

[0036] Fig. 9 A schematic diagram showing the structure of a target detection device provided by an embodiment of the present application is shown.

[0037] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application is shown;

[0038] Fig.11 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0039] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0040] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.

[0041] The model training method of the present application can be executed by a computing device, which can be a server, such as one server, multiple servers, a server cluster, a cloud computing platform, etc. Optionally, the computing device can also be a terminal device, such as a mobile phone, a tablet computer, a game console, a portable computer, a desktop computer, an advertising machine, an all-in-one machine, etc. The present application does not limit the device type and number of computing devices.

[0042] Based on the above background technology, the target detection algorithm for low-quality images can include indirect methods and direct methods. The indirect method uses image restoration to enhance the image quality through denoising, deblurring and super-resolution reconstruction, so that the features extracted by the network are richer, thereby improving the detection accuracy of the target detection algorithm. Depending on whether the image restoration and target detection processes are independent, it can be divided into algorithms in which restoration and detection are independent and detection algorithms in which restoration and detection are unified.

[0043] However, image restoration algorithms for multiple degradation factors often have cumbersome steps or large networks. Using them as preprocessing methods will make the detection time too long. Image restoration itself is an ill-posed problem, and there is a big difference between human perception and machine perception. Therefore, independent research on restoration and detection networks has limited optimization of target detection performance, and even cannot ensure that target detection performance is improved.

[0044] The method of unifying the restoration and detection networks directly connects the image restoration network with the detection network and jointly optimizes the network. This can enrich the image features while enabling the features to be correctly perceived by the machine, thereby improving detection accuracy. However, due to the mechanism of restoration before detection, the detection speed is greatly reduced.

[0045] The direct method is to improve the detection accuracy of the target detection algorithm by extracting image features more accurately. This method can perform target detection alone during the test phase without being affected by other network branches on the detection speed. According to different principles, the current mainstream methods can be divided into detection methods based on image complex atomic networks, detection methods based on degenerate equivariant representations, and detection methods based on transfer learning.

[0046] However, the complex atomic network part requires high-quality images for pixel-level supervision, which will inevitably cause the network to learn unnecessary detail features, thus affecting target detection.

[0047] Transfer learning-based methods learn low-quality image features or extract domain-invariant features for detection to reduce the impact of degradation, but the effect is poor when the difference between domains is too large.

[0048] In order to solve the above problems, the present application proposes a model training method, a target detection method, a device and a storage medium, including: inputting a first sample image into a first network model and inputting a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image; inputting a first feature image output by the first network model and a second feature image output by the second network model into a degradation parameter prediction model; obtaining the degradation parameter prediction result output by the degradation parameter prediction model; calculating a first loss function value based on the degradation parameter and the degradation parameter prediction result; adjusting the model parameters of the second network model based on the first loss function value, and continuing training until the preset training completion conditions are met to obtain a trained second network model. The embodiment of the present application calculates the degradation parameter loss function and optimizes the network parameters based on the loss function, so that it can recognize the discriminative features under different degradation degrees and improve the detection accuracy.

[0049] A model training method proposed according to an embodiment of the present application is described below in conjunction with the accompanying drawings.

[0050] See also Figure 1, the method specifically comprises the following steps:

[0051] S101, inputting a first sample image into a first network model and inputting a second sample image into a second network model.

[0052] The second sample image is generated by adding degradation parameters to the first sample image, and the degradation parameters include: motion blur parameters and noise information.

[0053] The first sample image is a high-quality image, the second sample image is a low-quality image, the degradation parameter includes a motion blur parameter and noise information. The first sample image may be an image that does not include the degradation parameter, and the second sample image is generated by adding the degradation parameter to the first sample image.

[0054] The first network model and the second network model have the same model structure and can both be used to detect target objects in sample images.

[0055] S102: Input the first feature image output by the first network model and the second feature image output by the second network model into a degradation parameter prediction model.

[0056] S103: Obtain a degradation parameter prediction result output by the degradation parameter prediction model.

[0057] The first feature image does not include degradation parameters, and the second feature image includes degradation parameters. Therefore, the degradation parameter prediction model can determine the difference information between the first feature image and the second feature image, that is, the degradation parameter prediction result.

[0058] S104. Calculate a first loss function value based on the degradation parameter and the degradation parameter prediction result.

[0059] S105. Adjust the model parameters of the second network model based on the first loss function value, and continue training until a preset training completion condition is met to obtain a trained second network model.

[0060] The degradation parameter is the true value of the degradation parameter prediction result. Therefore, the first loss function value can be calculated based on the difference information between the degradation parameter and the degradation parameter prediction result, so as to train the second network model using the first loss function value, so that it can learn the equivariant representation of the features before and after degradation, extract the discriminative features for detection and recognition under different degradation degrees, and improve the detection accuracy.

[0061] The training completion condition may be that the number of training times reaches a preset training times threshold.

[0062] The present application proposes a model training method, including: inputting a first sample image into a first network model and inputting a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image; inputting a first feature image output by the first network model and a second feature image output by the second network model into a degradation parameter prediction model; obtaining a degradation parameter prediction result output by the degradation parameter prediction model; calculating a first loss function value based on the degradation parameter and the degradation parameter prediction result; adjusting the model parameters of the second network model based on the first loss function value, and continuing training until a preset training completion condition is met to obtain a trained second network model. The embodiment of the present application calculates the degradation parameter loss function, optimizes the network parameters based on the loss function, so that it can recognize discriminative features under different degradation degrees and improve detection accuracy.

[0063] In some embodiments, the model parameters of the second network model are adjusted based on the first loss function value, and training is continued until a preset training completion condition is met. Before obtaining the trained second network model, it also includes: obtaining the first label result and the first prediction result of the first network model and the second label result and the second prediction result of the second network model; calculating the second loss function value based on the first label result and the first prediction result; calculating the third loss function value based on the second label result and the second prediction result.

[0064] It can be understood that in addition to enabling the second network model to recognize the discriminative features under different degrees of degradation, it is also necessary to improve the second network model's ability to recognize the target object. Therefore, it is necessary to calculate the second loss function value of the first network model and the third loss function value of the second network model.

[0065] The first label result is the true value of the first prediction result, the second label result is the true value of the second prediction result, the second loss function value can be calculated based on the first label result and the first prediction result, and the third loss function value can be calculated based on the second label result and the second prediction result. The first network model is further trained by the second loss function value and the second network model is trained by the third loss function value.

[0066] In some embodiments, adjusting the model parameters of the second network model based on the first loss function value, continuing training until a preset training completion condition is met, and obtaining a trained second network model includes:

[0067] The model parameters of the second network model are adjusted based on the first loss function value, the second loss function value and the third loss function value, and the training is continued until the preset training completion condition is met to obtain a trained second network model.

[0068] Among them, the second network model can be supervised by the first loss function value, the second loss function value and the third loss function value to optimize the network parameters so that it can learn the equivariant representation of features before and after degradation, extract discriminative features for detection and recognition under different degrees of degradation, and learn to recognize target objects from images to improve detection accuracy.

[0069] In some embodiments, adjusting the model parameters of the second network model based on the first loss function value, the second loss function value, and the third loss function value, continuing the training until a preset training completion condition is met, and obtaining a trained second network model includes:

[0070] Determine the loss weights corresponding to the first loss function value, the second loss function value, and the third loss function value; determine the total loss function value of the second network model based on the first loss function value, the second loss function value, the third loss function value, and the corresponding loss weights; adjust the model parameters of the second network model based on the total loss function value, continue training until the preset training completion conditions are met, and obtain a trained second network model.

[0071] It can be understood that the total loss function includes two parts, namely, the target detection loss function and the degradation parameter prediction loss, wherein the target detection loss function is obtained based on the second loss function value and the third loss function value, and the degradation parameter prediction loss is the first loss function value.

[0072] In some embodiments, the total loss function is as shown in formula (1):

[0073] L total =L obj +λ dpp ·L dpp (1)

[0074] Among them, L total is the total loss function, L obj is the target detection loss function, L dpp is the degradation parameter prediction loss function, λ dpp Predict the weights of the loss function for the degradation parameters.

[0075] The target monitoring loss function is shown in formula (2):

[0076] L obj =λ obj-H ·L obj-H +λ obj-L ·L obj-L (2)

[0077] L obj-H and L obj-L are the second loss function and the third loss function respectively, λ obj-H and λobj-L are the weights of the second loss function and the third loss function respectively.

[0078] Among them, the second loss function and the third loss function adopt the same calculation method, including confidence loss L conf , classification loss L cls and the localization loss L box Three parts, among which the second loss function or the third loss function is shown in formula (3):

[0079] L=λ conf ·L conf +λ cls ·L cls +λ box ·L box (3)

[0080] Among them, L is the second loss function or the third loss function, λ conf is the weight of the confidence loss, λ cls is the weight of classification loss, λ box is the weight of the positioning loss.

[0081] In some embodiments, inputting the first feature image output by the first network model and the second feature image output by the second network model into the degradation parameter prediction model includes:

[0082] The first feature image output by the first preset downsampling unit in the first network model and the second feature image output by the second preset downsampling unit in the second network model are input into the degradation parameter prediction model, the image sizes corresponding to the first feature image and the second feature image are consistent, the first preset downsampling unit exists in the first backbone network in the first network model, and the second preset downsampling unit exists in the second backbone network in the second network model.

[0083] Among them, the model structures of the first network model and the second network model both include a backbone network, a neck network and a detection layer, wherein the backbone network includes multiple downsampling units. In order to obtain accurate degradation parameter prediction results, the first feature image and the second feature image input into the degradation parameter prediction model are consistent in size, and motion blur and noise directly affect the shallow features of the image such as grayscale and texture. The closer the feature map is to the front, the richer the information of image degradation is retained, which is conducive to degradation parameter prediction; on the other hand, if the feature map participates in the multi-scale feature fusion of the subsequent neck network, it is beneficial to improve the overall performance of the network. Therefore, the preset downsampling unit is generally larger in size, and the retained image degradation information is richer, and the feature map output by the preset downsampling unit will participate in the multi-scale feature fusion of the subsequent neck network.

[0084] In some embodiments, the model structure of the target network model is as follows: Figure 2 As shown, Figure 2 As shown, the target network model includes a first network model, a second network model and a degradation parameter prediction model. The first network model and the second network model both include a backbone network, a neck network and a detection layer, and the backbone network includes a backbone network including 5 downsampling units C1, C2, C3, C4 and C5, where C i The downsampling factor is 2 i Downsampling unit. Since the feature image output by the downsampling unit C3 is relatively early, the retained image degradation information is relatively rich, and the feature image output by C3 participates in the multi-scale feature fusion of the subsequent neck network, the feature image output by the backbone network C3 is used to predict the degradation parameters.

[0085] In some embodiments, obtaining a degradation parameter prediction result output by a degradation parameter prediction model includes: using the degradation parameter prediction model to obtain a first feature vector corresponding to a first feature image and a second feature vector corresponding to a second feature image; using the degradation parameter prediction model to splice the first feature vector and the second feature vector to obtain a spliced ​​feature vector, the spliced ​​feature vector including difference information between the first feature vector and the second feature vector, the difference information including the size of a predicted motion blur parameter, the direction of the predicted motion blur parameter, and noise information; passing the spliced ​​feature vector through multiple fully connected layers in sequence to obtain the number of predicted degradation parameters; and obtaining a degradation parameter prediction result based on the difference information and the number of predicted degradation parameters.

[0086] The degradation parameters added in the first sample image mainly include two degradation factors: motion blur parameter and noise information. The degradation process can be expressed by formula (4):

[0087] g(x,y)=h(x,y)*f(x,y)+η(x,y) (4)

[0088] Among them, f(x, y) is the clear image, that is, the first sample image; g(x, y) is the degraded image, that is, the second sample image; h(x, y) is the motion blur parameter; η(x, y) is the noise information.

[0089] Motion blur is caused by the relative motion between the drone shooting equipment and the background and target during exposure. Due to the relative motion, the image of the object point goes through multiple pixels in the imaging plane during the exposure time. The drone flies at a high altitude. In a short exposure time, the blur caused by the ground target movement is much smaller than the blur caused by the drone movement. Therefore, this study mainly considers the motion blur caused by the drone movement. In a short exposure time, the motion blur caused by the drone movement can be simplified to uniform linear motion blur, and its point spread function can be expressed by formula (5):

[0090]

[0091] Among them, d is the number of pixels in the image reflected by the relative motion between the drone and the target, also known as the size of the blur kernel in the convolution operation; θ is the angle between the drone's motion direction and the x-axis when it is reflected in the image. From the formula, we can see that the two parameters of blur kernel size d and blur angle θ can be used to describe linear motion blur.

[0092] Image noise is diverse and difficult to predict, but in practical applications, such as transmission channel interference, electronic system noise, and noise caused by digital image processing can all be regarded as additive noise. Therefore, a zero-mean additive white Gaussian noise model is used to simulate camera sensor noise, and the standard deviation σ is used to describe noise information.

[0093] In some embodiments, Figure 3-Figure 5 is an example of image degradation, where Figure 3 For clear images and target objects identified based on clear images, Figure 4 For Figure 3 Based on the above, we introduce an image with linear motion blur of scale d=11 and θ=45° and the target object identified on the image. Figure 5 For Figure 4 Based on the above, an image with a noise standard deviation of σ = 10 / 255 is introduced and the target object is identified on the image. It can be seen that under the influence of motion blur and noise, the grayscale, texture and other features of the image are seriously distorted, making it difficult to extract discriminative features for target detection and recognition.

[0094] In practical applications, the blur kernel is obtained by interpolation of formula (5), and it is difficult to directly predict all the values ​​in the blur kernel. Therefore, for motion blur, this section does not directly predict the specific blur kernel, but uses the degradation parameter prediction model to predict the size d and direction θ of motion blur. For noise, the degradation parameter prediction model is used to predict the standard deviation σ of noise. Since the prediction parameters contain directional information, this section designs a degradation parameter prediction model that embeds directional information to further extract the directional information of the input feature map.

[0095] In some embodiments, the structure of the degradation parameter prediction model is as follows: Figure 6As shown, the first sample image and the second sample image are respectively extracted through the weight-sharing backbone network to obtain the first feature image F1 and the second feature image F2, both of which are of size C×H×W. In order to retain the directional information in the feature map, the global maximum pooling operations in the X direction and the Y direction are used to extract the directional information in the feature maps F1 and F2, and then two feature vectors of size (C×(H+W))×1×1 are obtained by flattening and connecting them respectively, and then the two are connected to obtain a feature vector of size (2×C×(H+W))×1×1. In this way, the difference information in the two F1 and F2 is compared, and then three fully connected layers are sequentially passed to obtain a feature vector of size N p The output of ×1×1 is the predicted degradation parameter and the predicted degradation parameter, where N p is the number of degenerate parameters. ReLU is used as the activation function between the three fully connected layers.

[0096] In some embodiments, the first loss function value is calculated based on the degradation parameter and the degradation parameter prediction result, as shown in formula (6):

[0097]

[0098] Among them, L dpp is the first loss function value, N p To predict the number of degradation parameters, p i is the true value corresponding to the i-th degradation parameter, is the predicted value of the i-th degradation parameter.

[0099] In order to illustrate the target detection method of the embodiment of the present application, the embodiment of the present application also provides a flow chart of the target detection method, such as Figure 7 As shown, the method comprises the following steps:

[0100] S701, inputting the image to be recognized into the second network model, where the second network model is obtained by the above training method.

[0101] S702: Determine, using the second network model, an equivariant representation feature image of a feature image corresponding to the image to be identified before degradation.

[0102] S703: Using the second network model, identify the target object in the image to be identified in the equivariant representation feature image.

[0103] Among them, degradation equivariance means that the degradation of the entire image is equivalent to the degradation of the image's internal visual structure. Degradation equivariance representation uses the encoded features of the original image and the transformed image to predict the conversion relationship between the two, so that the network can learn this feature, thereby reducing the impact of image degradation on other visual tasks such as image classification. Typical algorithms based on this method include MAET and RestoreDet.

[0104] In some embodiments, after extracting the feature image of the image to be identified, the second network model determines an equivariant representation feature image of the feature image before degradation based on the learned equivariant representation, and further identifies the target object in the image to be identified from the equivariant representation feature image.

[0105] The present application also provides a model training device, which is used to execute the model training method provided in any of the above embodiments. Figure 8 As shown, the device includes: an input module 801, an acquisition module 802, a calculation module 803 and an adjustment module 804.

[0106] An input module 801, used to input a first sample image into a first network model and a second sample image into a second network model, wherein the second sample image is generated by adding a degradation parameter to the first sample image;

[0107] The input module 801 is used to input the first feature image output by the first network model and the second feature image output by the second network model into a degradation parameter prediction model, wherein the degradation parameters include: motion blur parameters and noise information;

[0108] An acquisition module 802 is used to acquire a degradation parameter prediction result output by the degradation parameter prediction model;

[0109] A calculation module 803, configured to calculate a first loss function value based on the degradation parameter and the degradation parameter prediction result;

[0110] The adjustment module 804 is used to adjust the model parameters of the second network model based on the first loss function value, and continue training until a preset training completion condition is met to obtain a trained second network model.

[0111] The embodiment of the present application provides a model training device, inputting a first sample image into a first network model and a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image; inputting a first feature image output by the first network model and a second feature image output by the second network model into a degradation parameter prediction model; obtaining a degradation parameter prediction result output by the degradation parameter prediction model; calculating a first loss function value based on the degradation parameter and the degradation parameter prediction result; adjusting the model parameters of the second network model based on the first loss function value, and continuing training until a preset training completion condition is met to obtain a trained second network model. The embodiment of the present application calculates the degradation parameter loss function, and optimizes the network parameters based on the loss function, so that it can recognize discriminative features under different degradation degrees and improve detection accuracy.

[0112] In some embodiments, the above device further includes: a calculation module 803.

[0113] An acquisition module 802 is used to acquire a first label result and a first prediction result of the first network model and a second label result and a second prediction result of the second network model;

[0114] A calculation module 803, configured to calculate a second loss function value based on the first label result and the first prediction result;

[0115] A third loss function value is calculated based on the second label result and the second prediction result.

[0116] In some embodiments, the adjustment module 804 is specifically configured to:

[0117] The model parameters of the second network model are adjusted based on the first loss function value, the second loss function value and the third loss function value, and the training is continued until the preset training completion condition is met to obtain a trained second network model.

[0118] In some embodiments, the adjustment module 804 is further specifically configured to:

[0119] Determine the loss weights corresponding to the first loss function value, the second loss function value, and the third loss function value respectively;

[0120] Determine a total loss function value of the second network model based on the first loss function value, the second loss function value, the third loss function value, and their corresponding loss weights;

[0121] The model parameters of the second network model are adjusted based on the total loss function value, and the training is continued until a preset training completion condition is met to obtain a trained second network model.

[0122] In some embodiments, the input module 801 is specifically used to:

[0123] The first feature image output by the first preset downsampling unit in the first network model and the second feature image output by the second preset downsampling unit in the second network model are input into the degradation parameter prediction model, the image sizes corresponding to the first feature image and the second feature image are consistent, the first preset downsampling unit exists in the first backbone network in the first network model, and the second preset downsampling unit exists in the second backbone network in the second network model.

[0124] In some embodiments, the acquisition module 802 is specifically configured to:

[0125] Acquire a first feature vector corresponding to the first feature image and a second feature vector corresponding to the second feature image by using the degradation parameter prediction model;

[0126] Using the degradation parameter prediction model, the first feature vector and the second feature vector are spliced ​​to obtain a spliced ​​feature vector, wherein the spliced ​​feature vector includes difference information between the first feature vector and the second feature vector, and the difference information includes a size of a predicted motion blur parameter, a direction of the predicted motion blur parameter, and noise information;

[0127] Passing the concatenated feature vector through multiple fully connected layers in sequence to obtain the number of predicted degradation parameters;

[0128] A degradation parameter prediction result is obtained based on the difference information and the number of predicted degradation parameters.

[0129] In some embodiments, the calculation module 803 is specifically configured to:

[0130]

[0131] Among them, L dpp is the first loss function value, N p is the number of predicted degradation parameters, p i is the true value corresponding to the i-th degradation parameter, is the predicted value of the i-th degradation parameter.

[0132] The present application also provides a target detection device, which is used to execute the target detection method provided in any of the above embodiments. Fig. 9 As shown, the device includes: an input module 901, a determination module 902 and an identification module 902.

[0133] An input module 901 is used to input an image to be recognized into a second network model, wherein the second network model is obtained by the training method described in the first aspect;

[0134] A determination module 802 is used to determine an equivariant representation feature image of the feature image corresponding to the image to be identified before degradation using the second network model;

[0135] The recognition module 903 is used to recognize the target object in the image to be recognized in the equivariant representation feature image by using the second network model.

[0136] The present application also provides an electronic device to perform the above model training method. Fig.10 It shows a schematic diagram of an electronic device provided by some embodiments of the present application. Fig.10As shown, the electronic device 7 includes: a processor 700, a memory 701, a bus 702 and a communication interface 703, and the processor 700, the communication interface 703 and the memory 701 are connected via the bus 702; the memory 701 stores a computer program that can be run on the processor 700, and when the processor 700 runs the computer program, it executes the model training method or target detection method provided in any of the aforementioned embodiments of the present application.

[0137] The memory 701 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the device network element and at least one other network element is realized through at least one communication interface 703 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0138] The bus 702 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. Among them, the memory 701 is used to store programs, and the processor 700 executes the program after receiving the execution instruction. The model training method or target monitoring method disclosed in any implementation of the aforementioned embodiment of the present application can be applied to the processor 700, or implemented by the processor 700.

[0139] The processor 700 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 700. The above processor 700 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 701, and the processor 700 reads the information in the memory 701 and completes the steps of the above method in combination with its hardware.

[0140] The electronic device provided in the embodiment of the present application and the model training method or target detection method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0141] The present application also provides a computer-readable storage medium corresponding to the model training method provided in the above embodiment. Fig.11 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, it executes the model training method or target detection method provided by any of the aforementioned embodiments.

[0142] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0143] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the model training method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0144] It should be noted that:

[0145] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0146] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0147] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A model training method, characterized in that: include: Inputting a first sample image into a first network model and inputting a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image, and the degradation parameters include: motion blur parameters and noise information; Inputting a first feature image output by the first network model and a second feature image output by the second network model into a degradation parameter prediction module; Obtaining a degradation parameter prediction result output by the degradation parameter prediction module; Calculating a first loss function value based on the degradation parameter and the degradation parameter prediction result; The model parameters of the second network model are adjusted based on the first loss function value, and the training is continued until a preset training completion condition is met to obtain a trained second network model.

2. The method according to claim 1, characterized in that: The method further includes adjusting the model parameters of the second network model based on the first loss function value, and continuing the training until a preset training completion condition is met to obtain the trained second network model: Obtain a first label result and a first prediction result of the first network model and a second label result and a second prediction result of the second network model; Calculate a second loss function value based on the first label result and the first prediction result; A third loss function value is calculated based on the second label result and the second prediction result.

3. The method according to claim 2, characterized in that The adjusting the model parameters of the second network model based on the first loss function value, continuing the training until a preset training completion condition is met, and obtaining a trained second network model, includes: The model parameters of the second network model are adjusted based on the first loss function value, the second loss function value and the third loss function value, and the training is continued until the preset training completion condition is met to obtain a trained second network model.

4. The method according to claim 3, characterized in that The adjusting the model parameters of the second network model based on the first loss function value, the second loss function value and the third loss function value, continuing the training until a preset training completion condition is met, and obtaining a trained second network model, includes: Determine the loss weights corresponding to the first loss function value, the second loss function value, and the third loss function value respectively; Determine a total loss function value of the second network model based on the first loss function value, the second loss function value, the third loss function value, and their corresponding loss weights; The model parameters of the second network model are adjusted based on the total loss function value, and the training is continued until a preset training completion condition is met to obtain a trained second network model.

5. The method according to claim 1, characterized in that: The step of inputting the first feature image output by the first network model and the second feature image output by the second network model into a degradation parameter prediction module comprises: A first feature image output by a first preset downsampling unit in the first network model and a second feature image output by a second preset downsampling unit in the second network model are input into a degradation parameter prediction module, the image sizes corresponding to the first feature image and the second feature image are consistent, the first preset downsampling unit exists in a first backbone network in the first network model, and the second preset downsampling unit exists in a second backbone network in the second network model.

6. The method according to claim 1, characterized in that The obtaining the degradation parameter prediction result output by the degradation parameter prediction module includes: Using the degradation parameter prediction module to obtain a first feature vector corresponding to the first feature image and a second feature vector corresponding to the second feature image; Using the degradation parameter prediction module to splice the first feature vector and the second feature vector to obtain a spliced ​​feature vector, wherein the spliced ​​feature vector includes difference information between the first feature vector and the second feature vector, and the difference information includes a size of a predicted motion blur parameter, a direction of the predicted motion blur parameter, and noise information; Passing the concatenated feature vector through multiple fully connected layers in sequence to obtain the number of predicted degradation parameters; A degradation parameter prediction result is obtained based on the difference information and the number of predicted degradation parameters.

7. The method according to claim 6, characterized in that The calculating the first loss function value based on the degradation parameter and the degradation parameter prediction result includes: Among them, L dpp is the first loss function value, N p is the number of predicted degradation parameters, p i is the true value corresponding to the i-th degradation parameter, is the predicted value of the i-th degradation parameter.

8. A target detection method, characterized in that: include: Inputting the image to be recognized into a second network model, wherein the second network model is obtained by the training method according to any one of claims 1 to 7; Determine, by using the second network model, an equivariant representation feature image of the feature image corresponding to the image to be identified before degradation; The target object in the image to be identified is identified in the equivariant representation feature image using the second network model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor runs the computer program to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to any one of claims 1 to 8.

Images