A model training and target detection method, device and storage medium
By using a degradation parameter prediction model to adjust network parameters in object detection, the problem of low recognition accuracy caused by blurry or low-quality input images is solved, and high-precision object detection under different degradation levels is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-20
- Publication Date
- 2026-04-07
AI Technical Summary
In existing technologies, blurry or low-quality input images lead to low recognition accuracy during target detection.
By inputting the first sample image into the first network model and the second sample image (generated with degradation parameters added) into the second network model, the degradation parameter prediction results are calculated and the network parameters are adjusted until the training completion conditions are met, and the trained second network model is obtained.
It improves the accuracy of target detection and can identify discriminative features under different degrees of degradation.
Smart Images
Figure CN119559464B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of model training technology, specifically relating to a model training, object detection method, device, and storage medium. Background Technology
[0002] Object detection is a crucial task in computer vision, aiming to detect and identify the location and category of multiple objects in an image or video. Because object detection not only expands the capabilities of image processing but also enables computers to understand and process visual information more intelligently, it has wide applications in various fields.
[0003] However, if the input image is blurry during the target detection process, it may lead to the inability to identify the target in the image or result in low recognition accuracy. Summary of the Invention
[0004] This application proposes a model training method, a target detection method, a device, and a storage medium, which can solve the technical problem that in the current target detection process, the input image is relatively blurry, which may lead to the inability to identify the target in the image or the low recognition accuracy.
[0005] The first aspect of this application proposes a model training method, including:
[0006] The first sample image is input into the first network model and the second sample image is input into the second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image, and the degradation parameters include motion blur parameters and noise information;
[0007] The first feature image output by the first network model and the second feature image output by the second network model are input into the degradation parameter prediction module;
[0008] Obtain the degradation parameter prediction results output by the degradation parameter prediction module;
[0009] The first loss function value is calculated based on the degradation parameters and the prediction results of the degradation parameters;
[0010] The model parameters of the second network model are adjusted based on the first loss function value, and training continues until the preset training completion conditions are met, resulting in a well-trained second network model.
[0011] An embodiment of the second aspect of this application provides a model training apparatus, comprising:
[0012] The input module is used to input a first sample image into a first network model and a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image, and the degradation parameters include motion blur parameters and noise information;
[0013] The input module is used to input the first feature image output by the first network model and the second feature image output by the second network model into the degradation parameter prediction module;
[0014] The acquisition module is used to acquire the degradation parameter prediction results output by the degradation parameter prediction model;
[0015] The calculation module is used to calculate the value of the first loss function based on the degradation parameters and the prediction results of the degradation parameters;
[0016] The adjustment module is used to adjust the model parameters of the second network model based on the first loss function value, and continue training until the preset training completion conditions are met to obtain the trained second network model.
[0017] A third aspect of this application provides a target detection method, comprising:
[0018] The image to be recognized is input into a second network model, which is obtained by the training method described in the first aspect.
[0019] The second network model is used to determine the equivariant representation feature image of the feature image corresponding to the image to be identified before degradation;
[0020] The second network model is used to identify the target object in the image to be identified in the isovariant representation feature image.
[0021] A fourth aspect of this application provides a target detection device, comprising:
[0022] An input module is used to input the image to be recognized into a second network model, the second network model being obtained by the training method described in the first aspect;
[0023] The determination module is used to determine the equivariant representation feature image of the feature image corresponding to the image to be identified before degradation using the second network model;
[0024] The recognition module is used to identify the target object in the image to be recognized in the isomorphic representation feature image using the second network model.
[0025] An embodiment of the fifth aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first or third aspect above.
[0026] An embodiment of the sixth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method described in the first or third aspect above.
[0027] The technical solutions provided in this application embodiment have at least the following technical effects or advantages:
[0028] This application proposes a model training method, device, and storage medium, comprising: inputting a first sample image into a first network model and inputting a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image; inputting a first feature image output by the first network model and a second feature image output by the second network model into a degradation parameter prediction model; obtaining the degradation parameter prediction result output by the degradation parameter prediction model; calculating a first loss function value based on the degradation parameters and the degradation parameter prediction result; adjusting the model parameters of the second network model based on the first loss function value, and continuing training until a preset training completion condition is met to obtain a trained second network model. This application embodiment calculates a degradation parameter loss function and optimizes the network parameters based on this loss function, enabling it to identify discriminative features under different degrees of degradation, thereby improving detection accuracy.
[0029] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0030] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0031] Figure 1 A flowchart of a model training method provided in one embodiment of this application is shown;
[0032] Figure 2 This illustration shows a schematic diagram of the structure of a target network model provided in an embodiment of this application;
[0033] Figures 3-5An example diagram of image degradation provided in an embodiment of this application is shown;
[0034] Figure 6 This illustration shows a schematic diagram of the structure of a degradation parameter prediction model provided in an embodiment of this application;
[0035] Figure 7 A flowchart of a target detection method provided in an embodiment of this application is shown;
[0036] Figure 8 This illustration shows a schematic diagram of the structure of a model training device provided in one embodiment of this application;
[0037] Figure 9 This application shows a schematic diagram of the structure of a target detection device according to an embodiment of the present application.
[0038] Figure 10 This illustration shows a schematic diagram of the structure of an electronic device according to an embodiment of this application;
[0039] Figure 11 A schematic diagram of a storage medium provided in one embodiment of this application is shown. Detailed Implementation
[0040] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.
[0041] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.
[0042] The model training method described in this application can be executed by a computing device, which can be a server, such as a single server, multiple servers, a server cluster, a cloud computing platform, etc. Optionally, the computing device can also be a terminal device, such as a mobile phone, tablet computer, game console, portable computer, desktop computer, advertising machine, all-in-one machine, etc. This application does not limit the type or number of computing devices.
[0043] Building upon the aforementioned background technology, target detection algorithms for low-quality images can include indirect and direct methods. Indirect methods utilize image restoration, enhancing image quality through denoising, deblurring, and super-resolution reconstruction, thereby enriching the features extracted by the network and improving the detection accuracy of the target detection algorithm. Based on whether the image restoration and target detection processes are independent, they can be categorized into algorithms where restoration and detection are independent, and detection algorithms where restoration and detection are unified.
[0044] However, image restoration algorithms for various degradation factors are often cumbersome or have large networks, and using them as preprocessing methods will result in excessively long detection times. Image restoration itself is an ill-posed problem, and there are significant differences between human perception and machine perception. Therefore, studying restoration and detection networks independently offers limited optimization of object detection performance, and may not even guarantee an improvement in object detection performance.
[0045] The method of unifying restoration and detection networks directly connects the image restoration network and the detection network and jointly optimizes the network. This can enrich image features and enable the features to be correctly perceived by the machine, thereby improving detection accuracy. However, due to the mechanism of restoration before detection, the detection speed is greatly reduced.
[0046] The direct method improves the detection accuracy of object detection algorithms by extracting image features more accurately. This method can perform object detection independently during the testing phase, unaffected by the detection speed of other network branches. Based on different principles, current mainstream methods can be divided into detection methods based on image complex atomic networks, detection methods based on degenerate equivariant representations, and detection methods based on transfer learning.
[0047] However, complex atom networks require high-quality images for pixel-level supervision, which inevitably causes the network to learn unnecessary detailed features, thus affecting object detection.
[0048] Transfer learning-based methods can mitigate the effects of degradation by learning features from low-quality images or extracting domain-invariant features, but they perform poorly when there are large differences between domains.
[0049] To address the aforementioned problems, this application proposes a model training method, object detection method, device, and storage medium, comprising: inputting a first sample image into a first network model and inputting a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image; inputting a first feature image output by the first network model and a second feature image output by the second network model into a degradation parameter prediction model; obtaining the degradation parameter prediction result output by the degradation parameter prediction model; calculating a first loss function value based on the degradation parameters and the degradation parameter prediction result; adjusting the model parameters of the second network model based on the first loss function value, and continuing training until a preset training completion condition is met to obtain a trained second network model. This application embodiment calculates a degradation parameter loss function and optimizes the network parameters based on this loss function, enabling it to identify discriminative features under different degrees of degradation, thereby improving detection accuracy.
[0050] The following describes a model training method according to an embodiment of this application, with reference to the accompanying drawings.
[0051] See Figure 1 The method specifically includes the following steps:
[0052] S101. Input the first sample image into the first network model and the second sample image into the second network model.
[0053] The second sample image is generated by adding degradation parameters to the first sample image. The degradation parameters include motion blur parameters and noise information.
[0054] The first sample image is a high-quality image, and the second sample image is a low-quality image. The degradation parameters include motion blur parameters and noise information. The first sample image can be an image without degradation parameters, and the second sample image is generated by adding degradation parameters to the first sample image.
[0055] The first and second network models have the same model structure and can both be used to detect target objects in sample images.
[0056] S102. Input the first feature image output by the first network model and the second feature image output by the second network model into the degradation parameter prediction model.
[0057] S103. Obtain the degradation parameter prediction results output by the degradation parameter prediction model.
[0058] The first feature image does not include degradation parameters, while the second feature image does. Therefore, the degradation parameter prediction model can determine the difference information between the first feature image and the second feature image, i.e., the degradation parameter prediction result.
[0059] S104. Calculate the first loss function value based on the degradation parameters and the degradation parameter prediction results.
[0060] S105. Adjust the model parameters of the second network model based on the first loss function value, and continue training until the preset training completion conditions are met to obtain the trained second network model.
[0061] The degradation parameter is the true value of the degradation parameter prediction result. Therefore, the first loss function value can be calculated based on the difference information between the degradation parameter and the degradation parameter prediction result. The second network model can then be trained using the first loss function value, enabling it to learn the equivariant representation of features before and after degradation, extract discriminative features for detection and recognition under different degrees of degradation, and improve detection accuracy.
[0062] One of the conditions for completing training is that the number of training sessions reaches a preset threshold.
[0063] This application proposes a model training method, including: inputting a first sample image into a first network model and inputting a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image; inputting a first feature image output by the first network model and a second feature image output by the second network model into a degradation parameter prediction model; obtaining the degradation parameter prediction result output by the degradation parameter prediction model; calculating a first loss function value based on the degradation parameters and the degradation parameter prediction result; adjusting the model parameters of the second network model based on the first loss function value, and continuing training until a preset training completion condition is met to obtain a trained second network model. This application embodiment calculates a degradation parameter loss function and optimizes the network parameters based on this loss function, enabling it to identify discriminative features under different degrees of degradation and improving detection accuracy.
[0064] In some embodiments, before adjusting the model parameters of the second network model based on the first loss function value and continuing training until the preset training completion conditions are met to obtain the trained second network model, the method further includes: obtaining the first label result and the first prediction result of the first network model, and the second label result and the second prediction result of the second network model; calculating the second loss function value based on the first label result and the first prediction result; and calculating the third loss function value based on the second label result and the second prediction result.
[0065] Understandably, in addition to enabling the second network model to recognize discriminative features under different degrees of degradation, it is also necessary to improve the second network model's ability to recognize target objects. Therefore, it is necessary to calculate the second loss function value of the first network model and the third loss function value of the second network model.
[0066] In this model, the first label result is the true value of the first prediction result, and the second label result is the true value of the second prediction result. A second loss function value can be calculated based on the first label result and the first prediction result, and a third loss function value can be calculated based on the second label result and the second prediction result. The first network model is then trained using the second loss function value, and the second network model is trained using the third loss function value.
[0067] In some embodiments, the model parameters of the second network model are adjusted based on the first loss function value, and training continues until a preset training completion condition is met to obtain a trained second network model, including:
[0068] The model parameters of the second network model are adjusted based on the first, second, and third loss function values, and training continues until the preset training completion conditions are met, resulting in a well-trained second network model.
[0069] The second network model can be supervised by the first, second, and third loss function values to optimize network parameters, enabling it to learn equivariant representations of features before and after degradation, extract discriminative features for detection and recognition under different degrees of degradation, and learn to identify target objects from images, thereby improving detection accuracy.
[0070] In some embodiments, the model parameters of the second network model are adjusted based on the first loss function value, the second loss function value, and the third loss function value, and training continues until a preset training completion condition is met to obtain a trained second network model, including:
[0071] Determine the loss weights corresponding to the first, second, and third loss function values; based on the first, second, and third loss function values and their corresponding loss weights, determine the total loss function value of the second network model; adjust the model parameters of the second network model based on the total loss function value, and continue training until the preset training completion conditions are met, thus obtaining the trained second network model.
[0072] It is understandable that the total loss function consists of two parts: the object detection loss function and the degradation parameter prediction loss. The object detection loss function is obtained based on the second and third loss function values, while the degradation parameter prediction loss is the first loss function value.
[0073] In some embodiments, the total loss function is as shown in equation (1):
[0074] L total =L obj +λ dpp ·L dpp (1)
[0075] Among them, L total Let L be the total loss function. obj Let L be the target detection loss function. dpp For the loss function to predict degradation parameters, λ dpp Predict the weights of the loss function for the degradation parameters.
[0076] The target monitoring loss function is shown in equation (2):
[0077] L obj =λ obj-H ·L obj-H +λ obj-L ·L obj-L (2)
[0078] L obj-H and L obj-L These are the second and third loss functions, λ, respectively. obj-H and λ obj-L These are the weights for the second and third loss functions, respectively.
[0079] The second and third loss functions are calculated in the same way, and each includes the confidence loss L. conf Classification loss L cls and positioning loss L box The loss function consists of three parts, with the second or third loss function shown in equation (3):
[0080] L=λ conf ·L conf +λ cls ·L cls +λ box ·L box (3)
[0081] Where L is the second or third loss function, and λ conf λ is the weight of the confidence loss. cls λ is the weight of the classification loss. box The weights are used to determine the location loss.
[0082] In some embodiments, the first feature image output by the first network model and the second feature image output by the second network model are input into the degradation parameter prediction model, including:
[0083] The first feature image output by the first preset downsampling unit in the first network model and the second feature image output by the second preset downsampling unit in the second network model are input into the degradation parameter prediction model. The image sizes corresponding to the first feature image and the second feature image are the same. The first preset downsampling unit exists in the first backbone network of the first network model, and the second preset downsampling unit exists in the second backbone network of the second network model.
[0084] Both the first and second network models include a backbone network, a neck network, and a detection layer. The backbone network comprises multiple downsampling units. To obtain accurate degradation parameter prediction results, the first and second feature images input to the degradation parameter prediction model are of the same size. Motion blur and noise directly affect shallow features such as grayscale and texture of the image. The earlier the feature map is, the richer the image degradation information it retains, which is beneficial for degradation parameter prediction. On the other hand, if the feature map participates in the multi-scale feature fusion of the subsequent neck network, it is beneficial to improve the overall performance of the network. Therefore, the preset downsampling unit is generally relatively large in size, retains more image degradation information, and the feature map output by the preset downsampling unit will participate in the multi-scale feature fusion of the subsequent neck network.
[0085] In some embodiments, the model structure of the target network model is as follows: Figure 2 As shown, Figure 2 As shown, the target network model includes a first network model, a second network model, and a degradation parameter prediction model. Both the first and second network models include a backbone network, a neck network, and a detection layer. The backbone network comprises five downsampling units C1, C2, C3, C4, and C5, where C... i The downsampling factor is 2. i The downsampling unit C3 is used for degradation parameter prediction because its output feature image is relatively early in the network and retains rich information about image degradation. Furthermore, the feature image output by C3 participates in the multi-scale feature fusion of the subsequent neck network.
[0086] In some embodiments, obtaining the degradation parameter prediction result output by the degradation parameter prediction model includes: using the degradation parameter prediction model to obtain a first feature vector corresponding to a first feature image and a second feature vector corresponding to a second feature image; using the degradation parameter prediction model to concatenate the first feature vector and the second feature vector to obtain a concatenated feature vector, the concatenated feature vector including the difference information between the first feature vector and the second feature vector, the difference information including the magnitude of the predicted motion blur parameter, the direction of the predicted motion blur parameter, and noise information; passing the concatenated feature vector through multiple fully connected layers in sequence to obtain the number of predicted degradation parameters; and obtaining the degradation parameter prediction result based on the difference information and the number of predicted degradation parameters.
[0087] The degradation parameters added to the first sample image mainly include two degradation factors: motion blur parameters and noise information. The degradation process can be represented by equation (4):
[0088] g(x,y)=h(x,y)*f(x,y)+η(x,y) (4)
[0089] Where f(x,y) is the clear image, i.e. the first sample image; g(x,y) is the degraded image, i.e. the second sample image; h(x,y) is the motion blur parameter; and η(x,y) is the noise information.
[0090] Motion blur is caused by the relative motion between the UAV camera and the background and target during exposure. Due to the relative motion, the image of an object point travels through multiple pixels in the imaging plane during the exposure time. Since UAVs fly at relatively high altitudes, the blur caused by ground target motion is much smaller than the blur caused by UAV motion during a short exposure time. Therefore, this study mainly considers the motion blur caused by UAV motion. Within a short exposure time, the motion blur caused by UAV motion can be simplified to uniform linear motion blur, and its point spread function can be expressed by equation (5):
[0091]
[0092] Where d is the number of pixels in the image reflecting the relative motion between the UAV and the target, also known as the size of the blur kernel in convolution operations; θ is the angle between the UAV's motion direction reflected in the image and the x-axis. As the formula shows, linear motion blur can be described using the two parameters: the blur kernel size d and the blur angle θ.
[0093] Image noise is diverse and difficult to predict, but noise encountered in practical applications, such as transmission channel interference, electronic system noise, and noise caused by digital image processing, can all be regarded as additive noise. Therefore, a zero-mean additive white Gaussian noise model is used to simulate camera sensor noise, and the standard deviation σ is used to describe the noise information.
[0094] In some embodiments, Figures 3-5 Here is an example of image degradation, where, Figure 3 For clear images and target objects identified based on clear images, Figure 4 In order to be in Figure 3 Based on this, we introduce a linear motion-blurred image with scale d = 11 and θ = 45°, and the target object identified on this image. Figure 5 In order to be in Figure 4 Based on this, an image with noise of standard deviation σ = 10 / 255 and the target object identified on this image are introduced. It can be seen that under the influence of motion blur and noise, the grayscale, texture and other features of the image are severely distorted, making it difficult to extract discriminative features for target detection and recognition.
[0095] In practical applications, the fuzzy kernel is obtained by interpolation using equation (5), making it difficult to directly predict all values within the fuzzy kernel. Therefore, for motion fuzz, this section does not directly predict the specific fuzzy kernel, but instead uses a degenerate parameter prediction model to predict the magnitude d and direction θ of the motion fuzz. For noise, the standard deviation σ of the noise is predicted using a degenerate parameter prediction model. Since the prediction parameters contain directional information, this section designs a degenerate parameter prediction model that embeds directional information to further extract the directional information from the input feature map.
[0096] In some embodiments, the structure of the degradation parameter prediction model is as follows: Figure 6 As shown, the first and second sample images are processed by a backbone network with shared weights to extract the first feature image F1 and the second feature image F2, both of size C×H×W. To preserve the directional information in the feature maps, global max pooling operations in the X and Y directions are used to extract the directional information from feature maps F1 and F2, respectively. These are then flattened and concatenated to obtain two feature vectors of size (C×(H+W))×1×1. These two vectors are then concatenated to obtain a feature vector of size (2×C×(H+W))×1×1. The differences between F1 and F2 are then compared, and the result is passed through three fully connected layers to obtain a feature vector of size N. p The output of ×1×1 is the predicted degradation parameter and the predicted degradation parameter, where N p This represents the number of degradation parameters. ReLU is used as the activation function between the three fully connected layers.
[0097] In some embodiments, the first loss function value is calculated based on the degradation parameters and the degradation parameter prediction results, as shown in equation (6):
[0098]
[0099] Among them, L dpp N is the first loss function value. p To predict the number of degradation parameters, p i This represents the true value corresponding to the i-th degenerate parameter. Let be the predicted value of the i-th degradation parameter.
[0100] To illustrate the target detection method in this application, a flowchart of the target detection method is also provided, as shown below. Figure 7 As shown, the method includes the following steps:
[0101] S701. Input the image to be recognized into the second network model, which is obtained through the training method described above.
[0102] S702. Use the second network model to determine the equivariant representation feature image of the feature image corresponding to the image to be identified before degradation.
[0103] S703. Use the second network model to identify the target object in the image to be identified in the image with equivariant representation features.
[0104] In this context, degradation equivalence refers to the equivalence between the overall degradation of an image and the degradation of its intrinsic visual structure. Degradation equivalence representation uses the encoded features of the original and transformed images to predict the transformation relationship between them, enabling the network to learn these features and thus reduce the impact of image degradation on other visual tasks such as image classification. Typical algorithms based on this method include MAET and RestoreDet.
[0105] In some embodiments, after extracting the feature image of the image to be identified, the second network model determines the equivalent representation feature image of the feature image before degradation based on the learned equivalent representation, and further identifies the target object in the image to be identified from the equivalent representation feature image.
[0106] This application also provides a model training apparatus for executing the model training method provided in any of the above embodiments. Figure 8 As shown, the device includes: an input module 801, an acquisition module 802, a calculation module 803, and an adjustment module 804.
[0107] The input module 801 is used to input a first sample image into a first network model and a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image;
[0108] The input module 801 is used to input the first feature image output by the first network model and the second feature image output by the second network model into the degradation parameter prediction model, wherein the degradation parameters include: motion blur parameters and noise information;
[0109] The acquisition module 802 is used to acquire the degradation parameter prediction results output by the degradation parameter prediction model;
[0110] Calculation module 803 is used to calculate the first loss function value based on the degradation parameters and the prediction results of the degradation parameters;
[0111] The adjustment module 804 is used to adjust the model parameters of the second network model based on the first loss function value, and continue training until the preset training completion conditions are met to obtain the trained second network model.
[0112] This application provides a model training apparatus that inputs a first sample image into a first network model and a second sample image into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image; inputs a first feature image output by the first network model and a second feature image output by the second network model into a degradation parameter prediction model; obtains the degradation parameter prediction result output by the degradation parameter prediction model; calculates a first loss function value based on the degradation parameters and the degradation parameter prediction result; adjusts the model parameters of the second network model based on the first loss function value, and continues training until a preset training completion condition is met, thereby obtaining a trained second network model. This application embodiment optimizes network parameters by calculating a degradation parameter loss function, enabling it to identify discriminative features under different degrees of degradation, thereby improving detection accuracy.
[0113] In some embodiments, the above-described apparatus further includes a computing module 803.
[0114] The acquisition module 802 is used to acquire the first label result and the first prediction result of the first network model, and the second label result and the second prediction result of the second network model;
[0115] Calculation module 803 is used to calculate the second loss function value based on the first label result and the first prediction result;
[0116] The third loss function value is calculated based on the second label result and the second prediction result.
[0117] In some embodiments, the adjustment module 804 is specifically used for:
[0118] Based on the first loss function value, the second loss function value, and the third loss function value, adjust the model parameters of the second network model and continue training until the preset training completion conditions are met to obtain the trained second network model.
[0119] In some embodiments, the adjustment module 804 is further specifically used for:
[0120] Determine the loss weights corresponding to the first loss function value, the second loss function value, and the third loss function value;
[0121] Based on the first loss function value, the second loss function value, and the third loss function value, as well as their respective loss weights, the total loss function value of the second network model is determined;
[0122] Based on the total loss function value, adjust the model parameters of the second network model, and continue training until the preset training completion conditions are met to obtain the trained second network model.
[0123] In some embodiments, the input module 801 is specifically used for:
[0124] The first feature image output by the first preset downsampling unit in the first network model and the second feature image output by the second preset downsampling unit in the second network model are input into the degradation parameter prediction model. The image sizes corresponding to the first feature image and the second feature image are the same. The first preset downsampling unit exists in the first backbone network of the first network model, and the second preset downsampling unit exists in the second backbone network of the second network model.
[0125] In some embodiments, the acquisition module 802 is specifically used for:
[0126] The degradation parameter prediction model is used to obtain the first feature vector corresponding to the first feature image and the second feature vector corresponding to the second feature image;
[0127] The degradation parameter prediction model is used to concatenate the first feature vector and the second feature vector to obtain a concatenated feature vector. The concatenated feature vector includes the difference information between the first feature vector and the second feature vector. The difference information includes the magnitude of the predicted motion blur parameter, the direction of the predicted motion blur parameter, and noise information.
[0128] The concatenated feature vector is passed through multiple fully connected layers in sequence to obtain the number of predicted degradation parameters;
[0129] The degradation parameter prediction result is obtained based on the difference information and the number of predicted degradation parameters.
[0130] In some embodiments, the calculation module 803 is specifically used for:
[0131]
[0132] Among them, L dpp Let N be the value of the first loss function. p p is the number of predicted degradation parameters. i This represents the true value corresponding to the i-th degenerate parameter. Let be the predicted value of the i-th degradation parameter.
[0133] This application also provides a target detection device for performing the target detection method provided in any of the above embodiments. For example... Figure 9 As shown, the device includes: an input module 901, a determination module 902, and an identification module 902.
[0134] The input module 901 is used to input the image to be recognized into the second network model, which is obtained by the training method described in the first aspect.
[0135] The determination module 802 is used to determine the equivariant representation feature image of the feature image corresponding to the image to be identified before degradation using the second network model;
[0136] The recognition module 903 is used to identify the target object in the image to be recognized in the isomorphic representation feature image using the second network model.
[0137] This application also provides an electronic device for performing the above-described model training method. Please refer to... Figure 10 It illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 10 As shown, the electronic device 7 includes: a processor 700, a memory 701, a bus 702 and a communication interface 703. The processor 700, the communication interface 703 and the memory 701 are connected through the bus 702. The memory 701 stores a computer program that can run on the processor 700. When the processor 700 runs the computer program, it executes the model training method or the target detection method provided in any of the foregoing embodiments of this application.
[0138] The memory 701 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 703 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.
[0139] Bus 702 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. Memory 701 is used to store programs. After receiving execution instructions, processor 700 executes the programs. The model training method or target monitoring method disclosed in any of the foregoing embodiments of this application can be applied to processor 700, or implemented by processor 700.
[0140] The processor 700 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 700 or by instructions in software form. The processor 700 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 701. Processor 700 reads the information in memory 701 and, in conjunction with its hardware, completes the steps of the above method.
[0141] The electronic device provided in this application embodiment is based on the same inventive concept as the model training method or object detection method provided in this application embodiment, and has the same beneficial effects as the method it adopts, runs or implements.
[0142] This application also provides a computer-readable storage medium corresponding to the model training method provided in the foregoing embodiments. Please refer to... Figure 11 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the model training method or object detection method provided in any of the aforementioned embodiments.
[0143] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.
[0144] The computer-readable storage medium provided in the above embodiments of this application and the model training method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application stored therein.
[0145] It should be noted that:
[0146] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.
[0147] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
[0148] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A model training method, characterized in that, include: A first sample image is input into a first network model and a second sample image is input into a second network model, wherein the second sample image is generated by adding degradation parameters to the first sample image, and the degradation parameters include motion blur parameters and noise information; The first feature image output by the first network model and the second feature image output by the second network model are input into the degradation parameter prediction module; Obtain the degradation parameter prediction results output by the degradation parameter prediction module; The first loss function value is calculated based on the degradation parameters and the prediction results of the degradation parameters; The model parameters of the second network model are adjusted based on the first loss function value, and training continues until the preset training completion conditions are met, resulting in a well-trained second network model. The process of obtaining the degradation parameter prediction result output by the degradation parameter prediction module includes: extracting the directional information of the first feature image and the second feature image using global max pooling operations in the X and Y directions, then obtaining the first feature vector corresponding to the first feature image and the second feature vector corresponding to the second feature image through flattening and concatenation operations respectively; concatenating the first feature vector and the second feature vector using the degradation parameter prediction module to obtain a concatenated feature vector including the difference information between the first feature vector and the second feature vector; passing the concatenated feature vector through multiple fully connected layers to obtain the number of predicted degradation parameters; and obtaining the degradation parameter prediction result based on the difference information and the number of predicted degradation parameters.
2. The method according to claim 1, characterized in that, Before adjusting the model parameters of the second network model based on the first loss function value and continuing training until the preset training completion conditions are met to obtain the trained second network model, the process further includes: Obtain the first label result and the first prediction result of the first network model, and the second label result and the second prediction result of the second network model; Calculate the second loss function value based on the first label result and the first prediction result; The third loss function value is calculated based on the second label result and the second prediction result.
3. The method according to claim 2, characterized in that, The step of adjusting the model parameters of the second network model based on the first loss function value and continuing training until the preset training completion conditions are met to obtain a trained second network model includes: Based on the first loss function value, the second loss function value, and the third loss function value, adjust the model parameters of the second network model and continue training until the preset training completion conditions are met to obtain the trained second network model.
4. The method according to claim 3, characterized in that, The step of adjusting the model parameters of the second network model based on the first loss function value, the second loss function value, and the third loss function value, and continuing training until the preset training completion conditions are met to obtain a trained second network model, includes: Determine the loss weights corresponding to the first loss function value, the second loss function value, and the third loss function value; Based on the first loss function value, the second loss function value, and the third loss function value, as well as their respective loss weights, the total loss function value of the second network model is determined; Based on the total loss function value, adjust the model parameters of the second network model, and continue training until the preset training completion conditions are met to obtain the trained second network model.
5. The method according to claim 1, characterized in that, The step of inputting the first feature image output by the first network model and the second feature image output by the second network model into the degradation parameter prediction module includes: The first feature image output by the first preset downsampling unit in the first network model and the second feature image output by the second preset downsampling unit in the second network model are input to the degradation parameter prediction module. The image sizes corresponding to the first feature image and the second feature image are the same. The first preset downsampling unit exists in the first backbone network of the first network model, and the second preset downsampling unit exists in the second backbone network of the second network model.
6. The method according to claim 1, characterized in that, The calculation of the first loss function value based on the degradation parameters and the prediction results of the degradation parameters includes: Among them, L dpp Let N be the value of the first loss function. p p is the number of predicted degradation parameters. i This represents the true value corresponding to the i-th degenerate parameter. Let be the predicted value of the i-th degradation parameter.
7. A target detection method, characterized in that, include: The image to be identified is input into a second network model, which is obtained by the training method according to any one of claims 1-6; The second network model is used to determine the equivariant representation feature image of the feature image corresponding to the image to be identified before degradation; The second network model is used to identify the target object in the image to be identified in the isovariant representation feature image.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-7.