Target detection model establishment method and device

Abstract

The embodiment of the invention provides a target detection model establishment method and device. The method comprises the following steps: training a target detection large neural network by adopting a marked constant light image training set to obtain a target detection large model; a target detection large model is adopted to add pseudo labels to the dark light images, and a first dark light image training set is obtained; splicing the constant light image and the dark light image into a DAN image, inputting the DAN image into a large target detection model and a small target detection neural network at the same time, and calculating the current loss of the small target detection neural network according to predicted target information output by the large target detection model and the small target detection neural network and a label of the DAN image, and using the current loss to adjust parameters of the target detection small neural network until the target detection small neural network converges, and using the convergent target detection small neural network as a final target detection small model. According to the embodiment of the invention, the manpower and hardware cost of target detection is reduced.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a target detection model establishment method and device. Background Art

[0002] With the development of science and technology, artificial intelligence has been applied in many aspects of life, such as face recognition, vehicle detection, image super-resolution, etc. Object detection is one of the most widely used fields of artificial intelligence. Face detection, human shape detection, vehicle detection and other applications can be seen everywhere in life.

[0003] Although existing target detection methods have achieved good results in daily applications, the detection effect is often greatly reduced under low-light conditions at night. Since low-light data is complex and difficult to collect, most of the data used to train target detection models is data under normal light scenes. This makes the detection accuracy of the target detection model greatly reduced when facing more complex scenes, especially low-brightness overexposure under low-light conditions, because the transfer of data domains is inevitable in these scenes.

[0004] In practical applications, in order to improve the target detection effect of the target detection model under low-light conditions, more low-light data can be collected to expand the training data set, or a larger neural network structure can be used, or the image input to the neural network can be enhanced. However, the above methods either require greater manpower costs to collect and annotate data, or require greater hardware costs to run a larger network to train data to improve the model's capabilities. The latter is unacceptable for many edge devices. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for establishing a target detection model to reduce the manpower cost and hardware cost of target detection.

[0006] The technical solution of the embodiment of the present invention is achieved as follows:

[0007] A method for establishing a target detection model, the method comprising:

[0008] A. Obtain a normal light image training set including multiple normal light image frames, wherein each normal light image frame is added with a true label, and the true label is used to indicate the true category and true position of each target in the normal light image; obtain a dark light image set, wherein the dark light image set includes multiple dark light image frames, and each dark light image frame is not added with a label;

[0009] B. Use the normal light image training set to train the large neural network for target detection and obtain the large model for target detection;

[0010] C. Input each dark-light image in the dark-light image set into the target detection large model to obtain the predicted category and predicted position of each target in each dark-light image, add the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and put each dark-light image with the added pseudo-label into the first dark-light image training set;

[0011] D. sequentially read a frame of normal light image from the normal light image training set, randomly read a preset first number of frames of images from the normal light image training set and the first dark light image training set, and splice all the read images into a frame of day and night DAN image;

[0012] E. Input the DAN image into the target detection large model and the target detection small neural network at the same time, calculate the current loss of the target detection small neural network according to the predicted target information output by the target detection large model and the target detection small neural network respectively and the label of the DAN image, and use the current loss to adjust the parameters of the target detection small neural network; wherein the label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image;

[0013] F. Return to step D until the target detection small neural network converges, and use the converged target detection small neural network as the final target detection small model.

[0014] After randomly reading a preset first number of frames of images from the normal light image training set and the first dark light image training set in step D and before stitching all the read images into one frame of DAN image in step D, the method further includes:

[0015] Perform image enhancement on each read image;

[0016] The step of stitching all the read images into a frame of DAN image includes:

[0017] All the images after image enhancement are stitched into one DAN image.

[0018] Step E calculates the current loss of the small neural network for target detection, including:

[0019] According to the predicted target information output by the small target detection neural network and the label of the DAN image, the loss of the small target detection neural network on the DAN image is calculated;

[0020] According to the predicted target information output by the large target detection model and the predicted target information output by the small target detection neural network, the distillation loss from the large target detection model to the small target detection neural network is calculated;

[0021] The loss of the small target detection neural network for the DAN image is added to the distillation loss of the large target detection model to the small target detection neural network to obtain the current loss of the small target detection neural network.

[0022] Step E calculates the distillation loss of the target detection large model to the target detection small neural network, including:

[0023] According to the predicted foreground feature map output by the large target detection model and the predicted foreground feature map output by the small target detection neural network, the foreground distillation loss from the large target detection model to the small target detection neural network is calculated;

[0024] Calculate the target category distillation loss from the target detection large model to the target detection small neural network based on the predicted target category feature map output by the target detection large model and the predicted target category feature map output by the target detection small neural network.

[0025] Calculate the target position distillation loss from the target detection large model to the target detection small neural network based on the predicted target position feature map output by the target detection large model and the predicted target position feature map output by the target detection small neural network;

[0026] The foreground distillation loss, target category distillation loss, and target position distillation loss from the large target detection model to the small target detection neural network are added together to obtain the distillation loss from the large target detection model to the small target detection neural network.

[0027] The step B of obtaining the target detection large model is as follows: obtaining the initial large model of target detection;

[0028] Between step B and step C, further comprising:

[0029] B11, inputting each dark-light image in the dark-light image set into the initial large model for target detection, obtaining the predicted category and predicted position of each target in each dark-light image, adding the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and putting each dark-light image with the added pseudo-label into the second dark-light image training set;

[0030] B12, sequentially read a frame of normal light image from the normal light image training set, randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, and splice all the read images into a frame of DAN image; wherein the label of each frame of DAN image is composed of the true labels of each normal light image in the frame of DAN image, or is composed of the true labels of each normal light image in the frame of DAN image and the pseudo labels of each dark light image;

[0031] B13, input the DAN image into the target detection initial large model, calculate the current loss of the target detection initial large model according to the predicted target information output by the target detection initial large model and the label of the DAN image, and use the current loss to adjust the parameters of the target detection initial large model;

[0032] B14. Return to step B12 until the initial large model for target detection converges, and use the converged initial large model for target detection as the target detection large model for subsequent use.

[0033] The step B of obtaining the target detection large model is as follows: obtaining the initial large model of target detection;

[0034] Between step B and step C, further comprising:

[0035] B21, inputting each dark-light image in the dark-light image set into the initial large model for target detection, obtaining the predicted category and predicted position of each target in each dark-light image, adding the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and putting each dark-light image with the added pseudo-label into the second dark-light image training set;

[0036] B22. According to the preset proportion of normal light images in the current epoch and the selection rule of normal light images and DAN images under this proportion, determine whether the normal light image or the DAN image should be selected at present. If it is a normal light image, read one frame of normal light image from the normal light image training set in sequence, and input the normal light image into the initial large model of target detection; if it is a DAN image, read one frame of normal light image from the normal light image training set in sequence, and randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, splice all the read images into one frame of DAN image, and input the DAN image into the initial large model of target detection. The label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image;

[0037] B23. Calculate the current loss of the target detection initial large model according to the predicted target information output by the target detection initial large model and the label of the normal light image or DAN image input to the target detection initial large model. Determine the parameter frozen convolution layer of the current epoch according to the preset proportion of the parameter frozen convolution layer in the current epoch and the selection rule of the parameter frozen convolution layer at the proportion. Use the current loss to adjust the parameters of all batch sample normalization BN layers and parameter non-frozen convolution layers of the target detection initial large model. The epoch of the entire training process of the target detection initial large model is divided into a plurality of sequentially tangent ranges in advance. The proportion of the normal light image in the epoch in the closer range is to the front is smaller, and the proportion of the parameter frozen convolution layer is larger.

[0038] B24. Return to step B22 until the initial large model for target detection converges, and use the converged initial large model for target detection as the target detection large model for subsequent use.

[0039] After randomly reading a preset second number of frames of images from the normal light image training set and the second dark light image training set in step B22 and before stitching all the read images into one frame of DAN image in step B23, the method further includes:

[0040] Perform image enhancement on each read image;

[0041] The step of stitching all the read images into a frame of DAN image includes:

[0042] All the images after image enhancement are stitched into one DAN image.

[0043] A target detection model building device, the device comprising:

[0044] A training preparation module is used to obtain a normal light image training set including multiple normal light image frames, wherein each normal light image frame is added with a true label, and the true label is used to represent the true category and true position of each target in the normal light image; and obtain a dark light image set, wherein the dark light image set includes multiple dark light image frames, and each dark light image frame is not added with a label;

[0045] The target detection large model training module is used to train the target detection large neural network using the normal light image training set to obtain the target detection large model;

[0046] a dark-light image annotation module, for inputting each dark-light image in the dark-light image set into the target detection large model, obtaining the predicted category and predicted position of each target in each dark-light image, adding the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and putting each dark-light image with the pseudo-label added into the first dark-light image training set;

[0047] The target detection small model training module is used to sequentially read a frame of normal light image from the normal light image training set, randomly read a preset first number of frames of images from the normal light image training set and the first dark light image training set, and splice all the read images into a frame of day and night DAN image; input the DAN image into the target detection large model and the target detection small neural network at the same time, calculate the current loss of the target detection small neural network according to the predicted target information output by the target detection large model and the target detection small neural network respectively and the label of the DAN image, and use the current loss to adjust the parameters of the target detection small neural network; wherein the label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image; return to the action of sequentially reading a frame of normal light image from the normal light image training set until the target detection small neural network converges, and use the converged target detection small neural network as the target detection small model finally used.

[0048] The target detection small model training module calculates the current loss of the target detection small neural network, including:

[0049] According to the predicted target information output by the small target detection neural network and the label of the DAN image, the loss of the small target detection neural network on the DAN image is calculated;

[0050] According to the predicted target information output by the large target detection model and the predicted target information output by the small target detection neural network, the distillation loss from the large target detection model to the small target detection neural network is calculated;

[0051] The loss of the small target detection neural network for the DAN image is added to the distillation loss of the large target detection model to the small target detection neural network to obtain the current loss of the small target detection neural network.

[0052] The target detection large model training module obtains the target detection large model by: obtaining an initial large model of target detection;

[0053] After the target detection large model training module obtains the target detection large model, it further includes:

[0054] B11, inputting each dark-light image in the dark-light image set into the initial large model for target detection, obtaining the predicted category and predicted position of each target in each dark-light image, adding the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and putting each dark-light image with the added pseudo-label into the second dark-light image training set;

[0055] B12, sequentially read a frame of normal light image from the normal light image training set, randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, and splice all the read images into a frame of DAN image; wherein the label of each frame of DAN image is composed of the true labels of each normal light image in the frame of DAN image, or is composed of the true labels of each normal light image in the frame of DAN image and the pseudo labels of each dark light image;

[0056] B13, input the DAN image into the target detection initial large model, calculate the current loss of the target detection initial large model according to the predicted target information output by the target detection initial large model and the label of the DAN image, and use the current loss to adjust the parameters of the target detection initial large model;

[0057] B14. Return to step B12 until the initial large model for target detection converges, and use the converged initial large model for target detection as the target detection large model for subsequent use.

[0058] The target detection large model training module obtains the target detection large model by: obtaining an initial large model of target detection;

[0059] After the target detection large model training module obtains the target detection large model, it further includes:

[0060] B21, inputting each dark-light image in the dark-light image set into the initial large model for target detection, obtaining the predicted category and predicted position of each target in each dark-light image, adding the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and putting each dark-light image with the added pseudo-label into the second dark-light image training set;

[0061] B22. According to the preset proportion of normal light images in the current epoch and the selection rule of normal light images and DAN images under this proportion, determine whether the normal light image or the DAN image should be selected at present. If it is a normal light image, read one frame of normal light image from the normal light image training set in sequence, and input the normal light image into the initial large model of target detection; if it is a DAN image, read one frame of normal light image from the normal light image training set in sequence, and randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, splice all the read images into one frame of DAN image, and input the DAN image into the initial large model of target detection. The label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image;

[0062] B23. Calculate the current loss of the target detection initial large model according to the predicted target information output by the target detection initial large model and the label of the normal light image or DAN image input to the target detection initial large model. Determine the parameter frozen convolution layer of the current epoch according to the preset proportion of the parameter frozen convolution layer in the current epoch and the selection rule of the parameter frozen convolution layer at the proportion. Use the current loss to adjust the parameters of all batch sample normalization BN layers and parameter non-frozen convolution layers of the target detection initial large model. The epoch of the entire training process of the target detection initial large model is divided into a plurality of sequentially tangent ranges in advance. The proportion of the normal light image in the epoch in the closer range is to the front is smaller, and the proportion of the parameter frozen convolution layer is larger.

[0063] B24. Return to step B22 until the initial large model for target detection converges, and use the converged initial large model for target detection as the target detection large model for subsequent use.

[0064] In an embodiment of the present invention, a large target detection model is first obtained by training a labeled normal light image training set, and then the large target detection model is used to perform target prediction on the dark light image set to obtain a target pseudo label for each frame of the dark light image, and then the normal light image and the dark light image are spliced ​​to obtain a DAN image, and the DAN image is used to train a small target detection model, and at the same time, the large target detection model is used to distill the target detection result of the DAN image to the small target detection model, thereby: there is no need to manually label the dark light image, and a target detection model with excellent target detection effects for both normal light and dark light images can be obtained, saving labor costs; and, the DAN image is used to train the small target detection model, so that the model fully perceives the distribution of data in each domain to prevent overfitting, and the unsupervised and model distillation methods are combined to allow the small target detection model to learn the detection ability of the large target detection model, so that the small target detection model is close to the target detection ability of the large target detection model, which greatly improves the target detection effect under dark light conditions, and because the small target detection model has a small number of parameters and a small amount of calculation, it is very suitable for running on edge devices, reducing the hardware cost of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A flow chart of a method for establishing a target detection model provided by an embodiment of the present invention;

[0066] Figure 2 A flow chart of a method for establishing a target detection model provided by another embodiment of the present invention;

[0067] Figure 3 A flow chart of a method for establishing a target detection model provided by another embodiment of the present invention;

[0068] Figure 4A schematic diagram of the structure of a target detection model building device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0070] The inventor analyzed the existing target detection methods and found that although labeled dark light data requires a lot of manpower, unlabeled data can be obtained relatively easily. Based on this consideration, the inventor thought that unsupervised technology can be used to combine the existing labeled data set and unlabeled data in dark light complex scenes to train a large target detection model, and then use the large model to generate pseudo labels for the unlabeled data. At the same time, the results of the large model are used to distill the small target detection model, so that the small model is closer to the target detection ability of the large model, so as to greatly improve the target detection effect in dark light.

[0071] Figure 1 The following is a flow chart of a method for establishing a target detection model provided by an embodiment of the present invention, wherein the specific steps are as follows:

[0072] Step 101: obtaining a normal-light image training set comprising a plurality of normal-light image frames, wherein each normal-light image frame is added with a true label, and the true label is used to represent the true category and true position of each target in the normal-light image; obtaining a dark-light image set, wherein the dark-light image set comprises a plurality of dark-light image frames, and each dark-light image frame is not added with a label.

[0073] Generally, a normal light image refers to an image captured under a lighting condition with an illumination of not less than 1 lux, and a dark light image refers to an image captured under a lighting condition with an illumination of less than 1 lux.

[0074] Step 102: Use the normal light image training set to train the large neural network for target detection to obtain a large target detection model.

[0075] Step 103: Input each dark-light image in the dark-light image set into the target detection large model to obtain the predicted category and predicted position of each target in each dark-light image, add the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and put each dark-light image with the added pseudo-label into the dark-light image training set.

[0076] Step 104: first read a frame of normal light image from the normal light image training set in sequence, then randomly read a preset first number of frames of images from the normal light image training set and the dark light image training set, and splice all the read images into a frame of DAN (Day And Night) image; wherein the first number is not less than 2.

[0077] Among them, if the first number is 3, then in this step, first read a frame of normal light image in sequence from the normal light image training set, and then randomly read a preset first number of frames of images from the normal light image training set and the dark light image training set, that is, first read a frame of normal light image in sequence from the normal light image training set, and then randomly read 3 frames of images from the normal light image training set and the dark light image training set. These 3 frames of images may all be normal light images, or may all be dark light images, or may be 1 frame of normal light image + 2 frames of dark light images or 2 frames of normal light images + 1 frame of dark light image, then a total of 4 frames of images are read, and these 4 frames of images are spliced ​​into one frame of DAN image, that is, the DAN image may be composed entirely of normal light images, or may be composed of normal light images (1, 2 or 3 frames) + dark light images (3, 2 or 1 frame).

[0078] In an optional embodiment, in this step, after randomly reading a preset first number of frames of images from the normal light image training set and the dark light image training set and before stitching all the read images into a frame of DAN image, further comprising: performing image enhancement on each of the read images; and stitching all the read images into a frame of DAN image, including: stitching all the enhanced images into a frame of DAN image. The image enhancement method may adopt existing mature technology.

[0079] After all the read images are spliced ​​into a frame of DAN image, if the size of the DAN image is inconsistent with the size of the input image of the target detection small neural network, the size of the DAN image needs to be scaled to be consistent with the size of the input image of the target detection small neural network.

[0080] Step 105: Input the DAN image into the target detection large model and the target detection small neural network at the same time, calculate the current loss of the target detection small neural network according to the predicted target information output by the target detection large model and the target detection small neural network respectively and the label of the DAN image, and use the current loss to adjust the parameters of the target detection small neural network; wherein the label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image.

[0081] Among them, the computational amount and parameter amount of the small neural network for target detection are less than those of the large neural network for target detection. In practical applications, the structures of the small neural network for target detection and the large neural network for target detection can be set as needed. For example, the small neural network for target detection can use the YOLO (You Only Look Once) XS (Small) network, and the large neural network for target detection can use the YOLOX-M (Medium) network.

[0082] In an optional embodiment, in step 105, the current loss of the target detection small neural network is calculated, including: calculating the loss of the target detection small neural network to the DAN image based on the predicted target information output by the target detection small neural network and the label of the DAN image currently input to the target detection small neural network; calculating the distillation loss from the target detection large model to the target detection small neural network based on the predicted target information output by the target detection large model and the predicted target information output by the target detection small neural network; adding the loss of the target detection small neural network to the DAN image and the distillation loss from the target detection large model to the target detection small neural network, and the sum obtained is the current loss of the target detection small neural network.

[0083] Among them, the loss of the small neural network for target detection on the DAN image can be calculated using the following calculation formula:

[0084] l yolox = l obj (o gt ,o)+l cls (c gt ,c)+l bb (b gt ,b)

[0085] Among them, l yolox Loss of small neural network for DAN images for object detection;

[0086] l obj (o gt ,o) is the foreground loss of the target detection small neural network for the DAN image, which can be calculated using the cross entropy loss function;

[0087] o gt Represents the foreground feature map corresponding to the true label of the DAN image, o gt Each point in corresponds to a certain area in the DAN image, such as: gt If a point in is not 0, it indicates that the DAN image area corresponding to the point is the foreground. gt If a point in is 0, it indicates that the DAN image area corresponding to the point is the background;

[0088] o is the predicted foreground feature map, that is, the foreground feature map output by the small neural network for target detection. Similarly, each point in the predicted foreground feature map corresponds to a certain area in the DAN image, and the value of each point is used to indicate whether the corresponding area is the foreground or the background;

[0089] l cls (c gt ,c) is the target category loss of the small neural network for target detection on the DAN image, which can be calculated using the cross entropy loss function;

[0090] c gt It is the target category feature map indicated by the true label. Each point in the target category feature map corresponds to a certain area in the DAN image, and the value of each point is used to represent the category of the target contained in its corresponding area.

[0091] c is the predicted target category feature map, that is, the target category feature map output by the target detection small neural network;

[0092] l bb (b gt ,b) is the target position loss of the small neural network for target detection on the DAN image, which can be calculated using the intersection-over-union loss function;

[0093] b gt The target position feature map indicated by the true label. Each point in the target position feature map corresponds to a certain area in the DAN image, and the value of each point is used to represent the position of the target contained in its corresponding area.

[0094] b is the predicted target position feature map, that is, the target position feature map output by the small target detection neural network.

[0095] In an optional embodiment, the distillation loss from the target detection large model to the target detection small neural network is calculated, including: calculating the foreground distillation loss from the target detection large model to the target detection small neural network according to the predicted foreground feature map output by the target detection large model and the predicted foreground feature map output by the target detection small neural network; calculating the target category distillation loss from the target detection large model to the target detection small neural network according to the predicted target category feature map output by the target detection large model and the predicted target category feature map output by the target detection small neural network; calculating the target position distillation loss from the target detection large model to the target detection small neural network according to the predicted target position feature map output by the target detection large model and the predicted target position feature map output by the target detection small neural network; adding the foreground distillation loss, target category distillation loss, and target position distillation loss from the target detection large model to the target detection small neural network, and the sum obtained is the distillation loss from the target detection large model to the target detection small neural network.

[0096] In an optional embodiment, the distillation loss of the target detection large model to the target detection small neural network can be calculated using the following calculation formula:

[0097] l distll = l l2 (obj t ,obj)+l l2 (cls t ,cls)+l l2 (Bbox t ,Bbox)

[0098] Among them, l distll Distillation loss from large object detection model to small neural network for object detection;

[0099] l l2 (obj t ,obj) is the foreground distillation loss from the target detection large model to the target detection small neural network, which can be calculated using the L2 loss function;

[0100] obj t The predicted foreground feature map output by the target detection model;

[0101] obj is the predicted foreground feature map output by the small neural network for target detection;

[0102] l l2 (cls t ,cls) is the target category distillation loss from the target detection large model to the target detection small neural network, which can be calculated using the L2 loss function;

[0103] cls t The predicted target category feature map output by the target detection model;

[0104] cls is the predicted target category feature map output by the small target detection neural network;

[0105] l l2 (Bbox t ,Bbox) is the target position distillation loss from the target detection large model to the target detection small neural network, which can be calculated using the L2 loss function;

[0106] Bbox t The predicted target position feature map output by the target detection model;

[0107] Bbox is the predicted target position feature map output by the small neural network for target detection.

[0108] Among them, l l2 (obj t ,obj),l l2 (cls t ,cls),l l2 (Bbox t ,Bbox) can be calculated as follows:

[0109]

[0110] in, Represents the foreground distillation loss l from the large target detection model to the small target detection neural network l2 (obj t,obj) or target category distillation loss l l2 (cls t ,cls) or target position distillation loss l l2 (Bbox t ,Bbox);

[0111] y represents the predicted foreground feature obj output by the target detection model t map or predicted target category feature map cls t Or predict the target position feature map Bbox t ;

[0112] Represents the predicted foreground feature map obj or the predicted target category feature map cls or the predicted target position feature map Bbox output by the small neural network for target detection;

[0113] n is y, The size of y, The sizes of y are the same, for example, the width of y is W and the height is H, then n = W*H;

[0114] y i , They are y, The i-th point in , i is y i , In y, The serial number in .

[0115] Step 106: Return to step 104 until the target detection small neural network converges, and use the converged target detection small neural network as the final target detection small model.

[0116] After obtaining the target detection model, the normal light or dark light image to be detected can be input into the target detection model to obtain the predicted target information, which includes: predicted target category and target position.

[0117] In the above embodiment, a large target detection model is first obtained by training a labeled normal light image training set, and then the large target detection model is used to predict the target of the dark light image set to obtain a target pseudo label of each dark light image frame, and then the normal light image and the dark light image are spliced ​​to obtain a DAN image, and the DAN image is used to train a small target detection model, and at the same time, the large target detection model is used to distill the target detection result of the DAN image into the small target detection model, so that:

[0118] First, without manually labeling dark-light images, a target detection model with excellent target detection effects for both normal-light and dark-light images can be obtained, saving labor costs;

[0119] Second, the DAN image is used to train the small target detection model, so that the model can fully perceive the distribution of data in each domain and prevent overfitting. The unsupervised and model distillation methods are combined to allow the small target detection model to learn the detection ability of the large target detection model, so that the target detection ability of the small target detection model is close to that of the large target detection model, which greatly improves the target detection effect under dark light conditions. In addition, since the parameter amount and computational complexity of the small target detection model are relatively small, it is very suitable for running on edge devices, reducing the hardware cost of target detection.

[0120] Figure 2 A flow chart of a method for establishing a target detection model provided by another embodiment of the present invention, wherein the specific steps are as follows:

[0121] Step 201: Obtain a normal light image training set comprising multiple frames of normal light images, wherein each frame of the normal light image is added with a true label, and the true label is used to represent the true category and true position of each target in the normal light image; obtain a dark light image set, wherein the dark light image set comprises multiple frames of dark light images, and no label is added to each frame of the dark light image.

[0122] Step 202: Use the normal light image training set to train the target detection large neural network to obtain an initial large model for target detection.

[0123] Step 203: Input each dark-light image in the dark-light image set into the initial large model for target detection to obtain the predicted category and predicted position of each target in each dark-light image, add the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and put each dark-light image with the added pseudo-label into the first dark-light image training set.

[0124] Step 204: first read a frame of normal light image from the normal light image training set in sequence, then randomly read a preset first number of frames of images from the normal light image training set and the first dark light image training set, and splice all the read images into a frame of DAN image; wherein the first number is not less than 2; wherein the label of each frame of DAN image is composed of true labels of each normal light image in the frame of DAN image, or is composed of true labels of each normal light image in the frame of DAN image and pseudo labels of each dark light image.

[0125] Among them, if the first number is 3, then in this step, one frame of normal light image is first read in sequence from the normal light image training set, and then a preset first number of frames of images are randomly read from the normal light image training set and the first dark light image training set, that is, one frame of normal light image is first read in sequence from the normal light image training set, and then 3 frames of images are randomly read from the normal light image training set and the first dark light image training set. These 3 frames of images may all be normal light images, or may all be dark light images, or may be 1 frame of normal light image + 2 frames of dark light images or 2 frames of normal light images + 1 frame of dark light image, then a total of 4 frames of images are read, and these 4 frames of images are spliced ​​into one frame of DAN image, that is, the DAN image may be composed entirely of normal light images, or may be composed of normal light images (1, 2 or 3 frames) + dark light images (3, 2 or 1 frame).

[0126] For a frame of DAN image, if the frame of DAN image is composed entirely of normal light images, then the label of the frame of DAN image is composed of the true labels of each normal light image in the frame of DAN image; if the frame of DAN image is composed of normal light image + dark light image, then the label of the frame of DAN image is composed of the true labels of each normal light image in the frame of DAN image and the pseudo labels of each dark light image.

[0127] In an optional embodiment, in this step, after randomly reading a preset second number of frames of images from the normal light image training set and the first dark light image training set and before stitching all the read images into a frame of DAN image, further comprising: performing image enhancement on each of the read images; and stitching all the read images into a frame of DAN image, including: stitching all the enhanced images into a frame of DAN image. The image enhancement method may adopt existing mature technology.

[0128] After all the read images are spliced ​​into a frame of DAN image, if the size of the DAN image is inconsistent with the size of the input image of the target detection small neural network, the size of the DAN image needs to be scaled to be consistent with the size of the input image of the target detection small neural network.

[0129] Step 205: Input the DAN image into the target detection initial large model, calculate the current loss of the target detection initial large model according to the predicted target information output by the target detection initial large model and the label of the DAN image, and use the current loss to adjust the parameters of the target detection initial large model.

[0130] Step 206: Return to step 204 until the initial large model for target detection converges, and use the converged initial large model for target detection as the large model for target detection used in subsequent steps.

[0131] Step 207: Input each dark-light image in the dark-light image set into the target detection large model to obtain the predicted category and predicted position of each target in each dark-light image, add the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and put each dark-light image with the added pseudo-label into the second dark-light image training set.

[0132] Step 208: first read a frame of normal light image from the normal light image training set in sequence, then randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, and splice all the read images into a frame of DAN image; wherein the second number is not less than 2.

[0133] Among them, if the second number is 3, then in this step, first read a frame of normal light image in sequence from the normal light image training set, and then randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, that is, first read a frame of normal light image in sequence from the normal light image training set, and then randomly read 3 frames of images from the normal light image training set and the second dark light image training set. These 3 frames of images may all be normal light images, or may all be dark light images, or may be 1 frame of normal light image + 2 frames of dark light images or 2 frames of normal light images + 1 frame of dark light image, then a total of 4 frames of images are read, and these 4 frames of images are spliced ​​into one frame of DAN image, that is, the DAN image may be composed entirely of normal light images, or may be composed of normal light images (1, 2 or 3 frames) + dark light images (3, 2 or 1 frame).

[0134] In an optional embodiment, in this step, after randomly reading a preset second number of frames of images from the normal light image training set and the second dark light image training set and before stitching all the read images into a frame of DAN image, further comprising: performing image enhancement on each of the read images; and stitching all the read images into a frame of DAN image, including: stitching all the enhanced images into a frame of DAN image. The image enhancement method may adopt existing mature technology.

[0135] After all the read images are spliced ​​into a frame of DAN image, if the size of the DAN image is inconsistent with the size of the input image of the target detection large model or / and the target detection small neural network, the size of the DAN image needs to be scaled to be consistent with the size of the input image of the target detection large model or / and the target detection small neural network.

[0136] Step 209: Input the DAN image into the target detection large model and the target detection small neural network at the same time, calculate the current loss of the target detection small neural network according to the predicted target information output by the target detection large model and the target detection small neural network respectively and the label of the DAN image, and use the current loss to adjust the parameters of the target detection small neural network; wherein the label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image.

[0137] Among them, the computational amount and parameter amount of the small neural network for target detection are less than the computational amount and parameter amount of the large neural network for target detection. In practical applications, the structures of the small neural network for target detection and the large neural network for target detection can be set as needed. For example, the small neural network for target detection can use the YOLOX-S network, and the large neural network for target detection can use the YOLOX-M network.

[0138] Step 210: Return to step 208 until the target detection small neural network converges, and use the converged target detection small neural network as the final target detection small model.

[0139] In the above embodiment, the initial large target detection model is fine-tuned by using the DAN image, so that the large target detection model can adapt to different data distributions, thereby improving the robustness of the large target detection model, thereby ultimately improving the robustness of the small target detection model.

[0140] Figure 3 A flow chart of a method for establishing a target detection model provided by another embodiment of the present invention, wherein the specific steps are as follows:

[0141] Step 301: Obtain a normal light image training set comprising multiple frames of normal light images, wherein each frame of the normal light image is added with a true label, and the true label is used to represent the true category and true position of each target in the normal light image; obtain a dark light image set, wherein the dark light image set comprises multiple frames of dark light images, and no label is added to each frame of the dark light image.

[0142] Step 302: Use the normal light image training set to train the target detection large neural network to obtain an initial large model for target detection.

[0143] Step 303: Input each dark-light image in the dark-light image set into the initial large model for target detection to obtain the predicted category and predicted position of each target in each dark-light image, add the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and put each dark-light image with the added pseudo-label into the first dark-light image training set.

[0144] Step 304: According to the preset proportion of normal light images in the current epoch (round) and the selection rule of normal light images and DAN images under the proportion, determine whether the normal light image or the DAN image should be selected currently; if it is a normal light image, read a frame of normal light image from the normal light image training set in sequence, and input the normal light image into the initial large model of target detection; if it is a DAN image, read a frame of normal light image from the normal light image training set in sequence, and randomly read a preset first number of frames of images from the normal light image training set and the dark light image training set, splice all the read images into a frame of DAN image, and input the DAN image into the initial large model of target detection, wherein the first number is not less than 2, and the label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image.

[0145] When the step of "reading one frame of normal light image from the normal light image training set in sequence" in this step has been completed and all normal light images in the normal light image training set have been read and trained once, one epoch is completed.

[0146] In an optional embodiment, in this step, after randomly reading a preset first number of frames of images from the normal light image training set and the first dark light image training set and before stitching all the read images into a frame of DAN image, further comprising: performing image enhancement on each of the read images; and stitching all the read images into a frame of DAN image, including: stitching all the enhanced images into a frame of DAN image. The image enhancement method may adopt existing mature technology.

[0147] After all the read images are spliced ​​into a frame of DAN image, if the size of the DAN image is inconsistent with the size of the input image of the target detection small neural network, the size of the DAN image needs to be scaled to be consistent with the size of the input image of the target detection small neural network.

[0148] Step 305: Calculate the current loss of the target detection initial large model according to the predicted target information output by the target detection initial large model and the label of the normal light image or DAN image input to the target detection initial large model, determine the parameter frozen convolution layer of the current epoch according to the preset ratio of the parameter frozen convolution layer in the current epoch and the selection rule of the parameter frozen convolution layer at the ratio, and use the current loss to adjust the parameters of all BN (Batch Normalization) layers and parameter non-frozen convolution layers of the target detection initial large model; wherein, the epochs of the entire training process of the target detection initial large model are pre-divided into multiple sequentially tangent ranges, and the proportion of the normal light image in the epoch in the closer the range is to the front is smaller, and the proportion of the parameter frozen convolution layer is larger.

[0149] Among them, the epochs of the entire training process of the initial large model of target detection are pre-divided into multiple sequentially tangent ranges. The proportion of normal light images in the epochs in the earlier ranges is smaller, and the proportion of parameter frozen convolutional layers is larger. This is to reduce the impact of incorrect pseudo-labels on the accuracy and precision of the model during the fine-tuning of the initial large model of target detection. For example: the epochs of the entire training process of the initial large model of target detection are divided into the following 5 ranges: the first 10 epochs, the 11th-20th epochs, the 21st-50th epochs, the 51st-100th epochs, and the 101st and later epochs; the proportion of DAN images and the proportion of parameter frozen convolutional layers corresponding to each range are as follows:

[0150] 1) The first 10 epochs: DAN image ratio: 10%. At this time, after selecting 9 frames of normal light images for training, 1 frame of DAN image can be selected for training. The parameter freezing convolution layer ratio: 100%, that is, all convolution layers only perform calculations during training, but do not update parameters. The parameters always maintain the initial parameters of the initial large model of target detection;

[0151] 2) Epoch 11-20: DAN image ratio: 20%. At this time, after selecting 4 frames of normal light images for training, 1 frame of DAN image can be selected for training. The parameter-frozen convolution layer ratio: 80%, that is, all convolution layers perform calculations during training, but only 20% of the convolution layers perform parameter updates.

[0152] 3) Epoch 21-50: DAN image ratio: 30%. At this time, after selecting 7 frames of normal light images for training, 3 frames of DAN images can be selected for training. The parameter-frozen convolutional layer ratio: 60%, that is, all convolutional layers are calculated during the training process, but only 40% of the convolutional layers are updated with parameters.

[0153] 4) Epoch 51-100: DAN image ratio: 40%. At this time, after selecting 3 frames of normal light images for training, 2 frames of DAN images can be selected for training. The parameter-frozen convolution layer ratio: 50%, that is, all convolution layers perform calculations during training, but only 50% of the convolution layers perform parameter updates.

[0154] 5) After the 101th epoch: DAN image ratio: 50%. At this time, after selecting 1 frame of normal light image for training, another frame of DAN image can be selected for training. The parameter freezing convolution layer ratio: 0%, that is, all convolution layers perform calculations during the training process, and update the parameters of all convolution layers.

[0155] Step 306: Return to step 304 until the initial large model for target detection converges, and use the converged initial large model for target detection as the large model for target detection used in subsequent steps.

[0156] Step 307: Input each dark-light image in the dark-light image set into the target detection large model to obtain the predicted category and predicted position of each target in each dark-light image, add the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and put each dark-light image with the added pseudo-label into the second dark-light image training set.

[0157] Step 308: first read a frame of normal light image from the normal light image training set in sequence, then randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, and splice all the read images into a frame of DAN image; wherein the second number is not less than 2.

[0158] In an optional embodiment, in this step, after randomly reading a preset second number of frames of images from the normal light image training set and the second dark light image training set and before stitching all the read images into a frame of DAN image, further comprising: performing image enhancement on each of the read images; and stitching all the read images into a frame of DAN image, including: stitching all the enhanced images into a frame of DAN image. The image enhancement method may adopt existing mature technology.

[0159] After all the read images are spliced ​​into a frame of DAN image, if the size of the DAN image is inconsistent with the size of the input image of the target detection large model or / and the target detection small neural network, the size of the DAN image needs to be scaled to be consistent with the size of the input image of the target detection large model or / and the target detection small neural network.

[0160] Step 309: Input the DAN image into the target detection large model and the target detection small neural network at the same time, calculate the current loss of the target detection small neural network according to the predicted target information output by the target detection large model and the target detection small neural network and the label of the DAN image, and use the current loss to adjust the parameters of the target detection small neural network; wherein the label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image.

[0161] Among them, the computational amount and parameter amount of the small neural network for target detection are less than the computational amount and parameter amount of the large neural network for target detection. In practical applications, the structures of the small neural network for target detection and the large neural network for target detection can be set as needed. For example, the small neural network for target detection can use the YOLOX-S network, and the large neural network for target detection can use the YOLOX-M network.

[0162] Step 310: Return to step 308 until the target detection small neural network converges, and use the converged target detection small neural network as the final target detection small model.

[0163] In the above embodiment, the initial large target detection model is progressively fine-tuned by using the DAN image, so that the large target detection model can gradually adapt to different data distributions, thereby further improving the robustness of the large target detection model and further improving the robustness of the small target detection model.

[0164] The target detection model establishment method provided by the embodiment of the present invention is applicable to any scenario requiring target detection, for example, a scenario requiring 24-hour uninterrupted target detection, where the targets are such as people and / or animals and / or other moving objects such as vehicles.

[0165] Figure 4 The schematic diagram of the structure of the target detection model establishment device provided by the embodiment of the present invention mainly includes: a training preparation module 41, a target detection large model training module 42, a dark light image annotation module 43 and a target detection small model training module 44, wherein:

[0166] The training preparation module 41 is used to obtain a normal light image training set comprising multiple frames of normal light images, wherein each frame of the normal light image is added with a true label, and the true label is used to represent the true category and true position of each target in the normal light image; and obtain a dark light image set, which comprises multiple frames of dark light images, and no label is added to each frame of the dark light image.

[0167] The target detection large model training module 42 is used to train the target detection large neural network using the normal light image training set to obtain the target detection large model.

[0168] The dark-light image annotation module 43 is used to input each dark-light image in the dark-light image set into the target detection large model, obtain the predicted category and predicted position of each target in each dark-light image, add the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and put each dark-light image with the added pseudo-label into the first dark-light image training set.

[0169] The target detection small model training module 44 is used to sequentially read a frame of normal light image from the normal light image training set, randomly read a preset first number of frames of images from the normal light image training set and the first dark light image training set, and splice all the read images into a frame of day and night DAN image; wherein the first number is not less than 2; input the DAN image into the target detection large model and the target detection small neural network at the same time, calculate the current loss of the target detection small neural network according to the predicted target information output by the target detection large model and the target detection small neural network respectively and the label of the DAN image, and use the current loss to adjust the parameters of the target detection small neural network; wherein the label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image; return to the action of sequentially reading a frame of normal light image from the normal light image training set until the target detection small neural network converges, and use the converged target detection small neural network as the target detection small model finally used.

[0170] In an optional embodiment, after randomly reading a preset first number of frames of images from the normal light image training set and the first dark light image training set and before splicing all the read images into a frame of DAN image, the target detection small model training module 44 further includes: performing image enhancement on each of the read images;

[0171] The target detection small model training module 44 stitches all the read images into one frame of DAN image, including: stitching all the enhanced images into one frame of DAN image.

[0172] In an optional embodiment, the target detection small model training module 44 calculates the current loss of the target detection small neural network, including: calculating the loss of the target detection small neural network to the DAN image based on the predicted target information output by the target detection small neural network and the label of the DAN image; calculating the distillation loss from the target detection large model to the target detection small neural network based on the predicted target information output by the target detection large model and the predicted target information output by the target detection small neural network; adding the loss of the target detection small neural network to the DAN image and the distillation loss from the target detection large model to the target detection small neural network to obtain the current loss of the target detection small neural network.

[0173] In an optional embodiment, the target detection small model training module 44 calculates the distillation loss from the target detection large model to the target detection small neural network, including: calculating the foreground distillation loss from the target detection large model to the target detection small neural network according to the predicted foreground feature map output by the target detection large model and the predicted foreground feature map output by the target detection small neural network; calculating the target category distillation loss from the target detection large model to the target detection small neural network according to the predicted target category feature map output by the target detection large model and the predicted target category feature map output by the target detection small neural network; calculating the target position distillation loss from the target detection large model to the target detection small neural network according to the predicted target position feature map output by the target detection large model and the predicted target position feature map output by the target detection small neural network; adding the foreground distillation loss, target category distillation loss, and target position distillation loss from the target detection large model to the target detection small neural network to obtain the distillation loss from the target detection large model to the target detection small neural network.

[0174] In an optional embodiment, the target detection small model training module 44 obtains the target detection large model by: obtaining an initial large target detection model;

[0175] After the target detection large model training module 44 obtains the target detection large model, it further includes:

[0176] B11, inputting each dark-light image in the dark-light image set into the initial large model for target detection, obtaining the predicted category and predicted position of each target in each dark-light image, adding the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and putting each dark-light image with the added pseudo-label into the second dark-light image training set;

[0177] B12. sequentially read a frame of normal light image from the normal light image training set, randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, and splice all the read images into a frame of DAN image; wherein the second number is not less than 2; wherein the label of each frame of DAN image is composed of the true labels of each normal light image in the frame of DAN image, or is composed of the true labels of each normal light image in the frame of DAN image and the pseudo labels of each dark light image;

[0178] B13, input the DAN image into the target detection initial large model, calculate the current loss of the target detection initial large model according to the predicted target information output by the target detection initial large model and the label of the DAN image, and use the current loss to adjust the parameters of the target detection initial large model;

[0179] B14. Return to step B12 until the initial large model for target detection converges, and use the converged initial large model for target detection as the target detection large model for subsequent use.

[0180] In an optional embodiment, the target detection small model training module 44 obtains the target detection large model by: obtaining an initial large target detection model;

[0181] After the target detection large model training module 44 obtains the target detection large model, it further includes:

[0182] B21, inputting each dark-light image in the dark-light image set into the initial large model for target detection, obtaining the predicted category and predicted position of each target in each dark-light image, adding the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and putting each dark-light image with the added pseudo-label into the second dark-light image training set;

[0183] B22. According to the preset proportion of normal light images in the current round epoch and the selection rule of normal light images and DAN images under this proportion, determine whether the normal light image or the DAN image should be selected at present. If it is a normal light image, read one frame of normal light image from the normal light image training set in sequence, and input the normal light image into the initial large model of target detection; if it is a DAN image, read one frame of normal light image from the normal light image training set in sequence, and randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, splice all the read images into one frame of DAN image, and input the DAN image into the initial large model of target detection, wherein the second number is not less than 2, and the label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image;

[0184] B23. Calculate the current loss of the target detection initial large model according to the predicted target information output by the target detection initial large model and the label of the normal light image or DAN image input to the target detection initial large model. Determine the parameter frozen convolution layer of the current epoch according to the preset proportion of the parameter frozen convolution layer in the current epoch and the selection rule of the parameter frozen convolution layer at the proportion. Use the current loss to adjust the parameters of all batch sample normalization BN layers and parameter non-frozen convolution layers of the target detection initial large model. The epoch of the entire training process of the target detection initial large model is divided into a plurality of sequentially tangent ranges in advance. The proportion of the normal light image in the epoch in the closer range is to the front is smaller, and the proportion of the parameter frozen convolution layer is larger.

[0185] B24. Return to step B22 until the initial large model for target detection converges, and use the converged initial large model for target detection as the target detection large model for subsequent use.

[0186] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for establishing a target detection model, characterized in that: The method includes: A. Obtain a normal light image training set including multiple normal light image frames, wherein each normal light image frame is added with a true label, and the true label is used to indicate the true category and true position of each target in the normal light image; obtain a dark light image set, wherein the dark light image set includes multiple dark light image frames, and each dark light image frame is not added with a label; B. Use the normal light image training set to train the large neural network for target detection and obtain the large model for target detection; C. Input each dark-light image in the dark-light image set into the target detection large model to obtain the predicted category and predicted position of each target in each dark-light image, add the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and put each dark-light image with the added pseudo-label into the first dark-light image training set; D. sequentially read a frame of normal light image from the normal light image training set, randomly read a preset first number of frames of images from the normal light image training set and the first dark light image training set, and splice all the read images into a frame of day and night DAN image; E. Input the DAN image into the target detection large model and the target detection small neural network at the same time, calculate the current loss of the target detection small neural network according to the predicted target information output by the target detection large model and the target detection small neural network respectively and the label of the DAN image, and use the current loss to adjust the parameters of the target detection small neural network; wherein the label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image; F. Return to step D until the target detection small neural network converges, and use the converged target detection small neural network as the final target detection small model.

2. The method according to claim 1, characterized in that After randomly reading a preset first number of frames of images from the normal light image training set and the first dark light image training set in step D and before stitching all the read images into one frame of DAN image in step D, the method further includes: Perform image enhancement on each read image; The step of stitching all the read images into a frame of DAN image includes: All the images after image enhancement are stitched into one DAN image.

3. The method according to claim 1, characterized in that Step E calculates the current loss of the small neural network for target detection, including: According to the predicted target information output by the small target detection neural network and the label of the DAN image, the loss of the small target detection neural network on the DAN image is calculated; According to the predicted target information output by the large target detection model and the predicted target information output by the small target detection neural network, the distillation loss from the large target detection model to the small target detection neural network is calculated; The loss of the small target detection neural network for the DAN image is added to the distillation loss of the large target detection model to the small target detection neural network to obtain the current loss of the small target detection neural network.

4. The method according to claim 3, characterized in that Step E calculates the distillation loss of the target detection large model to the target detection small neural network, including: According to the predicted foreground feature map output by the large target detection model and the predicted foreground feature map output by the small target detection neural network, the foreground distillation loss from the large target detection model to the small target detection neural network is calculated; Calculate the target category distillation loss from the target detection large model to the target detection small neural network based on the predicted target category feature map output by the target detection large model and the predicted target category feature map output by the target detection small neural network. Calculate the target position distillation loss from the target detection large model to the target detection small neural network based on the predicted target position feature map output by the target detection large model and the predicted target position feature map output by the target detection small neural network; The foreground distillation loss, target category distillation loss, and target position distillation loss from the large target detection model to the small target detection neural network are added together to obtain the distillation loss from the large target detection model to the small target detection neural network.

5. The method according to claim 1, characterized in that: The step B of obtaining the target detection large model is as follows: obtaining the initial large model of target detection; Between step B and step C, further comprising: B11, inputting each dark-light image in the dark-light image set into the initial large model for target detection, obtaining the predicted category and predicted position of each target in each dark-light image, adding the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and putting each dark-light image with the added pseudo-label into the second dark-light image training set; B12, sequentially read a frame of normal light image from the normal light image training set, randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, and splice all the read images into a frame of DAN image; wherein the label of each frame of DAN image is composed of the true labels of each normal light image in the frame of DAN image, or is composed of the true labels of each normal light image in the frame of DAN image and the pseudo labels of each dark light image; B13, input the DAN image into the target detection initial large model, calculate the current loss of the target detection initial large model according to the predicted target information output by the target detection initial large model and the label of the DAN image, and use the current loss to adjust the parameters of the target detection initial large model; B14. Return to step B12 until the initial large model for target detection converges, and use the converged initial large model for target detection as the target detection large model for subsequent use.

6. The method according to claim 1, characterized in that The step B of obtaining the target detection large model is as follows: obtaining the initial large model of target detection; Between step B and step C, further comprising: B21, inputting each dark-light image in the dark-light image set into the initial large model for target detection, obtaining the predicted category and predicted position of each target in each dark-light image, adding the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and putting each dark-light image with the added pseudo-label into the second dark-light image training set; B22. According to the preset proportion of normal light images in the current epoch and the selection rule of normal light images and DAN images under this proportion, determine whether the normal light image or the DAN image should be selected at present. If it is a normal light image, read one frame of normal light image from the normal light image training set in sequence, and input the normal light image into the initial large model of target detection; if it is a DAN image, read one frame of normal light image from the normal light image training set in sequence, and randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, splice all the read images into one frame of DAN image, and input the DAN image into the initial large model of target detection. The label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image; B23. Calculate the current loss of the target detection initial large model according to the predicted target information output by the target detection initial large model and the label of the normal light image or DAN image input to the target detection initial large model. Determine the parameter frozen convolution layer of the current epoch according to the preset proportion of the parameter frozen convolution layer in the current epoch and the selection rule of the parameter frozen convolution layer at the proportion. Use the current loss to adjust the parameters of all batch sample normalization BN layers and parameter non-frozen convolution layers of the target detection initial large model. The epoch of the entire training process of the target detection initial large model is divided into a plurality of sequentially tangent ranges in advance. The proportion of the normal light image in the epoch in the closer range is to the front is smaller, and the proportion of the parameter frozen convolution layer is larger. B24. Return to step B22 until the initial large model for target detection converges, and use the converged initial large model for target detection as the target detection large model for subsequent use.

7. The method according to claim 6, characterized in that After randomly reading a preset second number of frames of images from the normal light image training set and the second dark light image training set in step B22 and before stitching all the read images into one frame of DAN image in step B23, the method further includes: Perform image enhancement on each read image; The step of stitching all the read images into a frame of DAN image includes: All the images after image enhancement are stitched into one DAN image.

8. A target detection model building device, characterized in that: The device includes: A training preparation module is used to obtain a normal light image training set including multiple normal light image frames, wherein each normal light image frame is added with a true label, and the true label is used to represent the true category and true position of each target in the normal light image; and obtain a dark light image set, wherein the dark light image set includes multiple dark light image frames, and each dark light image frame is not added with a label; The target detection large model training module is used to train the target detection large neural network using the normal light image training set to obtain the target detection large model; a dark-light image annotation module, for inputting each dark-light image in the dark-light image set into the target detection large model, obtaining the predicted category and predicted position of each target in each dark-light image, adding the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and putting each dark-light image with the pseudo-label added into the first dark-light image training set; The target detection small model training module is used to sequentially read a frame of normal light image from the normal light image training set, randomly read a preset first number of frames of images from the normal light image training set and the first dark light image training set, and splice all the read images into a frame of day and night DAN image; input the DAN image into the target detection large model and the target detection small neural network at the same time, calculate the current loss of the target detection small neural network according to the predicted target information output by the target detection large model and the target detection small neural network respectively and the label of the DAN image, and use the current loss to adjust the parameters of the target detection small neural network; wherein the label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image; return to the action of sequentially reading a frame of normal light image from the normal light image training set until the target detection small neural network converges, and use the converged target detection small neural network as the target detection small model finally used.

9. The device according to claim 8, characterized in that The target detection small model training module calculates the current loss of the target detection small neural network, including: According to the predicted target information output by the small target detection neural network and the label of the DAN image, the loss of the small target detection neural network on the DAN image is calculated; According to the predicted target information output by the large target detection model and the predicted target information output by the small target detection neural network, the distillation loss from the large target detection model to the small target detection neural network is calculated; The loss of the small target detection neural network for the DAN image is added to the distillation loss of the large target detection model to the small target detection neural network to obtain the current loss of the small target detection neural network.

10. The device according to claim 8, characterized in that The target detection large model training module obtains the target detection large model by: obtaining an initial target detection large model; After the target detection large model training module obtains the target detection large model, it further includes: B11, inputting each dark-light image in the dark-light image set into the initial large model for target detection, obtaining the predicted category and predicted position of each target in each dark-light image, adding the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and putting each dark-light image with the added pseudo-label into the second dark-light image training set; B12, sequentially read a frame of normal light image from the normal light image training set, randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, and splice all the read images into a frame of DAN image; wherein the label of each frame of DAN image is composed of the true labels of each normal light image in the frame of DAN image, or is composed of the true labels of each normal light image in the frame of DAN image and the pseudo labels of each dark light image; B13, input the DAN image into the target detection initial large model, calculate the current loss of the target detection initial large model according to the predicted target information output by the target detection initial large model and the label of the DAN image, and use the current loss to adjust the parameters of the target detection initial large model; B14. Return to step B12 until the initial large model for target detection converges, and use the converged initial large model for target detection as the target detection large model for subsequent use.

11. The device according to claim 8, characterized in that The target detection large model training module obtains the target detection large model by: obtaining an initial target detection large model; After the target detection large model training module obtains the target detection large model, it further includes: B21, inputting each dark-light image in the dark-light image set into the initial large model for target detection, obtaining the predicted category and predicted position of each target in each dark-light image, adding the predicted category and predicted position of each target in each frame of the dark-light image as a pseudo-label to the corresponding dark-light image, and putting each dark-light image with the added pseudo-label into the second dark-light image training set; B22. According to the preset proportion of normal light images in the current epoch and the selection rule of normal light images and DAN images under this proportion, determine whether the normal light image or the DAN image should be selected at present. If it is a normal light image, read one frame of normal light image from the normal light image training set in sequence, and input the normal light image into the initial large model of target detection; if it is a DAN image, read one frame of normal light image from the normal light image training set in sequence, and randomly read a preset second number of frames of images from the normal light image training set and the second dark light image training set, splice all the read images into one frame of DAN image, and input the DAN image into the initial large model of target detection. The label of the DAN image is composed of the true labels of each normal light image in the DAN image, or is composed of the true labels of each normal light image in the DAN image and the pseudo labels of each dark light image; B23. Calculate the current loss of the target detection initial large model according to the predicted target information output by the target detection initial large model and the label of the normal light image or DAN image input to the target detection initial large model. Determine the parameter frozen convolution layer of the current epoch according to the preset proportion of the parameter frozen convolution layer in the current epoch and the selection rule of the parameter frozen convolution layer at the proportion. Use the current loss to adjust the parameters of all batch sample normalization BN layers and parameter non-frozen convolution layers of the target detection initial large model. The epoch of the entire training process of the target detection initial large model is divided into a plurality of sequentially tangent ranges in advance. The proportion of the normal light image in the epoch in the closer range is to the front is smaller, and the proportion of the parameter frozen convolution layer is larger. B24. Return to step B22 until the initial large model for target detection converges, and use the converged initial large model for target detection as the target detection large model for subsequent use.

Images