A method, apparatus, device, and storage medium for training an image target detection model.

Abstract

This invention discloses an image target detection model training method, apparatus, device, and storage medium, applied in the field of target detection. The method includes: acquiring raw data; acquiring generated data where the occlusion area reaches a preset occlusion threshold to obtain generated training data; continuously using a basic target detection model to perform inner loop training on the generated training data and raw data; acquiring mixed data to perform iterative outer loop training on the basic detection model; stopping the inner and outer loop iterative training when the Loss3 value during training with mixed data does not change within a preset number of consecutive iterations, thus obtaining the image target detection model. This invention generates a large amount of generated data using computer technology and uses a basic model to perform inner and outer loop training on the acquired raw data, generated data, and mixed data, greatly reducing data collection time costs. Furthermore, due to the inner and outer loop iterations, the obtained image target detection training model can more accurately identify target objects.

Description

Technical Field

[0001] This invention relates to the field of target detection, and in particular to a method, apparatus, device, and storage medium for training an image target detection model. Background Technology

[0002] Customers upload facial images, and the backgrounds contain a wealth of information that can be used for fraud detection. For example, if a customer applies through a black market intermediary, the backgrounds in their facial images often show a certain similarity. The similarity in backgrounds between customers can help determine the likelihood of group fraud. However, when recognizing the background of a customer's image, issues such as objects being obscured by the image or angle problems preventing objects from fully entering the frame can cause some visually identifiable objects to be missed during target detection. Furthermore, existing target detection models require a large amount of image data for detection, which affects the accuracy of target recognition when there is insufficient image data. Therefore, the technical problem this invention aims to solve is how to improve the accuracy of target detection while reducing the number of samples required for model training. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, apparatus, device and storage medium for training an image target detection model, which solves the technical problems in the prior art that a large amount of raw data needs to be collected for training the target detection model, and that the existing target detection model has low accuracy in detecting targets.

[0004] To address the aforementioned technical problems, this invention provides a method for training an image target detection model, comprising:

[0005] Acquire raw data and generate data; wherein, the generated data refers to the data obtained by fusing the image in the raw data as the target image and the occluded image;

[0006] Data with occlusion areas reaching a preset occlusion threshold are obtained from the generated data to generate training data; wherein, the occlusion area refers to the area of ​​the target image occluded by the occluded image;

[0007] The generated training data and the original data are continuously trained using the basic object detection model to obtain Loss1 corresponding to the original data and Loss2 corresponding to the generated training data. Loss2-Loss1 are then continuously calculated to obtain LossD0...LossD n ,until When the time is reached, the next inner loop is triggered; where epoch represents the number of times the basic object detection model is used to train the generated training data and the original data; by continuously increasing the value of the preset occlusion threshold, the generated training data is adjusted to perform iterative training in the next inner loop;

[0008] Continuously obtain the loss corresponding to two consecutive inner loops Dn When Loss Dn The previous round of internal circulation When the outer loop is triggered, it is used to perform iterative training with mixed data. The mixed data is composed of the original data and the generated data. The outer loop obtains a Loss3 once. When the Loss3 of two consecutive rounds is equal to the Loss3 of the previous round, the inner loop is triggered.

[0009] When Loss3 does not change within a preset number of consecutive iterations, training using the basic object detection model is stopped, and an image object detection model is obtained.

[0010] Optionally, the image target detection model training method further includes:

[0011] The first upsampling result is subjected to max pooling to obtain the max pooling result; wherein, the training process using the basic target detection model includes upsampling;

[0012] The second upsampling result is subjected to average pooling, and the average pooling result is then subjected to cubic vector transpose to obtain the transposed result.

[0013] The max pooling result and the transpose result are added together to obtain the sum.

[0014] The summation result is added to the image vector from the upsampling process to obtain a vector summation result, which is then fed into the decoding network; wherein the decoding network is included in the training process using the basic detection model.

[0015] Optionally, before obtaining the generated data, the process further includes:

[0016] The original data is randomly augmented using Cycle-Gan to obtain the generated data.

[0017] Optionally, the step of randomly augmenting the original data using Cycle-GAN to obtain the generated data includes:

[0018] The original data is processed using a random point operator to obtain synthetic image data;

[0019] The synthesized image data is subjected to convolution processing to obtain convolutional image data;

[0020] The convolutional image data is processed using a residual network to obtain residual image data;

[0021] The convolutional image data and the residual image data are added together to obtain enhanced image data, and the enhanced image data is processed by an encoder to obtain coded image data.

[0022] The encoded image data is convolved to obtain the generated data.

[0023] Optionally, calculating the loss functions for Loss1 and Loss2 includes:

[0024] Wherein, A is the actual target region corresponding to the original data and the generated data, B is the inference target region obtained using the basic training model, and S... A∪B and S A∩B Let L represent the union and intersection of the areas of the actual target region and the inferred target region, respectively; L represent the length of the actual target region and the inferred target region; H represent the width of the actual target region and the inferred target region; and gt in the formula indicates that this data comes from the inference result. Pa represents the ratio of the Euclidean distance between the inference target region and the actual target region about the center point to the line connecting the maximum outer apex angle between the inference target region and the actual target region, where Pa represents the occlusion area.

[0025] Optionally, the loss function model in the outer loop includes:

[0026] Wherein, A is the actual target region corresponding to the original data and the generated data, B is the inference target region obtained using the basic training model, and S A∪B and S A∩B Let L represent the union and intersection of the areas of the actual target region and the inferred target region, respectively; L represent the length of the actual target region and the inferred target region; H represent the width of the actual target region and the inferred target region; and gt in the formula indicates that this data comes from the inference result. This represents the ratio of the Euclidean distance between the inference target region and the actual target region about the center point to the line connecting the maximum outer apex angle between the inference target region and the actual target region.

[0027] Optionally, the method for generating the mixed data includes:

[0028] The mixed data is generated by random sampling without replacement of a preset percentage from the original data and the generated data.

[0029] The present invention also provides an image target detection model training device, comprising:

[0030] The raw data and generated data acquisition module is used to acquire raw data and generated data; wherein, the generated data refers to the data obtained by fusing the image in the raw data as the target image and the occluded image;

[0031] A training data acquisition module is used to acquire data in the generated data whose occlusion area reaches a preset occlusion threshold, and obtain the generated training data; wherein, the occlusion area refers to the area of ​​the target image occluded by the occluded image;

[0032] The inner loop module continuously trains the generated training data and the original data using the basic object detection model, obtaining Loss1 for the original data and Loss2 for the generated training data, and continuously calculates Loss2-Loss1 to obtain LossD0...LossD n ,until When the time is reached, the next inner loop is triggered; where epoch represents the number of times the basic object detection model is used to train the generated training data and the original data; by continuously increasing the value of the preset occlusion threshold, the generated training data is adjusted to perform iterative training in the next inner loop;

[0033] The outer loop module is used to continuously obtain the loss corresponding to two consecutive inner loop iterations. Dn When Loss Dn The previous round of internal circulation When the outer loop is triggered, it is used to perform iterative training with mixed data. The mixed data is composed of the original data and the generated data. The outer loop obtains a Loss3 once. When the Loss3 of two consecutive rounds is equal to the Loss3 of the previous round, the inner loop is triggered.

[0034] The image target detection model calculation module is used to stop training the basic target detection model when Loss3 does not change within a preset number of consecutive times, thereby obtaining the image target detection model.

[0035] The present invention also provides an image target detection model training device, comprising:

[0036] Memory, used to store computer programs;

[0037] A processor is used to execute the computer program to implement the steps of the image target detection model training method described above.

[0038] The present invention also provides a readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described image target detection model training method.

[0039] As can be seen, this invention obtains raw data and generated data; wherein, the generated data refers to the data obtained by fusing the image in the raw data as the target image and the occluded image; obtaining data in the generated data where the occlusion area reaches a preset occlusion threshold to obtain generated training data; wherein, the occlusion area refers to the area of ​​the target image occluded by the occluded image; continuously using a basic target detection model to train the generated training data and the raw data to obtain Loss1 corresponding to the raw data and Loss2 corresponding to the generated training data, and continuously calculating Loss2-Loss1 to obtain LossD0......LossD n ,until When the time is reached, the next inner loop is triggered; where epoch represents the number of times the basic object detection model is trained on the generated training data and the original data; by continuously increasing the value of the preset occlusion threshold, the generated training data is adjusted to perform iterative training in the next inner loop; the loss corresponding to two adjacent inner loops is continuously obtained. Dn When Loss Dn The previous round of internal circulation When the outer loop is triggered, it uses mixed data for iterative training. The mixed data is composed of the original data and the generated data. Each iteration of the outer loop yields a Loss3. When the Loss3 of two consecutive iterations equals the Loss3 of the previous iteration, the inner loop is triggered. When the Loss3 remains unchanged for a preset number of consecutive iterations, training of the basic object detection model is stopped, resulting in an image object detection model. Compared to existing object training models that require developers to collect annotations themselves and can only use original data for training, this invention can process the original data to obtain a large amount of generated data. It can use both the original and generated data for iterative training in the inner loop and the mixed data for iterative training in the outer loop. This allows training of the basic object detection model with only a small amount of original data, resulting in an image object detection training model. Because a small amount of original data can generate a large amount of generated data, and the generated data and original data can be mixed to obtain the mixed data required for the outer loop, the data collection time required for training the model is greatly reduced. Furthermore, because it is possible to use raw data and generated data for inner loop iterative training, and mixed data for outer loop iterative training, the resulting image object detection training model can more accurately identify target objects after inner and outer loop iterative training of the basic object detection model.

[0040] In addition, the present invention also provides an image target detection training model apparatus, device and storage medium, which also have the above-mentioned beneficial effects. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0042] Figure 1 A flowchart illustrating an image target detection model training method provided in an embodiment of the present invention;

[0043] Figure 2 A flowchart illustrating a global feature optimization method provided in an embodiment of the present invention;

[0044] Figure 3 A flowchart illustrating a training set derivation method provided in an embodiment of the present invention;

[0045] Figure 4 A flowchart illustrating an optimized Cycle-GAN provided for an embodiment of the present invention;

[0046] Figure 5 A flowchart illustrating an image target detection model training method provided in an embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram of the structure of an image target detection model training device provided in an embodiment of the present invention;

[0048] Figure 7 This is a schematic diagram of the structure of an image target detection model training device provided in an embodiment of the present invention;

[0049] Figure 8 This is a schematic diagram of the structure of an image target detection model training device provided in an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Please refer to Figure 1 , Figure 1 A flowchart illustrating an image target detection model training method provided in an embodiment of the present invention. The method may include:

[0052] S100, acquire raw data and generated data; wherein, generated data refers to the data obtained by fusing the image in the raw data as the target image and the occluded image.

[0053] This embodiment does not limit the specific method for obtaining the raw data. For example, the required raw data can be collected by taking photos, or the relevant raw data can be downloaded from the internet. This embodiment does not limit the method for obtaining the generated data, as long as the generated data is obtained by expanding the raw data. For example, the image in the raw data can be used as a target image and an occluded image to synthesize from different angles to obtain the generated data, or the image in the raw data can be used as a target image and multiple occluded images to synthesize from different angles to obtain the generated data. Since the generated data is obtained from the raw data, the raw data needs to be processed accordingly before obtaining the generated data. In this way, the generated data also has attributes corresponding to the raw data. Therefore, although the following content is about the processing of raw data, the generated data also has characteristic attributes corresponding to the raw data. This embodiment does not limit the tool for labeling the raw data. For example, the Python version of label-image (image labeling) can be used to label the raw data and the generated data; or the Windows version of label-image can also be used to label the raw data and the generated data. This embodiment does not limit the specific method for encoding and labeling the raw data. For example, the English name of the object + the English name of the sub-category + the total task number can be used. For example, a fine-bristled toothbrush could be assigned the numbers toothbrush-ine-00001, toothbrush-ine-00002, toothbrush-ine-00003, etc. Alternatively, the object's subcategories (in English) + English name + overall task number can be used, such as ine-toothbrush-00001, ine-toothbrush-00002, ine-toothbrush-00003, etc. The overall task number can start from 0 or 1. After determining the coding method, the raw and generated data for different categories can be created into separate files. This embodiment does not limit the specific number of raw data. For example, the number of raw data items could be 50, 60, or 70. This embodiment does not limit the method of numbering the raw data. For example, all the collected raw data can be put into a folder and then effectively encoded using MD5 (Information Digest Algorithm 5) (meaning that each photo has its own independent code, and deletion and addition will not assign the code of that photo to other photos); or unique numbers can be assigned based on timestamps, so that the code of a photo does not change when it is deleted and then added back to the training set (to prepare for subsequent annotation and merging).Regardless of whether MD5 or timestamp encoding is used, if a photo contains two or more objects that need to be identified (e.g., a toothbrush (toothbrush-ine-00001) and a thermos (cup-vacuum-00002), the photo should be placed in separate folders for the two target tasks, but the file names must remain consistent. This embodiment does not limit the method of annotating the raw data. For example, the raw data can be annotated using Label Studio (a named entity recognition annotator), or it can be annotated using LabelImg (an image annotation tool), adding object center anchor points. Then, using OpenCV (a cross-platform computer vision and machine learning software library), the centerofcontour method can be used to calculate the image center point position C(x, y) of the raw data, and the center point coordinates can be written to a txt annotation document.

[0054] S101, Obtain data from the generated data whose occlusion area reaches a preset occlusion threshold, and obtain the generated training data; where the occlusion area refers to the area of ​​the target image that is occluded by the occluded image.

[0055] This embodiment does not limit the specific value of the preset occlusion threshold. For example, the preset occlusion threshold could be 1%, or it could be 5%.

[0056] S102, continuously train the basic object detection model on the generated training data and the original data to obtain Loss1 corresponding to the original data and Loss2 corresponding to the generated training data, and continuously calculate Loss2-Loss1 to obtain LossD0...LossD n ,until When the time is reached, the next inner loop is triggered; where epoch represents the number of times the basic object detection model is used to train the generated training data and the original data; by continuously increasing the value of the preset occlusion threshold, the generated training data is adjusted to perform iterative training in the next inner loop.

[0057] This embodiment does not limit the specific model of the base detection model. For example, the base object detection model can be YoloV3, or it can be YoloV5; or it can be YoloV7. In this embodiment, the original data and the generated training data (the group with the smallest occlusion area) that reach the preset occlusion threshold are simultaneously entered into the training set. After one training iteration of the model, the loss functions Loss1 (original data) and Loss2 (generated data) corresponding to the validation set are obtained. Then, a new initial difference Loss function is obtained by subtracting Loss1 from the difference between the two loss functions (since Loss2 is generated data, the loss function is naturally larger than Loss1 for the original data).D0 At this point, the model enters the second iteration of the first inner loop. The second iteration repeats the above process to obtain the loss. Dn ,when When the preset occlusion threshold value is changed, the next round of the inner loop is triggered (the next preset occlusion threshold range is greater than the previous preset occlusion threshold range) to perform the next inner loop iteration.

[0058] S103, continuously obtain the Loss corresponding to two consecutive inner loop iterations. Dn When Loss Dn The previous round of internal circulation When the outer loop is triggered, it uses mixed data for iterative training. The mixed data is a mixture of the original data and the generated data. The outer loop obtains a Loss3 once. When the Loss3 of two consecutive rounds is equal to the Loss3 of the previous round, the inner loop is triggered.

[0059] This embodiment is not limited to Loss Dn The quantity, for example, Loss Dn It can be 3, or Loss Dn It can also be 10, or Loss Dn It can be 12. In this embodiment, multiple iterations of the inner loop are performed continuously, with each inner loop corresponding to LossD0...LossD n At this point, continuously obtain the loss corresponding to the two adjacent inner loops. Dn When Loss Dn The previous round of internal circulation When the outer loop is triggered, the mixed data used in the outer loop in this embodiment is a mixture of the original data and the generated data.

[0060] S104, when Loss3 does not change within a preset number of consecutive iterations, stop training the basic object detection model and obtain the image object detection model.

[0061] This embodiment does not limit the specific number of consecutive preset iterations. For example, the consecutive preset iterations could be 5, 6, or 8. In this embodiment, when the Loss3 corresponding to the consecutive iterations of the outer loop does not change within the consecutive preset iterations (with possible interleaving of the inner loop), the training of the basic object detection model is stopped, and the image object detection model is obtained.

[0062] Furthermore, to enhance the correlation between local and global features of the generated data, the above-mentioned image object detection model training method may also include:

[0063] The first upsampling result is subjected to max pooling to obtain the max pooling result; the training process using the basic object detection model includes upsampling.

[0064] The second upsampling result is subjected to average pooling, and the average pooling result is then subjected to cubic vector transpose to obtain the transposed result.

[0065] Add the max pooling result and the transpose result to get the sum;

[0066] The summation result is added to the image vector from the upsampling process to obtain a vector summation result, which is then fed into the decoding network; the decoding network is included in the training process using the basic detection model.

[0067] This embodiment takes into account that existing neural network structures related to object detection have weak local and global connections between features, which can easily prevent the loss function from decreasing and cause the inner and outer loops to stop. Therefore, a series of global feature optimizations are introduced in the decoding network stage of the object detection model, such as... Figure 2 As shown, this layer can optimize the receptive field of the model, enabling local or non-coherent features of the target to be better transmitted to later network layers, effectively solving the problem of the model's over-perception of local features of the target. Figure 2 This means that during the decoding process, the results of the upsampling process in the basic object detection model are passed to the subsequent decoding process through a pipeline. The first upsampling result is subjected to max pooling, and the max pooling result is passed to the next step. The second upsampling result is subjected to average pooling, and the result is transposed three times. The max pooling result and the transposed result are summed, and then the result is added to the image tensor obtained from the upsampling process. Finally, the image is passed to the decoding process.

[0068] Furthermore, in order to quickly expand the original data to obtain the generated data, the process before obtaining the generated data may also include:

[0069] The original data is randomly augmented using Cycle-Gan to obtain the generated data.

[0070] This embodiment does not limit the specific process of randomly augmenting the original data using the Cycle-GAN model, as long as the generated data can be obtained. For example, the original data for Cycle-GAN augmentation includes the original data A of the target image and the original data B including the occluded image m. The original data is occluded using the occluded image to obtain the generated data; or the original data for Cycle-GAN augmentation includes the original data B of the target image and the original data B including the occluded images m, n, and h. All occluded images are randomly encoded to obtain their silhouette material data, the corresponding silhouette coordinate data txt (text format) document, and a random operator corresponding to the target object. The image materials generated in the previous steps are used for image merging, and the merged image will be integrated into the original data B according to the surrounding environment.

[0071] For a clearer understanding of this embodiment, please refer to the specific details. Figure 3 , Figure 3 A flowchart illustrating a training set derivation method provided in this embodiment of the invention may specifically include:

[0072] The process of augmenting the original data is as follows: Original image A contains target object a, and original image B contains target object b (in this scheme, the number of B and b is not fixed and depends on the complexity of the scene with occlusions generated). B can also contain other types of target objects such as bcd, etc. The random operator module randomly encodes all obfuscated objects in B or all images from B to N and obtains their silhouette material data (4-channel RGBA format images with transparent backgrounds), the corresponding silhouette coordinate data in a txt document, and the corresponding random operator R for the target object. R(r1, r2) represents the initial position coordinates r1 of the obfuscated object's center in the image to be merged during image fusion. This coordinate position needs to fall within the coordinate contour range of a in image A. An object center anchor point is added. Using the OpenCV (software library) center of contour method (to determine the image center point), the image center point position C(x, y) is calculated and written to the txt annotation document. The target center position is confirmed, and r2 represents the rotation angle of the object; by default, no rotation is performed. The random operator module also includes a random operator validity check. When the area of ​​the obfuscated object occluding the target object 'a' is less than 95% or greater than 10% (a's labeled area - overlapping area), a new attribute, the occlusion percentage Pa, is added to the annotation file. CG represents the Cycle-GAN image synthesis model, which uses image materials generated in previous steps for image merging. The merged object will be integrated into the original image according to its surrounding environment, forming a relatively natural occlusion situation. Here, image synthesis is introduced instead of simply pasting objects directly into the image, aiming to ensure the realism of the synthesized image and reduce the interference of human factors on the model structure. Sharp and abrupt edges can easily cause a decrease in model performance and cannot be optimized at the algorithm level. After the CG process, n new images of 'a' occluded by different objects at different angles are generated. The result can obtain several times more training data than before, and it is generated data where the target is occluded by multiple objects, which is beneficial to the model's accuracy in occluded object recognition scenarios.

[0073] Furthermore, to improve the effectiveness of features when the synthetic data network structure is deep, and to prevent excessive or insufficient obfuscation in the generated data corresponding to the synthetic image, Cycle-GAN is used to randomly augment the original data to obtain the generated data, including:

[0074] The original data is processed using a random point operator to obtain synthetic image data;

[0075] The synthesized image data is convolved to obtain convolved image data;

[0076] The residual network is used to perform residual processing on the convolutional image data to obtain residual image data;

[0077] The convolutional image data and the residual image data are added together to obtain the enhanced image data, and the enhanced image data is then processed by an encoder to obtain the coded image data.

[0078] The encoded image data is convolved to obtain the generated data.

[0079] This embodiment optimizes the synthesized image data by performing residual processing on the synthesized image data. For a clearer understanding of this embodiment, please refer to [the relevant documentation / reference needed]. Figure 4 , Figure 4 This is a flowchart illustrating an optimized Cycle-GAN implementation provided by an embodiment of the present invention. Specifically, it illustrates the use of a residual network to perform residual processing on convolutional image data to obtain residual image data. After two rounds of convolution, synthetic image data is obtained. This synthetic image data is then subjected to further convolution processing to obtain convolutional image data. The residual network is then used to perform residual processing on the convolutional image data to obtain residual image data. Finally, the convolutional image data and the residual image data are added together to obtain enhanced image data. This operation effectively avoids gradient vanishing, improves feature effectiveness even with deep network structures, and prevents excessively large or small obfuscation in the synthesized image.

[0080] Furthermore, in order to quickly escape the current local optimum network structure and accelerate the attainment of the global optimum when training using the basic object detection model, the loss functions for Loss1 and Loss2 mentioned above can include:

[0081] Where A represents the actual target region corresponding to the original data and the generated data, B represents the inference target region obtained using the basic training model, and S represents the target region. A∪B and S A∩B Let L represent the union and intersection of the areas of the actual target region and the inferred target region, respectively; L represent the length of the actual target region and the inferred target region; H represent the width of the actual target region and the inferred target region; and gt in the formula indicates that the data comes from the inference result. Pa represents the ratio of the Euclidean distance between the inference target region and the actual target region about the center point to the line connecting the maximum outer apex angle between the inference target region and the actual target region, and Pa represents the occlusion area.

[0082] This embodiment does not limit the specific values ​​of L and H. For example, L can be 3cm and H can be 2cm, or L can be 5cm and H can be 4cm. This embodiment also does not limit the specific value of Pa representing the shading area. For example, the shading area can be 25%, or the shading area can be 19%, or the shading area can be 65%.

[0083] Furthermore, to simulate real-world conditions and check whether the inner loop model has reached the convergence criterion, the loss function model in the outer loop includes:

[0084] Wherein, A is the actual target region corresponding to the original data and the generated data, B is the inference target region obtained using the basic training model, and S A∪B and S A∩B Let L represent the union and intersection of the areas of the actual target region and the inferred target region, respectively; L represent the length of the actual target region and the inferred target region; H represent the width of the actual target region and the inferred target region; and gt in the formula indicates that this data comes from the inference result. This represents the ratio of the Euclidean distance between the inference target region and the actual target region about the center point to the line connecting the maximum outer apex angle between the inference target region and the actual target region.

[0085] This embodiment does not limit the specific values ​​of L and H. For example, L can be 2cm and H can be 1cm, or L can be 3cm and H can be 4cm.

[0086] Furthermore, when using mixed data for outer loop training, the data more closely resembles real-world conditions. The methods for generating this mixed data include:

[0087] Mixed data is generated by random sampling without replacement of a predetermined percentage from the original data and the generated data.

[0088] This embodiment does not limit the specific preset percentage. For example, the preset percentage can be 35%, or the preset percentage can be 40%, or the preset percentage can also be 65%.

[0089] The image target detection model training method provided in this invention includes acquiring original data and generated data; acquiring data in the generated data whose occlusion area reaches a preset occlusion threshold to obtain generated training data; wherein, the occlusion area refers to the area of ​​the target image occluded by the occluded image; continuously training the generated training data and original data using a basic target detection model to obtain Loss1 corresponding to the original data and Loss2 corresponding to the generated training data, and continuously calculating Loss2-Loss1 to obtain LossD0...LossD n ,until When the time limit is reached, the next inner loop is triggered; where epoch represents the number of times the base object detection model is trained on the generated training data and the original data; by continuously increasing the value of the preset occlusion threshold, the generated training data is adjusted to perform iterative training in the next inner loop; the loss corresponding to two adjacent inner loops is continuously obtained. DnWhen Loss Dn The previous round of internal circulation Each iteration triggers an outer loop, which uses mixed data for iterative training. This mixed data consists of a combination of original and generated data. Each iteration of the outer loop yields a Loss3 value. When the Loss3 value equals the previous Loss3 value for two consecutive iterations, the inner loop is triggered. When the Loss3 value remains unchanged for a preset number of iterations, training the basic object detection model stops, resulting in an image object detection model. Compared to existing object detection training models that require developers to collect and label original data, and can only use original data for training, this invention can process the original data to obtain a large amount of generated data. Furthermore, it allows for iterative training of the inner loop using both original and generated data, and it uses mixed data for iterative training of the outer loop. This means that only a small amount of original data is needed to train the basic object detection model, resulting in an image object detection training model. Because a small amount of original data can generate a large amount of generated data, and the generated and original data can be mixed to obtain the mixed data required for the outer loop, the data collection time for training the model is significantly reduced. Furthermore, because the model can use both raw and generated data for inner loop iterative training, and mixed data for outer loop iterative training, the resulting image object detection training model, after inner and outer loop iterative training of the basic object detection model, can more accurately identify target objects. Moreover, since max pooling can be used for global feature optimization during upsampling, the connection between local and global features in the generated data is greatly enhanced. Additionally, Cycle-GAN is used to augment the raw data, significantly increasing the amount of training data. Furthermore, the use of a random point operator for validity verification during Cycle-GAN data augmentation makes the generated data more realistic. The inclusion of occlusion area and occlusion position during the reconstruction of the inner loop loss function ensures global optimization during the inner loop, and the inclusion of occlusion position during the reconstruction of the outer loop loss function significantly improves the recognition accuracy of the resulting image object detection model.

[0090] For a clearer understanding of this invention, please refer to the following details. Figure 5 , Figure 5 This is a flowchart illustrating the image target detection model training method provided in this embodiment of the invention, which may specifically include:

[0091] like Figure 5 As shown, the image object detection model training method consists of two parts: the inner loop trains the basic object detection model using both raw and generated data, while the outer loop trains the basic object detection model using mixed data. The apparatus used for model training is as follows: Figure 6As shown, the original data is labeled using one or more computer devices. After labeling, the labeled original data is stored on a server. Then, sample derivation (generated data and mixed data) is performed on the server cluster, and iterative training is conducted using a basic object detection model. The trained image object detection model is deployed on a server with a graphics card, and object detection is performed using image equipment or directly input image data. In this image object detection model training method, in the inner loop, the original data and generated training data (one or several sets) that reach the minimum preset occlusion threshold are simultaneously input into the training set. After one training iteration of the basic object detection model, the loss functions Loss1 (original data) and Loss2 (generated data) corresponding to the validation set are obtained. The loss function used in the inner loop is... Wherein, A is the actual target region corresponding to the original data and the generated data, B is the inference target region obtained using the basic training model, and S A∪B and S A∩B Let L represent the union and intersection of the areas of the actual target region and the inferred target region, respectively; L represent the length of the actual target region and the inferred target region; H represent the width of the actual target region and the inferred target region; and gt in the formula indicates that this data comes from the inference result. This represents the ratio of the Euclidean distance between the inferred target region and the actual target region about the center point to the line connecting the maximum outer apex angle between the inferred target region and the actual target region. Then, a new initial loss function, LossD0, is obtained by subtracting Loss1 from the difference between the two loss functions (since Loss2 comes from the generated data, the loss function is naturally larger than Loss1 of the original data). At this point, iterative training is continuously performed using the basic object detection model. This iterative training process repeats the above steps, obtaining LossD0 for the first round of inner loop iterations...LossD... n ,when When the occlusion area reaches the next preset occlusion threshold range, the generated training data (the group of data with the lowest occlusion area values) and the original data are combined and fed into the iteration of the basic object detection model training. Here, epoch represents the preset number of iterations for model initialization, and this threshold indicates that the optimization step size of the loss function must meet a certain value before proceeding to the next round of inner loop iteration training. In the inner loop, because the existing neural network structure related to object detection has weak local and global connections to features, the loss function may not decrease, causing the loop to stop. Therefore, a layer of global feature optimization is introduced in the decoding network stage of the object detection model, such as... Figure 2As shown, this layer optimizes the model's receptive field, allowing local or disjoint features of the target to be better transmitted to later network layers, effectively addressing the problem of the model over-receiving local features of the target. It can significantly reduce the value of the loss function LossDn and accelerate the inner loop into the next iteration cycle. Figure 2 This means that during the decoding process, the target detection model passes the upsampling results through a pipeline to the subsequent decoding process. The first upsampling result is subjected to max pooling, and the max pooling result is passed to the next step. The second upsampling result is subjected to average pooling, and the result is transposed three times. The max pooling result and the transposed result are summed, and then the result is added to the image tensor obtained from the upsampling process. Finally, the image is passed to the decoding process.

[0092] The image object detection model training method uses a loss function LossD after multiple iterations in the inner loop using a basic object detection model. n LossD of the previous inner loop iteration n When the loss value is 1 / 2, an outer loop is triggered to train the model using mixed data, and multiple iterations are performed. Each iteration of the outer loop yields the loss function Loss3. When Loss3 equals the next Loss3 for two consecutive iterations, training begins, triggering an inner loop to continue optimizing the model. Training terminates when the loss function cannot be reduced after five consecutive iterations of both the inner and outer loops; at this point, the image object detection model is obtained.

[0093] The image target detection model training device provided in the embodiments of the present invention will be described below. The image target detection model training device described below can be referred to in correspondence with the image target detection model training method described above.

[0094] Please refer to the details. Figure 7 , Figure 7 A schematic diagram of the structure of an image target detection model training device provided in this embodiment of the invention may include:

[0095] The raw data and generated data acquisition module 100 is used to acquire raw data and generated data; wherein, the generated data refers to the data obtained by fusing the image in the raw data as the target image and the occluded image;

[0096] The training data acquisition module 200 is used to acquire data in the generated data whose occlusion area reaches a preset occlusion threshold, and obtain the generated training data; wherein, the occlusion area refers to the area of ​​the target image occluded by the occluded image;

[0097] The inner loop module 300 is used to continuously train the generated training data and the original data using the basic object detection model, to obtain Loss1 corresponding to the original data and Loss2 corresponding to the generated training data, and continuously calculates Loss2-Loss1 to obtain LossD0...LossD n ,until When the time is reached, the next inner loop is triggered; where epoch represents the number of times the basic object detection model is used to train the generated training data and the original data; by continuously increasing the value of the preset occlusion threshold, the generated training data is adjusted to perform iterative training in the next inner loop;

[0098] The outer loop module 400 is used to continuously obtain the loss corresponding to two consecutive inner loop iterations. Dn When Loss Dn The previous round of internal circulation When the outer loop is triggered, it is used to perform iterative training with mixed data. The mixed data is composed of the original data and the generated data. The outer loop obtains a Loss3 once. When the Loss3 of two consecutive rounds is equal to the Loss3 of the previous round, the inner loop is triggered.

[0099] The image target detection model calculation module 500 is used to stop training the basic target detection model when Loss3 does not change within a preset number of consecutive times, thereby obtaining the image target detection model.

[0100] Furthermore, based on the above embodiments, the image target detection model training device may further include:

[0101] The max pooling module is used to perform max pooling on the first upsampling result to obtain the max pooling result; wherein, the training process using the basic target detection model includes upsampling;

[0102] The transpose module is used to perform average pooling on the second upsampling result and then perform cubic vector transpose on the average pooling result to obtain the transpose result.

[0103] The summation unit is used to add the max pooling result and the transpose result to obtain the summation result;

[0104] The decoding unit is used to add the summation result to the image vector in the upsampling process to obtain a vector summation result, and then input it into the decoding network; wherein the decoding network is included in the training process of the basic detection model.

[0105] Furthermore, based on any of the above embodiments, the image target detection model training device may further include:

[0106] An expansion module is used to randomly expand the original data using Cycle-GAN to obtain the generated data.

[0107] Furthermore, based on any of the above embodiments, the expansion module may include:

[0108] A synthetic image data acquisition unit is used to process the original data using a random point operator to obtain synthetic image data;

[0109] A convolutional image data acquisition unit is used to perform convolution processing on the synthesized image data to obtain convolutional image data;

[0110] The residual image data acquisition unit is used to perform residual processing on the convolutional image data using a residual network to obtain residual image data;

[0111] The encoded image data acquisition unit is used to add the convolutional image data and the residual image data to obtain enhanced image data, and to process the enhanced image data using an encoder to obtain encoded image data.

[0112] A data generation unit is used to perform convolution processing on the encoded image data to obtain the generated data.

[0113] Furthermore, based on any of the above embodiments, the inner loop module 300 may include:

[0114] Inner loop loss function calculation unit, used for Wherein, A is the actual target region corresponding to the original data and the generated data, B is the inference target region obtained using the basic training model, and S... A∪B and S A∩B Let L represent the union and intersection of the areas of the actual target region and the inferred target region, respectively; L represent the length of the actual target region and the inferred target region; H represent the width of the actual target region and the inferred target region; and gt in the formula indicates that this data comes from the inference result. Pa represents the ratio of the Euclidean distance between the inference target region and the actual target region about the center point to the line connecting the maximum outer apex angle between the inference target region and the actual target region, where Pa represents the occlusion area.

[0115] Furthermore, based on any of the above embodiments, the outer circulation module 400 may include:

[0116] The outer loop loss function calculation unit is used for Wherein, A is the actual target region corresponding to the original data and the generated data, B is the inference target region obtained using the basic training model, and SA∪B and S A∩B Let L represent the union and intersection of the areas of the actual target region and the inferred target region, respectively; L represent the length of the actual target region and the inferred target region; H represent the width of the actual target region and the inferred target region; and gt in the formula indicates that this data comes from the inference result. This represents the ratio of the Euclidean distance between the inference target region and the actual target region about the center point to the line connecting the maximum outer apex angle between the inference target region and the actual target region.

[0117] Furthermore, based on any of the above embodiments, the image target detection model training device may include:

[0118] The mixed data acquisition module is used to obtain the mixed data by random sampling without replacement of the original data and the generated data at a preset percentage.

[0119] It should be noted that the order of the modules and units in the above-mentioned image target detection model training device can be changed without affecting the logic.

[0120] The image target detection model training device provided in this embodiment of the invention includes a raw data and generated data acquisition module 100, used to acquire raw data and generated data; wherein, the generated data refers to data obtained by fusing the image in the raw data as the target image and the occluded image; a generated training data acquisition module 200, used to acquire data in the generated data whose occlusion area reaches a preset occlusion threshold, to obtain generated training data; wherein, the occlusion area refers to the area of ​​the target image occluded by the occluded image; an inner loop module 300, used to continuously train the generated training data and the raw data using a basic target detection model, to obtain Loss1 corresponding to the raw data and Loss2 corresponding to the generated training data, and continuously calculate Loss2-Loss1 to obtain LossD0...LossD n ,until When the time is reached, the next inner loop is triggered; where epoch represents the number of times the basic object detection model is trained on the generated training data and the original data; the generated training data is adjusted by continuously increasing the value of the preset occlusion threshold to perform iterative training in the next inner loop; the outer loop module 400 is used to continuously obtain the loss corresponding to two adjacent inner loops. Dn When Loss Dn The previous round of internal circulation When the outer loop is triggered, it is used to perform iterative training with mixed data. The mixed data is composed of the original data and the generated data. The outer loop obtains a Loss3 once. When the Loss3 of two consecutive rounds is equal to the Loss3 of the previous round, the inner loop is triggered.

[0121] The image target detection model calculation module 500 is used to stop training the basic target detection model when Loss3 does not change within a preset number of consecutive iterations, thus obtaining the image target detection model. It is evident that, compared to existing target training models that require developers to collect and label raw data themselves and can only use raw data for training, this invention can utilize a small amount of raw data to generate a large amount of generated data, and mix the generated data with the raw data to obtain the mixed data required for the outer loop, greatly reducing the data collection time required for training the model. Furthermore, because it allows for iterative training using raw data and generated data in the inner loop, and iterative training using mixed data in the outer loop, the resulting image target detection training model, after iterative training of the basic target detection model through both inner and outer loops, can more accurately identify target objects. Furthermore, since max pooling can be used for global feature optimization during upsampling, the connection between local and global features of the generated data is greatly enhanced. Additionally, Cycle-GAN is used to augment the original data, significantly increasing the amount of training data. Moreover, the use of a random point operator for validity verification during Cycle-GAN data augmentation makes the generated data more realistic. Furthermore, the inclusion of occlusion area and occlusion position during the reconstruction of the inner loop loss function ensures global optimization during the inner loop. Finally, the inclusion of occlusion position during the reconstruction of the outer loop loss function significantly improves the recognition accuracy of the resulting image object detection model.

[0122] The image target detection model training device provided in the embodiments of the present invention will be described below. The image target detection model training device described below and the image target detection model training method described above can be referred to in correspondence.

[0123] Please refer to Figure 8 , Figure 8 A schematic diagram of the structure of the image target detection model training device provided in the embodiments of the present invention may include:

[0124] Memory 10 is used to store computer programs;

[0125] The processor 20 is used to execute computer programs to implement the steps of the image target detection model training method described above.

[0126] The memory 10, processor 20, and communication interface 31 all communicate with each other through the communication bus 32.

[0127] like Figure 8 The diagram shows the structural composition of an image target detection model training device. The image target detection model training device may include: a memory 10, a processor 20, a communication interface 30, and a communication bus 40. The memory 10, processor 20, and communication interface 30 all communicate with each other through the communication bus 40.

[0128] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include NVRAM. Memory stores the operating system and operating instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof. The operating instructions may include a variety of operation instructions for implementing various operations. The operating system may include various system programs for performing various basic tasks and handling hardware-based tasks.

[0129] The processor controls the application's replication of the terminal device's operations; the processor can also be referred to as a CPU. There can be one or more processors. In some embodiments of this application, the processor and memory can be connected via a bus or other means.

[0130] The methods disclosed in the embodiments of this application can be applied to a processor or implemented by a processor. A processor can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor can be a microprocessor or any conventional processor, etc.

[0131] Of course, it should be noted that, Figure 8 The structure shown does not constitute a limitation on the image target detection model training device in the embodiments of this application. In practical applications, the image target detection model training device may include more than Figure 8 More or fewer components as shown, or combinations of certain components.

[0132] The storage medium provided in the embodiments of the present invention will be described below. The storage medium described below can be referred to in correspondence with the image target detection model training method described above.

[0133] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described image target detection model training method.

[0134] The storage medium can include various media that can store program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0135] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0136] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0137] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0138] The image target detection model training method, apparatus, device, and storage medium provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for training an image target detection model, characterized in that, include: Acquire raw data and generate data; wherein, the generated data refers to the data obtained by fusing the image in the raw data as the target image and the occluded image; Data with occlusion areas reaching a preset occlusion threshold are obtained from the generated data to generate training data; wherein, the occlusion area refers to the area of ​​the target image occluded by the occluded image; continuously training the generated training data and the original data using a basic target detection model to obtain the original data corresponding to and the generated training data corresponding to , and continuously calculating , obtaining , until , triggering the next inner loop; wherein, epoch represents the number of times of training the generated training data and the original data using the basic target detection model; by continuously increasing the value of the preset occlusion threshold, the generated training data is adjusted for the next inner loop iteration training; Continuously obtain the corresponding values ​​of two adjacent inner loops ,when The previous round of internal circulation At any given time, an outer loop is triggered; wherein, the outer loop uses mixed data for iterative training; the mixed data is composed of the original data and the generated data; the outer loop obtains one result at a time. When two consecutive rounds Equal to the previous round When this happens, the inner loop is triggered; when If the target does not change within a preset number of consecutive iterations, training using the basic target detection model is stopped, and an image target detection model is obtained.

2. The image target detection model training method according to claim 1, characterized in that, Also includes: The first upsampling result is subjected to max pooling to obtain the max pooling result; wherein, the training process using the basic target detection model includes upsampling; The second upsampling result is subjected to average pooling, and the average pooling result is then subjected to cubic vector transpose to obtain the transposed result. The max pooling result and the transpose result are added together to obtain the sum. The summation result is added to the image vector from the upsampling process to obtain a vector summation result, which is then fed into the decoding network; wherein the decoding network is included in the training process of the basic object detection model.

3. The image target detection model training method according to claim 1, characterized in that, Before obtaining the generated data, the process also includes: The original data is randomly augmented using Cycle-Gan to obtain the generated data.

4. The image target detection model training method according to claim 3, characterized in that, The process of randomly augmenting the original data using Cycle-GAN to obtain the generated data includes: The original data is processed using a random point operator to obtain synthetic image data; The synthesized image data is subjected to convolution processing to obtain convolutional image data; The convolutional image data is processed using a residual network to obtain residual image data; The convolutional image data and the residual image data are added together to obtain enhanced image data, and the enhanced image data is processed by an encoder to obtain coded image data. The encoded image data is convolved to obtain the generated data.

5. The image target detection model training method according to any one of claims 1 to 4, characterized in that, Calculate the and stated loss function, include: Where A is the actual target region corresponding to the original data and the generated data, and B is the inferred target region obtained using the basic target detection model. and Let L represent the union and intersection of the areas of the actual target region and the inferred target region, respectively; L represent the length of the actual target region and the inferred target region; H represent the width of the actual target region and the inferred target region; and gt in the formula indicates that this data comes from the inference result. Pa represents the ratio of the Euclidean distance between the inference target region and the actual target region about the center point to the line connecting the maximum outer apex angle between the inference target region and the actual target region, where Pa represents the occlusion area.

6. The image target detection model training method according to any one of claims 1 to 4, characterized in that, The loss function model in the outer loop includes: Wherein, A is the actual target region corresponding to the original data and the generated data, and B is the inferred target region obtained using the basic target detection model. and Let L represent the union and intersection of the areas of the actual target region and the inferred target region, respectively; L represent the length of the actual target region and the inferred target region; H represent the width of the actual target region and the inferred target region; and gt in the formula indicates that this data comes from the inference result. This represents the ratio of the Euclidean distance between the inference target region and the actual target region about the center point to the line connecting the maximum outer apex angle between the inference target region and the actual target region.

7. The image target detection model training method according to claim 1, characterized in that, The methods for generating the hybrid data include: The mixed data is generated by random sampling without replacement of a preset percentage from the original data and the generated data.

8. An image target detection model training device, characterized in that, include: The raw data and generated data acquisition module is used to acquire raw data and generated data; wherein, the generated data refers to the data obtained by fusing the image in the raw data as the target image and the occluded image; A training data acquisition module is used to acquire data in the generated data whose occlusion area reaches a preset occlusion threshold, and obtain the generated training data; wherein, the occlusion area refers to the area of ​​the target image occluded by the occluded image; The inner loop module is used to continuously train the generated training data and the original data using the basic object detection model to obtain the corresponding data from the original data. The training data generated corresponds to and continuously calculate ,get ,until When the time is reached, the next inner loop is triggered; where epoch represents the number of times the basic object detection model is used to train the generated training data and the original data; by continuously increasing the value of the preset occlusion threshold, the generated training data is adjusted to perform iterative training in the next inner loop; The outer loop module is used to continuously obtain the values ​​corresponding to two adjacent inner loop iterations. ,when The previous round of internal circulation At any given time, an outer loop is triggered; wherein, the outer loop uses mixed data for iterative training; the mixed data is composed of the original data and the generated data; the outer loop obtains one result at a time. When two consecutive rounds Equal to the previous round When this happens, the inner loop is triggered; Image object detection model calculation module, used when If the target does not change within a preset number of consecutive iterations, training using the basic target detection model is stopped, and an image target detection model is obtained.

9. An image target detection model training device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the image target detection model training method as described in any one of claims 1 to 7 when executing the computer program.

10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the image target detection model training method as described in any one of claims 1 to 7.

Images