Training method, detection method and device, electronic equipment, medium and program product

Abstract

The invention provides a training method, a detection method and device, electronic equipment, a medium and a program product, and relates to the technical field of computer vision and the field of artificial intelligence. The training method comprises the steps that multiple rounds of iterative training are carried out on an original detection model until a preset iterative condition is met, and the original detection model comprises a teacher network and a student network; and determining the student network meeting a predetermined iteration condition as a target detection model.

Description

Technical Field

[0001] The present disclosure relates to the fields of computer vision technology and artificial intelligence, and more specifically, to a training method, a detection method, a device, an electronic device, a medium, and a program product. Background Art

[0002] With the popularization of artificial intelligence technology, computer vision technology based on deep learning is increasingly favored by technicians. Among them, target detection is an important basic research in computer vision technology. In related technologies, the training of target detection models can be processed by data enhancement strategies such as random cropping, rotation, and scaling of image samples, and the model is trained based on the enhanced image samples.

[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following problems in the related art: data enhancement strategies such as random cropping, rotation, and scaling of image samples may cause the position of the target to shift and the size to change, resulting in difficulty in accurately detecting the target position and target size when training the target detection model using the pattern samples processed by the data enhancement strategy, thereby reducing the robustness of the model and the accuracy of detection. Summary of the invention

[0004] In view of this, the present disclosure provides a training method, a detection method, an apparatus, an electronic device, a medium, and a program product.

[0005] One aspect of the present disclosure provides a target detection model training method, comprising: performing multiple rounds of iterative training on an original detection model until a predetermined iteration condition is met, wherein the original detection model includes a teacher network and a student network; and determining the student network that meets the predetermined iteration condition as the target detection model.

[0006] According to an embodiment of the present disclosure, multiple rounds of iterative training are performed on the original detection model, including: obtaining unlabeled image samples corresponding to the current round of iteration; using a teacher network to perform feature extraction on the unlabeled image samples to generate pseudo-label data for the unlabeled image samples; performing mixed data enhancement processing on the unlabeled image samples to obtain unlabeled mixed enhanced image samples; adjusting parameters of the student network according to the pseudo-label data of the unlabeled image samples and the unlabeled mixed enhanced image samples to obtain network parameters of the student network corresponding to the current round of iteration; obtaining network parameters of the teacher network corresponding to the next round of iteration according to the network parameters of the student network corresponding to the current round of iteration.

[0007] Another aspect of the present disclosure provides a target detection method, including: acquiring an image to be detected; performing target detection on the image to be detected using a target detection model to obtain a detection result of the image to be detected; wherein the target detection model is trained using the above-mentioned training method.

[0008] Another aspect of the present disclosure provides a target detection model training device, comprising: an iteration module, configured to perform multiple rounds of iterative training on an original detection model until a predetermined iteration condition is satisfied, wherein the original detection model comprises a teacher network and a student network; a determination module, configured to determine the student network satisfying the predetermined iteration condition as the target detection model;

[0009] Among them, the iteration module includes: an acquisition submodule, which is used to obtain unlabeled image samples corresponding to the current round of iteration; an extraction submodule, which is used to use the teacher network to extract features of the unlabeled image samples and generate pseudo-label data of the unlabeled image samples; an enhancement processing submodule, which is used to perform mixed data enhancement processing on the unlabeled image samples to obtain unlabeled mixed enhanced image samples; an adjustment submodule, which is used to adjust the parameters of the student network according to the pseudo-label data of the unlabeled image samples and the unlabeled mixed enhanced image samples to obtain the network parameters of the student network corresponding to the current round of iteration; and an acquisition submodule, which is used to obtain the network parameters of the teacher network corresponding to the next round of iteration according to the network parameters of the student network corresponding to the current round of iteration.

[0010] Another aspect of the present disclosure provides a target detection device, including: an acquisition module, used to acquire an image to be detected; a target detection module, used to perform target detection on the image to be detected using a target detection model to obtain a detection result of the image to be detected; wherein the target detection model is obtained by training according to the above-mentioned training device.

[0011] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above method.

[0012] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0013] Another aspect of the present disclosure provides a computer program product, which includes computer executable instructions, and the instructions are used to implement the above method when executed.

[0014] According to an embodiment of the present disclosure, a teacher network is used to extract features from a first unlabeled image sample to generate pseudo-label data, and mixed data enhancement is performed on the first unlabeled image sample and the second unlabeled image sample to obtain a first unlabeled mixed enhanced image sample and a second unlabeled mixed enhanced image sample. The student network is parameterized according to the pseudo-label data, the first unlabeled mixed enhanced image sample and the second unlabeled mixed enhanced image sample, and the teacher network parameters are obtained by adjusting the student network parameters, thereby training the student network. Because a mixed data enhancement strategy is used to perform mixed data enhancement processing on unlabeled image samples to train the student network using the unlabeled image samples after data enhancement processing, the technical problem of reducing the robustness of the model and inaccurate detection results caused by rotating, cropping, scaling and other operations on the image in the prior art is at least partially overcome, thereby achieving the technical effect of being able to better capture local information of the image, retaining the original position and original size of the target in the image sample, enhancing the robustness and generalization of the trained model, and improving the detection accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0016] Figure 1 An exemplary system architecture to which an object detection model training method, an object detection method, and an apparatus according to an embodiment of the present disclosure can be applied is schematically shown;

[0017] Figure 2 A flowchart of a method for training a target detection model according to an embodiment of the present disclosure is schematically shown;

[0018] Figure 3 A flowchart of a method for obtaining unlabeled mixed enhanced image samples according to an embodiment of the present disclosure is schematically shown;

[0019] Figure 4 A schematic diagram of obtaining an unlabeled mixed enhanced image sample according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 5 A flowchart of a method for obtaining network parameters of a student network corresponding to a current round of iteration according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 6 A schematic diagram schematically shows a method of obtaining a feature image of an intermediate first unlabeled image sample according to an embodiment of the present disclosure;

[0022] Figure 7A flowchart of a target detection method according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 8 A schematic diagram schematically shows a method for training a target detection model according to an embodiment of the present disclosure;

[0024] Fig. 9 A block diagram of a target detection model training device according to an embodiment of the present disclosure is schematically shown;

[0025] Fig.10 A block diagram schematically shows a target detection device according to an embodiment of the present disclosure; and

[0026] Fig.11 A block diagram of an electronic device suitable for implementing a target detection model training method and a target detection method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] In the embodiments of the present disclosure, the collection, acquisition, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0032] The consistent prediction of the teacher-student network can be achieved by applying different data augmentation transformations to the teacher network and the student network during the training process. For example, random cropping, rotation, scaling, etc. can be used to enhance the diversity of the data so that the teacher network and the student network can produce consistent prediction results under different input transformations.

[0033] However, in autonomous driving application scenarios, when the teacher-student network is applied to target detection tasks, many data augmentation methods related to image geometry and interpolation regularization are difficult to use. The data augmentation process will cause the position of the target in the image to shift or be partially cropped, or cause the size of the target in the image to change. This will make it difficult for the model to accurately locate the target position and detect the target size during the target detection task, thereby reducing the accuracy of the detection results.

[0034] To this end, an embodiment of the present disclosure provides a target detection model training method, comprising:

[0035] Performing multiple rounds of iterative training on the original detection model until a predetermined iteration condition is met, wherein the original detection model includes a teacher network and a student network; and determining the student network that meets the predetermined iteration condition as the target detection model;

[0036] Among them, multiple rounds of iterative training are performed on the original detection model, including: obtaining unlabeled image samples corresponding to the current round of iteration; using the teacher network to extract features of the unlabeled image samples to generate pseudo-label data of the unlabeled image samples; performing mixed data enhancement processing on the unlabeled image samples to obtain unlabeled mixed enhanced image samples; adjusting parameters of the student network according to the pseudo-label data of the unlabeled image samples and the unlabeled mixed enhanced image samples to obtain network parameters of the student network corresponding to the current round of iteration; according to the network parameters of the student network corresponding to the current round of iteration, obtaining the network parameters of the teacher network corresponding to the next round of iteration.

[0037] Figure 1 The exemplary system architecture to which the target detection model training method, target detection method and device according to the embodiments of the present disclosure can be applied is schematically shown. It should be noted that: Figure 1What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0038] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0039] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only for example).

[0040] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0041] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0042] It should be noted that the target detection model training method and the target detection method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the target detection model training device and the target detection device provided in the embodiment of the present disclosure can generally be set in the server 105. The target detection model training method and the target detection method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal device 101, 102, 103 and / or the server 105. Correspondingly, the target detection model training device and the target detection device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal device 101, 102, 103 and / or the server 105. Alternatively, the target detection model training method and the target detection method provided in the embodiment of the present disclosure can also be executed by the terminal device 101, 102, or 103, or can also be executed by other terminal devices different from the terminal device 101, 102, or 103. Correspondingly, the target detection model training device and the target detection device provided in the embodiments of the present disclosure may also be arranged in the terminal device 101, 102, or 103, or in other terminal devices different from the terminal device 101, 102, or 103.

[0043] For example, the unlabeled image sample may be originally stored in any one of the terminal devices 101, 102, or 103 (for example, the terminal device 101, but not limited thereto), or stored on an external storage device and may be imported into the terminal device 101. Then, the terminal device 101 may locally execute the target detection model training method and the target detection method provided in the embodiment of the present disclosure, or send the unlabeled image sample to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the unlabeled image sample may execute the target detection model training method and the target detection method provided in the embodiment of the present disclosure.

[0044] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0045] Figure 2 The flowchart of the target detection model training method according to an embodiment of the present disclosure is schematically shown.

[0046] like Figure 2 As shown, the method 200 may include operations S210 to S220.

[0047] In operation S210, multiple rounds of iterative training are performed on the original detection model until a predetermined iteration condition is met, wherein the original detection model includes a teacher network and a student network.

[0048] In operation S220 , a student network that meets a predetermined iteration condition is determined as a target detection model.

[0049] According to an embodiment of the present disclosure, operation S210 may include operations S211 to S215 .

[0050] In operation S211, unlabeled image samples corresponding to the current round iteration are obtained.

[0051] In operation S212, the teacher network is used to extract features from the unlabeled image samples to generate pseudo label data for the unlabeled image samples.

[0052] In operation S213, hybrid data enhancement processing is performed on the unlabeled image samples to obtain unlabeled hybrid enhanced image samples.

[0053] In operation S214, parameters of the student network are adjusted according to the pseudo-label data of the unlabeled image samples and the unlabeled mixed enhanced image samples to obtain network parameters of the student network corresponding to the current round of iteration.

[0054] In operation S215, network parameters of the teacher network corresponding to the next round of iteration are obtained according to the network parameters of the student network corresponding to the current round of iteration.

[0055] According to an embodiment of the present disclosure, a predetermined iteration condition can be used as a condition for determining whether the student network in the original detection model satisfies the target detection model. The predetermined iteration condition may include a preset number of iterations, which may be the number of iteration rounds that reaches the maximum number of iteration rounds, and may be customized according to actual needs, and is not limited in the present disclosure. The predetermined iteration condition may also include a stable state of the loss value obtained by the student network after each round of training. When the loss value reaches a stable state, it indicates that the predetermined iteration condition is met. The original detection model may be a model obtained after pre-training the teacher-student network using labeled data.

[0056] According to an embodiment of the present disclosure, for each round of multiple rounds of iterative training, unlabeled image samples corresponding to each round of iteration are obtained, and multiple rounds of iterative training are performed on the original detection model until a predetermined iteration condition is met, which may include: the ongoing round is referred to as the current round. The pseudo-label data of the unlabeled image samples corresponding to the current round of iteration and the unlabeled mixed enhanced image samples corresponding to the current round of iteration can be used to adjust the parameters of the student network to obtain the network parameters of the student network corresponding to the current round of iteration, and then the network parameters of the student network corresponding to the current round of iteration and the network parameters of the teacher network corresponding to the current round of iteration are used to obtain the network parameters of the teacher network corresponding to the next round of iteration, and based on the network parameters of the teacher network corresponding to the next round of iteration, the pseudo-label data of the unlabeled image samples corresponding to the next round of iteration are obtained, thereby starting the next round of iterative training of the student network.

[0057] According to an embodiment of the present disclosure, the pseudo-label data of the unlabeled image sample may be a prediction result of detecting a target in the unlabeled image sample using a teacher network.

[0058] According to an embodiment of the present disclosure, when the current round of iteration is the first round of iteration, unlabeled image samples corresponding to the first round of iteration are obtained, and features of the unlabeled image samples are extracted using the teacher network to obtain pseudo-label data of the unlabeled image samples corresponding to the first round of iteration. The unlabeled image samples are subjected to mixed data enhancement processing to obtain unlabeled mixed enhanced image samples corresponding to the first round of iteration. The student network can be parameterized based on the unlabeled mixed enhanced image samples and the pseudo-label data of the unlabeled image samples to obtain network parameters of the student network corresponding to the first round of iteration, and then the network parameters of the teacher network corresponding to the first round of iteration are updated based on the network parameters of the student network corresponding to the first round of iteration and the network parameters of the teacher network corresponding to the first round of iteration to obtain network parameters of the teacher network corresponding to the second round of iteration.

[0059] According to an embodiment of the present disclosure, when the current iteration is the second iteration, unlabeled image samples corresponding to the second iteration are obtained, and a teacher network based on network parameters corresponding to the second iteration is used to perform special extraction on the unlabeled image samples to obtain pseudo-label data of the unlabeled image samples corresponding to the second iteration. The unlabeled image samples are subjected to mixed data enhancement processing to obtain unlabeled mixed enhanced image samples corresponding to the second iteration. The student network can be parameterized based on the unlabeled mixed enhanced image samples and the pseudo-label data of the unlabeled image samples to obtain the network parameters of the student network corresponding to the second iteration, and then the network parameters of the teacher network corresponding to the second iteration are updated based on the network parameters of the student network corresponding to the second iteration and the network parameters of the teacher network corresponding to the second iteration to obtain the network parameters of the teacher network corresponding to the third iteration.

[0060] Based on the above iterative processing, multiple rounds of iterations are performed on the original detection model until the predetermined iteration conditions are met, that is, the predetermined number of iterations is met, or the loss value reaches a stable state, and the student network is determined as the target detection model.

[0061] According to the embodiment of the present disclosure, the teacher network is used to extract features from the first unlabeled image sample, generate pseudo-label data, perform mixed data enhancement on the first unlabeled image sample and the second unlabeled image sample, obtain the first unlabeled mixed enhanced image sample and the second unlabeled mixed enhanced image sample, adjust the parameters of the student network according to the pseudo-label data, the first unlabeled mixed enhanced image sample and the second unlabeled mixed enhanced image sample, and update the teacher network parameters by adjusting the student network parameters, thereby realizing the training of the student network. Because the mixed data enhancement strategy is used to perform mixed data enhancement processing on the unlabeled image samples to realize the training of the student network by using the unlabeled image samples after data enhancement processing, at least partially overcomes the technical problem of rotating, cropping, scaling and other operations on the image in the prior art, resulting in changes in the position or size of the target in the image, thereby causing the technical problem of reducing the robustness of the model and inaccurate detection results during model training, thereby achieving the technical effect of being able to better capture the local information of the image, retain the original position and original size of the target in the image sample, and realize enhancing the robustness and generalization of the trained model and improving the detection accuracy of the model.

[0062] According to an embodiment of the present disclosure, a teacher network is used to perform feature extraction on unlabeled image samples to generate pseudo-label data for the unlabeled image samples, including: performing feature extraction on unlabeled image samples using a teacher network to obtain feature images of the unlabeled image samples; performing regional feature extraction on the feature images of the unlabeled image samples to generate pseudo-label data for the unlabeled image samples.

[0063] According to an embodiment of the present disclosure, a feature extractor in a teacher network can be used to perform feature extraction on unlabeled image samples, which can include extracting texture features, shape features, spatial relationship features, etc. of the unlabeled image samples to generate a feature image of the unlabeled image samples.

[0064] According to an embodiment of the present disclosure, regional features are extracted from a feature image of an unlabeled image sample to generate pseudo-label data for the unlabeled image sample, including: regional features are extracted from a feature image of an unlabeled image sample to obtain candidate regions of the feature image of the unlabeled image sample; and candidate regions of the feature image of an unlabeled image sample are classified to generate pseudo-label data for the unlabeled image sample.

[0065] According to an embodiment of the present disclosure, regional features can be extracted from an unlabeled image sample feature image through a region extraction network, and a ROI (Region of Interest) region box of the unlabeled image sample feature image can be obtained.

[0066] According to an embodiment of the present disclosure, the classification layer branch can be used to perform binary classification of foreground and background on the ROI area box of the feature image of the unlabeled image sample, and the foreground represented as the target is determined as the predicted target detection box, and the label data corresponding to the predicted target detection box can be pseudo-label data.

[0067] According to an embodiment of the present disclosure, the pseudo label data of the unlabeled image sample obtained by using the teacher network may include size information of the predicted target detection frame, center point information of the predicted target detection frame, and the type of the target object in the target detection frame.

[0068] Figure 3 A flowchart of a method for obtaining unlabeled mixed enhanced image samples according to an embodiment of the present disclosure is schematically shown.

[0069] like Figure 3 As shown, the method 300 may include operations S310 to S330.

[0070] In operation S310, image decomposition is performed on the unlabeled image sample to obtain a plurality of image blocks.

[0071] In operation S320, a hybrid mask matrix is ​​generated according to decomposition information of the image block.

[0072] In operation S330, a hybrid data enhancement process is performed on each image block using a hybrid mask matrix to obtain an unlabeled hybrid enhanced image sample.

[0073] According to an embodiment of the present disclosure, an image decomposition algorithm may be used to decompose each unlabeled image sample into multiple image blocks, for example, into N×N blocks. The image decomposition algorithm may use an existing algorithm, which will not be specifically described in the present disclosure.

[0074] According to an embodiment of the present disclosure, an unlabeled image sample is decomposed into N×N blocks, and an N×N mixed mask matrix of the same shape can be generated according to image block decomposition information of the decomposed N×N blocks.

[0075] According to an embodiment of the present disclosure, a mixing mask matrix may be used to mix image blocks of each unlabeled image sample to obtain a mixed enhanced image sample.

[0076] According to an embodiment of the present disclosure, a mixed mask matrix can be used to perform operations on each image block in each unlabeled image sample to obtain a mixed enhanced image block at a corresponding position in each unlabeled image sample, thereby obtaining an unlabeled mixed enhanced image sample.

[0077] Figure 4 A schematic diagram of obtaining an unlabeled mixed enhanced image sample according to an embodiment of the present disclosure is schematically shown.

[0078] like Figure 4 As shown, in the schematic diagram 400, taking the unlabeled image samples as 4 (i.e., 401-404), each unlabeled image sample is decomposed into 4×4 image blocks as an example, a 4×4 mixed mask matrix is ​​generated (i.e., 405-408), and 4 unlabeled mixed enhanced image samples are generated (i.e., 409-412).

[0079] Specifically, taking the hybrid mask matrix 405 as an example, in the hybrid mask matrix 405, each number and the position of the number represent which specific unlabeled image sample, and the position of the image block corresponding to the unlabeled image sample. For example, taking the "1" in the first row and the first column as an example, other numbers are masked, and the "1" is operated with the image block at the position of the "1" corresponding to the first unlabeled image sample 401, and the hybrid enhanced image block at the position of the "1" corresponding to the unlabeled hybrid enhanced image sample 409 is generated; taking the "2" in the first row and the second column as an example, other numbers are masked, and the "2" is operated with the image block at the position of the "2" corresponding to the second unlabeled image sample 402, and the hybrid enhanced image block at the position of the "2" corresponding to the unlabeled hybrid enhanced image sample 409 is generated. By analogy, it can be ensured that each image block in the four unlabeled image samples is operated with the hybrid mask matrix, and finally the unlabeled hybrid enhanced image sample is obtained.

[0080] According to an embodiment of the present disclosure, hybrid data enhancement can be performed on unlabeled image samples through a hybrid mask matrix to ensure that each unlabeled image sample can retain the original geometric position information of the original image sample in the image space after hybrid data enhancement, so as to facilitate subsequent image reconstruction and restoration after decomposition.

[0081] Figure 5 The flowchart of the method for obtaining the network parameters of the student network corresponding to the current round iteration according to an embodiment of the present disclosure is schematically shown.

[0082] like Figure 5 As shown, the method 500 may include operations S510 to S550.

[0083] In operation S510, feature extraction is performed on a first unlabeled mixed enhanced image sample to obtain a first detection result of the first unlabeled mixed enhanced image sample.

[0084] In operation S520, feature extraction is performed on the second unlabeled mixed enhanced image sample to obtain a second detection result of the second unlabeled mixed enhanced image sample.

[0085] In operation S530 , a first loss value is determined according to the first detection result and the pseudo label data of the first unlabeled image sample.

[0086] In operation S540, a second loss value is determined according to the second detection result and the labeling result of the known target object of the second unlabeled image sample.

[0087] In operation S550, parameters of the student network are adjusted according to the first loss value and the second loss value to obtain network parameters of the student network corresponding to the current round iteration.

[0088] According to an embodiment of the present disclosure, the unlabeled mixed enhanced image sample includes a first unlabeled mixed enhanced image sample and a second unlabeled mixed enhanced image sample.

[0089] According to an embodiment of the present disclosure, the unlabeled image sample may include a first unlabeled image sample and a second unlabeled image sample. The first unlabeled image sample may be a large number of unlabeled images, which are used to generate pseudo-label data using a teacher network; on the other hand, the first unlabeled image sample is subjected to mixed data enhancement processing to obtain a first unlabeled mixed enhanced image sample, and the first unlabeled mixed enhanced image sample is input into the student network to obtain a first detection result, and a first loss value is obtained based on the first detection result and the generated label data.

[0090] According to an embodiment of the present disclosure, the second unlabeled image samples may be a small number of unlabeled images, and mixed data enhancement processing is performed on the second unlabeled image samples to obtain second unlabeled mixed enhanced image samples, and the second unlabeled mixed enhanced image samples are input into the student network to obtain a second detection result, and a second loss value is obtained based on the second detection result and the labeling result of the known target object of the second unlabeled image sample.

[0091] According to an embodiment of the present disclosure, feature extraction is performed on a first unlabeled mixed enhanced image sample to obtain a first detection result of the first unlabeled mixed enhanced image sample, including: feature extraction is performed on the first unlabeled mixed enhanced image sample to obtain a first unlabeled mixed enhanced image sample feature image; using a mixed mask matrix, inverse transformation processing is performed on the first unlabeled mixed enhanced image sample feature image to obtain an intermediate first unlabeled image sample feature image; regional feature extraction is performed on the intermediate first unlabeled image sample feature image to obtain a first detection result of the first unlabeled mixed enhanced image sample.

[0092] According to an embodiment of the present disclosure, a feature extractor in a student network can be used to perform feature extraction on a first unlabeled mixed enhanced image sample, which may include extracting texture features, shape features, spatial relationship features, etc. of the first unlabeled mixed enhanced image sample to obtain a feature image of the first unlabeled mixed enhanced image sample.

[0093] According to an embodiment of the present disclosure, a hybrid mask matrix can be used to perform inverse transformation processing on each image block in the first unlabeled hybrid enhanced image sample feature image, that is, by operating each image block in the first unlabeled hybrid enhanced image sample feature image with the hybrid mask matrix, a feature image corresponding to the first unlabeled image sample before hybrid data enhancement can be obtained, that is, the intermediate first unlabeled image sample feature image.

[0094] According to an embodiment of the present disclosure, a region extraction network may be used to extract region features from the middle first unlabeled image sample feature image, and a ROI (Region of Interest) region frame of the middle first unlabeled image sample feature image may be obtained.

[0095] According to an embodiment of the present disclosure, the classification layer branch can be used to perform binary classification of foreground and background on the ROI area box of the intermediate first unlabeled image sample feature image, and the foreground represented as the target is determined as the predicted target detection box, and the predicted target detection box data can be the first detection result of the first unlabeled mixed enhanced image sample.

[0096] Figure 6 A schematic diagram of obtaining a feature image of an intermediate first unlabeled image sample according to an embodiment of the present disclosure is schematically shown.

[0097] like Figure 6 As shown, in the schematic diagram 600, corresponding to Figure 4 The four unlabeled mixed enhanced image samples obtained are taken as the first unlabeled mixed enhanced image samples as examples, and feature extraction is performed on the first unlabeled mixed enhanced image sample 601 to obtain a first unlabeled mixed enhanced image sample feature image 602. Then, the first unlabeled mixed enhanced image sample feature image is inversely transformed using the mixed mask matrix to obtain an intermediate first unlabeled image sample feature image 603.

[0098] According to an embodiment of the present disclosure, a method for obtaining a second detection result of a second unlabeled mixed enhanced image sample is the same as a method for obtaining a first detection result of a first unlabeled mixed enhanced image sample, which is not described in detail herein.

[0099] According to an embodiment of the present disclosure, the first detection result obtained using the student network may include size information of the predicted target detection frame, center point information of the predicted target detection frame, and the type of the target object in the target detection frame.

[0100] According to an embodiment of the present disclosure, the target object type may be whether the target object is a person or an object; the object may be a specific type such as a car, a tricycle, or an electric vehicle.

[0101] According to an embodiment of the present disclosure, the first detection result and the pseudo-label data may be used to calculate the difference in size information and center point information of the predicted target detection frame and the type of the target object to determine the first loss value.

[0102] According to an embodiment of the present disclosure, the annotation result of the known target object of the second unlabeled image sample and the second detection result may be used to calculate the difference in size information and center point information of the predicted target detection box and the type of the target object to determine the second loss value.

[0103] According to an embodiment of the present disclosure, the total loss value determined by the first loss value and the second loss value can be used to adjust the parameters of the student network to obtain the network parameters of the student network corresponding to the current round of iteration.

[0104] According to an embodiment of the present disclosure, network parameters of a teacher network corresponding to a next round of iteration are obtained based on network parameters of a student network corresponding to a current round of iteration, including: acquiring network parameters of the teacher network corresponding to the current round of iteration; and obtaining network parameters of the teacher network corresponding to the next round of iteration based on network parameters of the student network corresponding to the current round of iteration and network parameters of the teacher network corresponding to the current round of iteration.

[0105] According to an embodiment of the present disclosure, the network parameters β of the teacher network can be set, and the network parameters of the student network can be set to θ. The network parameters of the teacher network corresponding to the next round of iteration can be determined based on the network parameters of the teacher network corresponding to the current round of iteration and the network parameters of the student network corresponding to the current round of iteration.

[0106] Specifically, when the current round of iteration is the first round, the network parameters of the teacher network corresponding to the second round of iteration can be determined based on the network parameters of the teacher network when the pseudo-label data of the unlabeled image samples corresponding to the first round of iteration is obtained through the teacher network in the first round, and the network parameters of the student network corresponding to the first round of iteration.

[0107] When determining the network parameters of the teacher network corresponding to the t+1th iteration, the network parameters of the teacher network corresponding to the t+1th iteration can be obtained based on the network parameters of the teacher network corresponding to the tth iteration and the network parameters of the student network corresponding to the tth iteration.

[0108] According to an embodiment of the present disclosure, the network parameter β of the teacher network can be obtained by the following formula (1):

[0109] (1);

[0110] in, are the network parameters of the teacher network corresponding to the t+1th iteration; are the network parameters of the teacher network corresponding to the tth iteration; are the network parameters of the student network corresponding to the tth iteration; is the exponential moving average decay rate.

[0111] According to an embodiment of the present disclosure, based on the current round of iterative training of the student network, the network parameters of the student network corresponding to the current round of iteration are obtained, and then based on the network parameters of the student network corresponding to the current round of iteration, the network parameters of the teacher network corresponding to the current round of iteration are updated to obtain the network parameters of the teacher network corresponding to the next round of iteration. It can be achieved that the teacher network can be generated according to the student network without training, thereby reducing the computing power consumption of the computer. The prediction results of the teacher network generated based on exponential moving average smoothing are more stable, and thus the prediction results of the student network are also more stable.

[0112] Figure 7 The flowchart of the target detection method according to the embodiment of the present disclosure is schematically shown.

[0113] like Figure 7 As shown, the method 700 may include operations S710-720.

[0114] In operation S710, an image to be detected is acquired.

[0115] In operation S720, target detection is performed on the image to be detected using the target detection model to obtain a detection result of the image to be detected; wherein the target detection model is trained using the above-mentioned training method.

[0116] According to an embodiment of the present disclosure, hybrid data enhancement processing can be performed on the image to be detected to obtain a hybrid enhanced image to be detected. The hybrid enhanced image to be detected is input into the target detection model trained by the above-mentioned target detection model training method to obtain the detection result of the image to be detected, for example, what the detection target of the detection result is.

[0117] According to the embodiments of the present disclosure, when processing the target detection model trained by the target detection model training method to process the image to be detected for target detection tasks in different application scenarios, it is possible to improve the generalization of the target detection model and the accuracy of the detection results of the image to be detected.

[0118] Figure 8 A schematic diagram of a target detection model training method according to an embodiment of the present disclosure is schematically shown.

[0119] like Figure 8 As shown, in the training method 800, during each iteration, an unlabeled image sample corresponding to the current iteration is obtained, and the unlabeled image sample may include a first unlabeled image sample 801 and a second unlabeled image sample 802. The feature extractor 803-1 in the teacher network 803 is used to extract features of the first unlabeled image sample 801 to obtain a first unlabeled image sample feature image 804. The first unlabeled image sample feature image 804 is input into the region extraction network 803-2 for region feature extraction to obtain a candidate region 805 of the first unlabeled image sample feature image. The branch network 803-3 is used to classify the candidate region 805 of the first unlabeled image sample feature image to generate pseudo label data 806 of the first unlabeled image sample.

[0120] The first unlabeled image sample 801 is subjected to mixed data enhancement processing to obtain a first unlabeled mixed enhanced image sample 807. The second unlabeled image sample 802 is subjected to mixed data enhancement processing to obtain a second unlabeled mixed enhanced image sample 808.

[0121] The feature extractor 809-1 in the student network 809 is used to extract features from the first unlabeled mixed enhanced image sample 807 to obtain a first unlabeled mixed enhanced image sample feature image 810. The first unlabeled mixed enhanced image sample feature image 810 is reversely transformed using a mixed mask matrix to obtain an intermediate first unlabeled image sample feature image 811. The intermediate first unlabeled image sample feature image 811 is input into the region extraction network 809-2 of the student network to extract regional features to obtain a candidate region 812 of the intermediate first unlabeled image sample feature image. The branch network 809-3 is used to classify the candidate region 812 of the intermediate first unlabeled image sample feature image to generate a first detection result 813 for the first unlabeled mixed enhanced image sample.

[0122] The feature extractor 809-1 in the student network 809 is used to extract features from the second unlabeled mixed enhanced image sample 808 to obtain a second unlabeled mixed enhanced image sample feature image 814. The second unlabeled mixed enhanced image sample feature image 814 is reversely transformed using a mixed mask matrix to obtain an intermediate second unlabeled image sample feature image 815. The intermediate second unlabeled image sample feature image 815 is input into the region extraction network 809-2 of the student network to extract regional features to obtain a candidate region 816 of the intermediate second unlabeled image sample feature image. The branch network 809-3 is used to classify the candidate region 816 of the intermediate first unlabeled image sample feature image to generate a second detection result 817 for the second unlabeled mixed enhanced image sample.

[0123] A first loss value 819 is determined based on the pseudo-label data 806 of the first unlabeled image sample and the first detection result 813 of the first unlabeled mixed enhanced image sample. A second loss value 820 is determined based on the second detection result 817 of the second unlabeled mixed enhanced image sample and the labeling result 818 of the known target object of the second unlabeled image sample. A network parameter 821 of the student network is determined based on the first loss value 819 and the second loss value 820. Based on the network parameter 821 of the student network, the network parameter of the teacher network is updated to implement iterative training of the target detection model, and the trained student network is determined as the target detection model.

[0124] Fig. 9 A block diagram of a target detection model training device according to an embodiment of the present disclosure is schematically shown.

[0125] like Fig. 9 As shown, the device 900 may include: an iteration module 910 and a determination module 920 .

[0126] The iteration module 910 is used to perform multiple rounds of iterative training on the original detection model until a predetermined iteration condition is met, wherein the original detection model includes a teacher network and a student network.

[0127] The determination module 920 is used to determine the student network that meets the predetermined iteration condition as the target detection model.

[0128] According to an embodiment of the present disclosure, the iteration module 910 may include: an acquisition submodule 911 , an extraction submodule 912 , an enhancement processing submodule 913 , an adjustment submodule 914 and an acquisition submodule 915 .

[0129] The acquisition submodule 911 is used to acquire unlabeled image samples corresponding to the current round of iteration.

[0130] The extraction submodule 912 is used to extract features from the unlabeled image samples using the teacher network to generate pseudo-label data for the unlabeled image samples.

[0131] The enhancement processing submodule 913 is used to perform mixed data enhancement processing on the unlabeled image samples to obtain unlabeled mixed enhanced image samples.

[0132] The adjustment submodule 914 is used to adjust the parameters of the student network according to the pseudo-label data of the unlabeled image samples and the unlabeled mixed enhanced image samples to obtain the network parameters of the student network corresponding to the current round iteration.

[0133] The obtaining submodule 915 is used to obtain the network parameters of the teacher network corresponding to the next round of iteration according to the network parameters of the student network corresponding to the current round of iteration.

[0134] According to an embodiment of the present disclosure, the extraction submodule 912 may include: a first obtaining unit and a first generating unit.

[0135] The first obtaining unit is used to extract features of unlabeled image samples using a teacher network to obtain feature images of the unlabeled image samples.

[0136] The first generating unit is used to extract regional features from the feature image of the unlabeled image sample to generate pseudo label data of the unlabeled image sample.

[0137] According to an embodiment of the present disclosure, the first generating unit may include: a first obtaining subunit and a generating subunit.

[0138] The first obtaining subunit is used to extract regional features from the unlabeled image sample feature image to obtain a candidate region of the unlabeled image sample feature image.

[0139] The generating subunit is used to classify the candidate regions of the feature images of the unlabeled image samples and generate pseudo label data of the unlabeled image samples.

[0140] According to an embodiment of the present disclosure, the enhancement processing submodule 913 may include: a second obtaining unit, a second generating unit and a processing unit.

[0141] The second obtaining unit is used to perform image decomposition on the unlabeled image sample to obtain multiple image blocks.

[0142] The second generating unit is used to generate a hybrid mask matrix according to the decomposition information of the image block.

[0143] The processing unit is used to perform hybrid data enhancement processing on each image block by using a hybrid mask matrix to obtain a first unlabeled hybrid enhanced image sample.

[0144] According to an embodiment of the present disclosure, the unlabeled mixed enhanced image sample includes a first unlabeled mixed enhanced image sample and a second unlabeled mixed enhanced image sample.

[0145] The adjustment submodule 915 may include: a third obtaining unit, a fourth obtaining unit, a first determining unit, a second determining unit, and an adjustment unit.

[0146] The third obtaining unit is used to extract features from the first unlabeled mixed enhanced image sample to obtain a first detection result of the first unlabeled mixed enhanced image sample.

[0147] The fourth obtaining unit is used to extract features from the second unlabeled mixed enhanced image sample to obtain a second detection result of the second unlabeled mixed enhanced image sample.

[0148] The first determining unit is used to determine a first loss value according to the first detection result and the pseudo label data of the unlabeled image sample.

[0149] The second determining unit is used to determine a second loss value according to the second detection result and the labeling result of the known target object of the second unlabeled image sample.

[0150] The adjustment unit is used to adjust the parameters of the student network according to the first loss value and the second loss value to obtain the network parameters of the student network corresponding to the current round iteration.

[0151] According to an embodiment of the present disclosure, the third obtaining unit may include: a second obtaining sub-unit, a third obtaining sub-unit and a fourth obtaining sub-unit.

[0152] The second obtaining subunit is used to extract features from the first unlabeled mixed enhanced image sample to obtain a first unlabeled mixed enhanced image sample feature image.

[0153] The third obtaining subunit is used to perform inverse transformation processing on the first unlabeled mixed enhanced image sample feature image by using the mixed mask matrix to obtain the intermediate first unlabeled image sample feature image.

[0154] The fourth obtaining subunit is used to extract regional features from the intermediate first unlabeled image sample feature image to obtain a first detection result of the first unlabeled mixed enhanced image sample.

[0155] According to an embodiment of the present disclosure, the obtaining submodule 916 may include: a fifth obtaining unit and a sixth obtaining unit.

[0156] The fifth obtaining unit is used to obtain the network parameters of the teacher network corresponding to the current round iteration.

[0157] The sixth obtaining unit is used to obtain the network parameters of the teacher network corresponding to the next round of iteration according to the network parameters of the student network corresponding to the current round of iteration and the network parameters of the teacher network corresponding to the current round of iteration.

[0158] Fig.10 A block diagram of a target detection device according to an embodiment of the present disclosure is schematically shown.

[0159] like Fig.10 As shown, the device 1000 may include: an acquisition module 1010 and a target detection module 1020 .

[0160] The acquisition module 1010 is used to acquire the image to be detected.

[0161] The target detection module 1020 is used to perform target detection on the image to be detected using the target detection model to obtain the detection result of the image to be detected. The target detection model is obtained by training according to the above device.

[0162] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0163] For example, any multiple of the iteration module 910 and the determination module 920 or the acquisition module 1010 and the target detection module 1020 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the iteration module 910 and the determination module 920 or the acquisition module 1010 and the target detection module 1020 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the iteration module 910 and the determination module 920 or the acquisition module 1010 and the target detection module 1020 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be performed.

[0164] It should be noted that the target detection model training device and the target detection device part in the embodiments of the present disclosure correspond to the target detection model training method and the target detection method part in the embodiments of the present disclosure. The description of the target detection model training device and the target detection device part specifically refers to the target detection model training method and the target detection method part, which will not be repeated here.

[0165] Fig.11 A block diagram of an electronic device suitable for implementing a target detection model training method and a target detection method according to an embodiment of the present disclosure is schematically shown. Fig.11 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0166] like Fig.11 As shown, the electronic device 1100 according to an embodiment of the present disclosure includes a processor 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage part 1108 to a random access memory (RAM) 1103. The processor 1101 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1101 may also include an onboard memory for caching purposes. The processor 1101 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0167] In RAM 1103, various programs and data required for the operation of electronic device 1100 are stored. Processor 1101, ROM 1102 and RAM 1103 are connected to each other through bus 1104. Processor 1101 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1102 and / or RAM 1103. It should be noted that the program can also be stored in one or more memories other than ROM 1102 and RAM 1103. Processor 1101 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0168] According to an embodiment of the present disclosure, the electronic device 1100 may further include an input / output (I / O) interface 1105, which is also connected to the bus 1104. The system 1100 may further include one or more of the following components connected to the input / output (I / O) interface 1105: an input portion 1106 including a keyboard, a mouse, etc.; an output portion 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1108 including a hard disk, etc.; and a communication portion 1109 including a network interface card such as a LAN card, a modem, etc. The communication portion 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the input / output (I / O) interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed, so that a computer program read therefrom is installed into the storage portion 1108 as needed.

[0169] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109, and / or installed from the removable medium 1111. When the computer program is executed by the processor 1101, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0170] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0171] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0172] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 1102 and / or the RAM 1103 described above and / or one or more memories other than the ROM 1102 and the RAM 1103 .

[0173] The embodiments of the present disclosure also include a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiments of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiments of the present disclosure.

[0174] When the computer program is executed by the processor 1101, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0175] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1109, and / or installed from a removable medium 1111. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0176] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0177] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0178] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A target detection model training method, comprising: Performing multiple rounds of iterative training on the original detection model until a predetermined iteration condition is met, wherein the original detection model includes a teacher network and a student network; Determining the student network that meets the predetermined iteration condition as a target detection model; The performing of multiple rounds of iterative training on the original detection model includes: Get the unlabeled image samples corresponding to the current iteration; Using the teacher network to extract features from the unlabeled image samples to generate pseudo label data for the unlabeled image samples; Performing mixed data enhancement processing on the unlabeled image samples to obtain unlabeled mixed enhanced image samples; According to the pseudo-label data of the unlabeled image samples and the unlabeled mixed enhanced image samples, the parameters of the student network are adjusted to obtain the network parameters of the student network corresponding to the current round of iteration; According to the network parameters of the student network corresponding to the current round of iteration, the network parameters of the teacher network corresponding to the next round of iteration are obtained.

2. The method according to claim 1, wherein: The step of extracting features from the unlabeled image samples using the teacher network to generate pseudo label data for the unlabeled image samples includes: Using the teacher network to extract features from the unlabeled image samples to obtain feature images of the unlabeled image samples; Performing regional feature extraction on the feature image of the unlabeled image sample to generate pseudo label data of the unlabeled image sample.

3. The method according to claim 2, wherein: The step of extracting regional features from the feature image of the unlabeled image sample to generate pseudo label data of the unlabeled image sample includes: Performing regional feature extraction on the unlabeled image sample feature image to obtain a candidate region of the unlabeled image sample feature image; Classify the candidate areas of the feature image of the unlabeled image sample to generate pseudo label data of the unlabeled image sample.

4. The method according to claim 1, wherein: The performing mixed data enhancement processing on the unlabeled image samples to obtain unlabeled mixed enhanced image samples includes: Performing image decomposition on the unlabeled image sample to obtain multiple image blocks; generating a hybrid mask matrix according to the decomposition information of the image block; Using the hybrid mask matrix, hybrid data enhancement processing is performed on each of the image blocks to obtain unlabeled hybrid enhanced image samples.

5. The method according to claim 4, wherein: The unlabeled mixed enhanced image samples include a first unlabeled mixed enhanced image sample and a second unlabeled mixed enhanced image sample; The step of adjusting parameters of the student network according to the pseudo-label data of the unlabeled image samples and the unlabeled mixed enhanced image samples to obtain network parameters of the student network corresponding to the current iteration includes: Performing feature extraction on the first unlabeled mixed enhanced image sample to obtain a first detection result of the first unlabeled mixed enhanced image sample; Performing feature extraction on the second unlabeled mixed enhanced image sample to obtain a second detection result of the second unlabeled mixed enhanced image sample; Determining a first loss value according to the first detection result and the pseudo label data of the unlabeled image sample; Determining a second loss value according to the second detection result and a labeling result of the known target object of the second unlabeled image sample; According to the first loss value and the second loss value, the parameters of the student network are adjusted to obtain the network parameters of the student network corresponding to the current round iteration.

6. The method according to claim 5, wherein: The extracting features of the first unlabeled mixed enhanced image sample to obtain a first detection result of the first unlabeled mixed enhanced image sample includes: Performing feature extraction on the first unlabeled mixed enhanced image sample to obtain a first unlabeled mixed enhanced image sample feature image; Using the mixed mask matrix, performing inverse transformation processing on the first unlabeled mixed enhanced image sample feature image to obtain an intermediate first unlabeled image sample feature image; Performing regional feature extraction on the intermediate first unlabeled image sample feature image to obtain a first detection result of the first unlabeled mixed enhanced image sample.

7. The method according to claim 1, wherein: The obtaining, according to the network parameters of the student network corresponding to the current iteration, the network parameters of the teacher network corresponding to the next iteration, comprises: Obtaining network parameters of the teacher network corresponding to the current round iteration; According to the network parameters of the student network corresponding to the current round of iteration and the network parameters of the teacher network corresponding to the current round of iteration, the network parameters of the teacher network corresponding to the next round of iteration are obtained.

8. A target detection method, comprising: Acquire the image to be detected; Performing target detection on the image to be detected using the target detection model to obtain a detection result of the image to be detected; The target detection model is trained using the method according to any one of claims 1 to 7.

9. A target detection model training device, comprising: An iteration module, used to perform multiple rounds of iterative training on the original detection model until a predetermined iteration condition is met, wherein the original detection model includes a teacher network and a student network; A determination module, used to determine the student network that meets the predetermined iteration condition as a target detection model; Wherein, the iteration module includes: The acquisition submodule is used to obtain unlabeled image samples corresponding to the current round iteration; An extraction submodule, configured to perform feature extraction on the unlabeled image samples using the teacher network to generate pseudo label data for the unlabeled image samples; An enhancement processing submodule, used for performing mixed data enhancement processing on the unlabeled image samples to obtain unlabeled mixed enhanced image samples; An adjustment submodule, configured to adjust parameters of the student network according to the pseudo-label data of the unlabeled image samples and the unlabeled mixed enhanced image samples, so as to obtain network parameters of the student network corresponding to the current round of iteration; The acquisition submodule is used to obtain the network parameters of the teacher network corresponding to the next round of iteration based on the network parameters of the student network corresponding to the current round of iteration.

10. A target detection device, comprising: An acquisition module, used for acquiring an image to be detected; A target detection module, used to perform target detection on the image to be detected using the target detection model to obtain a detection result of the image to be detected; Wherein, the target detection model is trained using the device according to claim 9.

11. An electronic device, comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

12. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 8.

13. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images