Image detection network training method and image detection method, system and device

Abstract

The present invention discloses a training method for an image detection network and an image detection method, system and device, and relates to the field of image detection technology. The method comprises: obtaining multiple image data sets and weights corresponding to the image data sets, and obtaining an image detection network to be trained according to the image data sets; sampling the image data sets according to the weights to obtain a sampled image; detecting the sampled image using the image detection network to be trained to obtain a first category recognition result and a first region recognition result of the detection object in the sampled image; obtaining a first loss value according to the first region recognition result, and obtaining a second loss value according to the first category recognition result; adjusting the network parameters of the image detection network to be trained according to the first loss value and the second loss value, and adjusting the weight according to the second loss value, and returning to the step of sampling the image data set according to the weights, until the number of times the network parameters are adjusted reaches a first preset number, and a trained image detection network is obtained.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a training method for an image detection network and an image detection method, system and device. Background Art

[0002] With the development of large models, visual large models have demonstrated excellent performance in tasks such as image recognition, object detection, and semantic segmentation. By learning complex patterns and features in large-scale data sets, they can automatically extract abstract, high-level feature representations from data, achieving higher accuracy and better generalization than traditional methods. Therefore, data has become the cornerstone of large models. Large models usually require tens of thousands of data to support them, and these data usually contain a large number of samples and labels for model learning and generalization. Therefore, detection based on large models is of great significance. Summary of the invention

[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, a first object of the present invention is to propose a training method for an image detection network to support detection based on a large model.

[0004] The second objective of the present invention is to provide an image detection method.

[0005] A third objective of the present invention is to provide an electronic device.

[0006] A fourth objective of the present invention is to provide an image detection system.

[0007] To achieve the above-mentioned purpose, the first aspect of the present invention proposes a training method for an image detection network, the method comprising: obtaining multiple image data sets and weights corresponding to the image data sets, and obtaining an image detection network to be trained based on the image data sets; sampling the image data sets according to the weights to obtain a sampled image; using the image detection network to be trained to detect the sampled image to obtain a first category recognition result and a first area recognition result of the detection object in the sampled image; obtaining a first loss value based on the first area recognition result, and obtaining a second loss value based on the first category recognition result; adjusting the network parameters of the image detection network to be trained based on the first loss value and the second loss value, and adjusting the weights based on the second loss value, and returning to the step of sampling the image data set based on the weights, until the number of times the network parameters are adjusted has reached a first preset number of times, thereby obtaining a trained image detection network.

[0008] In addition, the training method of the image detection network according to the embodiment of the present invention may also have the following additional technical features:

[0009] In one embodiment of the present invention, obtaining the image detection network to be trained based on the image dataset includes: determining a classification head based on a first label of the image dataset, and obtaining the image detection network to be trained based on the classification head, wherein the first label is a label representing the category of the image dataset.

[0010] In one embodiment of the present invention, the detection of the sampled image using the image detection network to be trained includes: obtaining a first embedding vector based on a second label of the sampled image, wherein the second label is a label representing the category of the detection object in the sampled image; extracting features of the sampled image using a feature extraction module of the image detection network to be trained to obtain a feature map; and processing the feature map and the first embedding vector using the classification head to obtain the first category recognition result and the first region recognition result.

[0011] In one embodiment of the present invention, the method also includes: when the number of times the network parameters are adjusted reaches a second preset number, detecting images in multiple image data sets according to the current image detection network to obtain a second category recognition result and a second region recognition result, wherein the second preset number of times is less than the first preset number of times; updating the first label and the second label according to the second category recognition result and the second region recognition result to redetermine the classification head according to the updated first label, and re-obtain the first embedding vector according to the updated second label.

[0012] In one embodiment of the present invention, obtaining the first embedding vector according to the second label of the sampled image includes: processing the second label using a text encoder to obtain a second embedding vector; obtaining a third embedding vector according to the second embedding vector, wherein the third embedding vector is a second embedding vector of an empty string; calculating the difference between the second embedding vector and the third embedding vector, and normalizing the difference to obtain the first embedding vector.

[0013] In one embodiment of the present invention, the classification head includes a decoder, a classifier and a regressor, and the processing of the feature map and the first embedding vector by the classification head includes: using the decoder to decode the feature map and the first embedding vector to obtain a decoding result; using the classifier to classify the decoding result to obtain the first category recognition result; using the regressor to regress the decoding result to obtain the first region recognition result.

[0014] In one embodiment of the present invention, the first preset number is 64.

[0015] To achieve the above-mentioned purpose, the second aspect of the present invention proposes an image detection method, which includes: obtaining an image to be detected; inputting the image to be detected into an image detection network trained by the above-mentioned image detection network training method to obtain an image detection result.

[0016] To achieve the above objectives, the third aspect of the present invention proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the above image detection method is implemented.

[0017] To achieve the above-mentioned purpose, a fourth aspect of the present invention provides an image detection system, including the above-mentioned electronic device.

[0018] According to the training method of the image detection network and the image detection method, system and device of the embodiment of the present invention, multiple image data sets and weights corresponding to the image data sets are obtained, and the image detection network to be trained is obtained according to the image data set; the image data set is sampled according to the weights to obtain a sampled image; the sampled image is detected by the image detection network to be trained to obtain a first category recognition result and a first area recognition result of the detection object in the sampled image; a first loss value is obtained according to the first area recognition result, and a second loss value is obtained according to the first category recognition result; the network parameters of the image detection network to be trained are adjusted according to the first loss value and the second loss value, and the weight is adjusted according to the second loss value, and the step of sampling the image data set according to the weight is returned until the number of times the network parameters are adjusted has reached a first preset number of times, and a trained image detection network is obtained. Thus, it is possible to obtain an image detection network based on big data training, and the image detection network is a large model, thereby supporting image detection using a large model.

[0019] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a training method for an image detection network according to an embodiment of the present invention;

[0021] Figure 2 is a schematic diagram of a training method for an image detection network according to an example of the present invention;

[0022] Figure 3 is a schematic diagram of a training method for an image detection network of another example of the present invention;

[0023] Figure 4 is a schematic diagram of a training method for an image detection network of another example of the present invention;

[0024] Figure 5 is a schematic diagram of a training method for an image detection network of another example of the present invention;

[0025] Figure 6 is a flow chart of an image detection method according to an embodiment of the present invention;

[0026] Figure 7 is a structural block diagram of an electronic device according to an embodiment of the present invention;

[0027] Figure 8 It is a structural block diagram of an image detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following describes the training method of the image detection network and the image detection method, system and device of the embodiments of the present invention with reference to the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described with reference to the accompanying drawings are exemplary and should not be construed as limiting the present invention.

[0029] In actual work, because data is usually collected in batches, and the data distribution and labels of each collected data vary greatly due to different tasks, they are usually unified before training; because the amount of data in large models is huge, if it is unified manually, it will take a lot of human resources. How to unify the labels automatically has become a prerequisite for large model training.

[0030] The automatic unified labeling method generally unifies different datasets into a single dataset by classifying them. Although these methods are automatic, they still require carefully designed components and lack scalability in expanding to more datasets. The mapping from the dataset-specific label space to the automatically learned unified label space will be noisy due to the change in the label space, affecting the algorithm effect. There are also methods to unify datasets by designing specific loss values. For example, if there are conflicting labels in different datasets, the loss value will not be calculated. However, if the labels of the two datasets do not conflict, the positive samples in one dataset will be negative samples in another dataset, which will become noise data for training and increase the difficulty of training.

[0031] Therefore, the present invention proposes a training method for an image detection network.

[0032] Figure 1 It is a flowchart of a training method for an image detection network according to an embodiment of the present invention.

[0033] like Figure 1 As shown, the training method of the image detection network includes:

[0034] S11, obtaining multiple image data sets and weights corresponding to the image data sets, and obtaining an image detection network to be trained according to the image data sets.

[0035] Specifically, in order to obtain an image detection network that can perform image detection, first, multiple image data sets for training the image detection network are obtained according to the types of images that the image detection network may detect, and then the image detection network to be trained is obtained according to the image data sets.

[0036] In order to obtain the image detection network to be trained according to the image data set, the classification head in the image detection network to be trained can be designed according to the detection object category of the image in the image data set, or the feature extraction module can be designed according to the image in the image data set. Further, the image detection network to be trained including the feature extraction module and multiple classification heads is obtained.

[0037] After obtaining the multiple image data sets, weights are assigned to the multiple image data sets, and the multiple weights correspond to the multiple image data sets one by one, that is, each image data set corresponds to one weight.

[0038] S12, sampling the image data set according to the weights to obtain a sampled image.

[0039] Specifically, after obtaining a plurality of image data sets and an initial weight corresponding to each image data set, the image data sets are sampled according to the initial weight to obtain a sampled image.

[0040] S13, using the image detection network to be trained to detect the sampled image, and obtain a first category recognition result and a first region recognition result of the detection object in the sampled image.

[0041] S14, obtaining a first loss value according to the first region recognition result, and obtaining a second loss value according to the first category recognition result.

[0042] S15, adjusting the network parameters of the image detection network to be trained according to the first loss value and the second loss value, adjusting the weight according to the second loss value, and returning to the step of sampling the image data set according to the weight, until the number of times the network parameters are adjusted reaches a first preset number, thereby obtaining a trained image detection network.

[0043] Specifically, after obtaining the first region recognition result and the first category recognition result output by each classification head, the first region recognition result and the first category recognition result are respectively compared with corresponding standard values, and the first loss value and the second loss value are obtained according to the comparison results.

[0044] After obtaining the first loss value and the second loss value, the network parameters of the image detection network to be trained are adjusted according to the first loss value and the second loss value.

[0045] Moreover, since the second loss value is a loss value obtained based on the first category recognition result, the weight of each image data set can also be adjusted according to the second loss value, so that each image data set can be resampled according to the adjusted weight, so that the resampled sampled image can be re-input into the image detection network to be trained.

[0046] The above adjustment of the weights of the image data sets may be performed so that different weights are obtained for the proportions of the second loss values ​​of different image data sets:

[0047] ,

[0048] in, is the number of data in the i-th data set, is the second loss value of the i-th image dataset, For each image dataset, the current weight is given. Based on the weight, the weight of the image dataset is updated. For example, assuming that the number of data in dataset 1 is 10,000 images, and the second loss value is 1.0; the number of data in dataset 2 is 20,000 images, and the second loss value is 2.0; the number of data in dataset 3 is 30,000 images, and the second loss value is 3.0. Substitute the above data quantity and second loss value into the above formula to get the weight , , , and use this weight as the new weight of the corresponding image dataset.

[0049] In some embodiments of the present invention, obtaining an image detection network to be trained based on an image dataset includes: determining a classification head based on a first label of the image dataset, and obtaining the image detection network to be trained based on the classification head, wherein the first label is a label representing a category of the image dataset.

[0050] Specifically, after obtaining multiple image data sets, it is also necessary to add label information of the data set category to each image data set, and use the label information as the first label. After obtaining the first label, the category of the detection object in the image that the network needs to identify can be known according to the first label.

[0051] After determining the category of the detection object in the image that the network needs to recognize, select a query-based object detection network, and determine the classification head according to the category. Add the above classification head determined according to the category to the query-based object detection network to obtain the final image detection network to be trained. Subsequently, the image detection network to be trained can be trained, and image recognition can be performed based on the trained network.

[0052] The above-mentioned query-based object detection networks include but are not limited to DETR (Detection Transformer), deformable DETR (Deformable DEtection Transformer), Def-DETR (Deformable Transformers for End-to-End Object Detection), DINO (DETR with Improved deNoising anchor boxes, an improved object detection model based on DETR), etc.

[0053] In some embodiments of the present invention, a sampled image is detected using an image detection network to be trained, including: obtaining a first embedding vector based on a second label of the sampled image, wherein the second label is a label representing a category of a detection object in the sampled image; extracting features from the sampled image using a feature extraction module of the image detection network to be trained to obtain a feature map; and processing the feature map and the first embedding vector using a classification head to obtain a first category recognition result and a first region recognition result.

[0054] Specifically, after obtaining the above-mentioned multiple image data sets, basic information of the image data sets can be counted. The basic information includes but is not limited to the data flow of the image data set and the labels contained therein. The labels contained are the labels of each image in the image data set, indicating the category of the detected object in the image.

[0055] Based on this, after sampling the image data set to obtain sampled images, a label indicating the category of the detected object in each sampled image can be obtained, and the label is used as the second label, and the first embedding vector is obtained according to the second label.

[0056] Furthermore, the sampled image is input into the feature extraction module of the image detection network to be trained to obtain a feature map, and the first embedding vector and the feature map are combined. Figure 1 The weak supervision signal is then processed by multiple classification heads to obtain the category and region of the detected object in the sampled image, which are respectively used as the first category recognition result and the first region recognition result.

[0057] Among them, the feature extraction module in the target detection network includes but is not limited to CNN (Convolutional Neural Network) and transformer model.

[0058] In some embodiments of the present invention, the training method of the image detection network also includes: when the number of times the network parameters are adjusted reaches a second preset number, the images in the multiple image data sets are detected according to the current image detection network to obtain a second category recognition result and a second region recognition result, wherein the second preset number is less than the first preset number; the first label and the second label are updated according to the second category recognition result and the second region recognition result, so as to redetermine the classification head according to the updated first label, and obtain the first embedding vector according to the updated second label. Based on this step, the label padding operation of the image data set can be realized.

[0059] Specifically, when the number of times the network parameters have been adjusted reaches a second preset number, based on the current network, a full amount of pseudo labels are output for all image data sets, which are complemented with the labels of the current image data sets, and the corrected labels are used in the training. The iteration time of the training can be set to descend according to the exponential gradient.

[0060] Among them, the above-mentioned completion operation on the labels of the image dataset can output all the predicted category labels and corresponding regression boxes of all image datasets based on the current network, save the prediction information of the non-native label categories of the dataset, and use it together with the native label information of the dataset as the label for the next training and participate in the training.

[0061] The above-mentioned pseudo label includes the above-mentioned predicted category label and regression box. The predicted category label is the second category recognition result obtained by detecting the image in the image data set using the current network, and the above-mentioned regression box is the second region recognition result obtained by detecting the image in the image data set using the current network.

[0062] As an example, suppose that dataset 1 contains two labels, "person" and "bear", and dataset 2 contains two labels, "person" and "cup". When completing the labels, the network outputs three predicted labels, "person", "bear", and "cup". For dataset 1, the two original labels "person" and "bear" are kept, and the "cup" label is added as a pseudo-label; for dataset 2, the two original labels "person" and "cup" are kept, and the "bear" label is added as a pseudo-label.

[0063] The second preset number can be obtained according to the following formula:

[0064] .

[0065] in, . It means to update at this number of iterations, which is the second preset number mentioned above. The above number of iterations refers to the number of times the network parameters are adjusted. Int() means to convert the value into an integer. epochs represents the above first preset number.

[0066] The second preset number can also be obtained by presetting. For example, the second preset number can be set to half of the first preset number. For another example, assuming that the first preset number is 64 times, the label will be updated at 1, 2, 4, 8, 16, 32, 40, 50, and 60 times.

[0067] In some embodiments of the present invention, obtaining a first embedding vector according to a second label of a sampled image includes: processing the second label using a text encoder to obtain a second embedding vector; obtaining a third embedding vector according to the second embedding vector, wherein the third embedding vector is a second embedding vector of an empty string; calculating a difference between the second embedding vector and the third embedding vector, and normalizing the difference to obtain a first embedding vector.

[0068] The following is an explanation with an example.

[0069] In this example, the text encoder may be any text encoder, including but not limited to BETR (Pre-training of Deep Bidirectional Transformers for Language Understanding), FastText (a text classification and word vector learning library), Embeddingsfrom Language Models (a pre-trained language model for generating dynamic word vectors), and other text encoders.

[0070] .

[0071] in, The prompt word corresponding to each category is the second label mentioned above. For text encoder, is the second embedding vector for each category of the output.

[0072] After obtaining the second embedding vector, the second embedding vector may be corrected to achieve correction of the text encoder to obtain the above-mentioned first embedding vector.

[0073] Correction of the above text encoder, that is, correcting the bias of the pre-trained encoder model:

[0074] .

[0075] in is the second embedding vector of the empty string, which is the third embedding vector mentioned above. Norm is L2 normalization. is the corrected encoder vector, which is the first embedding vector mentioned above.

[0076] Combine the following Figure 2 Provide explanation.

[0077] Specifically, in Figure 2 The above-mentioned multiple image data sets include data set 1, data set 2, ..., data set M. Figure 2 In the , including pre-trained text encoder and correction text encoder.

[0078] exist Figure 2 , vehicles, Flower, Tulip, dataset-prior, Teddy bear, and bears are examples of second labels obtained according to the labels of each image in the image dataset.

[0079] After obtaining the second label according to the image data set, the second label is used to perform pre-training based on the text encoder to implement the pre-trained text encoder, obtain an embedding (embedding vector), and use the embedding vector as the second embedding vector.

[0080] After the embedding is obtained, the text encoder is corrected using the embedding to correct the text encoder, and an embedding is obtained again, and the embedding vector is used as the first embedding vector.

[0081] In some embodiments of the present invention, the classification head includes a decoder, a classifier and a regressor, and the classification head is used to process the feature map and the first embedding vector, including: using the decoder to decode the feature map and the first embedding vector to obtain a decoding result; using the classifier to classify the decoding result to obtain a first category recognition result; using the regressor to regress the decoding result to obtain a first region recognition result.

[0082] Specifically, the classification head can be calculated as follows:

[0083] .

[0084] 。

[0085] .

[0086] in, is the feature map, Q is the first embedding vector, () is the decoder, Based on the decoding of both inputs, is a classifier, where M is the total number of image datasets, is the output of the classifier after the input feature, which is the recognition result of the first category mentioned above. is the regressor, is the output of the regressor after the input feature, that is, the first region recognition result mentioned above, which is not affected by the data set category.

[0087] Combine the following Figure 3 , 4 , and the examples shown in 5 are used for explanation.

[0088] exist Figure 3 In it, multiple data sets are included, namely data set 1, data set 2, data set 3, ..., data set M, and the multiple data sets are the multiple image data sets mentioned above.

[0089] According to the multiple image data sets, specific data (ie, images in the image data sets) and category labels can be obtained.

[0090] For the images in the above image dataset, the features of the detected objects in the images can be obtained (i.e., the above feature maps), and then the obtained feature maps are input into the query-based detector (i.e., the above classification head), and the Bbox Head and Loss A, Loss B, and Loss C are obtained according to the output of the detection head. Among them, Bbox Head is the above first region recognition result, Loss A, Loss B, and Loss C are the second loss values ​​obtained according to the above first category recognition result, and the resampling is realized by substituting the second loss value into f. f is the above specific method of resampling according to the second loss value.

[0091] Moreover, for the above-mentioned category label, the above-mentioned second label can also be obtained according to the category label of each image in the image data set, and the first embedding vector can be obtained according to the second label, so as to process the first embedding vector using the classification head.

[0092] exist Figure 3 In the figure, the query synthesizer and Spares Querues are text decoders. The query synthesizer fuses the category labels of various image data sets; Sparse Queries is a correction function for the text encoder, CLS is the above-mentioned first category recognition result, and box is the above-mentioned first region recognition result.

[0093] For details on how to use the classification head to process the feature map and the first embedding vector, see Figure 4 .

[0094] exist Figure 4For the above multiple image datasets, the feature extractor (i.e., the feature extraction module in the image detection network to be trained) is used to extract the above feature maps. Each classification head includes a feature decoder (i.e., the above decoder), a Box regressor (i.e., the above regressor), a dataset 1 classifier, a dataset 2 classifier, ..., a dataset M classifier (i.e., the above classifier).

[0095] That is to say, Figure 4 The dataset 1loss, dataset 2loss, ..., dataset Mloss in are the second loss values ​​obtained according to the first category recognition results. Figure 4 The regression loss in is the first loss value obtained according to the first region recognition result. Moreover, the first loss value and the second loss value are loss values ​​obtained by referring to the data set information.

[0096] Thus, the weight can be adjusted according to the second loss value for resampling, and when the number of times the network parameters are adjusted reaches a second preset number, the data set label can be updated according to the second category recognition result and the second region recognition result output by the classification head.

[0097] For details on updating dataset labels, see Figure 5 .exist Figure 5 In , the predicted category is the second category recognition result, and the predicted box is the second region recognition result.

[0098] In summary, the training method of the image detection network of the embodiment of the present invention obtains multiple image data sets and weights corresponding to the image data sets, and obtains the image detection network to be trained according to the image data sets; samples the image data sets according to the weights to obtain sampled images; uses the image detection network to be trained to detect the sampled images to obtain the first category recognition result and the first area recognition result of the detection object in the sampled image; obtains the first loss value according to the first area recognition result, and obtains the second loss value according to the first category recognition result; adjusts the network parameters of the image detection network to be trained according to the first loss value and the second loss value, and adjusts the weights according to the second loss value, and returns to the step of sampling the image data set according to the weights until the number of times the network parameters are adjusted reaches the first preset number, and the trained image detection network is obtained. In this way, it is possible to obtain an image detection network based on big data training, thereby realizing image detection using a large model. Moreover, it is also set that when the number of times the network parameters are adjusted reaches the second preset number, the images in the multiple image data sets are detected according to the current image detection network to obtain the second category recognition result and the second region recognition result, wherein the second preset number is less than the first preset number; the first label and the second label are updated according to the second category recognition result and the second region recognition result, so as to re-determine the classification head according to the updated first label, and re-obtain the first embedding vector according to the updated second label. Thus, it is possible to update the first label and the second label when the number of times the network parameters are adjusted reaches the second preset number, thereby realizing automatic unified labels, and because the label is updated according to the trained network, it has high scalability, and can maximize the use of the data set to improve the generalization ability of the network. Moreover, using the classification head method, each image data set is classified and trained to ensure the independence of the label semantic space, and solve the problem of positive and negative sample conflicts caused by different sample spaces when multiple data sets are merged for training. The text encoder and image features are input into the detection decoder for training, and the weak supervision information of the text is used to accelerate the convergence of the model, and the text encoder is biased and corrected at the same time to increase the distance between the labels of each data set. By using the resampling method, the training loss of each data set is calculated and the distribution of the resampled training data set is resolved to solve the problems of imbalanced number of data sets and uneven training difficulty of data sets.

[0099] Furthermore, the present invention provides an image detection method.

[0100] Figure 6 is a flow chart of an image detection method according to an embodiment of the present invention.

[0101] like Figure 6 As shown, the image detection method includes:

[0102] S61, obtaining an image to be detected.

[0103] S62, inputting the image to be detected into the image detection network trained by the above-mentioned image detection network training method to obtain an image detection result.

[0104] The image detection method of the embodiment of the present invention uses the image detection network trained by the image detection network training method of the above-mentioned embodiment for detection. It can realize image detection using the image detection network trained based on big data, thereby realizing image detection using a large model.

[0105] Furthermore, the present invention provides an electronic device.

[0106] Figure 7 It is a structural block diagram of an electronic device according to an embodiment of the present invention.

[0107] like Figure 7 As shown, the electronic device 500 includes: a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, such as through a bus 502. Optionally, the electronic device 500 may also include a transceiver 504. It should be noted that in actual applications, the transceiver 504 is not limited to one, and the structure of the electronic device 500 does not constitute a limitation on the embodiments of the present invention.

[0108] The processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. The processor 501 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0109] The bus 502 may include a path to transmit information between the above components. The bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 502 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0110] The memory 503 is used to store a computer program corresponding to the image detection method of the above embodiment of the present invention, and the computer program is controlled and executed by the processor 501. The processor 501 is used to execute the computer program stored in the memory 503 to implement the content shown in the above method embodiment.

[0111] in, Figure 7 The electronic device 500 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0112] The electronic device of the embodiment of the present invention, by implementing the above-mentioned image detection method, can realize image detection using an image detection network obtained by training based on big data, thereby realizing image detection using a large model.

[0113] Furthermore, the present invention provides an image detection system.

[0114] Figure 8 It is a structural block diagram of an image detection system according to an embodiment of the present invention.

[0115] like Figure 8 As shown, the image detection system 10 includes the electronic device 500 mentioned above.

[0116] The image detection system of the embodiment of the present invention can realize image detection using an image detection network trained based on big data through the above-mentioned electronic device, thereby realizing image detection using a large model.

[0117] It should be noted that the logic and / or steps represented in the flowchart or described in other ways herein can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or equipment (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or equipment and execute instructions), or in combination with these instruction execution systems, devices or equipment. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or equipment, or in combination with these instruction execution systems, devices or equipment. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0118] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0119] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0120] In the description of this specification, the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and cannot be understood as a limitation on the present invention.

[0121] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0122] In the description of this specification, unless otherwise specified, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0123] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0124] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A training method for an image detection network, characterized in that: The method comprises: Acquire multiple image data sets and weights corresponding to the image data sets, and obtain an image detection network to be trained according to the image data sets; Sampling the image data set according to the weight to obtain a sampled image; Detecting the sampled image using the image detection network to be trained to obtain a first category recognition result and a first region recognition result of the detection object in the sampled image; Obtaining a first loss value according to the first region recognition result, and obtaining a second loss value according to the first category recognition result; adjusting the network parameters of the image detection network to be trained according to the first loss value and the second loss value, adjusting the weight according to the second loss value, and returning to the step of sampling the image data set according to the weight, until the number of times of adjusting the network parameters reaches a first preset number, thereby obtaining a trained image detection network; The step of obtaining the image detection network to be trained according to the image data set includes: Determine a classification head according to a first label of the image data set, and obtain the image detection network to be trained according to the classification head, wherein the first label is a label representing a category of the image data set; The detecting the sampled image using the to-be-trained image detection network includes: Obtaining a first embedding vector according to a second label of the sampled image, wherein the second label is a label representing a category of a detected object in the sampled image; Using the feature extraction module of the image detection network to be trained to extract features from the sampled image to obtain a feature map; Using the classification head to process the feature map and the first embedding vector to obtain the first category recognition result and the first region recognition result; The method further comprises: When the number of times the network parameters are adjusted reaches a second preset number of times, detecting the images in the plurality of image data sets according to the current image detection network to obtain a second category recognition result and a second region recognition result, wherein the second preset number of times is less than the first preset number of times; The first label and the second label are updated according to the second category recognition result and the second region recognition result, so as to redetermine the classification head according to the updated first label, and to obtain the first embedding vector again according to the updated second label.

2. The method for training an image detection network according to claim 1, characterized in that: The obtaining a first embedding vector according to the second label of the sampled image includes: Processing the second label using a text encoder to obtain a second embedding vector; Obtain a third embedding vector according to the second embedding vector, wherein the third embedding vector is the second embedding vector of the empty string; A difference between the second embedding vector and the third embedding vector is calculated, and the difference is normalized to obtain the first embedding vector.

3. The training method of the image detection network according to claim 1, characterized in that: The classification head includes a decoder, a classifier, and a regressor, and the processing of the feature map and the first embedding vector by using the classification head includes: Using the decoder to decode the feature map and the first embedding vector to obtain a decoding result; Using the classifier to classify the decoding result to obtain the first category recognition result; The decoding result is regressed using the regressor to obtain the first region recognition result.

4. The method for training an image detection network according to claim 1, characterized in that: The first preset number is 64.

5. An image detection method, characterized in that: The method comprises: Acquire the image to be detected; The image to be detected is input into an image detection network trained by the image detection network training method according to any one of claims 1 to 4 to obtain an image detection result.

6. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the image detection method according to claim 5 is implemented.

7. An image detection system, characterized in that: Comprising the electronic device as claimed in claim 6.

Images