Target detection method and device, equipment and storage medium

Abstract

The invention discloses a target detection method, apparatus and device, and a storage medium. The target detection method comprises the steps of obtaining a to-be-detected image of a target object; based on the to-be-detected image, image detection processing is carried out through a preset target detection model to obtain a detection result of the target object, the target detection model is obtained by carrying out iterative training on a preset first detection model based on a training sample set, a test sample set and a mining sample pool, and in the training process of the target detection model, the detection result of the target object is obtained; and selecting a first sample and a second sample from the mining sample pool by adopting an image retrieval mode, and respectively expanding the first sample and the second sample to a training sample set and a test sample set. In the training process of the target detection model, the high-value samples are selected from the mining sample pool and expanded to the training sample set and the test sample set, so that the target detection model continuously perfects the test set while expanding the training set, and the test set can effectively evaluate the performance improvement of the model.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a target detection method, device, equipment and storage medium. Background Art

[0002] At present, with the development of deep learning technology, deep learning technology is widely used in vehicle perception systems. Among them, deep learning requires sufficient manually labeled data to ensure the reliability of the deep learning model. However, the manual labeling method takes a lot of time and cost. Therefore, how to reduce the amount of manually labeled data while ensuring the reliability of the deep learning model is a problem to be solved.

[0003] Related technologies usually use active learning methods to mine valuable samples from uncalibrated samples and expand the scale of training samples, so as to achieve better learning performance of deep learning models with a small amount of manually calibrated samples.

[0004] However, this active learning method only expands the scale of training samples, while the test samples are not expanded, resulting in the test samples being dissimilar to the expanded training samples. As a result, when the model is evaluated on the test samples, the performance improvement of the model under active learning cannot be reflected. Summary of the invention

[0005] The main purpose of this application is to provide a target detection method, device, equipment and storage medium, aiming to solve the technical problem that the prior art cannot reflect the performance improvement of the model under active learning.

[0006] To achieve the above objectives, the present application provides a target detection method, which includes:

[0007] Acquire the image of the target object to be detected;

[0008] Based on the image to be detected, image detection processing is performed through a preset target detection model to obtain a detection result of the target object;

[0009] Among them, the target detection model is obtained by iteratively training a preset first detection model based on a training sample set, a test sample set and a mining sample pool. During the training process of the target detection model, an image retrieval method is used to select a first sample and a second sample from the mining sample pool and expand them to the training sample set and the test sample set respectively.

[0010] Optionally, before the step of acquiring the image to be detected of the target object, the method includes:

[0011] Obtain training sample sets, test sample sets and mining sample pools;

[0012] Based on the preset first detection model, the training sample set and the test sample set, a first sample is selected from the mining sample pool by image retrieval, and the first sample is expanded to the training sample set to obtain an expanded training sample set;

[0013] Based on the expanded training sample set, iteratively train the first detection model to obtain a second detection model;

[0014] Based on the expanded training sample set and the test sample set, a second sample is selected from the mining sample pool by a value search method, and the second sample is expanded to the test sample set to obtain an expanded test sample set;

[0015] Based on the second detection model, the expanded training sample set and the expanded test sample set, the above sample set expansion and model training process is repeated until the trained detection model meets the preset training conditions and the training is stopped to obtain the target detection model.

[0016] Optionally, the step of selecting a first sample from the mining sample pool by image retrieval based on the preset first detection model, the training sample set and the test sample set, and expanding the first sample to the training sample set to obtain the expanded training sample set includes:

[0017] Based on the test sample set, a third sample is obtained by evaluating the test sample set through a preset first detection model, wherein the third sample refers to a sample in the test sample set that has a detection error problem;

[0018] Based on the third sample, performing image retrieval on the mining sample pool to obtain a first sample;

[0019] After manually annotating the first samples, the samples are expanded to the training sample set to obtain an expanded training sample set.

[0020] Optionally, the step of performing image retrieval on the mining sample pool based on the third sample to obtain the first sample includes:

[0021] Extract target features from the training sample set, the mining sample pool, and the third sample respectively to obtain a first target feature corresponding to the training sample set, a second target feature corresponding to the mining sample pool, and a third target feature corresponding to the third sample;

[0022] Calculating a first cosine similarity between the first target feature and the second target feature, and retaining sample target features in the mining sample pool whose first cosine similarity is less than a preset first similarity threshold, to obtain a fourth target feature;

[0023] Calculating the second cosine similarity between the fourth target feature and the third target feature, and retaining the target features in the mining sample pool whose second cosine similarity is greater than or equal to a preset second similarity threshold, to obtain a fifth target feature;

[0024] The sample corresponding to the fifth target feature is determined as the first sample.

[0025] Optionally, after the step of retaining the target features in the mining sample pool whose second cosine similarity is greater than or equal to a preset second similarity threshold to obtain the fifth target feature, the method includes:

[0026] Calculating the third cosine similarity between target features in the fifth target feature;

[0027] Selecting a target feature having the highest second cosine similarity among at least two target features whose third cosine similarity is greater than or equal to a preset third similarity threshold, to obtain a screened sixth target feature;

[0028] The step of determining the sample corresponding to the fifth target feature as the first sample includes:

[0029] The sample corresponding to the sixth target feature is determined as the first sample.

[0030] Optionally, the step of selecting a second sample from the mining sample pool based on the expanded training sample set and the test sample set by a value retrieval method, and expanding the second sample to the test sample set to obtain the expanded test sample set includes:

[0031] Performing combined full-image feature extraction on the expanded training sample set and the test sample set to obtain a first full-image feature, and performing full-image feature extraction on the mining sample pool to obtain a second full-image feature;

[0032] Determining a value score of each sample in the mining sample pool based on the first full-image feature and the second full-image feature;

[0033] The sample with the highest value score in the mining sample pool is determined as the second sample, and the second sample is manually annotated and then expanded to the test sample set to obtain an expanded training sample set.

[0034] Optionally, the step of determining the value score of each sample in the mining sample pool based on the first full-image feature and the second full-image feature includes:

[0035] Based on the first full-image feature and the second full-image feature, a feature distance set from each sample in the mining sample pool to each sample in the expanded training sample set and the test sample set is calculated, and a target feature distance with the smallest feature distance in the feature distance set is selected;

[0036] Obtaining the target confidence of each sample in the mining sample pool, the expected number of categories, and the sample uncertainty of the mining sample pool;

[0037] The target confidence, the sample uncertainty, the expected number of categories and the target feature distance are weightedly calculated to obtain a value score for each sample in the mining sample pool.

[0038] The present application also provides a target detection device, the target detection device comprising:

[0039] An acquisition module, used for acquiring an image to be detected of a target object;

[0040] A detection module, used to perform image detection processing based on the image to be detected through a preset target detection model to obtain a detection result of the target object;

[0041] The target detection model is obtained by iteratively training a preset first detection model based on a training sample set, a test sample set and a mining sample pool. During the training process of the target detection model, an image retrieval method is adopted to select a first sample and a second sample from the mining sample pool, and expand them to the training sample set and the test sample set respectively.

[0042] And / or, the target detection device also includes: a sample acquisition module, which is used to acquire a training sample set, a test sample set and a mining sample pool; a first retrieval module, which is used to select a first sample from the mining sample pool by image retrieval based on a preset first detection model, the training sample set and the test sample set, and expand the first sample to the training sample set to obtain an expanded training sample set; a training module, which is used to iteratively train the first detection model based on the expanded training sample set to obtain a second detection model; a second retrieval module, which is used to select a second sample from the mining sample pool by value retrieval based on the expanded training sample set and the test sample set, and expand the second sample to the test sample set to obtain an expanded test sample set; an iterative training module, which is used to repeat the above sample set expansion and model training process based on the second detection model, the expanded training sample set and the expanded test sample set, until the trained detection model meets the preset training conditions, then stop the training and obtain the target detection model;

[0043] And / or, the first retrieval module includes: a problem evaluation module, which is used to evaluate the test sample set through a preset first detection model to obtain a third sample, wherein the third sample refers to a sample in the test sample set that has a detection error problem; an image retrieval module, which is used to perform image retrieval on the mining sample pool based on the third sample to obtain a first sample; a first expansion module, which is used to expand the first sample to the training sample set after manually annotating it to obtain an expanded training sample set;

[0044] And / or, the image retrieval module includes: a target feature extraction module, used to extract target features from the training sample set, the mining sample pool and the third sample respectively, to obtain a first target feature corresponding to the training sample set, a second target feature corresponding to the mining sample pool and a third target feature corresponding to the third sample; a first deduplication module, used to calculate the first cosine similarity between the first target feature and the second target feature, and retain the sample target features in the mining sample pool whose first cosine similarity is less than a preset first similarity threshold, to obtain a fourth target feature; a similar image retrieval module, used to calculate the second cosine similarity between the fourth target feature and the third target feature, and retain the target features in the mining sample pool whose second cosine similarity is greater than or equal to a preset second similarity threshold, to obtain a fifth target feature; and a sample determination module, used to determine the sample corresponding to the fifth target feature as the first sample.

[0045] And / or, the target detection device further includes: a similarity calculation module, used to calculate the third cosine similarity between the target features in the fifth target feature; a second deduplication module, used to select the target feature with the highest second cosine similarity among at least two target features whose third cosine similarity is greater than or equal to a preset third similarity threshold, to obtain a screened sixth target feature;

[0046] And / or, the second retrieval module includes: a full-image feature extraction module, which is used to perform full-image feature extraction on the expanded training sample set and the test sample set to obtain a first full-image feature, and perform full-image feature extraction on the mining sample pool to obtain a second full-image feature; a scoring module, which is used to determine the value score of each sample in the mining sample pool based on the first full-image feature and the second full-image feature; a second expansion module, which is used to determine the sample with the highest value score in the mining sample pool as the second sample, and after manually annotating the second sample, expand it to the test sample set to obtain an expanded training sample set.

[0047] And / or, the scoring module includes: a feature distance calculation module, which is used to calculate the feature distance set from each sample in the mining sample pool to the expanded training sample set and the test sample set based on the first full-image feature and the second full-image feature, and select the target feature distance with the smallest feature distance in the feature distance set; a weight acquisition module, which is used to obtain the target confidence, the expected number of categories and the sample uncertainty of each sample in the mining sample pool; and a weighted calculation module, which is used to perform weighted calculation on the target confidence, the sample uncertainty, the expected number of categories and the target feature distance to obtain the value score of each sample in the mining sample pool.

[0048] The present application also provides a target detection device, the target detection device comprising: a memory, a processor, and a program stored in the memory for implementing the target detection method.

[0049] The memory is used to store a program for implementing the target detection method;

[0050] The processor is used to execute a program for implementing the target detection method to implement the steps of the target detection method.

[0051] The present application also provides a storage medium, on which is stored a program for implementing the target detection method, and the program for implementing the target detection method is executed by a processor to implement the steps of the target detection method.

[0052] The target detection method, device, equipment and storage medium provided by the present application are compared with the related art that only expands the scale of training samples, makes the test samples dissimilar to the expanded training samples, and causes the performance improvement of the model under active learning to be unable to be reflected when evaluating the model on the test samples. In the present application, an image to be detected of the target object is obtained; based on the image to be detected, image detection processing is performed by a preset target detection model to obtain the detection result of the target object, wherein the target detection model is obtained by iteratively training a preset first detection model based on a training sample set, a test sample set and a mining sample pool. During the training process of the target detection model, an image retrieval method is used to select a first sample and a second sample from the mining sample pool, and expand them to the training sample set and the test sample set respectively. That is, in the present application, by using an image retrieval method during the training process of the target detection model, a high-value sample is selected from the mining sample pool to expand to the training sample set and the test sample set, so that the target detection model can continuously improve the test set while expanding the training set during the active learning process, so as to achieve that the test set can effectively evaluate the performance improvement of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0054] Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application;

[0055] Figure 2 This is a flow chart of the first embodiment of the target detection method of the present application;

[0056] Figure 3 This is a module schematic diagram of the target detection device of this application;

[0057] Figure 4 This is a schematic diagram of the overall process scenario of the target detection method of this application;

[0058] Figure 5 This is a schematic diagram of a process scenario of the second embodiment of the target detection method of the present application;

[0059] Figure 6 This is a schematic diagram of the process scenario of the third embodiment of the target detection method of the present application.

[0060] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0061] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] like Figure 1 As shown, Figure 1 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application.

[0063] The terminal in the embodiment of the present application can be a PC, or it can be a smart phone, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a portable computer, or other portable terminal devices with display function.

[0064] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0065] Optionally, the terminal may also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among them, sensors include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and / or backlight when the mobile terminal is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; of course, the mobile terminal may also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which will not be repeated here.

[0066] Those skilled in the art will understand that Figure 1 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0067] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating device, a network communication module, a user interface module, and a target detection program.

[0068] exist Figure 1In the terminal shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the target detection program stored in the memory 1005.

[0069] Reference Figure 2 , an embodiment of the present application provides a target detection method, the target detection method comprising:

[0070] Step S100, obtaining an image of a target object to be detected;

[0071] Step S200, based on the image to be detected, image detection processing is performed by using a preset target detection model to obtain a detection result of the target object;

[0072] Among them, the target detection model is obtained by iteratively training a preset first detection model based on a training sample set, a test sample set and a mining sample pool. During the training process of the target detection model, an image retrieval method is used to select a first sample and a second sample from the mining sample pool and expand them to the training sample set and the test sample set respectively.

[0073] In this embodiment, the application scenarios are:

[0074] As an example, the scene of target detection can be the detection of vehicles / or pedestrians / or other obstacles during driving. In the related art, active learning is usually used to mine valuable samples from uncalibrated samples and expand the scale of training samples, so as to achieve better learning performance of deep learning models with a small amount of manually calibrated samples. However, this active learning method only expands the scale of training samples, so that the test samples are not similar to the expanded training samples, resulting in the inability to reflect the performance improvement of the model under active learning when evaluating the model on the test samples. For this scenario, the target detection method of this embodiment uses image retrieval to select high-value samples from the mining sample pool to expand them to the training sample set and the test sample set during the training process of the target detection model, so that the target detection model can continuously improve the test set while expanding the training set during the active learning process, so as to achieve the test set can effectively evaluate the performance improvement of the model.

[0075] As an example, the application scenario of target detection is not only the detection of vehicles / pedestrians / or obstacles during driving of the above-mentioned car, but also includes various target detection scenarios, which are not specifically limited here.

[0076] The purpose of this embodiment is to achieve that the test set can effectively evaluate the performance improvement of the model.

[0077] In this embodiment, the target detection method is applied to a target detection device.

[0078] The specific steps are as follows:

[0079] Step S100, obtaining an image of a target object to be detected;

[0080] In this embodiment, the target object refers to objects such as vehicles / pedestrians / or obstacles that can be detected within the range of the vehicle during driving, and the image to be detected refers to a video image or photo containing the target object captured by the vehicle's camera in the scene of vehicle driving.

[0081] In this embodiment, the device may obtain the image of the target object to be detected by using a camera pre-installed on the moving vehicle, or by uploading the image by the user.

[0082] Step S200, based on the image to be detected, image detection processing is performed by using a preset target detection model to obtain a detection result of the target object;

[0083] Among them, the target detection model is obtained by iteratively training a preset first detection model based on a training sample set, a test sample set and a mining sample pool. During the training process of the target detection model, an image retrieval method is used to select a first sample and a second sample from the mining sample pool and expand them to the training sample set and the test sample set respectively.

[0084] In this embodiment, the device performs image detection processing based on the image to be detected through a preset target detection model to obtain a detection result of the target object, wherein the detection result may be a detection frame of the target object. For example, the image A to be detected includes vehicle B and pedestrian C, and the detection result is vehicle B and pedestrian C with detection frames, indicating that vehicle B and pedestrian C are detected in the image to be detected. The detection result may also be specific information of the target object, including location information, distance information, category information, etc.

[0085] In this embodiment, the target detection model is used to perform target detection on the image to be detected, including but not limited to 2D target detection and 3D target detection. The target detection model is obtained by iteratively training a preset first detection model based on a training sample set, a test sample set and a mining sample pool. During the training process of the target detection model, an image retrieval method is used to select a first sample and a second sample from the mining sample pool, and expand them to the training sample set and the test sample set respectively, wherein the training sample set is a data set containing multiple data sets used for training models and model fitting in deep learning tasks; the test sample set is a data set containing multiple data sets used for testing model detection accuracy and calculating evaluation indicators in deep learning tasks; the mining sample pool is a set of image samples containing multiple image samples to be mined; the samples in the training sample set and the test sample set are all (manually) labeled samples, and the samples in the mining sample pool are all unlabeled samples or pseudo-label samples.

[0086] In this embodiment, the active learning of the model is to mine valuable unlabeled samples from the mining sample pool and expand them to the training set, so as to achieve the purpose of achieving better learning performance of the deep learning model with a small amount of manually calibrated samples. The related technology only expands the scale of training samples, so that the test samples are not similar to the expanded training samples, resulting in the inability to reflect the performance improvement of the model under active learning when evaluating the model on the test samples. In the process of active learning, the present application continuously improves the test set while expanding the training set, so that the test set can effectively evaluate the performance improvement of the model, ensuring that the performance benefits brought by the active learning of the model can be reflected in the test set.

[0087] In this embodiment, it should be noted that the target detection model can be built into a server distributedly deployed inside the terminal device computer system, and the target detection model can also be deployed in a GPU hardware parallel computing accelerator inside the terminal device computer system to improve computing efficiency.

[0088] Before the step of obtaining the image to be detected of the target object in step S100, the method refers to Figure 4 , including the following steps A100-A400:

[0089] Step A100, obtaining a training sample set, a test sample set and a mining sample pool;

[0090] In this embodiment, the training sample set is a data set containing multiple data sets used to train models and provide model fitting in deep learning tasks; the test sample set is a data set containing multiple data sets used to test model detection accuracy and calculate evaluation indicators in deep learning tasks; the mining sample pool is a data set containing multiple image samples to be mined; the samples in the training sample set and the test sample set are all (manually) labeled samples, and the samples in the mining sample pool are all unlabeled samples or pseudo-label samples.

[0091] In this embodiment, the device obtains the training sample set, the test sample set and the mining sample pool by shooting the set scene with a preset camera and then manually labeling or not labeling, or by receiving historical samples uploaded by users or saved in a database.

[0092] Step A200, based on a preset first detection model, the training sample set and the test sample set, a first sample is selected from the mining sample pool by image retrieval, and the first sample is expanded to the training sample set to obtain an expanded training sample set;

[0093] In this embodiment, the device selects a first sample from the mining sample pool based on a preset first detection model, the training sample set and the test sample set, and expands the first sample to the training sample set to obtain an expanded training sample set, wherein the image retrieval refers to retrieving high-value samples, i.e., the first samples, from the mining sample pool based on features in an instance image (i.e., an image representing a target object) or a full-image image in an image sample. High-value samples in the training sample set expansion scenario refer to samples for which there are problems to be solved and which can improve model performance after solving the problems. That is, the target detection model is iteratively trained through high-value samples, and the parameters obtained by model training are more accurate, and the detection accuracy of the target detection model is higher.

[0094] Step A300, iteratively training the first detection model based on the expanded training sample set to obtain a second detection model;

[0095] In this embodiment, the device iteratively trains the first detection model based on the expanded training sample set to obtain a second detection model, that is, after expanding the high-value samples to the training sample set, the first detection model is trained again, and the detection accuracy of the obtained second detection model is higher, wherein the first detection model is a model obtained by iterative training based on the initial training sample set before expansion, that is, a preset initial model with basic target detection capabilities, which only differs from the second detection model or the target detection model in terms of accuracy.

[0096] Step A400, based on the expanded training sample set and the test sample set, a second sample is selected from the mining sample pool by a value search method, and the second sample is expanded to the test sample set to obtain an expanded test sample set;

[0097] In this embodiment, the device selects a second sample from the mining sample pool based on the expanded training sample set and the test sample set, and expands the second sample to the test sample set to obtain an expanded test sample set, wherein the value retrieval refers to retrieving high-value samples, i.e., second samples, from the mining sample pool based on features in instance images (i.e., images representing target objects) or full-image images in image samples. Unlike the first samples, high-value samples in the test sample set expansion scenario refer to new problems other than the problems to be solved in the test sample set, and samples that can also improve model performance after solving the problems.

[0098] Step A500, based on the second detection model, the expanded training sample set and the expanded test sample set, repeat the above sample set expansion and model training process until the trained detection model meets the preset training conditions and stops training to obtain the target detection model.

[0099] In this embodiment, the device repeats the above sample set expansion and model training process based on the second detection model, the expanded training sample set and the expanded test sample set, until the trained detection model meets the preset training conditions and stops training to obtain the target detection model, wherein the preset training conditions can be a preset mining and / or iteration number, or the number of expanded samples is lower than a certain threshold. For example, the number of samples mined from the mining sample pool and expanded to the test sample set is 6 images, and the mining sample number threshold is set to 8, then the above sample set expansion and model training are stopped, and the current detection model is the target detection model. Reference Figure 4, the specific scheme is as follows: Step 1, the target detection model is evaluated on the test set, and some problems are found, including missed detection and false detection, etc., and the evaluation results are obtained; Step 2, using the training mining module 101, according to the evaluation results, unlabeled samples are selected in the mining pool to solve the problems in the test set, these samples are manually labeled to add to the training set, and these samples are deleted from the mining pool; Step 3, the model is retrained using the updated data; Step 4, the target detection model is evaluated on the test set to detect whether the original problem is solved, that is, whether the detection effect after feedback expansion is improved; Step 5, using the test mining module 102, unlabeled samples are selected in the mining pool to find new problems, these samples are manually labeled to add to the test set, and these samples are deleted from the mining pool; Step 6, return to step 1, perform the next round of iteration, or wait for the mining pool to add new samples before starting the next round of iteration. The above overall process illustrates the overall idea of ​​the present invention, that is, each time the training set is mined, only samples similar to the problems in the test set are found, so as to reflect that the mining algorithm can effectively solve the problems in the test set, and continuously expand the problems in the test set so that the model can solve problems of rich enough types.

[0100] The target detection method provided by the present application is compared with the related art that only expands the scale of training samples, so that the test samples are not similar to the expanded training samples, resulting in the inability to reflect the performance improvement of the model under active learning when evaluating the model on the test samples. In the present application, an image to be detected of the target object is obtained; based on the image to be detected, image detection processing is performed through a preset target detection model to obtain the detection result of the target object, wherein the target detection model is obtained by iteratively training a preset first detection model based on a training sample set, a test sample set and a mining sample pool. During the training process of the target detection model, an image retrieval method is used to select a first sample and a second sample from the mining sample pool, and expand them to the training sample set and the test sample set respectively. That is, in the present application, by using an image retrieval method during the training process of the target detection model, high-value samples are selected from the mining sample pool to expand them to the training sample set and the test sample set, so that the target detection model can continuously improve the test set while expanding the training set during the active learning process, so as to achieve that the test set can effectively evaluate the performance improvement of the model.

[0101] Based on the above first embodiment, the present application also provides another embodiment, wherein the target detection method includes:

[0102] Specifically, the step A100 includes the following steps A110-A130:

[0103] Step A110, based on the test sample set, evaluating by a preset first detection model to obtain a third sample, wherein the third sample refers to a sample in the test sample set that has a detection error problem;

[0104] In this embodiment, the device evaluates the test sample set through a preset first detection model to obtain a third sample, wherein the third sample refers to a sample in the test sample set that has a detection error problem. Specifically, refer to Figure 5 The device selects the wrong samples in the test set as the instance objects to be retrieved through the wrong example screening unit 201. For example, the labeled boxes with IoU less than 0.5 and confidence less than 0.5 are regarded as missed detection instances, and the predicted boxes with IoU less than 0.3 and confidence greater than 0.5 are regarded as false detection instances.

[0105] Step A120, performing image retrieval on the mining sample pool based on the third sample to obtain a first sample;

[0106] In this embodiment, the device performs image retrieval on the mining sample pool based on the third sample to obtain a first sample, that is, the device retrieves an image similar to the third sample from the mining sample pool as a high-value first sample.

[0107] Specifically, the step A120 includes the following steps A121-A124:

[0108] Step A121, extracting target features from the training sample set, the mining sample pool, and the third sample respectively, to obtain a first target feature corresponding to the training sample set, a second target feature corresponding to the mining sample pool, and a third target feature corresponding to the third sample;

[0109] In this embodiment, the device extracts target features from the training sample set, the mining sample pool, and the third sample, respectively, to obtain a first target feature corresponding to the training sample set, a second target feature corresponding to the mining sample pool, and a third target feature corresponding to the third sample, wherein target feature extraction refers to feature extraction of each target object on each sample in the training sample set, the mining sample pool, and the third sample, namely, instance features. For the mining sample pool, an instance refers to a prediction box of a target detection model; for the training set, an instance includes an annotation box and a false alarm box; for the target set, an instance refers to a false alarm prediction box and a missed annotation box that have been screened out. The instance feature is an ROI feature. For the feature pyramid output by FPN, if it is a calibration box, the ROI extracts the ROI feature from the feature layer where the anchor with the largest IoU is located; if it is a prediction box, the ROI feature of the feature layer of the output prediction box is taken.

[0110] Step A122, calculating the first cosine similarity between the first target feature and the second target feature, and retaining the sample target features in the mining sample pool whose first cosine similarity is less than a preset first similarity threshold, to obtain a fourth target feature;

[0111] In this embodiment, the device calculates the first cosine similarity between the first target feature and the second target feature, and retains the sample target features whose first cosine similarity in the mining sample pool is less than the preset first similarity threshold, to obtain a fourth target feature, wherein the cosine similarity is to draw the target feature vector into the vector space, and evaluate the similarity between the two target feature vectors by calculating the cosine value of the angle between them. Specifically, the device removes the cosine similarity between the mining pool and the training set features exceeding T through the deduplication unit 203. rep (i.e., the first similarity threshold is preset) rep is a hyperparameter, preferably set to 0.95), and the remaining features in the sample pool are mined and retained and named as temporary feature A (i.e., the fourth target feature). The purpose of this step is to exclude instances similar to the training set. If some problems already exist in both the training set and the test set, but are difficult to fit (for example, instances with poor imaging quality), resulting in poor performance in the test set, more similar samples there may not necessarily allow the model to fit.

[0112] Step A123, calculating the second cosine similarity between the fourth target feature and the third target feature, and retaining the target features in the mining sample pool whose second cosine similarity is greater than or equal to a preset second similarity threshold, to obtain a fifth target feature;

[0113] In this embodiment, the device calculates the second cosine similarity between the fourth target feature and the third target feature, and retains the target features whose second cosine similarity in the mining sample pool is greater than or equal to the preset second similarity threshold value, to obtain the fifth target feature, that is, the device retrieves images similar to the third sample from the mining sample pool as high-value first samples, and similar images use the preset second similarity threshold value as a feature similarity standard to reflect the similarity between image samples. Specifically, the device retains the temporary feature A (the fourth target feature) and the target set feature (the third target feature) whose cosine similarity exceeds T through the retrieval unit 204. sim Characteristics (T sim is a hyperparameter, preferably set to 0.8), and the feature is named temporary feature B, which is the fifth target feature. This step screens out samples similar to the problems in the test set, and it is expected that these samples can solve the problems in the test set.

[0114] Step A124: determine the sample corresponding to the fifth target feature as the first sample.

[0115] In this embodiment, the device determines the sample corresponding to the fifth target feature as the first sample, that is, the sample corresponding to the fifth target feature is a sample similar to the problem in the test set.

[0116] Specifically, in step A123, after retaining the target features in the mining sample pool whose second cosine similarity is greater than or equal to the preset second similarity threshold to obtain the fifth target feature, the method includes the following steps B100-B300:

[0117] Step B100, calculating the third cosine similarity between target features in the fifth target feature;

[0118] In this embodiment, the device calculates the third cosine similarity between the target features in the fifth target feature to determine the similarity between the target features in the fifth target feature.

[0119] Step B200, selecting a target feature with the highest second cosine similarity among at least two target features whose third cosine similarity is greater than or equal to a preset third similarity threshold, to obtain a selected sixth target feature;

[0120] In this embodiment, the device selects the target feature with the highest second cosine similarity among at least two target features whose third cosine similarity is greater than or equal to the preset third similarity threshold, and obtains the sixth target feature after screening, that is, the target feature screening is performed on the fifth target feature. Specifically, the device removes duplicates from the fifth target feature, and for any two similarities exceeding the preset third similarity threshold T rep features, and only retains the one with higher similarity to the test set. This step is because there may be a lot of redundancy in the mining pool sometimes. For example, the pictures in the mining pool may come from the interval sampling of the same video.

[0121] Step B300: determine the sample corresponding to the sixth target feature as the first sample.

[0122] In this embodiment, the device determines the sample corresponding to the sixth target feature after deduplication screening as the first sample.

[0123] Step A130: After manually annotating the first sample, expand it to the training sample set to obtain an expanded training sample set.

[0124] In this embodiment, the device manually labels the first sample and expands it to the training sample set to obtain an expanded training sample set. After the first sample is expanded to the training sample set, the first sample is deleted from the mining pool.

[0125] Based on the above first and second embodiments, the present application also provides another embodiment, wherein the target detection method includes:

[0126] Specifically, the step A300 includes the following steps A310-A330:

[0127] Step A310, performing full-image feature extraction on the expanded training sample set and the test sample set to obtain a first full-image feature, and performing full-image feature extraction on the mining sample pool to obtain a second full-image feature;

[0128] In this embodiment, the device performs a combined full-image feature extraction on the expanded training sample set and the test sample set to obtain a first full-image feature, and performs a full-image feature extraction on the mining sample pool to obtain a second full-image feature, wherein, since the samples in the training sample set and the test sample set are (manually) labeled samples, the samples in the two are subjected to full-image feature extraction together to obtain the first full-image feature, and the samples in the mining sample pool are unlabeled samples or pseudo-label samples, and the second full-image feature is obtained after the full-image feature extraction, referring to Figure 6 , the first full-image feature is the labeled sample feature. The second full-image feature is the mining pool feature. Specifically, the device uses the feature extraction unit 302 to obtain the features of the mining pool and the labeled samples, which are full-image features, that is, the features of the last layer of the backbone part of the model are averaged in length and width.

[0129] Step A320, determining a value score of each sample in the mining sample pool based on the first full-image feature and the second full-image feature;

[0130] In this embodiment, the device determines the value score of each sample in the mining sample pool based on the first full-image feature and the second full-image feature. Specifically, referring to Figure 6 The device performs weighted calculation on the first full-image feature, the second full-image feature and the sample uncertainty by running the weighted CoreSet unit 303, wherein Coreset is a classic active method based on diversity. This application upgrades this method and combines selection strategies such as category balance and uncertainty.

[0131] Specifically, the step A320 includes the following steps A321-A323:

[0132] Step A321, based on the first full-image feature and the second full-image feature, calculate a feature distance set from each sample in the mining sample pool to each sample in the expanded training sample set and the test sample set, and select a target feature distance with the smallest feature distance in the feature distance set;

[0133] In this embodiment, the device calculates a set of feature distances from each sample in the mining sample pool to each sample in the expanded training sample set and the test sample set based on the first full-image feature and the second full-image feature, and selects a target feature distance with the smallest feature distance in the feature distance set. Specifically, the device calculates the nearest feature L2 distance from all samples in the mining pool to the labeled samples, that is, if there are N labeled samples and M samples to be mined, N×M distances need to be calculated, and the minimum value is taken to obtain a minimum distance L with a length of M. min .

[0134] Step A322, obtaining the target confidence of each sample in the mining sample pool, the expected number of categories, and the sample uncertainty of the mining sample pool;

[0135] In this embodiment, the device obtains the target confidence of each sample in the mining sample pool, the expected number of categories, and the sample uncertainty of the mining sample pool, wherein the sample uncertainty is obtained by the device using the uncertainty unit 301 to calculate the uncertainty, and the sample uncertainty includes, for example, entropy (entropy), mutual information (MI), etc., and the confidence is the interval estimate of a population parameter of the sample by the model.

[0136] In this embodiment, in the process of selecting high-value samples, in order to ensure the balance of the number of instances in each category in the sample (that is, to ensure that there are no too many or too few instances in each category), the present application proposes a category balance strategy, that is, by adding the expectation of the number of categories of each sample in the mining sample pool to ensure the balance of the number of instances in each category in the sample, wherein the expectation of the number of categories is calculated based on the category information of the selected samples in the mining sample pool and the corresponding confidence. If there are no selected samples before the current sample mining cycle, the expectation of the number of categories of the samples in the current sample mining cycle is [0,0,0], wherein the value in the expectation represents the confidence, and the number represents the number of categories. For example, the sample instance categories include three categories: pedestrians, vehicles, and obstacles, and the expectations are [0,0,0] respectively; if there are selected samples before the current sample mining cycle, the expectation of the number of categories of the samples in the current sample mining cycle is [X1,Y1,Z1], where X1, Y1, and Z1 represent the expectations of the pedestrian, vehicle, and obstacle categories, respectively. That is, the above expression is that the category of the selected sample affects the value of the subsequent sample.

[0137] Step A323, performing weighted calculation on the target confidence, the sample uncertainty, the expected number of categories and the target feature distance to obtain a value score for each sample in the mining sample pool.

[0138] In this embodiment, the device performs weighted calculation on the target confidence, the sample uncertainty, the expected number of categories and the target feature distance to obtain a value score for each sample in the mining sample pool, wherein the weighted calculation also includes sample instance categories (pedestrians, vehicles, obstacles, etc.), and the number of instance categories can be set by oneself, for example, three categories include pedestrians, vehicles, and obstacles, and the number C of selected samples for each class is initialized before the weighted calculation. For example, if there are three categories, then C = [0,0,0], and the score of each instance in each sample in the mining sample pool is calculated = confidence × feature distance × uncertainty ÷ (the expected number of sample categories + 1), and then the sum is calculated, that is, the score of a single sample image is calculated as the sum of all instance scores, to obtain the value score of each sample in the mining sample pool, wherein the calculation of the expected number of sample categories + 1 is due to preventing the expectation from being 0.

[0139] Step A330, determining the sample with the highest value score in the mining sample pool as the second sample, and expanding the second sample to the test sample set after manual annotation to obtain an expanded training sample set.

[0140] In this embodiment, the device determines the sample with the highest value score in the mining sample pool as the second sample, and after manually annotating the second sample, expands it to the test sample set to obtain an expanded training sample set, that is, selects the picture with the largest value score and adds the newly selected sample. When the mining quantity is greater than 1, it is necessary to repeat the above steps A310-A330 until the mining quantity condition is met, and delete the selected picture from the mining pool. Figure 6 , the specific scheme is as follows: Step 1: Use the uncertainty unit 301 to calculate the uncertainty, such as entropy (entropy), mutual information (MI), etc.; Step 2: Use the feature extraction unit 302 to obtain the features of the mining pool and the labeled samples. The feature is the full-image feature, that is, the features of the last layer of the backbone part of the model are averaged in length and width; Step 3: Run the weighted CoreSet unit 303. Coreset is a classic active method based on diversity. The present invention upgrades this method and combines selection strategies such as category balance and uncertainty; wherein, step 3 also includes the following steps 3.1-3.9: Step 3.1: Calculate the nearest feature L2 distance from all samples in the mining pool to the labeled samples, that is, if there are N labeled samples and M samples to be mined, it is necessary to calculate N×M distances and take the minimum value to obtain the minimum distance L with a length of M.min ; Step 3.2: Initialize the number C of selected samples for each class. For example, if there are three categories, then C = [0,0,0] Step 3.3: Calculate the score of each instance = confidence × feature distance × uncertainty ÷ (the number of newly selected samples of this category + 1); Step 3.4: Calculate the score of a single image as the sum of the scores of all instances; Step 3.5: Select the image with the largest score and add the newly selected sample; Step 3.6: If the cutoff condition is met, that is, the number of selected images reaches the set upper limit (for example, set to select 200 images), stop selecting, instead of deleting the selected images from the mining pool, otherwise, continue; Step 3.7: For the newly selected images, add the confidence of each prediction box of the image to the corresponding category, that is, update C; Step 3.8: Calculate the distance L from the newly selected image to each sample in the mining pool, and update L min =min(L min ,L); Step 3.9: Return to 3.3; Step 4: Output the selected uncalibrated image.

[0141] In this embodiment, the device solves the problems existing in the test set in a targeted manner through image retrieval, ensuring that the performance benefits brought by active learning can be reflected in the test set. In the active learning process, not only the training set is expanded, but also the test set is continuously improved, so that the test set can effectively evaluate the performance of the model.

[0142] The present application also provides a target detection device, referring to Figure 3 , the target detection device comprises:

[0143] An acquisition module 10 is used to acquire an image to be detected of a target object;

[0144] The detection module 20 is used to perform image detection processing based on the image to be detected by using a preset target detection model to obtain a detection result of the target object;

[0145] Among them, the target detection model is obtained by iteratively training a preset first detection model based on a training sample set, a test sample set and a mining sample pool. During the training process of the target detection model, an image retrieval method is used to select a first sample and a second sample from the mining sample pool and expand them to the training sample set and the test sample set respectively.

[0146] And / or, the target detection device also includes: a sample acquisition module, which is used to acquire a training sample set, a test sample set and a mining sample pool; a first retrieval module, which is used to select a first sample from the mining sample pool by image retrieval based on a preset first detection model, the training sample set and the test sample set, and expand the first sample to the training sample set to obtain an expanded training sample set; a training module, which is used to iteratively train the first detection model based on the expanded training sample set to obtain a second detection model; a second retrieval module, which is used to select a second sample from the mining sample pool by value retrieval based on the expanded training sample set and the test sample set, and expand the second sample to the test sample set to obtain an expanded test sample set; an iterative training module, which is used to repeat the above sample set expansion and model training process based on the second detection model, the expanded training sample set and the expanded test sample set, until the trained detection model meets the preset training conditions, then stop the training and obtain the target detection model;

[0147] And / or, the first retrieval module includes: a problem evaluation module, which is used to evaluate the test sample set through a preset first detection model to obtain a third sample, wherein the third sample refers to a sample in the test sample set that has a detection error problem; an image retrieval module, which is used to perform image retrieval on the mining sample pool based on the third sample to obtain a first sample; a first expansion module, which is used to expand the first sample to the training sample set after manually annotating it to obtain an expanded training sample set;

[0148] And / or, the image retrieval module includes: a target feature extraction module, used to extract target features from the training sample set, the mining sample pool and the third sample respectively, to obtain a first target feature corresponding to the training sample set, a second target feature corresponding to the mining sample pool and a third target feature corresponding to the third sample; a first deduplication module, used to calculate the first cosine similarity between the first target feature and the second target feature, and retain the sample target features in the mining sample pool whose first cosine similarity is less than a preset first similarity threshold, to obtain a fourth target feature; a similar image retrieval module, used to calculate the second cosine similarity between the fourth target feature and the third target feature, and retain the target features in the mining sample pool whose second cosine similarity is greater than or equal to a preset second similarity threshold, to obtain a fifth target feature; and a sample determination module, used to determine the sample corresponding to the fifth target feature as the first sample.

[0149] And / or, the target detection device further includes: a similarity calculation module, used to calculate the third cosine similarity between the target features in the fifth target feature; a second deduplication module, used to select the target feature with the highest second cosine similarity among at least two target features whose third cosine similarity is greater than or equal to a preset third similarity threshold, to obtain a screened sixth target feature;

[0150] And / or, the second retrieval module includes: a full-image feature extraction module, which is used to perform full-image feature extraction on the expanded training sample set and the test sample set to obtain a first full-image feature, and perform full-image feature extraction on the mining sample pool to obtain a second full-image feature; a scoring module, which is used to determine the value score of each sample in the mining sample pool based on the first full-image feature and the second full-image feature; a second expansion module, which is used to determine the sample with the highest value score in the mining sample pool as the second sample, and after manually annotating the second sample, expand it to the test sample set to obtain an expanded training sample set.

[0151] And / or, the scoring module includes: a feature distance calculation module, which is used to calculate the feature distance set from each sample in the mining sample pool to the expanded training sample set and the test sample set based on the first full-image feature and the second full-image feature, and select the target feature distance with the smallest feature distance in the feature distance set; a weight acquisition module, which is used to obtain the target confidence, the expected number of categories and the sample uncertainty of each sample in the mining sample pool; and a weighted calculation module, which is used to perform weighted calculation on the target confidence, the sample uncertainty, the expected number of categories and the target feature distance to obtain the value score of each sample in the mining sample pool.

[0152] The specific implementation of the target detection device of the present application is basically the same as the above-mentioned embodiments of the target detection method, and will not be repeated here.

[0153] Reference Figure 1 , Figure 1 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application.

[0154] like Figure 1As shown, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0155] Optionally, the target detection device may also include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, etc. The rectangular user interface may include a display screen (Display), an input submodule such as a keyboard (Keyboard), and the optional rectangular user interface may also include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0156] Those skilled in the art will understand that Figure 1 The target detection device structure shown in the figure does not constitute a limitation on the target detection device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0157] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module and a target detection program. The operating system is a program that manages and controls the hardware and software resources of the target detection device, and supports the operation of the target detection program and other software and / or programs. The network communication module is used to realize the communication between the components inside the memory 1005, and the communication between other hardware and software in the target detection system.

[0158] exist Figure 1 In the target detection device shown, the processor 1001 is used to execute the target detection program stored in the memory 1005 to implement the steps of any of the target detection methods described above.

[0159] The specific implementation of the target detection device of the present application is basically the same as the above-mentioned embodiments of the target detection method, and will not be repeated here.

[0160] The present application also provides a storage medium, on which a program for implementing the target detection method is stored, and the program for implementing the target detection method is executed by a processor to implement the target detection method described below:

[0161] Acquire the image of the target object to be detected;

[0162] Based on the image to be detected, image detection processing is performed through a preset target detection model to obtain a detection result of the target object;

[0163] Among them, the target detection model is obtained by iteratively training a preset first detection model based on a training sample set, a test sample set and a mining sample pool. During the training process of the target detection model, an image retrieval method is used to select a first sample and a second sample from the mining sample pool and expand them to the training sample set and the test sample set respectively.

[0164] Optionally, before the step of acquiring the image to be detected of the target object, the method includes:

[0165] Obtain training sample sets, test sample sets and mining sample pools;

[0166] Based on the preset first detection model, the training sample set and the test sample set, a first sample is selected from the mining sample pool by image retrieval, and the first sample is expanded to the training sample set to obtain an expanded training sample set;

[0167] Based on the expanded training sample set, iteratively train the first detection model to obtain a second detection model;

[0168] Based on the expanded training sample set and the test sample set, a second sample is selected from the mining sample pool by a value search method, and the second sample is expanded to the test sample set to obtain an expanded test sample set;

[0169] Based on the second detection model, the expanded training sample set and the expanded test sample set, the above sample set expansion and model training process is repeated until the trained detection model meets the preset training conditions and the training is stopped to obtain the target detection model.

[0170] Optionally, the step of selecting a first sample from the mining sample pool by image retrieval based on the preset first detection model, the training sample set and the test sample set, and expanding the first sample to the training sample set to obtain the expanded training sample set includes:

[0171] Based on the test sample set, a third sample is obtained by evaluating the test sample set through a preset first detection model, wherein the third sample refers to a sample in the test sample set that has a detection error problem;

[0172] Based on the third sample, performing image retrieval on the mining sample pool to obtain a first sample;

[0173] After manually annotating the first samples, the samples are expanded to the training sample set to obtain an expanded training sample set.

[0174] Optionally, the step of performing image retrieval on the mining sample pool based on the third sample to obtain the first sample includes:

[0175] Extract target features from the training sample set, the mining sample pool, and the third sample respectively to obtain a first target feature corresponding to the training sample set, a second target feature corresponding to the mining sample pool, and a third target feature corresponding to the third sample;

[0176] Calculating a first cosine similarity between the first target feature and the second target feature, and retaining sample target features in the mining sample pool whose first cosine similarity is less than a preset first similarity threshold, to obtain a fourth target feature;

[0177] Calculating the second cosine similarity between the fourth target feature and the third target feature, and retaining the target features in the mining sample pool whose second cosine similarity is greater than or equal to a preset second similarity threshold, to obtain a fifth target feature;

[0178] The sample corresponding to the fifth target feature is determined as the first sample.

[0179] Optionally, after the step of retaining the target features in the mining sample pool whose second cosine similarity is greater than or equal to a preset second similarity threshold to obtain the fifth target feature, the method includes:

[0180] Calculating the third cosine similarity between target features in the fifth target feature;

[0181] Selecting a target feature having the highest second cosine similarity among at least two target features whose third cosine similarity is greater than or equal to a preset third similarity threshold, to obtain a screened sixth target feature;

[0182] The step of determining the sample corresponding to the fifth target feature as the first sample includes:

[0183] The sample corresponding to the sixth target feature is determined as the first sample.

[0184] Optionally, the step of selecting a second sample from the mining sample pool based on the expanded training sample set and the test sample set by a value retrieval method, and expanding the second sample to the test sample set to obtain the expanded test sample set includes:

[0185] Performing combined full-image feature extraction on the expanded training sample set and the test sample set to obtain a first full-image feature, and performing full-image feature extraction on the mining sample pool to obtain a second full-image feature;

[0186] Determining a value score of each sample in the mining sample pool based on the first full-image feature and the second full-image feature;

[0187] The sample with the highest value score in the mining sample pool is determined as the second sample, and the second sample is manually annotated and then expanded to the test sample set to obtain an expanded training sample set.

[0188] Optionally, the step of determining the value score of each sample in the mining sample pool based on the first full-image feature and the second full-image feature includes:

[0189] Based on the first full-image feature and the second full-image feature, a feature distance set from each sample in the mining sample pool to each sample in the expanded training sample set and the test sample set is calculated, and a target feature distance with the smallest feature distance in the feature distance set is selected;

[0190] Obtaining the target confidence of each sample in the mining sample pool, the expected number of categories, and the sample uncertainty of the mining sample pool;

[0191] The target confidence, the sample uncertainty, the expected number of categories and the target feature distance are weightedly calculated to obtain a value score for each sample in the mining sample pool.

[0192] The specific implementation of the storage medium of the present application is basically the same as the above-mentioned embodiments of the target detection method, and will not be repeated here.

[0193] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned target detection method when executed by a processor.

[0194] The specific implementation of the computer program product of the present application is basically the same as the above-mentioned target detection method embodiments, and will not be repeated here.

[0195] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0196] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0197] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0198] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A target detection method, characterized in that: The target detection method comprises: Acquire the image of the target object to be detected; Based on the image to be detected, image detection processing is performed through a preset target detection model to obtain a detection result of the target object; Among them, the target detection model is obtained by iteratively training a preset first detection model based on a training sample set, a test sample set and a mining sample pool. During the training process of the target detection model, an image retrieval method is used to select a first sample and a second sample from the mining sample pool and expand them to the training sample set and the test sample set respectively.

2. The target detection method according to claim 1, characterized in that: Before the step of acquiring the image to be detected of the target object, the method includes: Obtain training sample sets, test sample sets and mining sample pools; Based on the preset first detection model, the training sample set and the test sample set, a first sample is selected from the mining sample pool by image retrieval, and the first sample is expanded to the training sample set to obtain an expanded training sample set; Based on the expanded training sample set, iteratively train the first detection model to obtain a second detection model; Based on the expanded training sample set and the test sample set, a second sample is selected from the mining sample pool by a value search method, and the second sample is expanded to the test sample set to obtain an expanded test sample set; Based on the second detection model, the expanded training sample set and the expanded test sample set, the above sample set expansion and model training process is repeated until the trained detection model meets the preset training conditions and the training is stopped to obtain the target detection model.

3. The target detection method according to claim 2, characterized in that: The step of selecting a first sample from the mining sample pool by image retrieval based on the preset first detection model, the training sample set and the test sample set, and expanding the first sample to the training sample set to obtain the expanded training sample set includes: Based on the test sample set, a third sample is obtained by evaluating the test sample set through a preset first detection model, wherein the third sample refers to a sample in the test sample set that has a detection error problem; Based on the third sample, performing image retrieval on the mining sample pool to obtain a first sample; After manually annotating the first samples, the samples are expanded to the training sample set to obtain an expanded training sample set.

4. The target detection method according to claim 3, characterized in that: The step of performing image retrieval on the mining sample pool based on the third sample to obtain the first sample includes: Extract target features from the training sample set, the mining sample pool, and the third sample respectively to obtain a first target feature corresponding to the training sample set, a second target feature corresponding to the mining sample pool, and a third target feature corresponding to the third sample; Calculating a first cosine similarity between the first target feature and the second target feature, and retaining sample target features in the mining sample pool whose first cosine similarity is less than a preset first similarity threshold, to obtain a fourth target feature; Calculating the second cosine similarity between the fourth target feature and the third target feature, and retaining the target features in the mining sample pool whose second cosine similarity is greater than or equal to a preset second similarity threshold, to obtain a fifth target feature; The sample corresponding to the fifth target feature is determined as the first sample.

5. The target detection method according to claim 4, characterized in that: After the step of retaining the target features whose second cosine similarity in the mining sample pool is greater than or equal to the preset second similarity threshold to obtain the fifth target feature, the method includes: Calculating the third cosine similarity between target features in the fifth target feature; Selecting a target feature having the highest second cosine similarity among at least two target features whose third cosine similarity is greater than or equal to a preset third similarity threshold, to obtain a screened sixth target feature; The step of determining the sample corresponding to the fifth target feature as the first sample includes: The sample corresponding to the sixth target feature is determined as the first sample.

6. The target detection method according to claim 2, characterized in that: The step of selecting a second sample from the mining sample pool based on the expanded training sample set and the test sample set by a value search method, and expanding the second sample to the test sample set to obtain the expanded test sample set includes: Performing combined full-image feature extraction on the expanded training sample set and the test sample set to obtain a first full-image feature, and performing full-image feature extraction on the mining sample pool to obtain a second full-image feature; Determining a value score of each sample in the mining sample pool based on the first full-image feature and the second full-image feature; The sample with the highest value score in the mining sample pool is determined as the second sample, and the second sample is manually annotated and then expanded to the test sample set to obtain an expanded training sample set.

7. The target detection method according to claim 6, characterized in that: The step of determining the value score of each sample in the mining sample pool based on the first full-image feature and the second full-image feature includes: Based on the first full-image feature and the second full-image feature, a feature distance set from each sample in the mining sample pool to each sample in the expanded training sample set and the test sample set is calculated, and a target feature distance with the smallest feature distance in the feature distance set is selected; Obtaining the target confidence of each sample in the mining sample pool, the expected number of categories, and the sample uncertainty of the mining sample pool; The target confidence, the sample uncertainty, the expected number of categories and the target feature distance are weightedly calculated to obtain a value score for each sample in the mining sample pool.

8. A target detection device, characterized in that: The target detection device comprises: An acquisition module, used for acquiring an image to be detected of a target object; A detection module, used to perform image detection processing based on the image to be detected through a preset target detection model to obtain a detection result of the target object; Among them, the target detection model is obtained by iteratively training a preset first detection model based on a training sample set, a test sample set and a mining sample pool. During the training process of the target detection model, an image retrieval method is used to select a first sample and a second sample from the mining sample pool and expand them to the training sample set and the test sample set respectively.

9. The target detection device according to claim 8, characterized in that: The target detection device also includes: A sample acquisition module is used to acquire a training sample set, a test sample set and a mining sample pool; a first retrieval module is used to select a first sample from the mining sample pool by image retrieval based on a preset first detection model, the training sample set and the test sample set, and expand the first sample to the training sample set to obtain an expanded training sample set; a training module is used to iteratively train the first detection model based on the expanded training sample set to obtain a second detection model; a second retrieval module is used to select a second sample from the mining sample pool by value retrieval based on the expanded training sample set and the test sample set, and expand the second sample to the test sample set to obtain an expanded test sample set; an iterative training module is used to repeat the above sample set expansion and model training process based on the second detection model, the expanded training sample set and the expanded test sample set, until the trained detection model meets the preset training conditions, then stop the training and obtain the target detection model; And / or, the first retrieval module includes: a problem evaluation module, which is used to evaluate the test sample set through a preset first detection model to obtain a third sample, wherein the third sample refers to a sample in the test sample set that has a detection error problem; an image retrieval module, which is used to perform image retrieval on the mining sample pool based on the third sample to obtain a first sample; a first expansion module, which is used to expand the first sample to the training sample set after manually annotating it to obtain an expanded training sample set; And / or, the image retrieval module includes: a target feature extraction module, used to extract target features from the training sample set, the mining sample pool and the third sample respectively, to obtain a first target feature corresponding to the training sample set, a second target feature corresponding to the mining sample pool and a third target feature corresponding to the third sample; a first deduplication module, used to calculate the first cosine similarity between the first target feature and the second target feature, and retain the sample target features in the mining sample pool whose first cosine similarity is less than a preset first similarity threshold, to obtain a fourth target feature; a similar image retrieval module, used to calculate the second cosine similarity between the fourth target feature and the third target feature, and retain the target features in the mining sample pool whose second cosine similarity is greater than or equal to a preset second similarity threshold, to obtain a fifth target feature; and a sample determination module, used to determine the sample corresponding to the fifth target feature as the first sample. And / or, the target detection device further includes: a similarity calculation module, used to calculate the third cosine similarity between the target features in the fifth target feature; a second deduplication module, used to select the target feature with the highest second cosine similarity among at least two target features whose third cosine similarity is greater than or equal to a preset third similarity threshold, to obtain a screened sixth target feature; And / or, the second retrieval module includes: a full-image feature extraction module, which is used to perform full-image feature extraction on the expanded training sample set and the test sample set to obtain a first full-image feature, and perform full-image feature extraction on the mining sample pool to obtain a second full-image feature; a scoring module, which is used to determine the value score of each sample in the mining sample pool based on the first full-image feature and the second full-image feature; a second expansion module, which is used to determine the sample with the highest value score in the mining sample pool as the second sample, and after manually annotating the second sample, expand it to the test sample set to obtain an expanded training sample set. And / or, the scoring module includes: a feature distance calculation module, which is used to calculate the feature distance set from each sample in the mining sample pool to the expanded training sample set and the test sample set based on the first full-image feature and the second full-image feature, and select the target feature distance with the smallest feature distance in the feature distance set; a weight acquisition module, which is used to obtain the target confidence, the expected number of categories and the sample uncertainty of each sample in the mining sample pool; and a weighted calculation module, which is used to perform weighted calculation on the target confidence, the sample uncertainty, the expected number of categories and the target feature distance to obtain the value score of each sample in the mining sample pool.

10. A target detection device, characterized in that: The target detection device comprises: a memory, a processor, and a program stored in the memory for implementing the target detection method. The memory is used to store a program for implementing the target detection method; The processor is used to execute a program for implementing the target detection method to implement the steps of the target detection method as claimed in any one of claims 1 to 7.

11. A storage medium, characterized in that: The storage medium stores a program for implementing the target detection method, and the program for implementing the target detection method is executed by a processor to implement the steps of the target detection method as described in any one of claims 1 to 7.

Images