A method for constructing a target detection model and an image target detection method

Abstract

The present invention discloses a method for constructing a target detection model and an image target detection method, which relates to the field of image data processing technology. The method can construct a target detection model so that when performing target detection on an image through the target detection model, the image annotation efficiency and accuracy can be improved. The method comprises: obtaining multiple first sample images; obtaining a first bounding box of a first object in each first sample image through an initial detection model; obtaining a second bounding box of the first object in each first sample image annotated by a user; determining a first coincidence parameter corresponding to each first sample image based on the first bounding box and the second bounding box in each first sample image; determining a second sample image from the multiple first sample images based on the number of second bounding boxes in each first sample image and the corresponding first coincidence parameter; and training the initial detection model using the second sample image having the second bounding box to obtain a target detection model.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a method for constructing a target detection model and an image target detection method. Background Art

[0002] Today's machine learning, especially deep learning models, are extremely powerful and have been widely applied in various fields. However, training these models requires large labeled datasets. Traditionally, manual labeling is costly, inefficient, and error-prone, and it is also impossible to construct large-scale standard datasets. Summary of the Invention

[0003] The present invention provides a method for constructing a target detection model and a method for detecting targets in an image, which can construct a target detection model so that when target detection is performed on an image using the target detection model, the efficiency and accuracy of image annotation can be improved.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present invention provides a method for constructing a target detection model, the method comprising:

[0006] Acquire a plurality of first sample images, each of the first sample images including a first object;

[0007] Performing target detection on each first sample image using the initial detection model to obtain a first bounding box of a first object in each first sample image;

[0008] Obtaining a second bounding box of the first object in each first sample image annotated by the user;

[0009] determining, based on the first bounding box and the second bounding box of the first object in each first sample image, a first overlap parameter corresponding to each first sample image, where the first overlap parameter is used to indicate a degree of overlap between all second bounding boxes in the first sample image and the corresponding first bounding box;

[0010] determining a second sample image from the plurality of first sample images according to the number of second bounding boxes in each first sample image and a first coincidence parameter corresponding to each first sample image;

[0011] The initial detection model is trained using a second sample image with a second bounding box to obtain an object detection model.

[0012] The method for constructing a target detection model provided by the present invention is used. In the process of constructing the target detection model, the first sample image is annotated by an initial detection model that only has basic detection capabilities. At the same time, the annotator annotates the same first sample image to obtain the first bounding box of the first object in each first sample image, and the second bounding box of the first object in each first sample image. According to the positional relationship between the first bounding box and the second bounding box (i.e., the degree of overlap), and the number of second bounding boxes, the first sample images can be classified into difficulty levels, and the second sample images with a higher difficulty coefficient in the first sample images can be determined. When training the initial detection model, only the second sample image with the second bounding box is used. Since the second sample image is a sample image with a higher difficulty coefficient, the detection precision, detection accuracy and detection efficiency of the target detection model finally obtained will be higher, especially for sample images with a higher difficulty coefficient.

[0013] In a possible implementation, determining the first coincidence parameter corresponding to each first sample image according to the first bounding box and the second bounding box of the first object in each first sample image includes:

[0014] determining an intersection-over-union ratio of each second bounding box in each first sample image with the corresponding first bounding box;

[0015] The sum of the intersection-and-union ratios corresponding to each second bounding box in the first sample image is determined to obtain a first coincidence parameter corresponding to each first sample image.

[0016] In one possible implementation, determining the second sample image from the plurality of first sample images according to the number of second bounding boxes in each first sample image and the first coincidence parameter corresponding to each first sample image includes:

[0017] determining a difference between the number of second bounding boxes in each first sample image and a first coincidence parameter corresponding to the first sample image;

[0018] When the difference value corresponding to the first sample image is greater than a preset threshold, the first sample image is determined as the second sample image.

[0019] In one possible implementation, the method for constructing the target detection model further includes:

[0020] Acquire a plurality of third sample images, each third sample image including the second object and a bounding box of each second object;

[0021] A plurality of third sample images are used to train the neural network model to obtain an initial detection model.

[0022] In a second aspect, the present invention provides a method for detecting an object in an image, the method comprising:

[0023] Acquire target images, each target image including a target object;

[0024] A target detection model is used to perform target detection on the target object in each target image to obtain a third bounding box of the target object in each target image. The target detection model is a model trained by the target detection model construction method of the first aspect and any possible implementation method thereof.

[0025] In one possible implementation, the above-mentioned object detection method for an image further includes:

[0026] Obtaining a first number of target images annotated by the target user within a preset time period, and a fourth bounding box of the target object in each target image;

[0027] Obtaining a second number of target images annotated by the target detection model within a preset time period;

[0028] determining, based on the third bounding box and the fourth bounding box of the target object in each target image, a second coincidence parameter corresponding to each target image, where the second coincidence parameter is used to indicate a degree of coincidence between all third bounding boxes in the target image and the corresponding fourth bounding box;

[0029] A labeling evaluation parameter of the target user is determined according to the first number, the second number, the second overlap parameter corresponding to each target image, and the number of third bounding boxes of each target image.

[0030] In one possible implementation, determining the target user's annotation evaluation parameter based on the first number, the second number, the second overlap parameter corresponding to each target image, and the number of third bounding boxes of each target image includes:

[0031] determining a speed parameter based on the first quantity and the second quantity;

[0032] determining a difference between the number of third bounding boxes in each target image and a second coincidence parameter corresponding to the target image;

[0033] Determine the sum of the difference between the number of third bounding boxes in each target image and the second coincidence parameter corresponding to the target image to obtain a quality parameter;

[0034] Determine the target user's annotation evaluation parameters based on the speed parameters and quality parameters.

[0035] In a possible implementation, determining the second coincidence parameter corresponding to each target image based on the third bounding box and the fourth bounding box of the target object in each target image includes:

[0036] determining an intersection-over-union (IoU) of each third bounding box in each target image with a corresponding fourth bounding box in the target image;

[0037] Determine the sum of the intersection-over-union (IoU) ratios for all third bounding boxes in each target image.

[0038] In a third aspect, the present invention provides a device for constructing a target detection model, the device comprising:

[0039] A first acquiring unit is configured to acquire a plurality of first sample images, each of which includes a first object;

[0040] a first processing unit, configured to perform target detection on each first sample image using an initial detection model to obtain a first bounding box of a first object in each first sample image;

[0041] The first obtaining unit is further configured to obtain a second bounding box of the first object in each first sample image annotated by the user;

[0042] The above-mentioned first processing unit is also used to determine the first overlap parameter corresponding to each first sample image based on the first bounding box and the second bounding box of the first object in each first sample image, where the first overlap parameter is used to indicate the degree of overlap between all second bounding boxes in the first sample image and the corresponding first bounding box. It is also used to determine the second sample image from multiple first sample images based on the number of second bounding boxes in each first sample image and the first overlap parameter corresponding to each first sample image, and use the second sample image with the second bounding box to train the initial detection model to obtain the target detection model.

[0043] In a fourth aspect, the present invention provides an image object detection device, the device comprising:

[0044] a second acquiring unit, configured to acquire target images, each target image including a target object;

[0045] The second processing unit is used to use a target detection model to perform target detection on the target object in each target image to obtain a third bounding box of the target object in each target image. The target detection model is a model trained by the target detection model construction method of the first aspect and any possible implementation method thereof.

[0046] In a fifth aspect, the present invention provides an image processing device comprising: a processor and a memory. The memory is configured to store computer program code, the computer program code comprising computer instructions. When the processor executes the computer instructions, the image processing device performs the method for constructing an object detection model according to the first aspect and any possible implementation thereof, or performs the method for detecting an object in an image according to the second aspect and any possible implementation thereof.

[0047] In a sixth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are run on an image processing device, the image processing device executes a method for constructing a target detection model such as the first aspect or any one of the possible implementations of the first aspect, or the image processing device executes a method for detecting an image such as the second aspect or any one of the possible implementations of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic structural diagram of an image processing device provided by an embodiment of the present invention;

[0049] Figure 2 One of the flowcharts of the method for constructing an object detection model provided by an embodiment of the present invention;

[0050] Figure 3 Schematic diagram of the positional relationship between the actual bounding box of the first object and the first bounding box after target detection is performed on the first sample image using the initial detection model in an embodiment of the present invention;

[0051] Figure 4 The second flowchart of the method for constructing an object detection model provided by an embodiment of the present invention;

[0052] Figure 5 The third flowchart of the method for constructing a target detection model provided by an embodiment of the present invention;

[0053] Figure 6 This is a flow chart of a method for detecting an object in an image according to an embodiment of the present invention;

[0054] Figure 7 The second flowchart of the method for detecting an object in an image provided by an embodiment of the present invention;

[0055] Figure 8 The third flowchart of the method for detecting an object in an image provided by an embodiment of the present invention;

[0056] Figure 9 A schematic diagram of the structure of a device for constructing an object detection model provided by an embodiment of the present invention;

[0057] Figure 10 A schematic diagram of the structure of an image target detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0059] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "multiple" means two or more. In addition, the use of "based on" or "according to" means openness and inclusiveness, because the process, steps, calculations or other actions "based on" or "according to" one or more of the conditions or values ​​may be based on additional conditions or values ​​beyond the stated in practice.

[0060] In order to be able to construct a target detection model and improve the detection accuracy, detection accuracy and detection efficiency of the target object in the target image when performing target detection on the image through the target detection model, that is, to improve the labeling accuracy, labeling accuracy and labeling efficiency of the target image, the embodiment of the present invention provides a method for constructing a target detection model and a target detection method for an image. In the process of constructing the target detection model, in order to improve the labeling accuracy, labeling accuracy and labeling efficiency of the target detection model for the target image, the sample images used to train the model are classified into difficulty levels. In the process of gradual iteration of the model, each time a sample image with a higher difficulty coefficient is used to train the model after the previous training, the robustness of the model can be gradually improved, so that the target detection model finally obtained has the ability to label difficult samples, thereby improving the labeling accuracy, labeling accuracy and labeling efficiency of the target image.

[0061] Figure 1 is one of the structural diagrams of the image processing device, such as Figure 1 As shown, the image processing apparatus may include: a processor 11, a memory 12, a communication interface 13, and a bus 14. The processor 11, the memory 12, and the communication interface 13 may be connected via the communication bus 14.

[0062] The processor 11 is the control center of the image processing device and can be a single processor 11 or a collective term for multiple processing elements. For example, the processor 11 can be a general-purpose central processing unit (CPU) or another general-purpose processor 11. The general-purpose processor 11 can be a microprocessor 11 or any conventional processor 11.

[0063] As an embodiment, the processor 11 may include one or more CPUs, for example, Figure 1 CPU0 and CPU1 are shown.

[0064] The memory 12 may be a read-only memory 12 (ROM) or other type of static storage device that can store static information and instructions, a random access memory 12 (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory 12 (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0065] In one possible implementation, the memory 12 may exist independently of the processor 11 and may be connected to the processor 11 via a bus 14 for storing instructions or program codes. When the processor 11 calls and executes the instructions or program codes stored in the memory 12, the method for constructing a target detection model or the method for detecting targets in an image provided in the following embodiments of the present invention may be implemented.

[0066] In another possible implementation, the memory 12 may also be integrated with the processor 11 .

[0067] The communication interface 13 is used to connect the image processing apparatus to other devices via a communication network, such as Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 23 may include a receiving unit for receiving data and a sending unit for sending data.

[0068] The bus 14 may be an Industry Standard Architecture (ISA) bus 14, a Peripheral Component Interconnect (PCI) bus 14, or an Extended Industry Standard Architecture (EISA) bus 14. The bus 14 may be divided into an address bus 14, a data bus 14, a control bus 14, and the like. For ease of representation, Figure 1 Only one thick line is used in the figure, but it does not mean that there is only one bus 14 or one type of bus 14.

[0069] It should be pointed out that Figure 1 The structure shown in the figure does not constitute a limitation on the image processing device. Figure 1 In addition to the components shown, the image processing apparatus may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0070] The execution subject of the target detection model construction method provided in the embodiment of the present invention is the target detection model construction device, and the execution subject of the image target detection method is the image target detection device. The target detection model construction device and the image target detection device can be integrated into one image processing device, or can be provided in different image processing devices. The image processing device can be a terminal device, or can be a CPU in the above-mentioned terminal device, or can be a control module in the above-mentioned terminal device for constructing a target detection model or performing target detection on an image. The embodiment of the present invention is described by taking the target detection model construction device and the image target detection device as an example in which they are integrated into one terminal device.

[0071] like Figure 2 As shown, the method for constructing a target detection model provided by an embodiment of the present invention includes the following steps 201 to 206.

[0072] 201. A terminal device obtains multiple first sample images, each of which includes a first object.

[0073] The first sample image may be an image captured by an image acquisition device on a vehicle in a driving scene. The first object of the first sample image may include but is not limited to a vehicle, a road, and a traffic sign.

[0074] 202. The terminal device performs target detection on each first sample image using the initial detection model to obtain a first bounding box of the first object in each first sample image.

[0075] The initial detection model refers to a target detection model that is trained during the initial training process and has only basic detection capabilities. That is, the initial detection model has poor detection capabilities and cannot accurately and quickly detect each first object included in each first sample image. This makes it possible that the number of first bounding boxes in each first sample image may be inconsistent with the number of first objects, or the position of a first bounding box does not correspond to the position of the corresponding first object.

[0076] Figure 3 FIG. 1 is a schematic diagram showing the positional relationship between the actual bounding box of the first object and the first bounding box after the initial detection model is used to perform target detection on the first sample image in an embodiment of the present invention. Figure 3 As shown, the position mismatch can be reflected in: the size of a first bounding box A1 is the same as the size of the actual bounding box B of the first object, but the positions do not overlap; or the size of a first bounding box A2 is different from the size of the corresponding actual bounding box B of the first object.

[0077] Optionally, the initial detection model may be obtained by training a neural network model multiple times. Specifically, the process of obtaining the initial detection model may include: the terminal device acquiring multiple third sample images, each third sample image including a second object and a bounding box of each second object; and training a neural network model pre-stored in the terminal device using the multiple third sample images to obtain the initial detection model.

[0078] It should be noted that the third sample image can be obtained from two sources. One is by annotators manually annotating bounding boxes on unlabeled sample images; the other is by obtaining annotated sample images from a public dataset. For example, the third sample image can be obtained from the China Traffic Sign Detection Dataset (CCTSDB).

[0079] If the third sample image is obtained by a human annotator manually annotating bounding boxes, the terminal device needs to preprocess the sample image before the annotator annotates the bounding boxes to facilitate subsequent annotators in marking the bounding boxes of the objects. For example, preprocessing methods may include, but are not limited to, distortion correction and resizing. After the annotator completes the bounding box annotation, a quality inspector will conduct a quality inspection of the annotator's annotation results to ensure that each object in the sample image is annotated with the correct bounding box. This ensures that the detection accuracy and precision of the first initial detection model trained by the neural network model are maintained to the greatest extent possible.

[0080] After the first initial detection model is trained, it is used to perform object detection on the corresponding unlabeled sample images, obtaining bounding boxes for each object in each sample image. However, due to the low accuracy of the first initial detection model, some objects in complex sample images may not be detected. In other words, when the first initial detection model is used to detect objects on sample images, some object bounding boxes may be missed or mislabeled. Therefore, the annotation results of the first initial detection model cannot be directly used to train subsequent initial detection models. In this case, human annotators are required to supplement or correct the missed or mislabeled objects. After the annotators complete the annotation, quality inspectors will conduct another quality inspection to ensure that each object in the sample image is correctly labeled with a bounding box. This ensures the detection accuracy and precision of each subsequently trained initial detection model. This iterative cycle continuously improves the detection accuracy of the next initial detection model until an initial detection model with excellent detection accuracy, precision, and efficiency is obtained.

[0081] In order to further improve the detection accuracy of the initial detection model finally obtained by the initial training, during the initial training process, the sample images can be gradually expanded to train the initial detection model.

[0082] It should be noted that during the training of the initial detection model, the annotator and the model trained each time will annotate the same sample image, and the annotator's annotation results are considered to be absolutely correct.

[0083] 203. The terminal device obtains a second bounding box of the first object in each first sample image annotated by the user.

[0084] In order to obtain a target detection model with higher detection accuracy, detection precision and detection efficiency, further mid-term training is still required for the initial detection model obtained in the initial training process.

[0085] During the mid-term training process, the labeler (ie, the user) will also label the first object in each first sample image, obtain the second bounding box of the first object, and input the second bounding box of the first object in each first sample image into the terminal device.

[0086] 204. The terminal device determines a first coincidence parameter corresponding to each first sample image according to the first bounding box and the second bounding box of the first object in each first sample image.

[0087] The first overlap parameter is used to indicate the overlap degree between all the second bounding boxes in the first sample image and the corresponding first bounding boxes.

[0088] 205. The terminal device determines a second sample image from the plurality of first sample images according to the number of second bounding boxes in each first sample image and the first coincidence parameter corresponding to each first sample image.

[0089] After selecting the second sample image from the first sample image, the annotator manually annotates the first objects in the second sample image to obtain a second bounding box for each first object in the second sample image.

[0090] 206. The terminal device uses the second sample image with the second bounding box to train the initial detection model to obtain a target detection model.

[0091] It should be noted that the mid-term training process is similar to the initial training process. The initial detection model must be trained multiple times before the final target detection model can be obtained. During the mid-term training process, after the terminal device uses the second sample image of the second bounding box to train the initial model, it will first obtain the first target detection model. Then, the terminal device will use the first target detection model to perform target detection on other sample images, obtain the bounding box of the object in each other sample image, and obtain the bounding box of the object in each other sample image annotated by the user. Next, the terminal device determines the target image with the bounding box annotated by the annotator from the other images based on the annotation results of the first target detection model and the annotation results of the annotator. Finally, the terminal device uses the target image to train the first target detection model. This cycle is iterated until the final target detection model is obtained.

[0092] In actual applications, the object detection model obtained after mid-term training has achieved the expected detection accuracy, precision, and efficiency, and the object detection model's image annotation results are superior to those of human annotators. However, in subsequent use, the object detection model obtained after mid-term training will need to be iterated based on actual conditions.

[0093] The method for constructing a target detection model provided by an embodiment of the present invention is used. In the process of constructing the target detection model, an initial detection model with only basic detection capabilities is used to annotate the first sample image. At the same time, the annotator annotates the same first sample image to obtain the first bounding box of the first object in each first sample image and the second bounding box of the first object in each first sample image. According to the positional relationship between the first bounding box and the second bounding box (i.e., the degree of overlap) and the number of second bounding boxes, the first sample images can be classified into difficulty levels, and the second sample images with a higher difficulty coefficient in the first sample images can be determined. When training the initial detection model, only the second sample image with the second bounding box is used. Since the second sample image is a sample image with a higher difficulty coefficient, the target detection model finally obtained can automatically annotate the target image, and the annotation precision, annotation accuracy and annotation efficiency are all high, especially for sample images with a higher difficulty coefficient.

[0094] Combine Figure 2 ,like Figure 4 As shown, the above step 204 may further include the following steps 401 and 402 .

[0095] 401. The terminal device determines an intersection-over-union ratio between each second bounding box and the corresponding first bounding box in each first sample image.

[0096] The higher the overlap between a first bounding box and a corresponding second bounding box, the higher the detection precision and accuracy of the initial detection model. The overlap can be represented by the intersection-over-union ratio of a second bounding box to the corresponding first bounding box.

[0097] For example, the intersection-over-union ratio of a second bounding box and the corresponding first bounding box can be calculated using the following formula (1).

[0098]

[0099] Among them, IOU represents intersection-over-union ratio; S intersection represents the overlapping area of ​​a second bounding box and the corresponding first bounding box, that is, the intersection area; S union Represents the union area of ​​a second bounding box and the corresponding first bounding box.

[0100] 402. The terminal device determines the sum of intersection-over-union ratios corresponding to each second bounding box in the first sample image to obtain a first coincidence parameter corresponding to each first sample image.

[0101] Combine Figure 4 ,like Figure 5 As shown, the above step 205 may further include the following steps 501 and 502.

[0102] 501. A terminal device determines a difference between the number of second bounding boxes in each first sample image and a first coincidence parameter corresponding to the first sample image.

[0103] Since the detection capability of the initial detection model is limited, the number of second bounding boxes annotated by the annotators is used as the criterion during the training of the initial detection model.

[0104] Exemplarily, the difference between the number of second bounding boxes in each first sample image and the first coincidence parameter corresponding to the first sample image can be calculated using the following formula (2).

[0105]

[0106] Among them, score sample represents the difference between the number of second bounding boxes in each first sample image and the first coincidence parameter corresponding to the first sample image, and the difference can indicate the detection capability of the initial detection model for a first sample image; n 人 represents the number of second bounding boxes in each first sample image; represents the first coincidence parameter; k1 represents the kth second bounding box in the first sample image.

[0107] 502. When the difference value corresponding to the first sample image is greater than a preset threshold, the terminal device determines the first sample image as the second sample image.

[0108] If the difference value corresponding to a first sample image is greater than a preset threshold, it indicates that the initial detection model has poor detection capabilities for that first sample image. The larger the difference value, the worse the initial detection model's detection capabilities for that first sample image. In other words, if the current initial detection model is unable to accurately detect each first object in that first sample image, then that first sample image is a sample image with a high difficulty coefficient for the current initial detection model, i.e., the second sample image.

[0109] If the difference value corresponding to a first sample image is less than a preset threshold, it indicates that the initial detection model has strong detection capabilities for that first sample image. The smaller the difference value, the better the initial detection model's detection capabilities for that first sample image. In other words, if the current initial detection model can accurately detect each first object in that first sample image, then that first sample image is a sample image with a low difficulty coefficient for the current initial detection model. During subsequent training, it will be filtered out and will no longer be used as a sample image to train the initial detection model.

[0110] The embodiment of the present invention also provides a method for performing target detection on an image using the target detection model constructed by the above construction method. Figure 6 As shown, the method for detecting an object in an image provided by an embodiment of the present invention includes the following steps 601 to 602.

[0111] 601. The terminal device obtains target images, each target image including a target object.

[0112] The target image is an image that needs to be detected, that is, annotated with the target object.

[0113] 602. The terminal device performs target detection on the target object in each target image using the target detection model to obtain a third bounding box of the target object in each target image.

[0114] Combine Figure 6 ,like Figure 7 As shown, the method for detecting an object in an image provided by the embodiment of the present invention may further include the following steps 701 to 704 .

[0115] 701. A terminal device obtains a first number of target images marked by a target user within a preset time period, and a fourth bounding box of a target object in each target image.

[0116] 702. The terminal device obtains a second number of target images marked by the target detection model within a preset time period.

[0117] 703. The terminal device determines a second coincidence parameter corresponding to each target image according to the third bounding box and the fourth bounding box of the target object in each target image.

[0118] The second overlap parameter is used to indicate the overlap degree between all third bounding boxes in the target image and the corresponding fourth bounding boxes.

[0119] In one embodiment, the terminal device may determine the intersection-and-union ratio of each third bounding box in each target image with the corresponding fourth bounding box in the target image, and use the sum of the intersection-and-union ratios corresponding to all third bounding boxes in each target image as the second coincidence parameter corresponding to each target image.

[0120] 704. The terminal device determines a labeling evaluation parameter of the target user based on the first number, the second number, the second overlap parameter corresponding to each target image, and the number of third bounding boxes of each target image.

[0121] Since the detection accuracy, detection precision and detection efficiency of the target detection model are better than those of the labelers, the labeling results of the target detection model can be used as a standard to evaluate the manual labeling results of the labelers (i.e., the target users).

[0122] During the evaluation process, the target detection model and the labeler simultaneously label a batch of target images within a preset time period, and the number of target images in the batch may be a second number. That is, the target detection model can label the second number of target images within the preset time period, and the second number corresponding to the target detection model is the rated workload of the labeler within the preset time period. The labeler can label the first number of target images within the preset time period, and the second number may be greater than or equal to the first number. When evaluating the manual labeling results of the labeler, the labeler's labeling evaluation parameters can be determined based on the first number completed by the labeler within the preset time period, the rated second number, the second overlap parameter corresponding to each target image, and the number of third bounding boxes for each target image.

[0123] It should be noted that the evaluation of annotators can be based on two aspects: annotation quality and annotation speed, and annotation quality is considered to be more important than annotation speed.

[0124] Combine Figure 7 ,like Figure 8 As shown, the above step 704 may specifically include the following steps 801 to 804.

[0125] 801. The terminal device determines a speed parameter according to the first quantity and the second quantity.

[0126] The speed parameter can be used to indicate the labeling efficiency of the labeler within a preset time period.

[0127] For example, the speed parameter may be determined by referring to the following formula (3).

[0128]

[0129] Among them, V person Indicates the speed parameter; Index indicates the second quantity; total person Represents the first quantity; 0.2 is the speed evaluation coefficient.

[0130] 802. The terminal device determines a difference between the number of third bounding boxes in each target image and the second coincidence parameter corresponding to the target image based on the number of third bounding boxes in each target image and the second coincidence parameter corresponding to each target image.

[0131] For example, the following formula (4) may be referred to to determine the difference between the number of third bounding boxes in each target image and the second coincidence parameter corresponding to the target image.

[0132]

[0133] Among them, score manun represents the difference between the number of third bounding boxes in each target image and the second coincidence parameter corresponding to the target image; 自 represents the number of third bounding boxes in each target image; represents the intersection-over-union ratio of each third bounding box in each target image to the corresponding fourth bounding box in the target image; k2 represents the kth third bounding box in the target image.

[0134] It should be noted that the smaller the difference between the number of third bounding boxes in a target image and the second overlap parameter corresponding to the target image, the closer the annotator's annotation results are to the annotation results of the target detection model, that is, the higher the quality of the annotator's annotation results for the target image. If the difference is less than the gap threshold, it can be confirmed that the annotator's annotation results for the target image are qualified, and the quality inspector does not need to perform manual quality inspection on the annotation results of the target image, thereby improving the quality inspection efficiency of the quality inspector; if the difference is greater than the gap threshold, it can be confirmed that the annotator's annotation results for the target image are unqualified. At this time, the quality inspector can notify the annotator to re-annotate the target image. Based on the difference between the number of third bounding boxes in each target image and the second overlap parameter corresponding to the target image, as well as the gap threshold, it is easy to distinguish target images with high-quality annotation results from those with low-quality annotation results.

[0135] 803. The terminal device determines the sum of the number of third bounding boxes in each target image and the difference between the second coincidence parameters corresponding to the target image to obtain a quality parameter.

[0136] For example, the quality parameter may be determined by referring to the following formula (5).

[0137]

[0138] Among them, M person represents the quality parameter; i represents the i-th target image; 0.8 is the quality evaluation coefficient.

[0139] 804. The terminal device determines the target user's marked evaluation parameters based on the speed parameter and the quality parameter.

[0140] According to the above formula (4) and formula (5), the following formula (6) can be obtained to determine the annotation evaluation parameters.

[0141] score person =M person +V person (6)

[0142] Among them, score person Represents the annotation evaluation parameters of a target user (i.e., annotator).

[0143] The above mainly introduces the solution provided by the embodiment of the present invention from the perspective of the device. It can be understood that in order to realize the above functions, the device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0144] Figure 9 A possible schematic diagram of the composition of the target detection model construction device 900 involved in the above embodiment is shown. Figure 9 As shown, the target detection model construction device 900 may include: a first acquisition unit 901 and a first determination unit 902.

[0145] The first acquisition unit 901 is configured to acquire multiple first sample images, each of which includes a first object. The first processing unit 902 is configured to perform target detection on each first sample image using an initial detection model to obtain a first bounding box of the first object in each first sample image. The first acquisition unit 901 is also configured to acquire a second bounding box of the first object in each first sample image annotated by a user. The first processing unit 902 is further configured to determine a first overlap parameter corresponding to each first sample image based on the first bounding box and the second bounding box of the first object in each first sample image, the first overlap parameter being used to indicate the degree of overlap between all second bounding boxes in the first sample image and the corresponding first bounding box. The first processing unit 902 is further configured to determine a second sample image from the multiple first sample images based on the number of second bounding boxes in each first sample image and the first overlap parameter corresponding to each first sample image, and to train the initial detection model using the second sample image having the second bounding box to obtain a target detection model.

[0146] Optionally, the first processing unit 902 is specifically configured to:

[0147] Determine an intersection-and-union ratio of each second bounding box in each first sample image with the corresponding first bounding box, and determine a sum of the intersection-and-union ratios corresponding to each second bounding box in the first sample image to obtain a first coincidence parameter corresponding to each first sample image.

[0148] Optionally, the first processing unit 902 is specifically configured to:

[0149] Determine the difference between the number of second bounding boxes in each first sample image and the first coincidence parameter corresponding to the first sample image. When the difference corresponding to the first sample image is greater than a preset threshold, determine the first sample image as the second sample image.

[0150] Optionally, the first acquiring unit 901 is further configured to acquire a plurality of third sample images, each of which includes a second object and a bounding box of each second object.

[0151] Optionally, the first processing unit 902 is further configured to train a neural network model using a plurality of third sample images to obtain an initial detection model.

[0152] Figure 10 A possible schematic diagram of the composition of the image target detection device 1000 involved in the above embodiment is shown. Figure 10 As shown, the image object detection device 1000 may include: a second acquisition unit 1001 and a second determination unit 1002.

[0153] The second acquisition unit 1001 is configured to acquire target images, each target image including a target object. The second processing unit is configured to perform target detection on the target object in each target image using a target detection model to obtain a third bounding box of the target object in each target image. The target detection model is the model trained by the first processing unit in the target detection model construction device described above.

[0154] Optionally, the above-mentioned second acquisition unit 1001 is also used to obtain a first number of target images marked by the target user within a preset time period, and a fourth bounding box of the target object in each target image; and obtain a second number of target images marked by the target detection model within a preset time period.

[0155] Optionally, the second processing unit 1002 is further configured to determine a second overlap parameter corresponding to each target image based on the third bounding box and the fourth bounding box of the target object in each target image, and determine a target user's annotation evaluation parameter based on the first number, the second number, the second overlap parameter corresponding to each target image, and the number of third bounding boxes in each target image. The second overlap parameter indicates the degree of overlap between all third bounding boxes in the target image and their corresponding fourth bounding boxes.

[0156] Optionally, the second processing unit 1002 is specifically configured to:

[0157] determining a speed parameter based on the first quantity and the second quantity;

[0158] determining, based on the number of third bounding boxes in each target image and the second coincidence parameter corresponding to each target image, a difference between the number of third bounding boxes in each target image and the second coincidence parameter corresponding to the target image;

[0159] Determine the sum of the difference between the number of third bounding boxes in each target image and the second coincidence parameter corresponding to the target image to obtain a quality parameter;

[0160] Determine the target user's annotation evaluation parameter according to the speed parameter and the quality parameter.

[0161] Optionally, the second processing unit 1002 is specifically configured to:

[0162] An intersection-of-union (IoU) ratio of each third bounding box in each target image to a corresponding fourth bounding box in the target image is determined, and a sum of the IoU ratios corresponding to all third bounding boxes in each target image is determined.

[0163] Of course, the object detection model construction device 900 and the image object detection device 1000 provided by the embodiment of the present invention include but are not limited to the above modules.

[0164] In actual implementation, the first acquisition unit 901, the first processing unit 902, the second acquisition unit 1001 and the second processing unit 1002 can be composed of Figure 1 The processor 11 shown is implemented by calling program codes in the memory 12 .

[0165] Another embodiment of the present invention further provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on the object detection model construction device 900, the object detection model construction device 900 executes each step executed by the terminal device in the method flow shown in the above method embodiment. Alternatively, when the computer instructions are executed on the image object detection device 1000, the image object detection device 1000 executes each step executed by the terminal device in the method flow shown in the above method embodiment.

[0166] Another embodiment of the present invention further provides a chip system, which is applied to the device for constructing a target detection model 900 and the device for detecting a target image 1000. The chip system includes one or more interface circuits and one or more processors 11. The interface circuit and the processor 11 are interconnected via a line. The interface circuit is used to receive a signal from the device for constructing a target detection model 900 or the memory 12 of the device for detecting a target image 1000, and to send the signal to the processor 11, wherein the signal includes a computer instruction stored in the memory 12. When the processor 11 executes the computer instruction, the device for constructing a target detection model 900 or the device for detecting a target image 1000 performs each step performed by the terminal device in the method flow shown in the above method embodiment.

[0167] In another embodiment of the present invention, a computer program product is also provided, which includes instructions. When the instructions are run on the target detection model construction device 900 or the image target detection device 1000, the target detection model construction device 900 or the image target detection device 1000 executes each step executed by the terminal device in the method flow shown in the above method embodiment.

[0168] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using a software program, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer-executable instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0169] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed by the present invention shall be covered by the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for constructing a target detection model, characterized in that: include: Acquire a plurality of first sample images, each of the first sample images including a first object; Performing target detection on each first sample image using the initial detection model to obtain a first bounding box of a first object in each first sample image; Obtaining a second bounding box of the first object in each first sample image annotated by the user; determining, based on the first bounding box and the second bounding box of the first object in each first sample image, a first coincidence parameter corresponding to each first sample image, wherein the first coincidence parameter is used to indicate a degree of coincidence between all second bounding boxes in the first sample image and the corresponding first bounding box; determining a second sample image from the plurality of first sample images according to the number of second bounding boxes in each first sample image and a first coincidence parameter corresponding to each first sample image; Training the initial detection model using the second sample image having the second bounding box to obtain the object detection model; The determining the second sample image from the plurality of first sample images according to the number of second bounding boxes in each first sample image and the first coincidence parameter corresponding to each first sample image includes: determining a difference between the number of second bounding boxes in each first sample image and a first coincidence parameter corresponding to the first sample image; When the difference value corresponding to the first sample image is greater than a preset threshold, the first sample image is determined as the second sample image.

2. The method for constructing a target detection model according to claim 1, wherein: The determining, based on the first bounding box and the second bounding box of the first object in each first sample image, a first coincidence parameter corresponding to each first sample image includes: determining an intersection-over-union ratio of each second bounding box in each first sample image with the corresponding first bounding box; The sum of the intersection-and-union ratios corresponding to each second bounding box in the first sample image is determined to obtain a first coincidence parameter corresponding to each first sample image.

3. The method for constructing a target detection model according to claim 1 or 2, characterized in that: The method for constructing the target detection model further includes: Acquire a plurality of third sample images, each third sample image including the second object and a bounding box of each second object; The neural network model is trained using the multiple third sample images to obtain the initial detection model.

4. A method for detecting an object in an image, characterized in that: include: Acquire target images, each target image including a target object; A target detection model is used to perform target detection on the target object in each target image to obtain a third bounding box of the target object in each target image. The target detection model is a model trained by the target detection model construction method described in any one of claims 1 to 3.

5. The method for detecting an object in an image according to claim 4, wherein: The target detection method of the image further includes: Obtaining a first number of target images annotated by the target user within a preset time period, and a fourth bounding box of the target object in each target image; Obtaining a second number of target images marked by the target detection model within the preset time period; determining, based on the third bounding box and the fourth bounding box of the target object in each target image, a second coincidence parameter corresponding to each target image, wherein the second coincidence parameter is used to indicate a degree of coincidence between all third bounding boxes in the target image and corresponding fourth bounding boxes; The labeling evaluation parameter of the target user is determined according to the first number, the second number, the second overlap parameter corresponding to each target image, and the number of third bounding boxes of each target image.

6. The method for detecting an object in an image according to claim 5, wherein: The determining the target user's annotation evaluation parameter according to the first number, the second number, the second overlap parameter corresponding to each target image, and the number of third bounding boxes of each target image includes: determining a speed parameter based on the first number and the second number; determining a difference between the number of third bounding boxes in each target image and a second coincidence parameter corresponding to the target image; Determine the sum of the difference between the number of third bounding boxes in each target image and the second coincidence parameter corresponding to the target image to obtain a quality parameter; Determine the target user's annotation evaluation parameter according to the speed parameter and the quality parameter.

7. The method for detecting an object in an image according to claim 5, wherein: The determining, based on the third bounding box and the fourth bounding box of the target object in each target image, a second coincidence parameter corresponding to each target image includes: determining an intersection-over-union (IoU) of each third bounding box in each target image with a corresponding fourth bounding box in the target image; Determine the sum of the intersection-over-union (IoU) ratios for all third bounding boxes in each target image.

8. A device for constructing a target detection model, characterized in that: include: A first acquiring unit is configured to acquire a plurality of first sample images, each of which includes a first object; a first processing unit, configured to perform target detection on each first sample image using an initial detection model to obtain a first bounding box of a first object in each first sample image; The first acquisition unit is further configured to acquire a second bounding box of the first object in each first sample image annotated by the user; The first processing unit is further configured to determine a first coincidence parameter corresponding to each first sample image based on the first bounding box and the second bounding box of the first object in each first sample image, where the first coincidence parameter indicates a degree of coincidence between all second bounding boxes in the first sample image and the corresponding first bounding box; and to determine a second sample image from the plurality of first sample images based on the number of second bounding boxes in each first sample image and the first coincidence parameter corresponding to each first sample image, and to train the initial detection model using the second sample image having the second bounding box to obtain the object detection model; When determining the second sample image from the plurality of first sample images according to the number of second bounding boxes in each first sample image and the first coincidence parameter corresponding to each first sample image, the first processing unit is specifically configured to: determining a difference between the number of second bounding boxes in each first sample image and a first coincidence parameter corresponding to the first sample image; When the difference value corresponding to the first sample image is greater than a preset threshold, the first sample image is determined as the second sample image.

9. An image target detection device, characterized in that: include: a second acquiring unit, configured to acquire target images, each target image including a target object; The second processing unit is used to use a target detection model to perform target detection on the target object in each target image to obtain a third bounding box of the target object in each target image, wherein the target detection model is a model trained by the target detection model construction method described in any one of claims 1 to 3.

10. An image processing device, characterized in that: The image processing device includes: a processor and a memory; the memory is used to store computer program code, and the computer program code includes computer instructions; when the processor executes the computer instructions, the image processing device executes the method for constructing a target detection model as described in any one of claims 1 to 3, or the image processing device executes the image target detection method as described in any one of claims 4 to 7.

11. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed on an image processing device, cause the image processing device to execute the method for constructing a target detection model as described in any one of claims 1 to 3, or cause the image processing device to execute the method for detecting a target image as described in any one of claims 4 to 7.

Images