Model training method, image labeling method, device, electronic equipment and medium

Abstract

The application provides a model training method, an image labeling method, a device, electronic equipment and a medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring an original image data set composed of a plurality of sample images; inputting the sample images in the original image data set into an initial first target detection model and an initial second target detection model to obtain first labeling results and second labeling results; dividing the original image data set into a first data set and a second data set according to the matching conditions of the first labeling results and the second labeling results; training a target detection model to be deployed according to the first data set to obtain an optimized target detection model; verifying the labeling accuracy of the optimized target detection model according to the second data set; and determining a final deployed target detection model according to the size relationship between the labeling accuracy and a preset threshold. In the present application, the labeling accuracy and speed of the target detection model can be greatly improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a model training method, an image annotation method, a model training device, an image annotation device, an electronic device, and a computer storage medium. Background Technology

[0002] Tags are highly refined feature identifiers obtained by applying algorithms such as abstraction, induction, and reasoning to target objects based on business scenario requirements, used for differentiated management and decision-making. Tagging is primarily applied to the processing of surveillance video data. However, in complex environments such as industrial parks and factories, existing object detection models perform poorly in recognizing human behavior in these environments.

[0003] In related technologies, existing target detection models have poor accuracy in labeling targets in images. Therefore, manual quality inspection is the only way to label them. However, the large amount of video image data can cause fatigue among quality inspectors, leading to missed or incorrect detections. Furthermore, it is time-consuming, labor-intensive, and has low speed and accuracy. Summary of the Invention

[0004] The present invention provides a model training method, an image annotation method, an apparatus, an electronic device, and a medium, which aim to solve or partially solve the problems existing in the background art.

[0005] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0006] In a first aspect, embodiments of the present invention provide a model training method, the method comprising:

[0007] Obtain the original image dataset consisting of multiple sample images;

[0008] The sample images in the original image dataset are input into the initial first target detection model to obtain the first annotation result, and the sample images in the original image dataset are input into the initial second target detection model to obtain the second annotation result;

[0009] Based on the matching results of the first and second annotations, the original image dataset is divided into a first dataset and a second dataset.

[0010] Based on the first dataset, the target detection model to be deployed is trained to obtain an optimized target detection model, and based on the second dataset, the annotation accuracy of the optimized target detection model is verified, wherein the target detection model to be deployed is the initial first target detection model and / or the initial second target detection model;

[0011] Based on the relationship between the annotation accuracy and the preset threshold, the final target detection model is determined.

[0012] Optionally, the step of inputting sample images from the original image dataset into an initial first object detection model to obtain a first annotation result, and inputting sample images from the original image dataset into an initial second object detection model to obtain a second annotation result, includes:

[0013] Each of the sample images is input into the initial first target detection model to obtain the number of first recognition boxes and the coordinates of the first recognition boxes for the sample images;

[0014] Each sample image is input into the initial second target detection model to obtain the number of second recognition boxes and the coordinates of the second recognition boxes for the sample image.

[0015] Optionally, the step of dividing the original image dataset into a first dataset and a second dataset based on the matching of the first annotation result and the second annotation result includes:

[0016] If the first annotation result and the second annotation result match, a filter mark is added to the sample image;

[0017] Based on whether the sample images carry filtering markers, the original image dataset is divided into a first dataset and a second dataset.

[0018] Optionally, the step of dividing the original image dataset into a first dataset and a second dataset based on whether the sample images carry filtering markers includes:

[0019] The sample images carrying the filtering tags in the original image dataset are integrated to obtain the first dataset;

[0020] The sample images in the original image dataset that do not carry the filtering markers are integrated to obtain the second dataset.

[0021] Optionally, the step of training the target detection model to be deployed based on the first dataset to obtain an optimized target detection model, and verifying the annotation accuracy of the optimized target detection model based on the second dataset, includes:

[0022] The first dataset is divided into a training dataset and a test dataset;

[0023] The target detection model to be deployed is trained based on the training set and the test set to adjust the model parameters of the target detection model to be deployed, so as to obtain the optimized target detection model to be deployed.

[0024] Using the second dataset as a validation dataset, the sample images from the second dataset are input into the optimized target detection model to be deployed, and the detection accuracy of the optimized target detection model to be deployed is obtained.

[0025] Optionally, the step of determining the final deployed target detection model based on the relationship between the annotation accuracy and the preset threshold includes:

[0026] If the detection accuracy of the optimized target detection model to be deployed is greater than or equal to a preset threshold, the optimized target detection model to be deployed is determined as the final deployed target detection model.

[0027] If the detection accuracy of the optimized target detection model to be deployed is less than a preset threshold, the step of dividing the original image dataset into a first dataset and a second dataset based on the matching of the first annotation result and the second annotation result continues.

[0028] Secondly, embodiments of the present invention provide an image annotation method, the method comprising:

[0029] Obtain a target image dataset consisting of multiple images to be labeled;

[0030] The images to be labeled in the target image dataset are input into the final deployed target detection model to obtain the labeling results corresponding to the images to be labeled. The final deployed target detection model is trained according to the model training method described in any one of the first aspects of this application.

[0031] Thirdly, embodiments of the present invention provide a model training apparatus, the apparatus comprising:

[0032] The sample image acquisition module is used to acquire the original image dataset consisting of multiple sample images;

[0033] The annotation module is used to input sample images from the original image dataset into an initial first target detection model to obtain a first annotation result, and to input sample images from the original image dataset into an initial second target detection model to obtain a second annotation result;

[0034] The partitioning module is used to partition the original image dataset into a first dataset and a second dataset based on the matching of the first annotation result and the second annotation result.

[0035] An optimization module is used to train the target detection model to be deployed based on the first dataset to obtain an optimized target detection model, and to verify the annotation accuracy of the optimized target detection model based on the second dataset, wherein the target detection model to be deployed is the initial first target detection model and / or the initial second target detection model;

[0036] The determination module is used to determine the final target detection model based on the relationship between the annotation accuracy and the preset threshold.

[0037] Optionally, the annotation module includes:

[0038] The first annotation submodule is used to input each of the sample images into the initial first target detection model to obtain the number of first recognition boxes and the coordinates of the first recognition boxes of the sample images;

[0039] The second annotation submodule is used to input each of the sample images into the initial second target detection model to obtain the number of second recognition boxes and the coordinates of the second recognition boxes of the sample images.

[0040] Optionally, the partitioning module includes:

[0041] The label-adding submodule is used to add filter tags to the sample image when the first annotation result and the second annotation result match;

[0042] The dataset partitioning submodule is used to partition the original image dataset into a first dataset and a second dataset based on whether the sample images carry filtering labels.

[0043] Optionally, the dataset is divided into sub-modules, including:

[0044] The first integration unit is used to integrate the sample images carrying the screening tags in the original image dataset to obtain the first dataset;

[0045] The second integration unit is used to integrate the sample images in the original image dataset that do not carry the filtering markers to obtain the second dataset.

[0046] Optional optimization modules include:

[0047] The first dataset partitioning submodule is used to partition the first dataset into a training dataset and a test dataset;

[0048] The training submodule is used to train the target detection model to be deployed based on the training set and the test set, so as to adjust the model parameters of the target detection model to be deployed, so as to obtain the optimized target detection model to be deployed.

[0049] The verification submodule is used to use the second dataset as a verification dataset, input sample images from the second dataset into the optimized target detection model to be deployed, and obtain the detection accuracy of the optimized target detection model to be deployed.

[0050] Optionally, the determination module includes:

[0051] The first execution submodule is used to determine the optimized target detection model to be deployed as the final deployed target detection model when the detection accuracy of the optimized target detection model to be deployed is greater than or equal to a preset threshold.

[0052] The second execution submodule is used to continue executing the step of dividing the original image dataset into a first dataset and a second dataset based on the matching of the first annotation result and the second annotation result when the detection accuracy of the optimized target detection model to be deployed is less than a preset threshold.

[0053] Fourthly, embodiments of the present invention provide an image annotation apparatus, the apparatus comprising:

[0054] The unlabeled image acquisition module is used to acquire a target image dataset consisting of multiple unlabeled images.

[0055] An image input module is used to input the images to be labeled in the target image dataset into the final deployed target detection model to obtain the labeling results corresponding to the images to be labeled. The final deployed target detection model is trained according to the model training method described in any one of the first aspects of this application.

[0056] Fifthly, embodiments of the present invention provide an electronic device, the electronic device comprising:

[0057] At least one processor; and a memory communicatively connected to the at least one processor; wherein,

[0058] The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform the method steps proposed in the first aspect of the present invention or the second aspect of the present invention.

[0059] In a sixth aspect, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method steps as proposed in the first aspect of the present invention or the second aspect of the present invention.

[0060] The embodiments of this invention include the following advantages: First, sample images from the original image dataset are input into an initial first object detection model and an initial second object detection model to obtain annotation results for the sample images under different object detection models, i.e., first annotation results and second annotation results. Then, based on the matching of the first annotation results and the second annotation results, the original image dataset is divided into a first dataset and a second dataset. That is, the annotation results of the main object detection model are verified by using the annotation results of the auxiliary object detection model, thereby greatly improving the annotation accuracy of the object detection model, and the original image dataset is divided into a first dataset with accurate recognition and a second dataset with inaccurate recognition. Finally, the object detection model to be deployed is trained based on the first dataset to obtain an optimized object detection model, and the annotation accuracy of the optimized object detection model is verified based on the second dataset. In other words, by using data with poor recognition performance of the main object detection model and the original data to strengthen the training of the main object detection model, the annotation accuracy of the object detection model is further improved, thereby obtaining the final object detection model. The final object detection model with high recognition accuracy is used to complete the annotation of the images to be annotated, thereby greatly reducing the workload of manual annotation. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart of the steps of a model training method in an embodiment of the present invention;

[0063] Figure 2 This is a flowchart of the steps of an image annotation method according to an embodiment of the present invention;

[0064] Figure 3 This is a schematic diagram of a model training device in an embodiment of the present invention;

[0065] Figure 4 This is a schematic diagram of an image annotation device according to an embodiment of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] In related technologies, in complex environments such as industrial parks and factories, existing target detection models perform poorly in recognizing human behavior in these environments, resulting in poor accuracy in labeling targets in images. Therefore, manual labeling is the only option. In order to improve the recognition accuracy of target detection models and reduce the corresponding manual workload and costs,

[0068] Based on this, embodiments of the present invention provide a model training method, see [link to relevant documentation]. Figure 1 , Figure 1 This application illustrates a flowchart of a model training method according to an embodiment of the present application. The method includes:

[0069] S101: Obtain the original image dataset consisting of multiple sample images.

[0070] In this embodiment, sample images can be obtained by performing frame extraction processing on video data acquired by an image acquisition device. For example, the image acquisition device acquires monitoring video data of a certain time period of the area to be monitored, then performs frame extraction processing on the monitoring video data to obtain single-frame sample images. These sample images are then aggregated to obtain the original image dataset. It should be noted that the sample images in the original image dataset can be acquired by a single camera or by multiple cameras. The type and source of the sample images can be determined according to the user's monitoring needs and task requirements; this application does not limit the type and source of the sample images.

[0071] S102: Input the sample images from the original image dataset into the initial first object detection model to obtain the first annotation result, and input the sample images from the original image dataset into the initial second object detection model to obtain the second annotation result.

[0072] In this embodiment, the initial first object detection model can be the main object detection model, which is the YOLOv5 algorithm model without parameter optimization. The YOLOv5 algorithm model can be used as the recognition model of the main object detection method. The initial second object detection model can be an auxiliary object detection model, which can be the Faster R-CNN algorithm model or the SDD-Net algorithm model without parameter optimization. The steps of inputting sample images from the original image dataset into the first object detection model and the second object detection model respectively to obtain the recognition results under different object detection models include:

[0073] S102-1: Input each sample image into the initial first target detection model to obtain the number of first recognition boxes and the coordinates of the first recognition boxes of the sample image;

[0074] S102-2: Input each sample image into the initial second object detection model to obtain the number of second recognition boxes and the coordinates of the second recognition boxes in the sample image;

[0075] In the embodiments S102-1 and S102-2, after inputting the sample image into the recognition model using the YOLOv5 algorithm as the object detection method, a first annotation result based on this object detection method is obtained, namely, the number of bounding boxes of the targets identified by the YOLOv5 algorithm model and the coordinate position information of each bounding box in the sample image. This information is then converted into first text information in txt format. After inputting the sample image into the recognition model using the Faster R-CNN algorithm model as the object detection method, a second annotation result based on this object detection method is obtained, namely, the number of bounding boxes of the targets identified by the Faster R-CNN algorithm and the coordinate position information of each bounding box in the sample image. This information is then converted into second text information in txt format.

[0076] In another feasible implementation, in order to improve the recognition accuracy of the second annotation result, the second annotation result can also be a weighted average of the annotation results of different auxiliary target detection models. That is, different weight coefficients are set for different auxiliary target detection models, and then the weighted sum is performed according to the weight coefficients to obtain a more accurate second annotation result.

[0077] S103: Based on the matching of the first annotation result and the second annotation result, the original image dataset is divided into the first dataset and the second dataset.

[0078] In this embodiment, after obtaining the first annotation result output by the recognition model using the YOLOv5 algorithm as the object detection method and the second annotation result output by the recognition model using the Faster R-CNN algorithm as the object detection method, it is necessary to divide the original image dataset according to the recognition situation represented by the first and second annotation results. The specific steps are as follows:

[0079] S103-1: If the first annotation result and the second annotation result match, add a filter mark to the sample image.

[0080] In this embodiment, it is necessary to determine the accuracy of the recognition results of the first text information representation and the second text information representation, so as to filter out the recognition results with credibility. The specific steps include:

[0081] S103-1-1: If the first text information and the second text information of the sample image match, add a filter mark to the sample image and retain its first text information;

[0082] S103-1-2: If the first text information and the second text information of a sample image do not match, do not add a filter mark to the sample image, and delete its first text information and second text information.

[0083] In the implementations of S103-1-1 to S103-1-2, two judgment conditions need to be met. First, the number of recognition boxes recorded in the first text information is the same as the number of recognition boxes recorded in the second text information. Second, the difference between the coordinate values ​​corresponding to each recognition box recorded in the first text information and the coordinate values ​​corresponding to each recognition box recorded in the second text information is less than a preset threshold. If these conditions are met, the first text information and the second text information are considered to match, and the initial first object detection model's annotation result for the sample image is considered reliable. Therefore, the first text information (i.e., the label information corresponding to the annotation result) is retained, and a filtering mark is added. Conversely, if any of the above judgment conditions are not met, it indicates that the first text information and the second text information do not match. Therefore, the initial first object detection model's annotation result for the sample image is considered unreliable, and the first and second text information are deleted. That is, the label information corresponding to the annotation result is not retained, and no filtering mark is added.

[0084] S103-2: Based on whether the sample images carry filtering markers, the original image dataset is divided into the first dataset and the second dataset.

[0085] In this embodiment, after judging the annotation results of each sample image, it is necessary to divide the original image dataset according to whether each sample image carries a filter marker. The specific steps include:

[0086] S103-2-1: Integrate the sample images with filter tags in the original image dataset to obtain the first dataset;

[0087] S103-2-2: Integrate the sample images without filter labels from the original image dataset to obtain the second dataset.

[0088] In the implementations of S103-2-1 to S103-2-2, the first dataset can be understood as a dataset consisting of sample images that have performed well in the initial first object detection model, while the second dataset can be understood as a dataset consisting of sample images that have performed poorly in the initial first object detection model. For any sample image, if it has a filtering label, it is placed in the set of the first dataset; if it does not have a filtering label, it is placed in the set of the second dataset. Based on this, the original image dataset can be divided.

[0089] S104: Based on the first dataset, train the object detection model to be deployed to obtain the optimized object detection model, and based on the second dataset, verify the annotation accuracy of the optimized object detection model.

[0090] In this embodiment, the target detection model to be deployed can be an initial first target detection model, or an initial first target detection model and an initial second target detection model. That is, only the initial first target detection model, i.e., the main target detection model, can be optimized, thus saving only the highest-precision YOLOv5 model. Alternatively, both the initial first and initial second target detection models can be optimized simultaneously, i.e., the main target detection model and the auxiliary target detection model are optimized simultaneously, thus saving both the highest-precision YOLOv5 model and the Faster R-CNN algorithm model. After obtaining the first and second datasets, it is necessary to continuously optimize the model parameters of the target detection model to be deployed based on the first dataset, thereby improving its detection accuracy on sample images in the second dataset. The specific steps include:

[0091] S104-1: Divide the first dataset into a training dataset and a test dataset;

[0092] S104-2: Train the target detection model to be deployed based on the training set and the test set, and adjust the model parameters of the target detection model to be deployed in order to obtain the optimized target detection model to be deployed.

[0093] S104-3: Use the second dataset as the validation dataset, input the sample images from the second dataset into the optimized target detection model to be deployed, and obtain the detection accuracy of the optimized target detection model to be deployed.

[0094] In the implementations of S104-1 to S104-3, the first dataset is divided into a training dataset and a test dataset according to a preset allocation ratio. Then, the model parameters of the target detection model to be deployed are optimized and adjusted based on the training dataset, and the model recognition accuracy of the optimized target detection model to be deployed is verified using the test dataset. The model recognition accuracy is the ratio of correctly labeled sample images to unlabeled sample images at the beginning.

[0095] If the model recognition accuracy meets the requirements, and only the YOLOv5 algorithm model is optimized, then the second dataset is used as the validation dataset. That is, the sample images in the second dataset are then input into the optimized first object detection model and the unoptimized second object detection model respectively, to obtain the third annotation result corresponding to the optimized first object detection model and the fourth annotation result corresponding to the unoptimized second object detection model. It should be noted that in this case, for the same sample image, its fourth annotation result and second annotation result are the same, only the annotation time is different.

[0096] If the model recognition accuracy meets the requirements, and it is necessary to optimize the YOLOv5 algorithm model and the Faster R-CNN algorithm model simultaneously, then the second dataset is used as the validation dataset. That is, the sample images in the second dataset are then input into the optimized first object detection model and the optimized second object detection model, respectively, to obtain the third annotation result corresponding to the optimized first object detection model and the fifth annotation result corresponding to the optimized second object detection model. It should be noted that in this case, for the same sample image, its fifth annotation result and second annotation result may be different, but the annotation time is different.

[0097] After obtaining the third and fourth annotation results or the third and fifth annotation results of the sample image, it is necessary to dynamically update the first and second datasets. That is, if the third and fourth annotation results match or the third and fourth annotation results of the sample image match, filter markers are added to the sample image, and the sample image is added from the second dataset to the first dataset, thereby realizing the dynamic update of the second and first datasets.

[0098] S105: Determine the final target detection model based on the relationship between the annotation accuracy and the preset threshold.

[0099] In this implementation, when the maximum number of iterations is reached or the accuracy of the target detection model to be deployed meets the requirements, the target detection model to be deployed obtained from the last iteration is output to obtain the final deployed target detection model, which is then used for target detection in subsequent images to be labeled. Simultaneously, the labeled sample images from the first and second datasets are saved to facilitate manual review. 。

[0100] The steps for determining the final target detection model based on the relationship between the annotation accuracy and the preset threshold can be as follows:

[0101] S105-1: If the detection accuracy of the optimized target detection model to be deployed is greater than or equal to the preset threshold, the optimized target detection model to be deployed will be determined as the final target detection model to be deployed.

[0102] S105-2: If the detection accuracy of the optimized target detection model to be deployed is less than the preset threshold, continue to execute the step of dividing the original image dataset into the first dataset and the second dataset according to the matching of the first annotation result and the second annotation result.

[0103] In the implementations of S105-1 to S105-2, if the detection accuracy of the optimized target detection model to be deployed is greater than or equal to the preset threshold, it indicates that the optimized target detection model has a good image annotation effect in complex environments. Therefore, the parameters of the optimized target detection model to be deployed can be solidified and output to obtain the target detection model. However, if the detection accuracy of the optimized target detection model to be deployed is less than the preset threshold, it indicates that the optimized target detection model has an insufficient image annotation effect in complex environments and needs to be further optimized. Therefore, the target detection model to be deployed needs to be further optimized based on the updated first dataset and second dataset.

[0104] The model training method provided in this application greatly improves the labeling accuracy of the target model by using the labeling results of the auxiliary target detection model to verify the labeling results of the main target detection model. Furthermore, by using data with poor recognition performance of the main target detection model and the original data to strengthen the training of the main target detection model, the labeling accuracy, i.e., the recognition precision, of the target model is further improved, thereby greatly reducing the workload of manual review.

[0105] After deploying the final target detection model, labels can be applied to the images acquired by the image acquisition device, helping users accurately identify the targets they want to detect. Based on this, this invention provides an image labeling method, see [link to relevant documentation]. Figure 2 , Figure 2 This paper illustrates a flowchart of an image annotation method according to an embodiment of the present application. The method includes:

[0106] S201: Obtain the target image dataset consisting of multiple images to be labeled.

[0107] In this embodiment, the image to be labeled refers to the image used for target recognition and labeling. The image to be labeled can be the image after frame extraction processing of the real-time monitoring video data collected by the monitoring equipment in the area to be monitored. The area to be monitored can be a complex environment such as a park or factory.

[0108] As an example, the target image dataset can be an image dataset composed of frame extraction processing of the monitoring video data of monitoring device B in time period C, located in area A to be monitored.

[0109] S202: Input the images to be labeled from the target image dataset into the final deployed target detection model to obtain the labeling results corresponding to the images to be labeled.

[0110] In this embodiment, the final deployed target detection model refers to the target detection model trained using the model training method described above. After obtaining the target image dataset, the target image dataset can be input into the final deployed target detection model to obtain the annotation result of each image to be labeled, that is, each target in the image to be labeled is labeled using a target bounding box.

[0111] The image annotation method provided in this embodiment inputs the images to be annotated in the target image dataset into the final deployed target detection model, thereby outputting the corresponding annotation results. Since the target detection model used in this embodiment is trained using the model training method of the above embodiment, it can realize the identification and annotation of targets in complex monitoring environments, thereby improving the accuracy and efficiency of target identification in complex monitoring environments and reducing the workload of manual annotation.

[0112] This invention also provides a model training device, with reference to... Figure 3 The diagram illustrates a functional block diagram of a model training device according to the present invention, which may include the following modules:

[0113] The sample image acquisition module 301 is used to acquire an original image dataset consisting of multiple sample images;

[0114] The annotation module 302 is used to input sample images from the original image dataset into the initial first target detection model to obtain the first annotation result, and input sample images from the original image dataset into the initial second target detection model to obtain the second annotation result;

[0115] The partitioning module 303 is used to partition the original image dataset into a first dataset and a second dataset based on the matching of the first annotation result and the second annotation result.

[0116] The optimization module 304 is used to train the target detection model to be deployed based on the first dataset to obtain the optimized target detection model, and to verify the annotation accuracy of the optimized target detection model based on the second dataset. The target detection model to be deployed is the initial first target detection model, or the initial first target detection model and the initial second target detection model.

[0117] The determination module 305 is used to determine the final target detection model based on the relationship between the annotation accuracy and the preset threshold.

[0118] In one feasible implementation, the annotation module includes:

[0119] The first annotation submodule is used to input each sample image into the initial first target detection model to obtain the number of first recognition boxes and the coordinates of the first recognition boxes of the sample image;

[0120] The second annotation submodule is used to input each sample image into the initial second target detection model to obtain the number of second recognition boxes and the coordinates of the second recognition boxes of the sample image.

[0121] In one feasible implementation, the modules are divided, including:

[0122] The label-adding submodule is used to add filter tags to sample images when the first and second annotation results match;

[0123] The dataset partitioning submodule is used to divide the original image dataset into a first dataset and a second dataset based on whether the sample images carry filter tags.

[0124] In one feasible implementation, the dataset is divided into sub-modules, including:

[0125] The first integration unit is used to integrate the sample images carrying filter tags in the original image dataset to obtain the first dataset;

[0126] The second integration unit is used to integrate sample images that do not carry filter labels in the original image dataset to obtain a second dataset.

[0127] In one feasible implementation, the optimization module includes:

[0128] The first dataset partitioning submodule is used to divide the first dataset into a training dataset and a test dataset;

[0129] The training submodule is used to train the target detection model to be deployed based on the training set and the test set, so as to adjust the model parameters of the target detection model to be deployed and obtain the optimized target detection model to be deployed.

[0130] The validation submodule is used to take the second dataset as the validation dataset, input the sample images in the second dataset into the optimized target detection model to be deployed, and obtain the detection accuracy of the optimized target detection model to be deployed.

[0131] In one feasible implementation, the determining module includes:

[0132] The first execution submodule is used to determine the optimized target detection model to be deployed as the final target detection model if the detection accuracy of the optimized target detection model to be deployed is greater than or equal to a preset threshold.

[0133] The second execution submodule is used to continue executing the step of dividing the original image dataset into a first dataset and a second dataset based on the matching of the first annotation result and the second annotation result if the detection accuracy of the optimized target detection model to be deployed is less than a preset threshold.

[0134] This invention also provides an image annotation device, referring to... Figure 4 The diagram illustrates a functional block diagram of an image annotation device according to the present invention, which may include the following modules:

[0135] The image acquisition module 401 is used to acquire a target image dataset consisting of multiple images to be labeled.

[0136] The image input module 402 is used to input the image to be labeled in the target image dataset into the final deployed target detection model to obtain the labeling result corresponding to the image to be labeled. The final deployed target detection model is trained according to the model training method of any one of the first aspects of this application.

[0137] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0138] Memory, used to store computer programs;

[0139] The processor, when executing a program stored in memory, implements the model training method or image annotation method of the present invention.

[0140] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned terminal and other devices. The memory can include Random Access Memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory can also be at least one storage system located remotely from the aforementioned processor.

[0141] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0142] In addition, to achieve the above objectives, embodiments of this application also propose a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the model training method or image annotation method of the embodiments of this application.

[0143] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable vehicles (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0145] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0147] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. "And / or" indicates that either one or both can be chosen. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0148] The above provides a detailed description of the model training method, image annotation method, apparatus, electronic device, and medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A model training method, characterized in that, The method includes: Obtain the original image dataset consisting of multiple sample images; The sample images in the original image dataset are input into the initial first target detection model to obtain the first annotation result, and the sample images in the original image dataset are input into the initial second target detection model to obtain the second annotation result; Based on the matching results of the first and second annotations, the original image dataset is divided into a first dataset and a second dataset. Based on the first dataset, the target detection model to be deployed is trained to obtain an optimized target detection model, and based on the second dataset, the annotation accuracy of the optimized target detection model is verified, wherein the target detection model to be deployed is the initial first target detection model and / or the initial second target detection model; Based on the relationship between the annotation accuracy and the preset threshold, the final target detection model to be deployed is determined; The step of dividing the original image dataset into a first dataset and a second dataset based on the matching of the first annotation result and the second annotation result includes: If the first annotation result and the second annotation result match, a filter mark is added to the sample image; Based on whether the sample images carry filtering markers, the original image dataset is divided into a first dataset and a second dataset; The step of dividing the original image dataset into a first dataset and a second dataset based on whether the sample images carry filtering markers includes: The sample images carrying the filtering tags in the original image dataset are integrated to obtain the first dataset; The sample images in the original image dataset that do not carry the filtering markers are integrated to obtain the second dataset.

2. The model training method according to claim 1, characterized in that, The steps of inputting sample images from the original image dataset into an initial first object detection model to obtain a first annotation result, and inputting sample images from the original image dataset into an initial second object detection model to obtain a second annotation result, include: Each of the sample images is input into the initial first target detection model to obtain the number of first recognition boxes and the coordinates of the first recognition boxes for the sample images; Each sample image is input into the initial second target detection model to obtain the number of second recognition boxes and the coordinates of the second recognition boxes for the sample image.

3. The model training method according to claim 1, characterized in that, The steps of training the target detection model to be deployed based on the first dataset to obtain an optimized target detection model, and verifying the annotation accuracy of the optimized target detection model based on the second dataset, include: The first dataset is divided into a training dataset and a test dataset; The target detection model to be deployed is trained based on the training set and the test set to adjust the model parameters of the target detection model to be deployed, so as to obtain the optimized target detection model to be deployed. Using the second dataset as a validation dataset, the sample images from the second dataset are input into the optimized target detection model to be deployed, and the detection accuracy of the optimized target detection model to be deployed is obtained.

4. The model training method according to claim 1, characterized in that, The steps for determining the final deployed target detection model based on the relationship between the annotation accuracy and the preset threshold include: If the detection accuracy of the optimized target detection model to be deployed is greater than or equal to a preset threshold, the optimized target detection model to be deployed is determined as the final deployed target detection model. If the detection accuracy of the optimized target detection model to be deployed is less than a preset threshold, the step of dividing the original image dataset into a first dataset and a second dataset based on the matching of the first annotation result and the second annotation result continues.

5. An image annotation method, characterized in that, The method includes: Obtain a target image dataset consisting of multiple images to be labeled; The images to be labeled in the target image dataset are input into the final deployed target detection model to obtain the labeling results corresponding to the images to be labeled. The final deployed target detection model is trained by the model training method according to any one of claims 1-4.

6. A model training device, characterized in that, The device includes: The sample image acquisition module is used to acquire the original image dataset consisting of multiple sample images; The annotation module is used to input sample images from the original image dataset into an initial first target detection model to obtain a first annotation result, and to input sample images from the original image dataset into an initial second target detection model to obtain a second annotation result; The partitioning module is used to partition the original image dataset into a first dataset and a second dataset based on the matching of the first annotation result and the second annotation result. An optimization module is used to train the target detection model to be deployed based on the first dataset to obtain an optimized target detection model, and to verify the annotation accuracy of the optimized target detection model based on the second dataset, wherein the target detection model to be deployed is the initial first target detection model and / or the initial second target detection model; The determination module is used to determine the final target detection model based on the relationship between the annotation accuracy and the preset threshold. The partitioning module includes: The label-adding submodule is used to add filter tags to the sample image when the first annotation result and the second annotation result match; The dataset partitioning submodule is used to divide the original image dataset into a first dataset and a second dataset based on whether the sample images carry filtering tags. The dataset partitioning submodule includes: The first integration unit is used to integrate the sample images carrying the screening tags in the original image dataset to obtain the first dataset; The second integration unit is used to integrate the sample images in the original image dataset that do not carry the filtering markers to obtain the second dataset.

7. An image annotation device, characterized in that, The device includes: The unlabeled image acquisition module is used to acquire a target image dataset consisting of multiple unlabeled images. An image input module is used to input the images to be labeled in the target image dataset into the final deployed target detection model to obtain the labeling results corresponding to the images to be labeled. The final deployed target detection model is trained by the model training method according to any one of claims 1-4.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor, when executing a program stored in memory, implements the model training method according to any one of claims 1-4 or the image annotation method according to claim 5.

9. A computer storage medium, characterized in that, The storage medium stores computer instructions, which, when executed by a processor, implement the model training method according to any one of claims 1-4 or the image annotation method according to claim 5.

Images