A labeling method and system based on multi-channel data

Abstract

The present application provides a kind of based on the labeling method and labeling system of multi-channel data, through the pre-labeling capability of fusing multi-person online collaborative labeling, unsupervised model and mapping strategy, realize the comprehensive optimization of multi-channel image data labeling task, significantly reduce the labor intensity of artificial labeling, improve the overall efficiency of labeling work and ensure the labeling quality, while through dynamic adjustment task allocation, real-time data synchronization and intelligent labeling quality evaluation, significantly improve the labeling efficiency and data quality.

Description

Technical Field

[0001] This invention relates to the field of training image annotation technology, and in particular to an annotation method and system based on multi-channel data. Background Technology

[0002] In the training of deep learning models, high-quality data annotation is a key factor in improving model performance. Especially in fields such as industrial defect detection, object detection, and medical image analysis, accurate image annotation data directly impacts the model's generalization ability and final application results. However, current image annotation methods primarily rely on manual annotation, a process that typically suffers from the following problems:

[0003] 1) Due to the massive amount of data, manual annotation often requires a significant investment of manpower and time. In the development of enterprise-level high-precision customized models, data annotation time can account for more than 50% of the entire project cycle, greatly slowing down the algorithm development progress.

[0004] 2) Inconsistencies in data can easily arise due to varying levels of experience among annotators, especially in scenarios requiring high-precision annotation, such as industrial defect detection and small object detection. Annotation errors or data noise can directly affect the accuracy of the trained model.

[0005] 3) Traditional annotation methods lack efficient multi-person collaboration tools, making it difficult to synchronize data annotated by different people, which can easily lead to data conflicts and chaotic version management. In addition, the allocation of annotation tasks and quality review require manual operation, increasing communication costs.

[0006] To address the aforementioned annotation challenges, methods for automatic image annotation based on algorithmic models have been proposed, such as those described in patent application CN118429279A. However, the annotated images used in that patent still require manual annotation. Although the trained model can ultimately help with image annotation, a significant amount of manual annotation is still needed during model training, perpetuating the aforementioned problems—it's time-consuming and labor-intensive. Therefore, an annotation method and system that can alleviate the workload of annotators is needed. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies and provide a labeling method and system based on multi-channel data.

[0008] To solve the above problems, the present invention adopts the following technical solution:

[0009] A labeling method based on multi-channel data includes the following steps:

[0010] Step 1: Acquire multi-channel images of multiple target objects for training using a 2.5D photometric stereo camera;

[0011] Step 2: Based on whether the target object has defects, the acquired multi-channel images are divided into two categories: one is target object image data with defects, and the other is target object image data without defects.

[0012] Step 3: Input the image data of the defect-free target object into the unsupervised learning model for training to obtain the unsupervised training model;

[0013] Step 4: Input the defective target image data into the completed unsupervised training model for pre-annotation;

[0014] Step 5: Assign the pre-annotated defect target images to different annotators for manual review;

[0015] Step 6: Map the annotation information in the re-inspected target image to the other channel image data of the corresponding multi-channel image;

[0016] Step 7: Obtain multi-channel images of several labeled targets for training a deep learning model. End of step.

[0017] Furthermore, in step 3, the image of the same channel from the multi-channel images of multiple targets is input into the unsupervised learning model for training to obtain an unsupervised training model; corresponding to the multi-channel images, an unsupervised training model corresponding to the number of channels is obtained.

[0018] Furthermore, in step 4, the defective multi-channel target image data is input into the unsupervised training model corresponding to its channel for pre-labeling.

[0019] Furthermore, in step 5, when assigning images to annotators, it is necessary to use the Q-learning algorithm and a reward mechanism, combined with the annotators' historical annotation data and the image annotation requirements corresponding to the target objects, to complete the image assignment.

[0020] Furthermore, the historical annotation data of the annotator includes annotation speed, annotation accuracy, number of historical erroneous annotations, and total annotation time; the image annotation requirements corresponding to the target object include the difficulty of image annotation and the required accuracy of annotation.

[0021] Furthermore, during the re-inspection process in step 6, if the pre-labeling is correct, the correct pre-labeling information is mapped onto the image data of other channels; if the labeling is incorrect or no labeling is performed, the incorrect labeling is manually corrected or a labeling is manually added, and the result is mapped onto the image data of other channels.

[0022] Furthermore, the multi-channel images obtained in step 1 are labeled with image acquisition time and channel name.

[0023] A multi-channel data annotation system is provided. Based on the above annotation method, the annotation system includes a server and at least one client, wherein the client is connected to the server to form a distributed collaborative system; the server is used to store data and allocate image annotation tasks; the client is used to complete annotation review.

[0024] Furthermore, the server includes a database and a task image allocation module; the database uses a MySQL database and is responsible for storing and managing the annotation data of images from all channels, as well as the historical annotation information of the annotators; the task image allocation module dynamically allocates image annotation review tasks based on the historical annotation information of the annotators and the image annotation requirements.

[0025] Furthermore, the client includes a UI interface, a server interaction module, an annotation specialist collaboration module, an annotation quality evaluation module, and a monitoring module;

[0026] The UI was developed using C++ and the Qt framework. The UI provides functions such as image loading, selection, annotation modification, and adding annotations to regions.

[0027] The server interaction module is responsible for data communication between the client and the server. The server interaction module interacts with the server based on the TCP / IP protocol. The operations of the client's UI interface will be synchronized to the server in real time through the server interaction module. The server will also issue image annotation re-inspection tasks through the server interaction module.

[0028] The annotation collaboration module uses long polling technology to synchronize annotation modifications of the same image from multiple clients; for annotations that are being modified, it will also "lock" them based on an optimistic locking mechanism.

[0029] The annotation quality assessment module uses a consensus mechanism to evaluate the annotated images reviewed by the annotators, and measures the annotation differences between different annotators for the same annotated image using the Kappa coefficient; if the annotation difference exceeds the threshold, it is considered that the difference is too large and there is an annotation error, and the administrator is reminded to review it.

[0030] The monitoring module includes an administrator interface, which is developed using the Qt framework. The administrator interface displays the annotator's task completion status, annotation speed, and annotation quality; the administrator interface also allows manual adjustment of the task allocation for annotated images.

[0031] The beneficial effects of this invention are as follows:

[0032] An unsupervised training model is obtained by training with defect-free target object image data to complete the initial image pre-annotation task, reducing the workload of manual annotation and transforming the original annotation work of annotators into review work, thereby improving efficiency and accuracy.

[0033] By using multi-channel images for annotation, various types of defects in the target object can be displayed more clearly. Furthermore, the multi-channel images can be used for mutual verification, thereby improving the accuracy of the annotation.

[0034] By employing the Q-learning algorithm and setting a reward mechanism, the annotation review task is allocated, and the image content to be annotated is reasonably distributed, thereby improving overall efficiency while ensuring the quality of annotation.

[0035] By employing long polling technology and optimistic locking mechanism, multiple annotators can simultaneously modify the same image, improving collaboration efficiency while avoiding the generation of erroneous data.

[0036] Through a consensus mechanism, multiple annotators conduct a consistency evaluation of the image annotation results, reducing the possibility of incorrect annotations. Attached Figure Description

[0037] Figure 1 This is a flowchart of the annotation method in Example 1. Detailed Implementation

[0038] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0039] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the figures only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0040] Example 1:

[0041] like Figure 1 As shown, a labeling method based on multi-channel data includes the following steps:

[0042] Step 1: Acquire multi-channel images of multiple target objects for training using a 2.5D photometric stereo camera;

[0043] Step 2: Based on whether the target object has defects, the acquired multi-channel images are divided into two categories: one is target object image data with defects, and the other is target object image data without defects.

[0044] Step 3: Input the image data of the defect-free target object into the unsupervised learning model for training to obtain the unsupervised training model;

[0045] Step 4: Input the defective target image data into the completed unsupervised training model for pre-annotation;

[0046] Step 5: Assign the pre-annotated defect target images to different annotators for manual review;

[0047] Step 6: Map the annotation information in the re-inspected target image to the other channel image data of the corresponding multi-channel image;

[0048] Step 7: Obtain multi-channel images of several labeled targets for training a deep learning model. End of step.

[0049] Traditional optical cameras only capture two-dimensional images of a target object from one angle, which can easily lead to some defects in the target object not being clearly displayed. Therefore, this application uses a 2.5D photometric stereo camera. The 2.5D photometric stereo camera is built based on the principle of photometric stereo processing. Compared to traditional optical cameras, it employs different processing methods, such as calculating the minimum, median, and maximum values ​​and normal vectors for each pixel and projecting them onto different x, y, and z directions to obtain rich multi-channel image data. This allows for a clearer display of various types of defects in the target object and also provides more training image data. It should be noted that, to facilitate data tracking and annotation, in this example, the acquired multi-channel images are labeled with "acquisition time + channel name." This facilitates the classification of images of different targets acquired in the same channel and allows them to be trained in the same model in subsequent steps.

[0050] In step 2, the image data is divided into defective target object image data and non-defective target object image data by manual screening.

[0051] In step 3, images of the same channel from multiple target objects in multi-channel images are input into an unsupervised learning model for training to obtain an unsupervised training model; corresponding to the multi-channel images, an unsupervised training model corresponding to the number of channels is obtained. The training process specifically includes the following steps:

[0052] Step 31: Preprocess the image, including normalizing the image pixel values;

[0053] Step 32: Construct the training model, including a pre-trained convolutional neural network and distribution modeling based on a Gaussian mixture model;

[0054] Step 33: Train the model by inputting the preprocessed image into the completed training model; the image training objective is set to minimize the reconstruction error of the normal image or feature distribution matching; the training parameters are set to Epochs: 100, and the learning rate is adjusted to 1 / 10 of the original when the loss stalls; a variational autoencoder is used to constrain the latent space distribution; it should be noted that a denoising autoencoder is also needed to denoise the input to the model.

[0055] Step 34: Obtain the completed unsupervised training model and end the process.

[0056] In step 4, the defective multi-channel target image data is input into the unsupervised training model corresponding to its channel for pre-labeling.

[0057] In step 5, when assigning images to annotators, the Q-learning algorithm is used, employing a reward mechanism that combines the annotators' historical annotation data with the image annotation requirements corresponding to the target objects to complete the image allocation. The reward mechanism includes rewards for correct annotation and rewards for annotation speed. It should be noted that annotated images requiring review will be assigned to several annotators. This allows for cross-verification based on the review results of each annotator, reducing the possibility of annotation errors. Furthermore, it facilitates collaborative annotation for reviewing complex annotated images.

[0058] The annotator's historical annotation data includes annotation speed, annotation accuracy, number of historical erroneous annotations, and total annotation time; the image annotation requirements for the target object include the difficulty of image annotation and the required accuracy of annotation.

[0059] In the re-inspection process of step 6, if the pre-labeling is correct, the correct pre-labeling information is mapped onto the image data of other channels; if the labeling is incorrect or no labeling is performed, the incorrect labeling is manually corrected or a labeling is manually added, and the result is mapped onto the image data of other channels. In this way, the labeling efficiency of multi-channel images of the same target object can be improved. In addition, by combining the labeling content displayed on the images of other channels, or by comparing the supplementary labeling content with the image content, the accuracy of the labeling can be better verified and the labeling quality can be guaranteed.

[0060] It should be noted that the multi-channel images in step 7 need to be manually reviewed before being used for training to prevent duplicate annotations and other issues.

[0061] A multi-channel data annotation system is provided. Based on the above annotation method, the annotation system includes a server and at least one client, wherein the client is connected to the server to form a distributed collaborative system; the server is used to store data and allocate image annotation tasks; the client is used to complete annotation review.

[0062] The server includes a database and a task image allocation module; the database uses MySQL and is responsible for storing and managing the annotation data of images from all channels, as well as the historical annotation information of the annotators; the task image allocation module dynamically allocates image annotation review tasks based on the historical annotation information of the annotators and the image annotation requirements.

[0063] The client includes a UI interface, a server interaction module, an annotation collaboration module, an annotation quality evaluation module, and a monitoring module.

[0064] The UI was developed using C++ and the Qt framework. It provides functions such as image loading, selection, annotation modification, and adding annotations to regions. The UI allows annotators to easily view and modify the content of images, and perform operations such as selection, annotation modification, and adding annotations to regions.

[0065] The server interaction module is responsible for data communication between the client and the server. The server interaction module interacts with the server based on the TCP / IP protocol. The operations of the client's UI interface will be synchronized to the server in real time through the server interaction module. The server will also distribute image annotation review tasks through the server interaction module. For example, the selected areas added by the annotator and the added or modified annotations will be transmitted to the server in real time.

[0066] The annotation collaboration module uses long polling technology to synchronize annotation modifications of the same image from multiple clients, enabling task collaboration among multiple annotators. For annotations being modified on a particular client, an optimistic locking mechanism is used to "lock" them, ensuring that only one annotator can modify the annotation at a time, preventing confusion caused by multiple modifications to the same data. However, there is no "locking" function for annotations added to regions; therefore, subsequent verification by the monitoring module is necessary to prevent duplicate annotations.

[0067] The annotation quality assessment module uses a consensus mechanism to evaluate the annotated images reviewed by the annotators, and measures the annotation differences between different annotators for the same annotated image using the Kappa coefficient; if the annotation difference exceeds the threshold, it is considered that the difference is too large and there is an annotation error, and the administrator is reminded to review it.

[0068] The monitoring module includes an administrator interface, which is developed using the Qt framework. The administrator interface displays the annotator's task completion status, annotation speed, and annotation quality. When necessary, the administrator interface can also manually adjust the task allocation of the annotated images, such as when the annotator is unable to continue completing the assigned annotation tasks. In addition, the monitoring module also provides data monitoring and statistical functions. The data monitoring displays the annotator's work information data, while the statistical function is used to analyze the work tasks completed by the annotator.

[0069] During implementation, an unsupervised training model was obtained by training on defect-free target object image data to complete the initial image pre-annotation task, reducing the workload of manual annotation and transforming the annotator's original annotation work into a review work, thus improving efficiency and accuracy. By using multi-channel images for annotation, various types of defects of the target object can be displayed more clearly, and the multi-channel images can also be used for mutual verification, improving the accuracy of annotation. By using the Q-learning algorithm and setting a reward mechanism, the annotation review task is allocated, and the image content to be annotated is reasonably distributed, improving overall efficiency while ensuring the quality of annotation. By using long polling technology and optimistic locking mechanism, multiple annotators can modify the same image simultaneously, improving collaboration efficiency while avoiding the generation of erroneous data. Through a consensus mechanism, multiple annotators conduct a consistency evaluation of the image annotation results, reducing the possibility of erroneous annotation.

[0070] The above description is merely a specific example of the present invention and does not constitute any limitation on the present invention. Obviously, those skilled in the art, after understanding the content and principles of the present invention, may make various modifications and changes in form and detail without departing from the principles and structure of the present invention; however, these modifications and changes based on the spirit of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A labeling method based on multi-channel data, characterized in that, Includes the following steps: Step 1: Acquire multi-channel images of multiple target objects for training using a 2.5D photometric stereo camera; Step 2: Based on whether the target object has defects, the acquired multi-channel images are divided into two categories: one is target object image data with defects, and the other is target object image data without defects. Step 3: Input the image data of the defect-free target object into the unsupervised learning model for training to obtain the unsupervised training model; Step 4: Input the defective target image data into the completed unsupervised training model for pre-annotation; Step 5: Assign the pre-annotated defect target images to different annotators for manual review; Step 6: Map the annotation information in the re-inspected target image to the other channel image data of the corresponding multi-channel image; Step 7: Obtain multi-channel images of several labeled targets for training the deep learning model, and end the step; In step 3, the image of the same channel in the multi-channel images of multiple targets is input into the unsupervised learning model for training to obtain an unsupervised training model; corresponding to the multi-channel image, an unsupervised training model corresponding to the number of channels is obtained. The training process specifically includes the following steps: Step 31: Preprocess the image, including normalizing the image pixel values; Step 32: Construct the training model, including a pre-trained convolutional neural network and distribution modeling based on a Gaussian mixture model; Step 33: Train the model by inputting the preprocessed image into the completed training model; The training objective is set to minimize the reconstruction error or feature distribution matching of normal images; the training parameters are set to 100 epochs and the learning rate is adjusted to 1 / 10 of the original when the loss stalls; a variational autoencoder is used to constrain the latent space distribution; and a denoising autoencoder is also used to denoise the input of the model. Step 34: Obtain the completed unsupervised training model and end the process; In the re-inspection process of step 6, if the pre-labeling is correct, the correct pre-labeling information is mapped onto the image data of other channels; if the labeling is incorrect or no labeling is performed, the incorrect labeling is manually corrected or the labeling is manually added, and the result is mapped onto the image data of other channels.

2. The annotation method based on multi-channel data according to claim 1, characterized in that, In step 4, the defective multi-channel target image data is input into the unsupervised training model corresponding to its channel for pre-labeling.

3. The annotation method based on multi-channel data according to claim 1, characterized in that, In step 5, when assigning images to annotators, the image assignment is completed by using the Q-learning algorithm, employing a reward mechanism, and combining the annotators' historical annotation data with the image annotation requirements corresponding to the target object.

4. The annotation method based on multi-channel data according to claim 3, characterized in that, The annotator's historical annotation data includes annotation speed, annotation accuracy, number of historical erroneous annotations, and total annotation time; the image annotation requirements for the target object include the difficulty of image annotation and the required accuracy of annotation.

5. The annotation method based on multi-channel data according to claim 1, characterized in that, The multi-channel images obtained in step 1 are labeled with image acquisition time and channel name.

6. A labeling system based on multi-channel data, characterized in that, Based on the annotation method according to any one of claims 1 to 5, the annotation system includes a server and at least one client, wherein the client is connected to the server to form a distributed collaborative system; the server is used to store data and allocate image annotation tasks. The client is used to complete the annotation review.

7. A labeling system based on multi-channel data according to claim 6, characterized in that, The server includes a database and a task image allocation module; the database uses MySQL and is responsible for storing and managing the annotation data of images from all channels, as well as the historical annotation information of the annotators; the task image allocation module dynamically allocates image annotation review tasks based on the historical annotation information of the annotators and the image annotation requirements.

8. The annotation system based on multi-channel data according to claim 6, characterized in that, The client includes a UI interface, a server interaction module, an annotation specialist collaboration module, an annotation quality evaluation module, and a monitoring module; The UI was developed using C++ and the Qt framework. The UI provides functions such as image loading, selection, annotation modification, and adding annotations to regions. The server interaction module is responsible for data communication between the client and the server. The server interaction module interacts with the server based on the TCP / IP protocol. The operations of the client's UI interface will be synchronized to the server in real time through the server interaction module. The server will also issue image annotation re-inspection tasks through the server interaction module. The annotation collaboration module uses long polling technology to synchronize annotation modifications of the same image from multiple clients; for annotations that are being modified, it will also "lock" them based on an optimistic locking mechanism. The annotation quality assessment module uses a consensus mechanism to evaluate the annotated images reviewed by the annotators, and measures the annotation differences between different annotators for the same annotated image using the Kappa coefficient; If the difference in labeling exceeds the threshold, it is considered that the difference is too large and there is a labeling error, and the administrator is reminded to review it. The monitoring module includes an administrator interface, which is developed using the Qt framework. The administrator interface displays the annotator's task completion status, annotation speed, and annotation quality; the administrator interface also allows manual adjustment of the task allocation for annotated images.

Images