A label fusion generation method for multiple annotations of image regions
By constructing annotation evaluation data set and calculating intelligent measurements, and generating label probability maps, the problems of inconsistency, single type limitations and low degree of automation in image annotation technology are solved, and more efficient and reliable annotation information fusion is achieved, improving the improvement of the improvement of the problem of inconsistency in image annotation technology. The performance and application effect of the model.
Patent Information
- Application Number
- CN202411170640.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-26
AI Technical Summary
The existing image annotation technology has problems such as label inconsistency, limitations of single annotation type, subjective deviation, insufficient uncertainty processing, low degree of automation and high cost, which affects the model training effect and performance.
By constructing an annotation evaluation data set, we calculate the mid-intelligence membership, uncertainty and non-membership of each annotator type of the annotator, generate a label probability map, fuse information from different annotators, and automatically process data of multiple annotation types.
It reduces labeling deviation, improves labeling consistency and accuracy, reduces labeling costs, and improves the model's adaptability and efficiency in complex scenarios.
Smart Images

Figure CN118692088B_ABST
Abstract
Description
[0001] The present invention relates to the field of computer vision technology, and in particular to a label fusion generation method for multiple annotations of image regions. Background Art
[0002] In the field of computer vision and image processing, image annotation is a key task, and its quality directly affects subsequent model training and performance. Existing image annotation methods and technologies mainly include manual annotation and semi-automatic annotation, such as strong region annotation, rectangular box annotation, ellipse annotation, rotated rectangular box annotation, rotated ellipse annotation, etc. These methods have the following major technical problems in practical applications:
[0003] Different annotators often produce different annotation results for the same image due to differences in experience, understanding, and subjective judgment. These differences lead to inconsistency in the annotation data, which in turn affects the accuracy and robustness of the training model.
[0004] Image annotation tasks often involve multiple annotation types (such as strong region annotation, rectangular box annotation, etc.), while existing technologies usually only support a single type of annotation. This limitation makes it difficult to comprehensively utilize different types of annotation information, thereby limiting the performance of the model when processing complex scenes.
[0005] Due to the subjectivity of the annotator, the annotation results of the same image by different annotators may have significant deviations. The annotation results of a single annotator are easily affected by his or her personal understanding and ability, thus introducing potential annotation bias and affecting the quality of training data.
[0006] In the case of multiple annotators and multiple annotation types, how to effectively fuse annotation information from different sources and different types to generate unified high-quality annotation results is a complex and unresolved technical problem. Existing technologies often lack systematic methods when dealing with multi-annotation information fusion, resulting in poor information fusion results.
[0007] Uncertainties in image annotation (such as blurred boundaries, insufficient annotator confidence, etc.) are often not adequately addressed in existing methods. This uncertainty affects the reliability of the annotation results, which in turn has a negative impact on model training and inference.
[0008] Most existing annotation methods rely on manual operations, with low automation and low efficiency. In the annotation task of large-scale image datasets, this inefficient annotation method cannot meet the demand for quickly generating high-quality annotated data.
[0009] The annotation of large-scale image datasets usually requires a lot of manpower and time, which is costly. Existing technologies lack effective methods to reduce the annotation cost while ensuring the quality and consistency of the annotated data.
[0010] In summary, existing image annotation technologies have many problems and challenges in terms of consistency, accuracy, diversity fusion, bias processing, uncertainty processing, degree of automation and annotation cost, which directly affect the training effect and performance of subsequent models. To address these issues, a new method is urgently needed to achieve more efficient and reliable image annotation information fusion and improve the performance and application effect of the model. Summary of the invention
[0011] In order to solve the technical problems of consistency, accuracy, diversity fusion, uncertainty processing, degree of automation and annotation cost of image annotation in the prior art, the present invention provides a label fusion generation method for multiple annotations of image regions.
[0012] The technical solution provided by the present invention is as follows:
[0013] First aspect:
[0014] The present invention provides a label fusion generation method for multiple annotations of image regions, comprising:
[0015] S1. Constructing a labeling evaluation data set, wherein the labeling evaluation data set includes original images and labels of different label types. After the labeling evaluation data set is constructed, the annotator labels the labeling evaluation data set with different label types;
[0016] S2. Calculate the labeling ability of each annotator for each label type by using the neutrosophic information characterization method. The neutrosophic information characterization method calculates the neutrosophic membership, uncertainty and non-membership of each label type of each annotator respectively. The method is as follows:
[0017]
[0018] in, T The neutrosophic membership of the label annotated by the annotator, I is the uncertainty, F is the non-membership degree, Used to calculate the total number of pixels in a pixel set. and Respectively i a set of pixels in the label area annotated by the annotator of the image, and a set of pixels in the standard label area corresponding to the label annotated by the annotator;
[0019] S3, calculate the neutrinometric metric of each pixel in the image and the ideal choice The distance measure between for:
[0020]
[0021] in, is the ideal neutroschiometric membership, is the ideal uncertainty, is the ideal non-membership degree, and the calculation method of the neutrosophic measure is as follows:
[0022]
[0023] in, d For the middle wisdom measure, is a positive definite matrix The square root of the largest non-negative eigenvalue of n is the total number of reference attributes for neutrosophic measurement, and the calculation methods of each matrix are as follows:
[0024]
[0025] Among them, the function is a strictly monotone binary function, , and Based on the i The above formula treats the distance measures related to the neutrosophic membership, uncertainty and non-membership of the attributes equally.
[0026] S4. Calculate the neutronometry strength as the credibility of each label type of each annotator through neutronometry:
[0027]
[0028] in, P is the neutrosophic metric strength, d is a neutrosophic measure, and the neutrosophic measure strength is a neutrosophic characterization graph after the neutrosophic uncertain information characterization is fused. x The strength at , the neutrosophic distance metric is x The neutrosophic distance metric at ;
[0029] S5, the annotator annotates an image other than the annotated evaluation data set with multiple label types, and calculates a label probability map x Neutrosophic membership T , uncertainty I and non-membership F , the calculation method is as follows:
[0030]
[0031] Among them, K is the total number of annotation labels associated with the image, The neutrosophic metric strength is iThe label credibility corresponding to the label, the label credibility corresponds to the credibility of the annotator's ability to annotate such labels, For labels in x The mask value at x If it is in the label marked area, it is 1, otherwise it is 0, and L is the set of pixels corresponding to the label marked area;
[0032] S6. Obtain label probability map through S4 calculation x After the intensity value at each pixel in the image is calculated, the label probability map generated by the label fusion of the image is obtained. The label probability map and the original image are used as training images and reference labels for model training;
[0033] S7. Complete the labeling of each image and calculate the corresponding label probability map to complete the multi-person multi-label fusion data set for model training.
[0034] The beneficial effects brought about by the technical solution provided by the present invention include at least:
[0035] (1) In the present invention, by fusing label information from different annotators and different label types, this method can effectively reduce the deviation caused by a single annotator and a single label type, generate a more consistent and accurate label probability map, and thus improve the effect and performance of model training;
[0036] (2) In the present invention, the comprehensive use of information of multiple annotation types such as strong regional annotation, rectangular frame annotation, ellipse annotation, etc. helps the model to provide a more comprehensive feature description when processing complex and changing image scenes, thereby improving the adaptability and accuracy of the model in various practical applications;
[0037] (3) In the present invention, by automatically fusing annotated data from different sources and types, the reliance on a large amount of manual annotation is reduced, the annotation cost is reduced, the data processing speed is accelerated, and the efficiency and scalability of large-scale image dataset annotation are improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0039] Figure 1 A flowchart of a label fusion generation method for multiple annotations of image regions provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0040] The technical solution of the present invention is described below in conjunction with the accompanying drawings.
[0041] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.
[0042] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.
[0043] In the embodiments of the present invention, sometimes the subscripts are as follows It may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meanings they express are the same.
[0044] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.
[0045] Reference Manual Attached Figure 1 , shows a schematic flow chart of a label fusion generation method for multiple annotations of image regions provided by an embodiment of the present invention.
[0046] The embodiment of the present invention provides a label fusion generation method for multiple annotations of image regions, and the processing flow may include the following steps:
[0047] S1. Construct a label evaluation dataset. The label evaluation dataset includes original images and labels of different label types. After the label evaluation dataset is constructed, the annotator will label the label evaluation dataset with different label types.
[0048] The purpose of this step is to create a benchmark dataset for evaluating the ability of annotators. The dataset contains original images and annotations of different label types (such as strong region annotations, rectangular box annotations, etc.). Annotators will annotate these images so that their annotation capabilities can be analyzed and compared later. Through this process, the actual annotation data of annotators under various label types can be obtained, laying the foundation for subsequent ability evaluation and label fusion.
[0049] S2. The annotation ability of each annotator for each label type is calculated by the Neutrosophic Information Characterization method. The Neutrosophic Information Characterization method calculates the Neutrosophic Membership, Uncertainty and Non-membership of each label type for each annotator respectively. The method is as follows:
[0050]
[0051] in, T The neutrosophic membership of the label annotated by the annotator, I is the uncertainty, F is the non-membership degree, Used to calculate the total number of pixels in a pixel set. For the i The image annotator annotates the set of pixels in the label area. The set of pixels in the standard label area that the annotator labels.
[0052] This step calculates the annotation ability of each annotator on each label type through neutrosophic information characterization. The annotation quality of the annotator is quantified using neutrosophic membership, uncertainty, and non-membership. These calculations help evaluate the accuracy and consistency of the annotators, so that the reliability of each annotation type and the annotation ability of the annotator can be quantified.
[0053] S3, calculate the neutrinometric metric of each pixel in the image and the ideal choice Ideal for measuring distance between for:
[0054]
[0055] in, is the ideal neutroschiometric membership, is the ideal uncertainty, For the ideal non-membership degree, the calculation method of the neutrosophic measure is as follows:
[0056]
[0057] in, d For the middle wisdom measure, is a positive definite matrix The square root of the largest non-negative eigenvalue of n is the total number of reference attributes for neutrosophic measurement, and the calculation methods of each matrix are as follows:
[0058]
[0059] Among them, the function is a strictly monotone binary function, , and Based on thei The above formula treats the distance measures related to the neutrosophic membership, uncertainty and non-membership degrees equally.
[0060] This stage involves calculating the distance between the neutroscopy metric of each pixel in the image and the ideal choice. The neutroscopy metric can quantify the position and state of each pixel in the annotation and the degree of deviation compared with the ideal annotation state. This step helps determine the accuracy and effectiveness of the annotation and provides a quantitative basis for label fusion.
[0061] S4. Calculate the neutrosophic strength as the credibility of each label type for each annotator through neutrosophic metric:
[0062]
[0063] in, P is the neutrosophic metric strength, d is the neutrosophic measure, and the neutrosophic measure strength is the neutrosophic characterization diagram after the fusion of the neutrosophic uncertain information characterization x The strength at , the neutrosophic distance metric is x The neutrosophic distance metric at .
[0064] In this phase, the credibility of each annotator on each label type is evaluated based on the calculation results of the neutrosophic metric. This helps to understand the performance and reliability of different annotators in the labeling task, and can be used to weight the labeling results of different annotators. This process ensures that the final generated label probability map can accurately reflect the labeling ability of the annotator.
[0065] S5. The annotator annotates an image other than the annotated evaluation dataset with multiple label types and calculates the label probability map x Neutrosophic membership T , uncertainty I and non-membership F , the calculation method is as follows:
[0066]
[0067] Among them, K is the total number of annotation labels associated with the image, The neutrosophic metric strength is i The label credibility corresponding to the label, the label credibility corresponds to the credibility of the annotator's ability to annotate such labels, For labels in x The mask value at x It is 1 when it is in the label marked area, otherwise it is 0, and L is the set of pixels corresponding to the label marked area.
[0068] By calculating the neutrosophic membership, uncertainty, and non-membership of each pixel, a label probability map is generated. This label probability map integrates the annotation information of all annotators and can effectively represent the probability and uncertainty of each pixel belonging to a certain label. This step is an important process for fusing multiple annotation information into the final annotation map, providing accurate training data for subsequent model training.
[0069] S6. Obtain label probability map through S4 calculation x After the intensity value at each pixel in the image is calculated, the label probability map generated by the label fusion of the image is obtained. The label probability map and the original image are used as training images and reference labels for model training.
[0070] In this stage, the final label probability map is generated by calculating the intensity value of each pixel in the label probability map. This label probability map summarizes the labeling information and calculation results of all annotators and is used as a reference label during model training. Through this step, more comprehensive and accurate labeling data can be obtained, thereby improving the training quality and effect of the model.
[0071] S7. After labeling each image and calculating the corresponding label probability map, a multi-person multi-label fusion dataset can be completed for model training.
[0072] It should be noted that this stage involves labeling each image and calculating the corresponding label probability map to build a multi-person multi-label fusion dataset. This dataset is used for subsequent model training to ensure that the training data is of high quality and reliability. Through multi-label fusion, the labeling information of different annotators can be fully utilized to improve the generalization ability and performance of the model.
[0073] In a possible implementation, the label types include a region strong label image, a rectangle frame label image, an ellipse label image, a rotated rectangle frame label image, a rotated ellipse label image, and the like.
[0074] It should be noted that by covering multiple annotation types, this implementation can make full use of information in different annotation forms, thereby providing a more comprehensive feature description in the label fusion process. This diverse annotation form can better handle various complex scenes, improve the model's ability to recognize different shapes and regions, and enhance the accuracy and robustness of the model in practical applications.
[0075] In a possible implementation, the neutrosophic membership, uncertainty, and non-membership correspond to hybrid attributes based on regional proportion, stability, and fusion statistics.
[0076] It should be noted that this implementation method calculates the neutrosophic membership, uncertainty and non-membership by combining grayscale, color and edge information, making the annotated neutrosophic information more comprehensive and accurate. By combining these attributes, the characteristics of each pixel in the image can be better described, and a more accurate label probability map can be generated, thereby improving the effect and reliability of model training.
[0077] In a possible implementation, S3 assigns different weights according to the contribution of neutrosophic membership, uncertainty, and non-membership.
[0078] It should be noted that this implementation method calculates the neutrosophic membership, uncertainty and non-membership by combining grayscale, color and edge information, making the annotated neutrosophic information more comprehensive and accurate. By combining these attributes, the characteristics of each pixel in the image can be better described, and a more accurate label probability map can be generated, thereby improving the effect and reliability of model training.
[0079] In a possible implementation, S3 assigns different weights to each neutrosophic metric under different attribute conditions to perform constraints.
[0080] It should be noted that when calculating the neutrosophic metric, weights are assigned to each neutrosophic metric according to different attribute conditions (such as grayscale, color, edge, etc.) to constrain the impact of different attributes on the final labeling results more accurately. This method allows weights to be adjusted according to the importance of the attribute and the characteristics of the data, making the label fusion results more representative and accurate, and enhancing the training effect of the model.
[0081] In a possible implementation, when the annotator in S5 does not completely annotate all types of label types, K changes accordingly, and K is the total number of label images annotated by all annotators on a single image.
[0082] It should be noted that when calculating the neutrosophic metric, weights are assigned to each neutrosophic metric according to different attribute conditions (such as grayscale, color, edge, etc.) to constrain the impact of different attributes on the final labeling results more accurately. This method allows weights to be adjusted according to the importance of the attribute and the characteristics of the data, making the label fusion results more representative and accurate, and enhancing the training effect of the model.
[0083] The present invention is further described below by a specific embodiment.
[0084] 600 gastrointestinal endoscopy images were collected to construct a gastrointestinal endoscopy image dataset, which uses polyp areas as regions of interest. 100 of them were randomly selected as the annotation evaluation dataset, and each image was labeled with polyp areas, rotated rectangles, and rotated ellipses, and all labeled images were confirmed by multiple experts in the field to ensure accurate labeling. The remaining 500 images can be used for regional detection model training.
[0085] Step 2: Recruit two annotators and evaluate their annotation capabilities. The two annotators perform strong annotation of polyp regions, rotated rectangles, and rotated ellipses on each image, according to the formula
[0086]
[0087] For each type of annotation method of each annotator, the corresponding neutrosophic membership degree T, uncertainty degree I, and non-membership degree F are calculated respectively. In this embodiment, N is 100. According to the formula
[0088]
[0089] Calculate the credibility of each annotator's ability to annotate each type of label Pi, where n is 1, the ideal choice Neutrosophic membership , uncertainty , non-membership The calculation methods of the neutrosophic membership degree T, the uncertainty degree I, and the non-membership degree F can be expanded later, and n increases by 1 for each expansion.
[0090] Step 3: Each annotator selects one image from the 500 images to complete the polyp region strong labeling, rotated rectangle labeling, and rotated ellipse labeling. Use the formula
[0091]
[0092] The neutrosophic membership T, uncertainty I, and non-membership F at the label probability map x are calculated. Here, K is 6 (a total of 2 annotators, each of whom completes 3 types of annotations). In practice, it also supports the situation where some annotators do not fully annotate, such as if an annotator does not annotate, or only completes one type of annotation for the image. At this time, K changes accordingly. K is the total number of labeled images annotated by all annotators for the image. According to the formula
[0093]
[0094] The intensity value at x in the label probability map is calculated. After the calculation is completed for each pixel in the image, the label probability map generated by the label fusion of the image is obtained. This label probability map can be used together with the original image as a training image and reference label for model training.
[0095] Step 4: Complete the labeling of each image according to step 3, and calculate the corresponding label probability map, that is, complete the multi-person multi-label fusion data set for subsequent model training.
[0096] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:
[0097] (1) In the present invention, by fusing label information from different annotators and different label types, this method can effectively reduce the deviation caused by a single annotator and a single label type, generate a more consistent and accurate label probability map, and thus improve the effect and performance of model training;
[0098] (2) In the present invention, the comprehensive use of information of multiple annotation types such as strong regional annotation, rectangular frame annotation, ellipse annotation, etc. helps the model to provide a more comprehensive feature description when processing complex and changing image scenes, thereby improving the adaptability and accuracy of the model in various practical applications;
[0099] (3) In the present invention, by automatically fusing annotated data from different sources and types, the reliance on a large amount of manual annotation is reduced, the annotation cost is reduced, the data processing speed is accelerated, and the efficiency and scalability of large-scale image dataset annotation are improved.
[0100] The above contents are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
[0101] There are a few points to note:
[0102] (1) The drawings of the embodiments of the present invention only involve structures related to the embodiments of the present invention. Other structures may refer to conventional designs.
[0103] (2) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.
[0104] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A label fusion generation method for multiple annotations of image regions, characterized in that: include: S1. Constructing a labeling evaluation data set, wherein the labeling evaluation data set includes original images and labels of different label types. After the labeling evaluation data set is constructed, the annotator labels the labeling evaluation data set with different label types; S2, calculating the labeling ability of each annotator for each label type by using neutrosophic information characterization, wherein the neutrosophic information characterization method calculates the neutrosophic membership, uncertainty and non-membership of each label type of each annotator respectively; S3, calculate the neutrinometric metric of each pixel in the image and the ideal choice The distance measure between for: in, is the ideal neutroschiometric membership, is the ideal uncertainty, is the ideal non-membership degree; S4. Calculating the neutronometry strength as the credibility of each label type of each annotator through neutronometry. The specific method includes: in, P is the neutrosophic metric strength, d is a neutrosophic measure, and the neutrosophic measure strength is a neutrosophic characterization graph after the neutrosophic uncertain information characterization is fused. x The strength at , the neutrosophic distance metric is x The neutrosophic distance metric at ; S5, the annotator annotates an image other than the annotated evaluation data set with multiple label types, and calculates a label probability map x Neutrosophic membership T , uncertainty I and non-membership F , the calculation method specifically includes: Among them, K is the total number of annotation labels associated with the image, The neutrosophic metric strength is i The label credibility corresponding to the label, the label credibility corresponds to the credibility of the annotator's ability to annotate such labels, For labels in x The mask value at x If it is in the label marked area, it is 1, otherwise it is 0, and L is the set of pixels corresponding to the label marked area; S6. Obtain a label probability map by calculating S4 x After the intensity value at each pixel in the image is calculated, the label probability map generated by the label fusion of the image is obtained. The label probability map and the original image are used as training images and reference labels for model training; S7. Complete the labeling of each image and calculate the corresponding label probability map to complete the multi-person multi-label fusion data set for model training.
2. The label fusion generation method for multiple annotations of image regions according to claim 1, characterized in that: The S2 specifically includes: The method of describing the information is as follows: in, T The neutrosophic membership of the label annotated by the annotator, I is the uncertainty, F is the non-membership degree, Used to calculate the total number of pixels in a pixel set. For the i The image annotator annotates the set of pixels in the label area. The set of pixels in the standard label area that the annotator labels.
3. The label fusion generation method for multiple annotations of image regions according to claim 1, characterized in that: The neutrosophic metrics in S3 specifically include: The calculation method of the neutrosophic metric is as follows: in, d For the middle wisdom measure, is a positive definite matrix The square root of the largest non-negative eigenvalue of n is the total number of reference attributes for neutrosophic measurement, and the calculation methods of each matrix are as follows: Among them, the function is a strictly monotone binary function, , and Based on the i The above formula treats the distance measures related to the neutrosophic membership, uncertainty and non-membership degrees equally.
4. The label fusion generation method for multiple annotations of image regions according to claim 1, characterized in that: include: The label types include region strong label image, rectangular frame label image, ellipse label image, rotated rectangular frame label image and rotated ellipse label image.
5. The label fusion generation method for multiple annotations of image regions according to claim 1, characterized in that: include: The neutrosophic membership, uncertainty and non-membership correspond to mixed attributes based on regional proportion, stability and fusion statistics.
6. The label fusion generation method for multiple annotations of image regions according to claim 3, characterized in that: The S3 specifically includes: Different weights are assigned according to the contribution of neutrosophic membership, uncertainty and non-membership.
7. The label fusion generation method for multiple annotations of image regions according to claim 3, characterized in that: The S3 specifically includes: Different weights are assigned to each neutrosophic measure under different attribute conditions to constrain it.
8. The label fusion generation method for multiple annotations of image regions according to claim 1, characterized in that: The S5 specifically includes: When the annotator does not completely annotate all types of the label types, K changes accordingly, and K is the total number of label images annotated by all the annotators on a single image.
