A target segmentation method and image annotation tool based on multimodal data fusion

Abstract

The present invention relates to the field of image segmentation technology, and in particular to a target segmentation method and image annotation tool based on multimodal data fusion, the method comprising: obtaining an initial RGB image, an initial infrared image, and an initial depth image containing a target; aligning them to obtain a first RGB image, a first infrared image, and a first depth image in the same coordinate system; specifying an initial prompt point in the first RGB image, and obtaining a first mask representing a target area in different modal images through the initial prompt point; based on the first mask of the first RGB image, the first infrared image, and the first depth image, fusing the pixel values ​​of the images in different modalities to obtain a second mask; based on the second mask, determining the minimum bounding box of the target, calibrating the minimum bounding box, and obtaining a segmentation result of the target. The present invention significantly improves the accuracy and robustness of target segmentation through multimodal data fusion and iterative optimization strategies, and is suitable for a variety of complex application scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a target segmentation method and an image annotation tool based on multimodal data fusion. Background Art

[0002] With the development of computer vision technology, object segmentation has become an important research direction in the field of image processing. Object segmentation refers to separating specific objects of interest from images. This process is crucial for many applications, such as autonomous driving, medical image analysis, and security monitoring systems. Although traditional single-modality image segmentation methods (such as using only RGB images) perform well in some scenarios, they often fail to achieve ideal segmentation results in complex environments. This is because single-modality images have limited information expression capabilities when faced with problems such as lighting changes, occlusion, and unclear textures.

[0003] In recent years, multimodal data fusion technology has gradually become a research hotspot. By combining data from multiple modalities such as RGB images, infrared images, and depth images, information about the target object can be obtained from different angles and dimensions, thereby improving the accuracy and robustness of segmentation. Specifically, RGB images provide rich color information, which helps to distinguish different objects; infrared images are not affected by light and can provide thermal radiation information of the target at night or in low light conditions; depth images provide distance information of the target, which helps to understand the spatial layout of the object.

[0004] However, there are many challenges in effectively fusing multimodal data and applying them to target segmentation, including key technical issues such as alignment between images of different modalities, feature extraction, and information fusion. In particular, how to accurately locate the target in multimodal images and generate high-quality segmentation masks is the key to achieving accurate segmentation. Although existing methods have made some progress, their performance in complex environments still needs to be improved. Summary of the invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the defects of the prior art in dealing with target segmentation in complex environments, which is often difficult to deal with problems such as illumination changes, occlusion and unclear texture due to the limitations of single modal information, resulting in insufficient segmentation accuracy and robustness.

[0006] In order to solve the above technical problems, the present invention provides a target segmentation method based on multimodal data fusion, comprising the following steps:

[0007] S1: Acquire an initial RGB image, an initial infrared image, and an initial depth image containing the target;

[0008] S2: aligning the initial RGB image, the initial infrared image and the initial depth image to obtain a first RGB image, a first infrared image and a first depth image in the same coordinate system respectively;

[0009] S3: specifying an initial cue point in the first RGB image, and obtaining first masks representing a target area in the first RGB image, the first infrared image, and the first depth image respectively through the initial cue point;

[0010] S4: Based on the first RGB image, the first infrared image, and the first mask of the first depth image, the pixel values ​​of the images in different modalities are fused to obtain a second mask;

[0011] S5: Based on the second mask, determine a minimum bounding box of the target, calibrate the minimum bounding box, and obtain a segmentation result of the target.

[0012] In one embodiment of the present invention, in S2, the method for obtaining the first RGB image, the first infrared image and the first depth image in the same coordinate system is as follows:

[0013] S21: extracting feature points from the initial RGB image, the initial infrared image, and the initial depth image respectively, each feature point having a feature descriptor, and the feature descriptor being a coding vector containing local information around the feature point;

[0014] S22: For each set of feature descriptors of each modality image, construct an approximate nearest neighbor search data structure;

[0015] S23: Select an approximate nearest neighbor search data structure of any modality image, and search in the current approximate nearest neighbor search data structure according to the feature point descriptor of another modality image to obtain a candidate matching point;

[0016] S24: obtaining a geometric relationship estimation matrix between any two modality images based on the candidate matching points;

[0017] S25: Based on the geometric relationship estimation matrix, all modality images are aligned to obtain a first RGB image, a first infrared image and a first depth image in the same coordinate system.

[0018] In one embodiment of the present invention, in S3, a method for obtaining a first mask representing a target area in the first RGB image, the first infrared image, and the first depth image is as follows:

[0019] S31: selecting one or more initial prompt points in the target area of ​​the first RGB image;

[0020] S32: Based on the initial cue point, obtain a first mask of the first RGB image, and map the coordinates of the initial cue point to the first infrared image and the first depth image respectively to obtain a first mask of the first infrared image and a first mask of the first depth image.

[0021] In one embodiment of the present invention, in S4, the method for obtaining the second mask is as follows:

[0022] S41: Obtaining information entropy of each modality image channel based on a first mask of the first RGB image, the first infrared image, and the first depth image;

[0023] S42: obtaining a weight of a first mask of a corresponding modality image based on the information entropy of each modality image channel;

[0024] S43: performing weighted fusion on the first masks of the three modal images according to the weights of the first masks of the corresponding modal images to obtain a fusion value of each pixel;

[0025] S44: Compare the fusion value of each pixel with an evaluation threshold, retain the pixel points whose fusion value is greater than the evaluation threshold, and obtain a second mask.

[0026] In one embodiment of the present invention, in S41, the method for calculating the information entropy of each modality image channel is as follows:

[0027] Information entropy of the first RGB image channel The calculation formula is:

[0028] ,

[0029] Information entropy of the first infrared image channel The calculation formula is:

[0030] ,

[0031] Information entropy of the first depth image channel The calculation formula is:

[0032] ,

[0033] in, , and They are the pixel values ​​of RGB images, depth images and infrared images. The probability distribution on .

[0034] In one embodiment of the present invention, in S43, the fusion value of each pixel is obtained. The method is as follows:

[0035] ,

[0036] in, is the weight of the first RGB image channel, is the coordinate of any pixel point in the first mask of the first RGB image; is the weight of the first infrared image channel, is the coordinate of any pixel point in the first mask of the first infrared image; is the weight of the first depth image channel, is the coordinate of any pixel point in the first mask of the first depth image.

[0037] In one embodiment of the present invention, the evaluation threshold The calculation method is:

[0038] ,

[0039] in, Represents the mean value of information entropy of each modality image channel, , is the information entropy of the first RGB image, is the information entropy of the first depth image, is the information entropy of the first infrared image; represents the standard deviation of the information entropy of each modality image channel, .

[0040] In one embodiment of the present invention, in S5, the method for obtaining the segmentation result of the target is as follows:

[0041] S51: superimposing the second mask on the initial RGB image to obtain a minimum bounding box of the target;

[0042] S52: traverse all the pixels in the minimum bounding box, and convert the RGB value of each pixel into its closest color feature according to the color mapping table;

[0043] S53: Processing the converted color features, clustering the areas within the minimum bounding box into N categories, and obtaining a clustering result;

[0044] S54: randomly selecting n points from the category with the largest amount of data in the clustering result as auxiliary points, and adding the auxiliary points to the prompt point set to obtain an updated prompt point set;

[0045] S55: generating a new mask based on the updated cue point set, and calculating a change in the intersection-and-union ratio between the current mask and the mask generated in the previous iteration;

[0046] S56: Determine whether the change is less than a preset threshold:

[0047] If not, return to step S54;

[0048] If so, stop the iteration, output the current mask, and superimpose the current mask on the initial RGB image to obtain the segmentation result of the target.

[0049] The present invention also provides an image annotation tool, comprising the following modules:

[0050] A receiving module, used for receiving at least one creation instruction input through a target interface, and when there are multiple creation instructions, queuing them for processing according to priority or submission order;

[0051] An acquisition module, used for acquiring a target number of images to be annotated based on the resource addresses included in the creation instruction;

[0052] A labeling module, used for automatically labeling the image to be labeled using the target segmentation method based on multimodal data fusion, and displaying the labeling result on the target interface;

[0053] The saving module saves the annotation results in a variety of file formats.

[0054] The above technical solution of the present invention has the following advantages compared with the prior art:

[0055] 1. Multimodal data fusion: By combining the multimodal data of RGB images, infrared images and depth images, this method effectively solves the inaccuracy and instability problems faced by traditional single-modal segmentation technology. It not only improves the segmentation accuracy, but also can provide more comprehensive target information under different conditions, and enhances the adaptability to complex environments. It is suitable for various application scenarios such as medical image analysis, autonomous driving, security monitoring, etc., showing excellent technical value and broad application prospects.

[0056] 2. Precise alignment processing: By extracting and matching feature points of images of different modalities and using the geometric relationship estimation matrix for alignment processing, the precise alignment of multimodal images in the same coordinate system is ensured, reducing the errors caused by inconsistent coordinates.

[0057] 3. Information entropy fusion: By calculating the information entropy of each modality image channel and determining the weight based on the inverse of the information entropy, this method can effectively fuse the information of different modal images and improve the rationality of mask generation.

[0058] 4. Robustness enhancement: This method not only considers the color information of RGB images, but also takes advantage of infrared images and depth images, and can provide stable segmentation effects under various lighting conditions and complex backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0060] Figure 1 is a flow chart of a target segmentation method based on multimodal data fusion provided in the first embodiment of the present invention;

[0061] Figure 2 is a specific implementation flow chart of the target segmentation method based on multimodal data fusion provided in the first embodiment of the present invention;

[0062] Figure 3 is a flow chart of a method for obtaining a first RGB image, a first infrared image and a first depth image in the same coordinate system provided in the first embodiment of the present invention;

[0063] Figure 4 is a flow chart of a method for obtaining a first mask representing a target area in the first RGB image, the first infrared image, and the first depth image provided in Embodiment 1 of the present invention;

[0064] Figure 5 is a flow chart of a method for obtaining a second mask provided in Embodiment 1 of the present invention;

[0065] Figure 6 is a flow chart of a method for obtaining a target segmentation result provided in the first embodiment of the present invention;

[0066] Figure 7 is a schematic diagram of the structure of an image annotation tool provided in the first embodiment of the present invention;

[0067] Description of the accompanying drawings in the specification: 10, receiving module; 20, acquiring module; 30, marking module; 40, saving module. DETAILED DESCRIPTION

[0068] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention. Embodiment 1

[0069] Reference Figure 1-Figure 2 As shown, the present invention provides a target segmentation method based on multimodal data fusion, comprising the following steps:

[0070] S1: Acquire an initial RGB image, an initial infrared image, and an initial depth image containing the target;

[0071] S2: aligning the initial RGB image, the initial infrared image and the initial depth image to obtain a first RGB image, a first infrared image and a first depth image in the same coordinate system respectively;

[0072] S3: specifying an initial cue point in the first RGB image, and obtaining first masks representing a target area in the first RGB image, the first infrared image, and the first depth image respectively through the initial cue point;

[0073] S4: Based on the first RGB image, the first infrared image, and the first mask of the first depth image, the pixel values ​​of the images in different modalities are fused to obtain a second mask;

[0074] S5: Based on the second mask, determine a minimum bounding box of the target, calibrate the minimum bounding box, and obtain a segmentation result of the target.

[0075] like Figure 3 As shown, in S2, the method of obtaining the first RGB image, the first infrared image and the first depth image in the same coordinate system is as follows:

[0076] S21: extracting feature points from the initial RGB image, the initial infrared image, and the initial depth image respectively, each feature point having a feature descriptor, and the feature descriptor being a coding vector containing local information around the feature point;

[0077] S22: For search efficiency, for each set of feature descriptors of each modality image, an approximate nearest neighbor search algorithm is used to construct an approximate nearest neighbor search data structure;

[0078] S23: Select an approximate nearest neighbor search data structure of any modality image, and search in the current approximate nearest neighbor search data structure according to the feature point descriptor of another modality image to obtain a candidate matching point; generally, the feature point with the smallest distance is selected as the candidate matching point;

[0079] S24: Based on the candidate matching points, in order to eliminate the influence of erroneous matching points, a random sampling consistency algorithm is used to obtain a geometric relationship estimation matrix between any two modal images; if the two images are taken by a small movement of a fixed distance, a basic matrix can be estimated; if they are different modal images under the same viewing angle, a homography matrix can be estimated;

[0080] S25: Based on the geometric relationship estimation matrix, align all modal images to obtain a first RGB image, a first infrared image and a first depth image in the same coordinate system.

[0081] like Figure 4 As shown, in S3, the method for obtaining the first mask representing the target area in the first RGB image, the first infrared image, and the first depth image is as follows:

[0082] S31: selecting one or more initial prompt points in the target area of ​​the first RGB image;

[0083] S32: Based on the initial cue point, obtain a first mask of the first RGB image, and map the coordinates of the initial cue point to the first infrared image and the first depth image respectively to obtain a first mask of the first infrared image and a first mask of the first depth image.

[0084] See also Figure 5 As shown, in S4, the method for obtaining the second mask is as follows:

[0085] S41: Based on the first RGB image, the first infrared image, and the first mask of the first depth image, obtain the information entropy of each modality image channel, and the calculation method thereof is as follows:

[0086] Information entropy of the first RGB image channel The calculation formula is:

[0087] ,

[0088] Information entropy of the first infrared image channel The calculation formula is:

[0089] ,

[0090] Information entropy of the first depth image channel The calculation formula is:

[0091] ,

[0092] in, , and They are the pixel values ​​of RGB images, depth images and infrared images. The probability distribution on ;

[0093] S42: based on the information entropy of each modality image channel, taking the inverse of the information entropy of each modality image channel as the weight of the first mask of the corresponding modality image;

[0094] S43: performing weighted fusion on the first masks of the three modal images according to the weight of the first mask of the corresponding modal image to obtain a fusion value of each pixel point , and its calculation formula is:

[0095] ,

[0096] in, is the weight of the first RGB image channel, is the coordinate of any pixel point in the first mask of the first RGB image; is the weight of the first infrared image channel, is the coordinate of any pixel point in the first mask of the first infrared image; is the weight of the first depth image channel, is the coordinate of any pixel point in the first mask of the first depth image;

[0097] S44: The fusion value of each pixel With evaluation threshold Compare and keep the fusion value Greater than the evaluation threshold The pixel points are obtained by the second mask.

[0098] Furthermore, the evaluation threshold The calculation method is:

[0099] ,

[0100] in, Represents the mean value of information entropy of each modality image channel, , is the information entropy of the first RGB image, is the information entropy of the first depth image, is the information entropy of the first infrared image; represents the standard deviation of the information entropy of each modality image channel, .

[0101] like Figure 6 As shown, in S5, the method for obtaining the segmentation result of the target is as follows:

[0102] S51: superimposing the second mask on the initial RGB image to obtain a minimum bounding box of the target;

[0103] S52: traverse all the pixels in the minimum bounding box, and based on the color mapping table, convert the RGB value of each pixel into its closest color feature in the form of ColorNames (CN for short) by calculating the Euclidean distance or other similarity measurement between the RGB value of each pixel and the color in the mapping table;

[0104] The type of the color mapping table includes but is not limited to a WEB standard color table, an X11 color name list, or other defined color classification systems, and the content of the color mapping table includes a series of common color names and their corresponding RGB value ranges;

[0105] S53: inputting the converted color features into the K-means algorithm, clustering the area within the minimum bounding box into N categories by calculating the distance between the color feature of each pixel and each cluster center, and obtaining a clustering result;

[0106] S54: randomly selecting n points from the category with the largest amount of data in the clustering result as auxiliary points, and adding the auxiliary points to the prompt point set to obtain an updated prompt point set;

[0107] S55: Based on the updated cue point set, a new mask is generated by the Segment Anything Model (SAM) algorithm, and the change in the Intersection over Union (IoU) between the current mask and the mask generated in the previous iteration is calculated. :

[0108] ,

[0109] Among them, A is the mask generated by the previous iteration, and B is the current mask;

[0110] S56: Determine the change amount Is it less than the preset threshold? :

[0111] If not, return to step S54;

[0112] If so, stop the iteration, output the current mask, and superimpose the current mask on the initial RGB image to obtain the segmentation result of the target. Embodiment 2

[0113] like Figure 7 As shown, the present invention also provides an image annotation tool, comprising the following modules:

[0114] A receiving module 10, configured to receive at least one creation instruction input through a target interface, and when there are multiple creation instructions, queue them for processing according to priority or submission order;

[0115] The acquisition module 20 is used to acquire a target number of images to be annotated based on the resource address included in the creation instruction; in addition to supporting a single resource address, it also supports batch import of image lists or directory paths, and provides basic image preprocessing options such as scaling, cropping, rotation, etc.;

[0116] The annotation module 30 is used to automatically annotate the image to be annotated by using the target segmentation method based on multimodal data fusion described in the first embodiment, and also allows the user to manually adjust the bounding box or segmentation area based on the automatic annotation, and displays the annotation result on the target interface, so that the user can view and verify it immediately;

[0117] The saving module 40 saves the annotation results in a variety of file formats, including but not limited to XML, TXT, JSON, CSV, and allows the user to select the output format and save different versions for each annotation task to facilitate tracking of modification history.

[0118] In addition, the image annotation tool provided in this embodiment can also analyze the annotation data and generate statistical reports to help users understand the annotation progress and quality. It can also be connected to the cloud storage service to achieve seamless data upload and download.

[0119] In summary, the present invention aims to improve the accuracy and stability of target segmentation by comprehensively using the information of RGB images, infrared images and depth images. This method achieves effective segmentation of the target through a series of steps such as image alignment, feature point matching, mask generation, information entropy calculation, weight fusion, and further improves the quality of the segmentation results through iterative optimization strategies. In addition, the present invention can achieve accurate segmentation of the target and can be used in multiple fields such as autonomous driving, medical image analysis, and security monitoring systems.

[0120] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0121] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0123] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0124] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A target segmentation method based on multimodal data fusion, characterized in that: The following steps are involved: S1: Acquire an initial RGB image, an initial infrared image, and an initial depth image containing the target; S2: aligning the initial RGB image, the initial infrared image and the initial depth image to obtain a first RGB image, a first infrared image and a first depth image in the same coordinate system respectively; S3: specifying an initial cue point in the first RGB image, and obtaining first masks representing a target area in the first RGB image, the first infrared image, and the first depth image respectively through the initial cue point; S4: Based on the first RGB image, the first infrared image, and the first mask of the first depth image, the pixel values ​​of the images in different modalities are fused to obtain a second mask; S5: determining a minimum bounding box of the target based on the second mask, calibrating the minimum bounding box, and obtaining a segmentation result of the target; In S2, the method for obtaining the first RGB image, the first infrared image, and the first depth image in the same coordinate system is as follows: S21: extracting feature points from the initial RGB image, the initial infrared image, and the initial depth image respectively, each feature point having a feature descriptor, and the feature descriptor being a coding vector containing local information around the feature point; S22: For each set of feature descriptors of each modality image, construct an approximate nearest neighbor search data structure; S23: Select an approximate nearest neighbor search data structure of any modality image, and search in the current approximate nearest neighbor search data structure according to the feature point descriptor of another modality image to obtain a candidate matching point; S24: obtaining a geometric relationship estimation matrix between any two modality images based on the candidate matching points; S25: Based on the geometric relationship estimation matrix, align all modality images to obtain a first RGB image, a first infrared image, and a first depth image in the same coordinate system; In S3, a method for obtaining a first mask representing a target area in the first RGB image, the first infrared image, and the first depth image is as follows: S31: selecting one or more initial prompt points in the target area of ​​the first RGB image; S32: Based on the initial cue point, obtain a first mask of the first RGB image, and map the coordinates of the initial cue point to the first infrared image and the first depth image respectively to obtain a first mask of the first infrared image and a first mask of the first depth image; In S4, the method of obtaining the second mask is as follows: S41: Obtaining information entropy of each modality image channel based on a first mask of the first RGB image, the first infrared image, and the first depth image; S42: obtaining a weight of a first mask of a corresponding modality image based on the information entropy of each modality image channel; S43: performing weighted fusion on the first masks of the three modal images according to the weights of the first masks of the corresponding modal images to obtain a fusion value of each pixel; S44: Compare the fusion value of each pixel with the evaluation threshold, retain the pixel whose fusion value is greater than the evaluation threshold, and obtain a second mask; In S5, the method for obtaining the segmentation result of the target is as follows: S51: superimposing the second mask on the initial RGB image to obtain a minimum bounding box of the target; S52: traverse all the pixels in the minimum bounding box, and convert the RGB value of each pixel into its closest color feature according to the color mapping table; S53: Processing the converted color features, clustering the areas within the minimum bounding box into N categories, and obtaining a clustering result; S54: randomly selecting n points from the category with the largest amount of data in the clustering result as auxiliary points, and adding the auxiliary points to the prompt point set to obtain an updated prompt point set; S55: generating a new mask based on the updated cue point set, and calculating a change in the intersection-and-union ratio between the current mask and the mask generated in the previous iteration; S56: Determine whether the change is less than a preset threshold: If not, return to step S54; If so, stop the iteration, output the current mask, and superimpose the current mask on the initial RGB image to obtain the segmentation result of the target.

2. The target segmentation method based on multimodal data fusion according to claim 1, characterized in that: In S41, the calculation method for obtaining the information entropy of each modality image channel is as follows: Information entropy of the first RGB image channel The calculation formula is: ， Information entropy of the first infrared image channel The calculation formula is: ， Information entropy of the first depth image channel The calculation formula is: ， in, and They are the pixel values ​​of RGB images, depth images and infrared images. The probability distribution on .

3. The target segmentation method based on multimodal data fusion according to claim 1, characterized in that: In S42, the method for obtaining the weight of the first mask corresponding to the modal image is: taking the inverse of the information entropy of each modal image channel as its corresponding weight.

4. The target segmentation method based on multimodal data fusion according to claim 1, characterized in that: In S43, the fusion value of each pixel is obtained. The method is as follows: ， in, is the weight of the first RGB image channel, is the coordinate of any pixel point in the first mask of the first RGB image; is the weight of the first infrared image channel, is the coordinate of any pixel point in the first mask of the first infrared image; is the weight of the first depth image channel, is the coordinate of any pixel point in the first mask of the first depth image.

5. The target segmentation method based on multimodal data fusion according to claim 1, characterized in that: The evaluation threshold The calculation method is: ， in, Represents the mean value of information entropy of each modality image channel, , is the information entropy of the first RGB image, is the information entropy of the first depth image, is the information entropy of the first infrared image; represents the standard deviation of the information entropy of each modality image channel, .

6. An image annotation tool, characterized in that: include: A receiving module, used for receiving at least one creation instruction input through a target interface, and when there are multiple creation instructions, queuing them for processing according to priority or submission order; An acquisition module, used for acquiring a target number of images to be annotated based on the resource addresses included in the creation instruction; A labeling module, used for automatically labeling the image to be labeled using the target segmentation method based on multimodal data fusion according to any one of claims 1 to 5, and displaying the labeling result on the target interface; The saving module saves the annotation results in a variety of file formats.

Images