Target segmentation method based on multi-modal data fusion, and image labeling tool

Abstract

The present invention relates to the technical field of image segmentation, and particularly relates to a target segmentation method based on multi-modal data fusion, and an image labeling tool. The method comprises: acquiring an initial RGB image, an initial infrared image and an initial depth image, each of which includes a target; performing alignment processing on the initial RGB image, the initial infrared image and the initial depth image, so as to obtain a first RGB image, a first infrared image and a first depth image in the same coordinate system; specifying an initial prompt point in the first RGB image, and respectively acquiring, by means of the initial prompt point, first masks representing target regions in the images of different modalities; on the basis of the first masks of the first RGB image, first infrared image and first depth image, fusing pixel values of the images of different modalities, so as to obtain a second mask; and on the basis of the second mask, determining a minimum bounding box for the target, and calibrating the minimum bounding box, so as to obtain a segmentation result of the target. The present invention significantly improves the precision and robustness of target segmentation by means of multi-modal data fusion and an iterative optimization strategy, and is applicable to a plurality of complex application scenarios.

Description

A target segmentation method based on multi-modal data fusion and image labeling tool TECHNICAL FIELD

[0001] The present application relates to the technical field of image segmentation, in particular to a target segmentation method based on multi-modal data fusion and an image labeling tool. BACKGROUND

[0002] With the development of computer vision technology, target segmentation has become an important research direction in the field of image processing. Target segmentation refers to separating specific objects of interest from images, which is crucial for many applications such as autonomous driving, medical image analysis, security monitoring systems, etc. Traditional single-modal image segmentation methods (such as using only RGB images) may perform well in some scenarios, but often fail to achieve ideal segmentation results in complex environments. This is because single-modal images have limited information expression capabilities when facing problems such as light changes, occlusions, and unclear textures.

[0003] In recent years, multi-modal data fusion technology has gradually become a research hotspot. By combining data from multiple modalities such as RGB images, infrared images, and depth images, information about target objects can be obtained from different angles and dimensions, thereby improving the accuracy and robustness of segmentation. Specifically, RGB images provide rich color information, which helps to distinguish different objects; infrared images are not affected by light and can provide thermal radiation information of targets at night or in low light conditions; depth images provide distance information about targets, which helps to understand the spatial layout of objects.

[0004] However, effectively fusing multi-modal data and applying it to target segmentation faces many challenges, including alignment between different modal images, feature extraction, information fusion, and other key technical issues. In particular, how to accurately locate targets in multi-modal images and generate high-quality segmentation masks (Masks) is the key to achieving accurate segmentation. Although existing methods have made some progress, their performance in complex environments still needs to be improved. SUMMARY

[0005] To this end, the technical problem to be solved by the present application is to overcome the limitations of single-modal information in handling target segmentation in complex environments, which often fails to cope with problems such as light changes, occlusions, and unclear textures, resulting in insufficient segmentation accuracy and robustness.

[0006] To solve the above technical problems, the present application provides a target segmentation method based on multi-modal data fusion, comprising the following steps:

[0007] S1: obtaining initial RGB images, initial infrared images, and initial depth images containing targets;

[0008] S2: performing alignment processing on the initial RGB image, the initial infrared image and the initial depth image to obtain a first RGB image, a first infrared image and a first depth image in a same coordinate system respectively;

[0009] S3: specifying an initial prompt point in the first RGB image, and acquiring a first mask representing a target region in the first RGB image, the first infrared image and the first depth image respectively through the initial prompt point;

[0010] S4: fusing pixel values of images in different modalities based on the first mask of the first RGB image, the first infrared image and the first depth image to obtain a second mask;

[0011] S5: determining a minimum bounding box of a target based on the second mask, and calibrating the minimum bounding box to obtain a segmentation result of the target.

[0012] In an embodiment of the present application, in S2, the method for obtaining the first RGB image, the first infrared image and the first depth image in the same coordinate system is as follows:

[0013] S21: extracting feature points from the initial RGB image, the initial infrared image and the initial depth image respectively, each feature point having a feature descriptor, and the feature descriptor being an encoding vector containing local information around the feature point;

[0014] S22: constructing an approximate nearest neighbor search data structure for a feature descriptor set of each modality image;

[0015] S23: searching in the approximate nearest neighbor search data structure of an optional modality image according to the feature point descriptor of another modality image to obtain a candidate matching point;

[0016] S24: obtaining a geometric relationship estimation matrix between any two modality images based on the candidate matching point;

[0017] S25: performing alignment processing on all modality images based on the geometric relationship estimation matrix to obtain the first RGB image, the first infrared image and the first depth image in the same coordinate system.

[0018] In an embodiment of the present application, in S3, the method for acquiring the first mask representing the target region in the first RGB image, the first infrared image and the first depth image is as follows:

[0019] S31: selecting one or more initial prompt points in the target region of the first RGB image;

[0020] S32: Obtain a first mask of the first RGB image based on the initial prompt point, map coordinates of the initial prompt point into the first infrared image and the first depth image respectively to obtain a first mask of the first infrared image and a first mask of the first depth image.

[0021] In an embodiment of the present application, in S4, the method for obtaining the second mask is as follows:

[0022] S41: Obtain information entropy of each modality image channel based on the first mask of the first RGB image, the first infrared image and the first depth image;

[0023] S42: Obtain a weight of the first mask of the corresponding modality image based on the information entropy of each modality image channel;

[0024] S43: Perform weighted fusion on the first masks of the three modality images according to the weight of the first mask of the corresponding modality image to obtain a fusion value of each pixel point;

[0025] S44: Compare the fusion value of each pixel point with an evaluation threshold, retain the pixel point with a fusion value greater than the evaluation threshold to obtain a second mask.

[0026] In an embodiment of the present application, in S41, the calculation method of the information entropy of each modality image channel is as follows:

[0027] The calculation formula of the information entropy H RGB of the first RGB image channel is as follows:

[0028] The calculation formula of the information entropy H IR of the first infrared image channel is as follows:

[0029] The calculation formula of the information entropy H Depth of the first depth image channel is as follows:

[0030] Wherein, P RGB (i RGB ), P Dept h(i Depth ) and P IR (i IR ) are the probability distribution of the RGB image, the depth image and the infrared image on the pixel value i RGB , i Depth , i IR .

[0031] In one embodiment of the present application, in S42, the method for obtaining the weight of the first mask corresponding to the modality image is: taking the reciprocal of the information entropy of each modality image channel as its corresponding weight.

[0032] In one embodiment of the present application, in S43, the method for obtaining the fusion value F(x,y of each pixel point is as follows:

[0033] wherein, W RGB is the weight of the first RGB image channel, RGB mask (x,y) is the coordinate of any pixel point in the first mask of the first RGB image; W IR is the weight of the first infrared image channel, IR mask (x,y) is the coordinate of any pixel point in the first mask of the first infrared image; W Depth is the weight of the first depth image channel, Depth mask (x,y) is the coordinate of any pixel point in the first mask of the first depth image.

[0034] In one embodiment of the present application, the method for calculating the evaluation threshold θ is:

[0035] wherein, represents the mean value of the information entropy of each modality image channel, H RGB is the information entropy of the first RGB image, H Depth is the information entropy of the first depth image, H IR is the information entropy of the first infrared image; σ H represents the standard deviation of the information entropy of each modality image channel,

[0036] In one embodiment of the present application, in S5, the method for obtaining the segmentation result of the target is as follows:

[0037] S51: superimposing the second mask on the initial RGB image to obtain the minimum bounding box of the target;

[0038] S52: traversing all pixel points in the minimum bounding box, and converting the RGB value of each pixel point into its closest color feature according to the color mapping table;

[0039] S53: processing the converted color feature, clustering the area within the minimum bounding box into N categories to obtain a clustering result;

[0040] S54: randomly selecting n points in the category with the largest total amount of data of the clustering result as auxiliary points, adding the auxiliary points to the prompt point set to obtain an updated prompt point set;

[0041] S55: generating a new mask based on the updated prompt point set, and calculating a change amount of an intersection over union between the current mask and a mask generated in a last iteration;

[0042] S56: determining whether the change amount is less than a preset threshold value;

[0043] If not, returning to step S54;

[0044] If yes, stopping iteration, outputting the current mask, and superimposing the current mask on the initial RGB image to obtain a segmentation result of the target.

[0045] The application further provides an image labeling tool, comprising the following modules:

[0046] A receiving module is configured to receive at least one creation instruction input through a target interface, and when the number of creation instructions is more than one, the creation instructions are queued for processing according to priority or submission order;

[0047] An obtaining module is configured to obtain a target number of images to be labeled based on a resource address included in the creation instruction;

[0048] A labeling module is configured to automatically label the images to be labeled by using the target segmentation method based on multi-modal data fusion, and display the labeling result on the target interface;

[0049] A saving module is configured to save the labeling result in multiple file formats.

[0050] The above technical solution of the application has the following advantages compared with the prior art:

[0051] 1. Multi-modal data fusion: by combining the multi-modal data of RGB images, infrared images and depth images, the method effectively solves the inaccurate and unstable problems faced by traditional single-modal segmentation technology, improves the segmentation accuracy, and can provide more comprehensive target information under different conditions, enhances the adaptability to complex environments, and is suitable for medical image analysis, automatic driving, security monitoring and other application scenarios, and shows excellent technical value and broad application prospect.

[0052] 2. Precise alignment processing: by extracting and matching feature points of different modal images, and using a geometric relationship estimation matrix for alignment processing, the precise alignment of multi-modal images in the same coordinate system is ensured, and the error caused by inconsistent coordinates is reduced.

[0053] 3. Information entropy fusion: By calculating the information entropy of each modality image channel, and determining the weight based on the reciprocal of the information entropy, this method can effectively fuse the information of different modal images, and improve the rationality of mask generation.

[0054] 4. Robustness enhancement: This method not only considers the color information of RGB images, but also takes advantage of infrared images and depth images, and can provide stable segmentation effect under various lighting conditions and complex backgrounds. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to make the content of the present application more easily understood, the following will be further described in detail according to the specific embodiments of the present application and in conjunction with the drawings, in which

[0056] Figure 1 is a flowchart of the target segmentation method based on multi-modal data fusion provided in embodiment one of the present application;

[0057] Figure 2 is a flowchart of the specific implementation of the target segmentation method based on multi-modal data fusion provided in embodiment one of the present application;

[0058] Figure 3 is a flowchart of the method for obtaining the first RGB image, the first infrared image and the first depth image in the same coordinate system provided in embodiment one of the present application;

[0059] Figure 4 is a flowchart of the method for obtaining the first mask representing the target region in the first RGB image, the first infrared image and the first depth image provided in embodiment one of the present application;

[0060] Figure 5 is a flowchart of the method for obtaining the second mask provided in embodiment one of the present application;

[0061] Figure 6 is a flowchart of the method for obtaining the segmentation result of the target provided in embodiment one of the present application;

[0062] Figure 7 is a structural schematic diagram of an image annotation tool provided in embodiment one of the present application;

[0063] Description of the drawings Reference signs in the specification: 10, receiving module; 20, obtaining module; 30, annotation module; 40, saving module. DETAILED DESCRIPTION

[0064] The present application will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it. However, the embodiments are not limiting to the present application.

[0065] Embodiment one

[0066] With reference to FIGS. 1-2, the present application provides a target segmentation method based on multi-modal data fusion, comprising the following steps:

[0067] S1: obtaining an initial RGB image, an initial infrared image and an initial depth image containing a target;

[0068] S2: performing alignment processing on the initial RGB image, the initial infrared image and the initial depth image to obtain a first RGB image, a first infrared image and a first depth image in the same coordinate system, respectively;

[0069] S3: specifying an initial prompt point in the first RGB image, and obtaining a first mask representing a target region in the first RGB image, the first infrared image and the first depth image through the initial prompt point, respectively;

[0070] S4: fusing pixel values of images under different modalities based on the first masks of the first RGB image, the first infrared image and the first depth image to obtain a second mask;

[0071] S5: determining a minimum bounding box of the target based on the second mask, and calibrating the minimum bounding box to obtain a segmentation result of the target.

[0072] As shown in FIG. 3, in S2, the method for obtaining the first RGB image, the first infrared image and the first depth image in the same coordinate system is as follows:

[0073] S21: extracting feature points from the initial RGB image, the initial infrared image and the initial depth image, respectively, each feature point having a feature descriptor, the feature descriptor being an encoding vector containing local information around the feature point;

[0074] S22: for the sake of search efficiency, for each set of feature descriptors of modal images, an approximate nearest neighbor search data structure is constructed using an approximate nearest neighbor search algorithm;

[0075] S23: searching for candidate matching points in the approximate nearest neighbor search data structure of an optional modal image according to the feature point descriptors of another modal image; generally, the feature point with the smallest distance is selected as the candidate matching point;

[0076] S24: based on the candidate matching points, a geometric relationship estimation matrix between any two modal images is obtained using a random sample consensus algorithm to exclude the influence of false matching points; if the two images are taken by small movements with a fixed distance, a fundamental matrix can be estimated; if they are different modal images under the same view angle, a homography matrix can be estimated;

[0077] S25: Aligning all modal images based on the estimated matrix of geometric relationship to obtain the first RGB image, the first infrared image and the first depth image in the same coordinate system.

[0078] As shown in FIG. 4, in S3, the method for obtaining the first mask representing the target region in the first RGB image, the first infrared image and the first depth image is as follows:

[0079] S31: Selecting one or more initial prompt points in the target region of the first RGB image;

[0080] S32: Obtaining the first mask of the first RGB image based on the initial prompt points, and mapping the coordinates of the initial prompt points to the first infrared image and the first depth image respectively to obtain the first mask of the first infrared image and the first mask of the first depth image.

[0081] Referring to FIG. 5, in S4, the method for obtaining the second mask is as follows:

[0082] S41: Obtaining the information entropy of each modal image channel based on the first masks of the first RGB image, the first infrared image and the first depth image, and the calculation method is as follows:

[0083] The calculation formula of the information entropy H RGB of the first RGB image channel is:

[0084] The calculation formula of the information entropy H IR of the first infrared image channel is:

[0085] The calculation formula of the information entropy H Depth of the first depth image channel is:

[0086] Wherein, P RGB (i RGB ), P Depth (i Depth ) and P IR (i IR ) are the probability distribution of the RGB image, the depth image and the infrared image at pixel value i RGB , i Depth , i IR ;

[0087] S42: Taking the reciprocal of the information entropy of each modal image channel as the weight of the first mask of the corresponding modal image based on the information entropy of each modal image channel;

[0088] S43: weight the first mask of the three modal images according to the weight of the first mask of the corresponding modal image, to obtain a fusion value F(x, y) of each pixel point, and the calculation formula is:

[0089] wherein, W RGB is the weight of the first RGB image channel, RGB mask (x, y) is the coordinate of any pixel point in the first mask of the first RGB image; W IR is the weight of the first infrared image channel, IR mask (x, y) is the coordinate of any pixel point in the first mask of the first infrared image; W Dept h is the weight of the first depth image channel, Depth mask (x, y) is the coordinate of any pixel point in the first mask of the first depth image.

[0090] S44: compare the fusion value F(x, y) of each pixel point with an evaluation threshold θ, and retain the pixel points with the fusion value F(x, y) greater than the evaluation threshold θ, to obtain a second mask.

[0091] Further, the calculation method of the evaluation threshold θ is:

[0092] wherein, represents the mean value of the information entropy of each modal image channel, H RGB is the information entropy of the first RGB image, H Depth is the information entropy of the first depth image, H IR is the information entropy of the first infrared image; σ H represents the standard deviation of the information entropy of each modal image channel,

[0093] As shown in FIG. 6, in S5, the method for obtaining the segmentation result of the target is as follows:

[0094] S51: superimpose the second mask on the initial RGB image to obtain a minimum bounding box of the target;

[0095] S52: traverse all pixel points in the minimum bounding box, and convert the RGB value of each pixel point into its closest color feature in the form of ColorNames (CN) based on a color mapping table by calculating the Euclidean distance or other similarity measure of the RGB value of each pixel point and the color in the mapping table;

[0096] The type of the color mapping table includes, but is not limited to, a WEB standard color table, an X11 color name list, or other defined color classification systems, and the content of the color mapping table includes a series of common color names and corresponding RGB value ranges;

[0097] S53: input the converted color features into a K-means algorithm, cluster the regions in the minimum bounding box into N categories by calculating the distance between the color features of each pixel point and each cluster center, and obtain a clustering result;

[0098] S54: randomly select n points in the category with the largest total amount of data in the clustering result as auxiliary points, add the auxiliary points to the prompt point set, and obtain an updated prompt point set;

[0099] S55: based on the updated prompt point set, generate a new mask through a SegmentAnything Model (SAM) algorithm, and calculate the change amount ΔIoU of the Intersection over Union (IoU) between the current mask and the mask generated in the last iteration:

[0100] Wherein, A is the mask generated in the last iteration, and B is the current mask.

[0101] S56: determine whether the change amount ΔIoU is less than a preset threshold ε:

[0102] If not, return to step S54;

[0103] If yes, stop iteration, output the current mask, and superimpose the current mask on the initial RGB image to obtain the segmentation result of the target.

[0104] Embodiment Two

[0105] As shown in FIG. 7, the application also provides an image labeling tool, which includes the following modules:

[0106] The receiving module 10 is configured to receive at least one creation instruction input through a target interface, and when the number of the creation instructions is more than one, the creation instructions are queued for processing according to priority or submission order.

[0107] The obtaining module 20 is configured to obtain a target number of images to be labeled based on a resource address included in the creation instruction. In addition to supporting a single resource address, the obtaining module 20 also supports batch importing of an image list or a directory path, and provides basic image preprocessing options such as scaling, cropping, and rotating.

[0108] The labeling module 30 is configured to automatically label the to-be-labeled image by using the target segmentation method based on multi-modal data fusion described in Embodiment 1, and also allows the user to manually adjust the bounding box or the segmentation region on the basis of the automatic labeling, and display the labeling result on the target interface, so as to facilitate the user to instantly check and verify.

[0109] The saving module 40 is configured to save the labeling result in a plurality of file formats, including but not limited to xml, txt, JSON, and CSV, and allows the user to select the output format, so as to save different versions for each labeling task and facilitate tracking of the modification history.

[0110] In addition, the image labeling tool provided by the embodiment can also analyze the labeling data and generate a statistical report to help the user to understand the labeling progress and quality, and can also be connected with a cloud storage service to realize seamless data uploading and downloading.

[0111] To sum up, the present application aims to improve the accuracy and stability of target segmentation by comprehensively using the information of RGB images, infrared images and depth images. The method realizes effective segmentation of the target through a series of steps such as image alignment, feature point matching, mask generation, information entropy calculation and weight fusion, and further improves the quality of the segmentation result through an iterative optimization strategy. In addition, the present application can realize accurate segmentation of the target and can be used in many fields such as automatic driving, medical image analysis and security monitoring system.

[0112] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0113] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems) and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0114] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart or flowsheets and / or block or blocks of the block diagrams.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheets and / or block or blocks of the block diagrams.

[0116] Obviously, the above-described embodiments are only examples for clearly illustrating the present application and are not intended to limit the present application. Based on the above description, one of ordinary skill in the art can make other different forms of changes or modifications. Here, it is not necessary and impossible to enumerate all the embodiments. The obvious changes or modifications derived therefrom are still within the protection scope of the present application.

Claims

1. A target segmentation method based on multi-modal data fusion, characterized in that, The method comprises the following steps: S1: obtaining an initial RGB image, an initial infrared image and an initial depth image containing a target; S2: performing alignment processing on the initial RGB image, the initial infrared image and the initial depth image to obtain a first RGB image, a first infrared image and a first depth image in a same coordinate system; S3: specifying an initial prompt point in the first RGB image, and obtaining a first mask representing a target region in the first RGB image, the first infrared image and the first depth image through the initial prompt point; S4: fusing pixel values of images in different modalities based on the first masks of the first RGB image, the first infrared image and the first depth image to obtain a second mask; S5: determining a minimum bounding box of the target based on the second mask, and calibrating the minimum bounding box to obtain a segmentation result of the target.

2. The target segmentation method based on multi-modal data fusion according to claim 1, characterized in that: In S2, the method for obtaining the first RGB image, the first infrared image and the first depth image in the same coordinate system is as follows: S21: extracting feature points from the initial RGB image, the initial infrared image and the initial depth image respectively, each feature point having a feature descriptor, and the feature descriptor being an encoding vector containing local information around the feature point; S22: constructing an approximate nearest neighbor search data structure for a feature descriptor set of each modality image; S23: searching in the approximate nearest neighbor search data structure of an optional modality image according to a feature point descriptor of another modality image to obtain a candidate matching point; S24: obtaining a geometric relationship estimation matrix between any two modality images based on the candidate matching point; S25: performing alignment processing on all modality images based on the geometric relationship estimation matrix to obtain the first RGB image, the first infrared image and the first depth image in the same coordinate system.

3. The target segmentation method based on multi-modal data fusion according to claim 1, characterized in that: In S3, the method for obtaining the first mask representing the target region in the first RGB image, the first infrared image and the first depth image is as follows: S31: selecting one or more initial prompt points in the target region of the first RGB image; S32: obtaining the first mask of the first RGB image based on the initial prompt points, and mapping the coordinates of the initial prompt points to the first infrared image and the first depth image respectively to obtain the first mask of the first infrared image and the first mask of the first depth image.

4. The target segmentation method based on multi-modal data fusion according to claim 1, characterized in that: In S4, the method for obtaining the second mask is as follows: S41: obtaining information entropy of each modality image channel based on the first masks of the first RGB image, the first infrared image and the first depth image; S42: obtaining a weight of the first mask of a corresponding modality image based on the information entropy of each modality image channel; S43: performing weighted fusion on the first masks of the three modality images according to the weight of the first mask of the corresponding modality image to obtain a fusion value of each pixel point. S44: compare the fusion value of each pixel point with the evaluation threshold, retain the pixel points with the fusion value greater than the evaluation threshold, and obtain a second mask.

5. The target segmentation method based on multi-modal data fusion according to claim 4, characterized in that: In S41, the calculation method of the information entropy of each modality image channel is as follows: The information entropy H of the first RGB image channel RGB The calculation formula is: The information entropy H of the first infrared image channel IR The calculation formula is: information entropy H of the first depth image channel Depth The formula for calculating H is: where P RGB (i RGB ), P Depth (i Depth ), and P IR (i IR ) are the probability distributions of the RGB image, the depth image, and the infrared image on the pixel value i RGB , i Depth , i IR , respectively.

6. The target segmentation method based on multi-modal data fusion according to claim 4, characterized in that: In S42, the method for obtaining the weight of the first mask corresponding to the modality image is: taking the reciprocal of the information entropy of each modality image channel as its corresponding weight.

7. The target segmentation method based on multi-modal data fusion according to claim 4, characterized in that: In S43, the method for obtaining the fusion value F(x, y) of each pixel point is as follows: wherein W RGB is a weight of the first RGB image channel, RGB mask (x,y) is a coordinate of any pixel point in the first mask of the first RGB image; W IR is a weight of the first infrared image channel, IR mask (x,y) is a coordinate of any pixel point in the first mask of the first infrared image; W Depth is a weight of the first depth image channel, Depth mask (x,y) is a coordinate of any pixel point in the first mask of the first depth image.

8. The target segmentation method based on multi-modal data fusion according to claim 4, characterized in that: The method for calculating the evaluation threshold θ is: wherein a mean value of the information entropy representing each modality image channel, H is the information entropy of the first RGB image Depth H is the information entropy of the first depth image IR H is the information entropy of the first infrared image H denotes the standard deviation of the information entropy of each modality image channel, 9.The target segmentation method based on multi-modal data fusion according to claim 1, characterized in that: In S5, the method for obtaining the segmentation result of the target is as follows: S51: superimpose the second mask on the initial RGB image to obtain a minimum bounding box of the target; S52: traverse all pixel points in the minimum bounding box, and convert the RGB value of each pixel point into its closest color feature according to the color mapping table; S53: process the converted color feature, cluster the area in the minimum bounding box into N categories, and obtain a clustering result; S54: randomly select n points in the category with the largest total amount of data in the clustering result as auxiliary points, add the auxiliary points to the prompt point set, and obtain an updated prompt point set; S55: based on the updated prompt point set, generate a new mask, and calculate the change amount of the intersection over union between the current mask and the mask generated in the last iteration; S56: determine whether the change amount is less than a preset threshold: If not, return to step S54; If yes, stop iteration, output the current mask, superimpose the current mask on the initial RGB image, and obtain the segmentation result of the target.

10. An image annotation tool, characterized by, Comprise: A receiving module configured to receive at least one creation instruction input through a target interface, and when the number of the creation instructions is more than one, queue processing according to a priority or a submission order; An obtaining module configured to obtain a target number of images to be labeled based on a resource address included in the creation instruction; A labeling module configured to automatically label the images to be labeled by using the target segmentation method based on multi-modality data fusion according to any one of claims 1-9, and display a labeling result on the target interface; A saving module configured to save the labeling result in multiple file formats.

Images