A deep learning-based semantic segmentation image data automatic labeling method

Abstract

The application discloses a kind of based on deep learning's semantic segmentation image data automatic labeling method. A plurality of scale semantic segmentation label file generation networks are used to generate a large number of semantic segmentation label files and the quality score of each label file, then the high-quality semantic segmentation label file is screened out through the score, and finally the label file lightweight algorithm is used to make the dataset lighter, to realize semantic segmentation image data automatic labeling. The present application can effectively generate a large number of high-precision semantic segmentation image data by combining deep learning with polygon approximation algorithm.

Description

Technical Field

[0001] This invention belongs to the field of computer science, specifically relating to a method for automatically labeling semantic segmentation image data based on deep learning. Background Technology

[0002] With modernization, computer vision and semantic segmentation algorithms have been widely applied. Using semantic segmentation to perform pixel-level annotation on unlabeled image datasets is of great significance for recognizing and detecting targets of different shapes, such as vehicle detection, target classification, and target color recognition.

[0003] Currently, datasets related to semantic image segmentation include KITTI, UA-DETRAC, and BDD100K, but these datasets are not suitable for all real-world semantic segmentation and recognition tasks. In certain specific scenarios, it is necessary to manually collect unlabeled image datasets in specific environments and perform semantic segmentation annotations on these images. For image semantic segmentation tasks, i.e., training a high-performance semantic segmentation neural network, the required dataset is enormous. Annotating semantic segmentation image datasets often takes hours, and training a high-precision semantic segmentation deep neural network requires approximately 6000 hours or more of data. When each image contains 70 targets that need annotation, semantic segmentation annotation of just 100 such images would require one person spending more than 12 hours on annotation alone. Manually annotating datasets is time-consuming and laborious, and the quality of semantic segmentation image data annotated under fatigue is also low. This method makes it difficult to quickly and easily obtain high-quality, large-scale, and satisfactory vehicle datasets. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by providing a deep learning-based method for automatically labeling semantic segmentation image data. This method can generate a large number of lightweight, high-quality semantic segmentation label files, which can be combined with unlabeled images to form a lightweight, high-quality semantic segmentation dataset, significantly reducing the time required for manual labeling.

[0005] To achieve the above objectives, the technical solution provided by this invention is an automatic semantic segmentation image data encoding method based on deep learning.

[0006] Step 1: Collect a large number of unlabeled images and divide them into two groups: a small number of unlabeled images and a large number of unlabeled images. Perform mean grayscale conversion and uniform size preprocessing on these two groups of images to obtain the preprocessed small number of unlabeled images and the preprocessed large number of unlabeled images.

[0007] Step 2: Manually label the small number of unlabeled images after preprocessing in Step 1 to obtain a small number of labeled image datasets;

[0008] Step 3: Construct a multi-scale semantic segmentation label file generation network and train it using the COCO dataset to obtain a pre-trained model for the COCO dataset.

[0009] Step 4: Use transfer learning to transfer the parameters in the pre-trained model of the COCO dataset to the constructed multi-scale semantic segmentation label file generation network. Then, use the small amount of labeled image dataset from Step 2 to train the network and obtain the trained multi-scale semantic segmentation label file generation network.

[0010] Step 5: Use the multi-scale semantic segmentation label file generation network trained in Step 4 to automatically label and score the large number of unlabeled images after preprocessing in Step 1, and retain the label files with high scores as high-quality label files.

[0011] Step 6: Use the semantic segmentation label file lightweighting algorithm to lightweight the high-quality label file obtained in Step 5 to obtain a lightweight high-quality semantic segmentation label file, and match it one-to-one with the large number of unlabeled images collected in Step 1 to form a lightweight high-quality semantic segmentation dataset, thereby realizing automatic labeling of semantic segmentation image data based on deep learning.

[0012] Step 6.1: Combine the beetle whisker algorithm and the Douglas-Peucker algorithm to construct a lightweight semantic segmentation label file algorithm;

[0013] Step 6.2: Feed the high-quality semantic segmentation label file dataset from Step 5 into the lightweight semantic segmentation label file algorithm built in Step 6.1 one by one to obtain a large number of lightweight high-quality semantic segmentation label files, which are denoted as the lightweight high-quality semantic segmentation label file dataset.

[0014] Step 6.3: Place the lightweight high-quality semantic segmentation label file dataset obtained in Step 6.2 and the large number of unlabeled images collected in Step 1 into a folder, denoted as the lightweight high-quality semantic segmentation dataset. This completes the automatic labeling of semantic segmentation image data based on deep learning.

[0015] Furthermore, the size of the sample collected in step 1 is... The images were divided into two groups: a small number of unlabeled images and a large number of unlabeled images. All the acquired unlabeled images were then averaged to grayscale. Finally, the averaged grayscale images (the small number of unlabeled images and the large number of unlabeled images) were scaled down. The size, if Then the scaling factor ,like Then the scaling factor The scaled image is then re-valued using the nearest neighbor interpolation algorithm. If either the length or width of the scaled image is insufficient... Then, use points with a pixel value of 0 to fill in the gaps. .

[0016] Furthermore, step 2 involves manually labeling the small number of preprocessed unlabeled images from step 1 using irregular rectangles, saving all the coordinates of the irregular rectangles in a JSON file, with each unlabeled image corresponding to a JSON file, and placing the small number of preprocessed unlabeled images and their corresponding JSON files in the same folder, which is referred to as a small number of labeled image datasets.

[0017] Furthermore, the multi-scale semantic segmentation label file generation network in step 3 combines ResNet-50 with a feature pyramid to extract high-level features from the image, forming a five-layer feature pyramid. The feature maps of the last two layers are subtracted to obtain a multi-scale feature map. Then, the multi-scale feature map is subjected to multiple convolution operations to obtain a deep convolutional feature map. The deep convolutional feature map is then pooled and input into two fully connected layers to obtain the semantic segmentation target category and semantic segmentation target contour. The obtained semantic segmentation target category and semantic segmentation target contour are stored in a JSON file, which is designated as the label file. Finally, the multi-scale feature map and the deep convolutional feature map are added to obtain the label file quality score feature map. The semantic segmentation quality score feature map is then subjected to multiple convolution and pooling operations and then passed through a fully connected layer to obtain the quality score of the label file.

[0018] Furthermore, step 5 involves assigning all parameters of the multi-scale semantic segmentation label file generation model trained in step 4 to the multi-scale semantic segmentation label file generation network built in step 3, forming a trained multi-scale semantic segmentation label file generation network. Then, the large number of preprocessed unlabeled images from step 1 are fed into the trained multi-scale semantic segmentation label file generation network to obtain label files and label file quality scores. Each image corresponds to one label file and one label file quality score. When the label file quality score corresponding to an image is greater than... If the image is not found, retain the image and its corresponding label file; otherwise, discard the image and its corresponding label file. The retained images are denoted as the high-quality image dataset, and the corresponding label files are denoted as the high-quality semantic segmentation label file dataset. The high-quality image dataset and the high-quality semantic segmentation label file dataset are placed in the same folder and denoted as the high-quality semantic segmentation dataset.

[0019] Furthermore, the input to the semantic segmentation label file lightweighting algorithm in step 6.1 is the semantic segmentation label file corresponding to the image in step 5, and the output is the lightweight semantic segmentation label file. The semantic segmentation label file includes two pieces of information: the target's category information and the target's contour information. While ensuring that the target's contour information covers the target, the number of contour points in the target's contour information is reduced, resulting in less information recorded in the label file and a lighter label file. This includes the following steps:

[0020] Step 6.1.1: Divide the information in the semantic segmentation label file corresponding to the image in Step 5 into target category information and target contour information. The target contour information records all points on the closed curve surrounding the target. Record all points on the closed curve of the polygon surrounding the target as edge contour points, initialize the Douglas-Peucker algorithm, and set the threshold D. threshold ;

[0021] Step 6.1.2: From the edge contour points of the closed curve enclosing the target in Step 6.1.1, find the two points that are farthest apart, and denote them as points M and N respectively. Connect points M and N to form line segment MN. The formula for calculating line segment MN is as follows:

[0022] (1)

[0023] In the formula, A , B , C These are the parameters of line segment MN. x It is the dependent variable. y It is the independent variable;

[0024] Then, the closed curve surrounding the target is split into two non-closed curves through points M and N, namely curve MN1 and curve MN2.

[0025] Step 6.1.3: Find the point Q on curve MN1 that is furthest from the perpendicular distance of line segment MN in step 6.1.3. The calculation formula is as follows:

[0026] (2)

[0027] In the formula, ( x o , y o Let Q be the coordinate of point Q. A , B , C These are the parameters of line segment MN. This represents the distance from point Q to MN;

[0028] The parameters of the beetle whisker algorithm are initialized, including the decay factor Eta, step size Step, step-whisker ratio c, iteration count n, and parameter k. A fitness function is also established. as follows:

[0029] (3)

[0030] In the formula, ( x o , y o Let Q be the coordinate of point Q. A , B , C These are the parameters of line segment MN;

[0031] According to formula (4), the direction of the random longhorn beetle. The distance between the left and right whiskers of the longhorn beetle Calculate the position of the longhorn beetle's left whiskers The corresponding position of the right whisker And calculate the position of the left whisker. Corresponding fitness function and right position corresponding function And the next location of the longhorn beetle. x ;

[0032] (4)

[0033] In the formula, when hour, ;when hour, ;when hour, ;

[0034] Repeat step 6.1.3. This time, the optimal fitness function value is obtained. The fitness function corresponds to the final position of the longhorn beetle whiskers, which is the position of point Q.

[0035] Step 6.1.4, if Then the coordinates of points Q, M, and N are recorded as contour points on the target contour after the reduction of curve MN1; if Then, use point Q to divide curve MN1 into two curves, namely curve MQ and curve NQ. Then connect point M and point Q and denote it as line segment MQ, and connect point N and point Q and denote it as line segment NQ.

[0036] Step 6.1.5: Perform steps 6.1.3 and 6.1.4 on points MQ and NQ respectively until all calculations are completed, and obtain all contour points on MN1. Then replace curve MN2 with MN1, and repeat steps 6.1.3 and 6.1.4 to obtain all contour points on MN2. Record all contour points on curves MN1 and MN2 as contour points on the reduced target contour.

[0037] Step 6.1.6: Create a new blank JSON file, and multiply the x and y coordinates of the contour points on the reduced target contour obtained in 6.1.4 by the scaling factor from Step 1. The target category information from 6.1.1 is then stored together in a newly created blank JSON file, which is called the lightweight semantic segmentation tag file.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] By leveraging the high-precision semantic segmentation capabilities of deep neural networks after transfer learning, and combining the advantages of the Douglas-Peucker algorithm and the beetle whisker algorithm in polygon optimization, a large number of high-quality semantic segmentation datasets are generated, and the semantic segmentation datasets are made more lightweight, thus improving the data requirements for other deep learning algorithms that require large amounts of semantic segmentation data. Attached Figure Description

[0040] Figure 1 This is an overall flowchart of an embodiment of the present invention.

[0041] Figure 2 This is an unlabeled image according to an embodiment of the present invention.

[0042] Figure 3 This is a flowchart of the multi-scale semantic segmentation tag file generation network according to an embodiment of the present invention.

[0043] Figure 4 This is a flowchart of the tag file lightweighting algorithm according to an embodiment of the present invention.

[0044] Figure 5 This is a tag file image generated by an embodiment of the present invention.

[0045] Figure 6 This is a result image of displaying a tagged file on an untagged image according to an embodiment of the present invention. Detailed Implementation

[0046] This invention provides a method for automatically labeling semantic segmentation image data based on deep learning. The technical solution of this invention will be further explained below using an unlabeled vehicle image as an example.

[0047] like Figure 1As shown, the process of this embodiment of the invention includes the following steps:

[0048] Step 1: Collect a large number of unlabeled images and divide them into two groups: a small number of unlabeled images and a large number of unlabeled images. Perform mean grayscale conversion and size uniformity preprocessing on these two groups of images to obtain the preprocessed small number of unlabeled images and the preprocessed large number of unlabeled images.

[0049] Step 1.1: Collect a large number of unlabeled images, first divide them into a small number of unlabeled images and a large number of unlabeled images, and then perform mean grayscale conversion.

[0050] 2100 images of size 2048×1206 were captured using a mobile phone camera or camera. 100 of them were selected as a small group of unlabeled images, and the remaining 2000 were selected as a large group of unlabeled images. The 2100 images were converted to grayscale using the mean grayscale method in [Reference 1] to obtain a small group of unlabeled grayscale images and a large group of unlabeled grayscale images.

[0051] Step 1.2: Scale the small number of unlabeled grayscale images and the large number of unlabeled grayscale images obtained in Step 1.1 to a size of 1024×1024, and use the nearest neighbor interpolation algorithm to assign values ​​to the scaled images. If either the width or height of the scaled image is less than 1024, use pixels with a value of 0 to pad it to 1024.

[0052] If the length and width of the unlabeled grayscale image are both less than 1024 and equal in size, then it is scaled up proportionally to 1024×1024. If the length and width of the unlabeled grayscale image are both less than 1024 and unequal in size, then it is scaled up proportionally, with the longest side being scaled to 1024. If the shorter side is still less than 1024, the portion of the shorter side shorter than 1024 is padded with a pixel value of 0, resulting in a grayscale image size of 1024×1024. If either the length or width of the labeled grayscale image is greater than 1024, then the image is scaled proportionally, with the longest side being scaled to 1024. If the shorter side is less than 1024, the portion of the shorter side shorter than 1024 is padded with a pixel value of 0, resulting in a grayscale image size of 1024×1024. If the length and width of the labeled grayscale images are equal, and the length and width are greater than 1024, then the length and width of these unlabeled grayscale images are scaled proportionally to 1024×1024. The scaling factor in this embodiment... =2048 / 1024=2, meaning that the 2048×1206 image is first scaled to 1024×603, and then the nearest neighbor interpolation algorithm is used to assign values ​​to the scaled image, padding pixels with width less than 1024 with 0.

[0053] Step 2: Manually label the small number of unlabeled images preprocessed in Step 1 to obtain a small number of labeled image datasets.

[0054] The 100 preprocessed unlabeled images from step 1 were labeled using irregular rectangles. All coordinates of the irregular rectangles were saved in a JSON file, with one JSON file for each unlabeled image. The 100 preprocessed unlabeled images and their corresponding JSON files were placed in the same folder, which was designated as a small number of labeled image datasets.

[0055] Step 3: Construct a multi-scale semantic segmentation label file generation network and train it using the COCO dataset to obtain a pre-trained model for the COCO dataset.

[0056] Step 3.1: Construct a multi-scale semantic segmentation label file generation network.

[0057] Step 3.1.1: The ResNet-50 combined with the feature pyramid in [Reference 2] is used as the feature extraction part of the multi-scale semantic segmentation label file generation network to extract high-level features from the image, forming a five-layer feature pyramid. The feature maps of the last two layers are subtracted to form a fused feature map, which is denoted as the multi-scale feature map.

[0058] Step 3.1.2: The multi-scale feature map obtained in step 3.1.1 is convolved twice with 14×14×256 kernels, and then convolved again with 28×28×256 kernels to obtain the depth convolution feature map.

[0059] Step 3.1.3: Pool the deep convolutional feature map from step 3.1.2, and then input it into two fully connected layers to obtain the target category and target contour of semantic segmentation. Store the obtained target category and target contour of semantic segmentation into a JSON file, which is referred to as the label file.

[0060] Step 3.1.4: Add the multi-scale feature map from step 3.1.1 and the depthwise convolutional feature map from step 3.1.2 to obtain the label file quality score feature map. Then, perform three convolution operations on the semantic segmentation quality score feature map with a convolution kernel of size 14×14×256, then perform a convolution operation with a convolution kernel of size 7×7×256, then perform two pooling operations, and then pass through a fully connected layer to obtain the quality score of the label file.

[0061] Step 3.2: Use the multi-scale semantic segmentation label file generation network constructed in step 2.1 using the COCO dataset in [Reference 3] to train the model, which is denoted as the pre-trained model of the COCO dataset.

[0062] Step 4: Use transfer learning to transfer the parameters in the pre-trained model of the COCO dataset to the multi-scale semantic segmentation label file generation network built in Step 3.1. Then, use the small amount of labeled image dataset from Step 2 to train the network and obtain the trained multi-scale semantic segmentation label file generation network.

[0063] In step 4.1, the initial values ​​of the parameters in the multi-scale semantic segmentation label file generation network built in step 3.1 are random numbers. All the parameters recorded in the pre-trained model of the COCO dataset in step 3.2 are transferred to the multi-scale semantic segmentation label file generation network built in step 3.1 to replace the initial random values ​​of the parameters. This is called the multi-scale semantic segmentation label file generation network after transfer learning.

[0064] Step 4.2: Input the small amount of labeled training set image data from Step 2 into the multi-scale semantic segmentation label file generation network [Reference 4] after transfer learning for training, and obtain the trained multi-scale semantic segmentation label file generation model.

[0065] Step 5: Use the multi-scale semantic segmentation label file generation network trained in Step 4 to automatically label and score the large number of unlabeled images after preprocessing in Step 1, and retain the label files with high scores as high-quality label files.

[0066] All parameters of the multi-scale semantic segmentation label file generation model trained in step 4.2 are assigned to the multi-scale semantic segmentation label file generation network built in step 3.1 to form a trained multi-scale semantic segmentation label file generation network. Then, a large number of preprocessed unlabeled images from step 1 are fed into the trained multi-scale semantic segmentation label file generation network to obtain the label files in step 3.1.3 and the label file quality scores in step 3.1.4. Each image corresponds to one label file and one label file quality score. When the quality score of the label file corresponding to an image is greater than 90, the image and its corresponding label file are retained; otherwise, the image and its corresponding label file are discarded. The retained images are denoted as the high-quality image dataset, and the label files corresponding to the retained images are denoted as the high-quality semantic segmentation label file dataset. The high-quality image dataset and the high-quality semantic segmentation label file dataset are placed in the same folder and denoted as the high-quality semantic segmentation dataset.

[0067] Step 6: Use the semantic segmentation label file lightweighting algorithm to lightweight the high-quality label file obtained in Step 5 to obtain a lightweight high-quality semantic segmentation label file. Then, match it one-to-one with the large number of unlabeled images collected in Step 1 to form a lightweight high-quality semantic segmentation dataset, thereby realizing automatic labeling of semantic segmentation image data based on deep learning.

[0068] Step 6.1: Combine the beetle whisker algorithm in [Reference 5] and the Douglas-Peucker algorithm in [Reference 6] to construct a lightweight algorithm for semantic segmentation tag files.

[0069] The input to the lightweight semantic segmentation label file algorithm is the semantic segmentation label file corresponding to the image in step 5, and the output is the lightweight semantic segmentation label file. The semantic segmentation label file includes two pieces of information: the target's category information and the target's contour information. While ensuring that the target's contour information covers the target, the number of contour points in the target's contour information is reduced, resulting in less information recorded in the label file and a lighter label file.

[0070] Step 6.1.1: Divide the information in the semantic segmentation label file corresponding to the image in Step 5 into target category information and target contour information. The target contour information records all points on the closed curve surrounding the target. Record all points on the closed curve of the polygon surrounding the target as edge contour points, initialize the Douglas-Peucker algorithm, and set the threshold D. threshold It is 25.

[0071] Step 6.1.2: From the edge contour points of the closed curve enclosing the target in Step 6.1.1, find the two points that are farthest apart, and denote them as points M and N respectively. Connect points M and N to form line segment MN. The formula for calculating line segment MN is as follows:

[0072] (1)

[0073] In the formula, A, B, and C are the parameters of line segment MN, which can be set with reference [7]; x It is the dependent variable. y It is the independent variable.

[0074] Then, the closed curve surrounding the target is split into two non-closed curves, namely curve MN1 and curve MN2, through the two points M and N.

[0075] Step 6.1.3: Find the point Q on curve MN1 that is furthest from the perpendicular distance of line segment MN in step 6.1.3. The calculation formula is as follows:

[0076] (2)

[0077] In the formula, ( x o , y o Let Q be the coordinate of point Q. A , B , C These are the parameters of line segment MN. This represents the distance from point Q to MN.

[0078] The parameters of the beetle whisker algorithm are initialized, including the decay factor Eta, step size Step, step-whisker ratio c, iteration count n, and parameter k. A fitness function is also established. as follows:

[0079] (3)

[0080] In the formula, ( x o , y o Let Q be the coordinate of point Q. A , B , C These are the parameters of line segment MN.

[0081] According to formula (4), the direction of the random longhorn beetle. The distance between the left and right whiskers of the longhorn beetle Calculate the position of the longhorn beetle's left whiskers The corresponding position of the right whisker And calculate the position of the left whisker. Corresponding fitness function and right position corresponding function And the next location of the longhorn beetle. x .

[0082] (4)

[0083] In the formula, when hour, ;when hour, ;when hour, .

[0084] Repeat step 6.1.3 20 times to obtain the optimal fitness function value. The fitness function corresponds to the final position of the longhorn beetle whiskers, which is the position of point Q.

[0085] Step 6.1.4, if Then the coordinates of points Q, M, and N are recorded as contour points on the target contour after the reduction of curve MN1; if Then, use point Q to divide curve MN1 into two curves, namely curve MQ and curve NQ. Then connect point M and point Q and denote it as line segment MQ, and connect point N and point Q and denote it as line segment NQ.

[0086] Step 6.1.5: Perform steps 6.1.3 and 6.1.4 on points MQ and NQ respectively until all calculations are completed, and obtain all contour points on MN1. Then replace curve MN2 with MN1, and repeat steps 6.1.3 and 6.1.4 to obtain all contour points on MN2. Record all contour points on curves MN1 and MN2 as contour points on the reduced target contour.

[0087] Step 6.1.6: Create a new blank JSON file, and multiply the x and y coordinates of the contour points on the reduced target contour obtained in 6.1.4 by the scaling factor in step 1.2. The target category information from 5.1.1 is then stored together in a newly created blank JSON file, which is called the lightweight semantic segmentation tag file.

[0088] Step 6.2: Feed the high-quality semantic segmentation label file dataset from Step 5 into the lightweight semantic segmentation label file algorithm built in Step 6.1 one by one to obtain a large number of lightweight high-quality semantic segmentation label files, which are denoted as the lightweight high-quality semantic segmentation label file dataset.

[0089] Step 6.3: Place the lightweight high-quality semantic segmentation label file dataset obtained in Step 6.2 and the large number of unlabeled images collected in Step 1 into a folder, denoted as the lightweight high-quality semantic segmentation dataset. This completes the automatic labeling of semantic segmentation image data based on deep learning.

[0090] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

[0091] [Reference 1] Tang Fashuai, Gao Qi, Du Zongzhan. High-voltage line target detection based on nearest neighbor weighted gray-scale mean algorithm [J]. Computer Applications and Software, 2021, 38(10):201-208+296.

[0092] [Reference 2] Lin Jinchao, Wen Pan, Pang Yu. Text detection in natural scene images based on feature pyramid network [J]. Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition), 2022, 34(01):155-163.

[0093] [Reference 3] Zhao Yongqiang, Rao Yuan, Dong Shipeng, Zhang Junyi. A review of deep learning object detection methods [J]. Journal of Image and Graphics, 2020, 25(04):629-654.

[0094] [Reference 4] Qi Guimei, Yu Zhihong, Wu Zhuoyue, Li Yanling, Zhao Jinming. Wild bird identification based on transfer learning and convolutional neural network [J]. Journal of Inner Mongolia Normal University (Natural Science Chinese Edition), 2022, 51(03):299-304.

[0095] [Reference 5] Wang Liang, Lu Yemin. Comparative study on the control of constant pressure water supply PID system by fuzzy and longhorn beetle optimization algorithms [J]. Industrial Control Computer, 2022, 35(03):77-79.

[0096] [Reference 6] Fei Lifan, He Jin, Ma Chenyan, Yan Huiwu. Research on 3D Douglas-Peucker algorithm and its application in DEM automatic generalization [J]. Acta Geodaetica et Cartographica Sinica, 2006(03):278-284.

[0097] [Reference 7] Gao Zunhai. A formula for the distance from a point to a straight line in space [J]. Research on Higher Mathematics, 2005(02):4-5.

Claims

1. A method for automatically labeling semantic segmentation image data based on deep learning, characterized in that, Includes the following steps: Step 1: Collect a large number of unlabeled images and divide them into two groups: a small number of unlabeled images and a large number of unlabeled images. Perform mean grayscale conversion and size uniformity preprocessing on these two groups of images to obtain the preprocessed small number of unlabeled images and the preprocessed large number of unlabeled images. The size of the collected data is The images were divided into two groups: a small number of unlabeled images and a large number of unlabeled images. All the acquired unlabeled images were then averaged to grayscale. Finally, the averaged grayscale images (the small number of unlabeled images and the large number of unlabeled images) were scaled down. The size, if Then the scaling factor ,like Then the scaling factor The scaled image is then re-valued using the nearest neighbor interpolation algorithm. If either the length or width of the scaled image is insufficient... Then, use points with a pixel value of 0 to fill in the gaps. ; Step 2: Manually label the small number of unlabeled images after preprocessing in Step 1 to obtain a small number of labeled image datasets; Irregular rectangles were used to manually label a small number of preprocessed unlabeled images in step 1. All coordinates of the irregular rectangles were saved in a JSON file. Each unlabeled image corresponded to a JSON file. The small number of preprocessed unlabeled images and their corresponding JSON files were placed in the same folder and denoted as a small number of labeled image datasets. Step 3: Construct a multi-scale semantic segmentation label file generation network and train it using the COCO dataset to obtain a pre-trained model for the COCO dataset. The multi-scale semantic segmentation label file generation network combines ResNet-50 with a feature pyramid to extract high-level features from images, forming a five-layer feature pyramid. The feature maps of the last two layers are subtracted to obtain a multi-scale feature map. The multi-scale feature map is then subjected to multiple convolution operations to obtain a deep convolutional feature map. The deep convolutional feature map is then pooled and fed into two fully connected layers to obtain the semantic segmentation target category and semantic segmentation target contour. The obtained semantic segmentation target category and semantic segmentation target contour are stored in a JSON file, which is designated as the label file. Finally, the multi-scale feature map and the deep convolutional feature map are added to obtain the label file quality score feature map. The semantic segmentation quality score feature map is then subjected to multiple convolution and pooling operations and passed through a fully connected layer to obtain the quality score of the label file. Step 4: Use transfer learning to transfer the parameters in the pre-trained model of the COCO dataset to the constructed multi-scale semantic segmentation label file generation network. Then, use the small amount of labeled image dataset from Step 2 to train the network and obtain the trained multi-scale semantic segmentation label file generation network. Step 5: Use the multi-scale semantic segmentation label file generation network trained in Step 4 to automatically label and score the large number of unlabeled images after preprocessing in Step 1, and retain the label files with high scores as high-quality label files. Step 6: Use the semantic segmentation label file lightweighting algorithm to lightweight the high-quality label file obtained in Step 5 to obtain a lightweight high-quality semantic segmentation label file. Then, match it one-to-one with the large number of unlabeled images collected in Step 1 to form a lightweight high-quality semantic segmentation dataset, thereby realizing automatic labeling of semantic segmentation image data based on deep learning.

2. The method for automatically labeling semantic segmentation image data based on deep learning as described in claim 1, characterized in that: Step 5 involves assigning all parameters of the multi-scale semantic segmentation label file generation model trained in Step 4 to the multi-scale semantic segmentation label file generation network built in Step 3, forming a trained multi-scale semantic segmentation label file generation network. Then, the large number of preprocessed unlabeled images from Step 1 are fed into the trained multi-scale semantic segmentation label file generation network to obtain label files and label file quality scores. Each image corresponds to one label file and one label file quality score. When the label file quality score of an image is greater than... If the image is not found, retain the image and its corresponding label file; otherwise, discard the image and its corresponding label file. The retained images are denoted as the high-quality image dataset, and the corresponding label files are denoted as the high-quality semantic segmentation label file dataset. The high-quality image dataset and the high-quality semantic segmentation label file dataset are placed in the same folder and denoted as the high-quality semantic segmentation dataset.

3. The method for automatically labeling semantic segmentation image data based on deep learning as described in claim 1, characterized in that: Step 6 includes the following steps: Step 6.1: Combine the beetle whisker algorithm and the Douglas-Peucker algorithm to construct a lightweight semantic segmentation label file algorithm; Step 6.2: Feed the high-quality semantic segmentation label file dataset from Step 5 into the lightweight semantic segmentation label file algorithm built in Step 6.1 one by one to obtain a large number of lightweight high-quality semantic segmentation label files, which are denoted as the lightweight high-quality semantic segmentation label file dataset. Step 6.3: Place the lightweight high-quality semantic segmentation label file dataset obtained in Step 6.2 and the large number of unlabeled images collected in Step 1 into a folder, denoted as the lightweight high-quality semantic segmentation dataset. This completes the automatic labeling of semantic segmentation image data based on deep learning.

4. The method for automatically labeling semantic segmentation image data based on deep learning as described in claim 3, characterized in that: The input to the semantic segmentation label file lightweighting algorithm in step 6.1 is the semantic segmentation label file corresponding to the image in step 5, and the output is the lightweight semantic segmentation label file; The semantic segmentation label file includes two pieces of information: the target's category information and the target's contour information. While ensuring that the target's contour information can cover the target, the number of contour points in the target's contour information is reduced, so that the label file records less information and becomes more lightweight.

5. The method for automatically labeling semantic segmentation image data based on deep learning as described in claim 4, characterized in that: Step 6.1 includes the following steps: Step 6.1.1: Divide the information in the semantic segmentation label file corresponding to the image in Step 5 into target category information and target contour information. The target contour information records all points on the closed curve surrounding the target. Record all points on the closed curve of the polygon surrounding the target as edge contour points, initialize the Douglas-Peucker algorithm, and set the threshold D. threshold ; Step 6.1.2: From the edge contour points of the closed curve enclosing the target in Step 6.1.1, find the two points that are farthest apart, and denote them as points M and N respectively. Connect points M and N to form line segment MN. The formula for calculating line segment MN is as follows: （1） In the formula, A , B , C These are the parameters of line segment MN. x It is the dependent variable. y It is the independent variable; Then, the closed curve surrounding the target is split into two non-closed curves through points M and N, namely curve MN1 and curve MN2. Step 6.1.3: Find the point Q on curve MN1 that is furthest from the perpendicular distance of line segment MN in step 6.1.

3. The calculation formula is as follows: （2） In the formula, ( x o , y o Let Q be the coordinate of point Q. A , B , C These are the parameters of line segment MN. This represents the distance from point Q to MN; The parameters of the beetle whisker algorithm are initialized, including the decay factor Eta, step size Step, step-whisker ratio c, iteration count n, and parameter k. A fitness function is also established. as follows: （3） In the formula, ( x o , y o Let Q be the coordinate of point Q. A , B , C These are the parameters of line segment MN; According to formula (4), the direction of the random longhorn beetle. The distance between the left and right whiskers of the longhorn beetle Calculate the position of the longhorn beetle's left whiskers The corresponding position of the right whisker And calculate the position of the left whisker. Corresponding fitness function and right position corresponding function And the next location of the longhorn beetle. x ; （4） In the formula, when hour, ;when hour, ;when hour, ; Repeat step 6.1.

3. This time, the optimal fitness function value is obtained. The fitness function corresponds to the final position of the longhorn beetle whiskers, which is the position of point Q. Step 6.1.4, if Then the coordinates of points Q, M, and N are recorded as contour points on the target contour after the reduction of curve MN1; if Then, use point Q to divide curve MN1 into two curves, namely curve MQ and curve NQ. Then connect point M and point Q and denote it as line segment MQ, and connect point N and point Q and denote it as line segment NQ. Step 6.1.5: Perform steps 6.1.3 and 6.1.4 on points MQ and NQ respectively until all calculations are completed, and obtain all contour points on MN1. Then replace curve MN2 with MN1, and repeat steps 6.1.3 and 6.1.4 to obtain all contour points on MN2. Record all contour points on curves MN1 and MN2 as contour points on the reduced target contour. Step 6.1.6: Create a new blank JSON file, and multiply the x and y coordinates of the contour points on the reduced target contour obtained in 6.1.4 by the scaling factor from Step 1. The target category information from 6.1.1 is then stored together in a newly created blank JSON file, which is called the lightweight semantic segmentation tag file.

Images