Image segmentation method and apparatus
By segmenting the image into multiple sub-regions using image segmentation methods, performing dilation processing, and merging regions with similar centroids, the problem of uneven segmentation when there are few samples is solved, thus improving the segmentation accuracy and stability of screen images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BOE TECHNOLOGY GROUP CO LTD
- Filing Date
- 2023-05-24
- Publication Date
- 2026-06-02
Smart Images
Figure CN116542994B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to an image segmentation method and apparatus. Background Technology
[0002] Image segmentation technology is one of the most important breakthroughs in the field of artificial intelligence in recent years, achieving great success in many areas such as computer vision, image and video analysis, multimedia, and machine vision defect detection. In real-time defect detection, the features of the region of interest within an image are not obvious, necessitating image segmentation into different regions for detection. This improves the subsequent defect recognition rate and facilitates the development of image processing and defect detection algorithms. Summary of the Invention
[0003] The technical problem to be solved by the present invention is to provide an image segmentation method and apparatus that can improve the accuracy of region segmentation.
[0004] To address the aforementioned technical problems, embodiments of the present invention provide the following technical solutions:
[0005] On the one hand, an image segmentation method is provided, including:
[0006] Obtain the image to be processed;
[0007] Feature extraction is performed on the image to be processed, and the image to be processed is divided into multiple sub-regions based on the feature extraction results;
[0008] The segmented sub-regions are then dilated.
[0009] Obtain the centroid of each sub-region after dilation;
[0010] The multiple sub-regions are merged based on the position of the centroid to obtain the segmented result.
[0011] In some embodiments, if the image to be processed is a color image, the method further includes the following steps before feature extraction:
[0012] The image to be processed is converted into a grayscale image.
[0013] In some embodiments, the step of extracting features from the image to be processed and segmenting the image to be processed into multiple sub-regions based on the feature extraction results includes:
[0014] The image to be processed is used to extract features through a convolutional neural network to obtain a three-dimensional feature map.
[0015] Based on the three-dimensional feature map and its corresponding label, cross-entropy loss is calculated for each pixel of the image to be processed to obtain the feature value of each pixel.
[0016] The feature value of each pixel is input into the classification network to obtain the classification result, which indicates whether each pixel belongs to the foreground region or the background region.
[0017] Based on the classification results, the image to be processed is segmented into multiple sub-regions, each of which belongs to either the foreground region or the background region.
[0018] In some embodiments, the dilation process on the segmented sub-regions includes:
[0019] Noise points in the sub-region are filtered out by opening operations.
[0020] In some embodiments, filtering noise points in the sub-region via opening operation includes:
[0021] The center point of the preset convolution template is compared with each pixel in the sub-region. If all points on the preset convolution template are within the range of the sub-region, the pixel in the sub-region is retained; otherwise, the pixel in the sub-region is removed.
[0022] In some embodiments, obtaining the centroid of each of the dilated sub-regions includes:
[0023] Convert the sub-region into a binary image;
[0024] All connected regions in the binary image are found using a connected component labeling algorithm;
[0025] The centroid is obtained by applying the geometric distance calculation algorithm to each of the connected regions.
[0026] In some embodiments, merging the plurality of sub-regions based on the position of the centroid includes:
[0027] If the distance between the centroids of two adjacent sub-regions in the column direction is less than M pixels, the two sub-regions are merged, where M is a positive integer.
[0028] Embodiments of the present invention also provide an image segmentation apparatus, comprising:
[0029] The acquisition module is used to acquire the image to be processed;
[0030] The segmentation module is used to extract features from the image to be processed and to segment the image into multiple sub-regions based on the feature extraction results.
[0031] An expansion module is used to expand the segmented sub-regions;
[0032] The centroid acquisition module is used to acquire the centroid of each of the sub-regions after expansion.
[0033] The merging module is used to merge the multiple sub-regions according to the position of the centroid to obtain the segmented result.
[0034] In some embodiments, if the image to be processed is a color image, the apparatus further includes:
[0035] A conversion module is used to convert the image to be processed into a grayscale image.
[0036] In some embodiments, the segmentation module includes:
[0037] The feature extraction unit is used to extract features from the image to be processed through a convolutional neural network to obtain a three-dimensional feature map;
[0038] The feature value calculation unit is used to perform cross-entropy loss calculation on each pixel of the image to be processed based on the three-dimensional feature map and its corresponding label to obtain the feature value of each pixel.
[0039] A classification unit is used to input the feature value of each pixel into a classification network to obtain a classification result, which indicates whether each pixel belongs to the foreground region or the background region.
[0040] The segmentation unit is used to segment the image to be processed into multiple sub-regions based on the classification result, each of the sub-regions belonging to either the foreground region or the background region.
[0041] In some embodiments, the apparatus further includes:
[0042] A filtering module is used to filter noise points in the sub-region through an opening operation.
[0043] The filtering module is specifically used to compare the center point of the preset convolution template with the pixels of the sub-region one by one. If all the points on the preset convolution template are within the range of the sub-region, the pixel in the sub-region is retained; otherwise, the pixel in the sub-region is removed.
[0044] In some embodiments, the centroid acquisition module includes:
[0045] A conversion unit is used to convert the sub-region into a binary image;
[0046] The first computational unit is used to find all connected regions in the binary image using a connected component labeling algorithm;
[0047] The second calculation unit is used to obtain the centroid of each connected region by applying the geometric distance calculation algorithm.
[0048] In some embodiments, the merging module is specifically used to merge two adjacent sub-regions if the distance between the centroids of two adjacent sub-regions in the column direction is less than M pixels, where M is a positive integer.
[0049] Embodiments of the present invention also provide an image segmentation apparatus, including a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and the program or instructions, when executed by the processor, implement the steps of the image segmentation method as described above.
[0050] Embodiments of the present invention also provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the image segmentation method described above.
[0051] The embodiments of the present invention have the following beneficial effects:
[0052] In the above scheme, feature extraction is performed on the image to be processed, and the image to be processed is divided into multiple sub-regions based on the feature extraction results. Then, the segmented sub-regions are dilated, and multiple sub-regions are merged based on the centroid of the dilated sub-regions. This can merge unevenly segmented regions into more accurate regions, so that images that might have been discarded due to uneven segmentation can meet the requirements of the training set samples, thereby reducing the amount of training samples used. This can improve the accuracy and stability of region segmentation when there are few image samples. Attached Figure Description
[0053] Figure 1 This is a flowchart illustrating the image segmentation method according to an embodiment of the present invention;
[0054] Figure 2 This is a schematic diagram of the image to be processed according to an embodiment of the present invention;
[0055] Figure 3 This is a schematic diagram of noise points generated after segmentation in an embodiment of the present invention;
[0056] Figure 4 This is a schematic diagram of the principle of using opening operation to filter noise points;
[0057] Figure 5 This is a schematic diagram of the region segmentation result after filtering noise points according to an embodiment of the present invention;
[0058] Figure 6 This is a structural block diagram of the image segmentation device according to an embodiment of the present invention. Detailed Implementation
[0059] To make the technical problems, technical solutions and advantages of the embodiments of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.
[0060] In related technologies, the accuracy of region segmentation is poor for images with few samples. Due to the limited number of samples in screen images, and the instability caused by lighting conditions such as reflections, dirt, and scratches, the expected ROI (region of interest) segmentation effect cannot be achieved when performing region segmentation on screen images.
[0061] This invention provides an image segmentation method and apparatus that can improve the accuracy of region segmentation.
[0062] Embodiments of the present invention provide an image segmentation method, such as... Figure 1 As shown, it includes:
[0063] Step 101: Obtain the image to be processed;
[0064] Step 102: Extract features from the image to be processed, and divide the image into multiple sub-regions based on the feature extraction results;
[0065] In this embodiment, the image to be processed can be a screen image, such as... Figure 2 As shown, the screen image is the image obtained by photographing the display screen of the display device. In the industrial production process, a camera device can be used to take pictures of the display screen to obtain industrial images. By segmenting the industrial images into regions, they can be divided into different regions for inspection, which can improve the defect recognition rate when performing defect identification later.
[0066] The image to be processed can be a color image or a black and white image.
[0067] In some embodiments, if the image to be processed is a color image, the method further includes converting the image to be processed into a grayscale image before feature extraction. Converting the image to be processed from a color image to a grayscale image can save image computation time and reduce the computational load of image processing.
[0068] In this embodiment, a convolutional neural network can be used to extract features from the image to be processed to obtain a three-dimensional feature map; based on the three-dimensional feature map and its corresponding label, cross-entropy loss is calculated for each pixel in the image to be processed to obtain the feature value of each pixel; the feature value of each pixel is input into a classification network to obtain a classification result, the classification result indicating whether each pixel belongs to a foreground region or a background region; based on the classification result, the image to be processed is divided into multiple sub-regions, each of which belongs to a foreground region or a background region.
[0069] Specifically, features can be extracted from the image to be processed using a deep convolutional neural network. Since screen images have a large resolution and require high accuracy, ResNet34 or ResNet18 networks can be used to extract features from the screen image.
[0070] Taking the ResNet18 network as an example, during feature extraction, the ResNet18 network is first used to extract features from the image to be processed, resulting in a 512x32x32 3D feature map. Then, the ASPP network layer is used to further convolve the 512x32x32 3D feature map from 5 branches to extract features, resulting in 5 256x32x32 feature maps. These 5 256x32x32 feature maps are then concatenated to obtain a 1280x32x32 feature map, which is then convolved twice to obtain a 20x32x32 feature map. The 20x32x32 feature map is then upsampled to obtain a 20x256x256 feature map. The cross-entropy loss is calculated for each pixel of the image to be processed using the loss function. Considering the class imbalance problem, corresponding weights can be added to the loss for each class. The calculation result is used as the feature value of each pixel of the image to be processed.
[0071] For each pixel, its feature value is input into a sigmoid classification network to obtain a score. This score determines whether the pixel belongs to the foreground or background region. For example, using 0.5 as the dividing line, a pixel with a score greater than 0.5 is considered to belong to the foreground region, while a pixel with a score less than 0.5 is considered to belong to the background region. This process divides multiple pixels in the image into foreground and background regions, separating them and obtaining preliminary region segmentation results.
[0072] The formula for sigmoid can be shown below, where x is the feature value of the pixel:
[0073]
[0074] The feature value of a pixel is input into the sigmoid formula, and a score is output. The score is between [0,1]. If the score is greater than the preset probability threshold, the pixel is considered to belong to a certain category; otherwise, it does not belong to a certain category.
[0075] Step 103: Dilate the segmented sub-regions;
[0076] like Figure 3As shown, the area within the rectangle represents noise points generated during segmentation. These noise points can lead to multiple split regions after segmentation, resulting in inaccurate image segmentation. Dilation and erosion can be used to filter out noise points in the sub-regions, thereby connecting the split regions as much as possible. Specifically, an opening operation can be used to filter noise points in the sub-regions, including: comparing the center point of a preset convolution template with each pixel in the sub-region; if all points on the preset convolution template are within the range of the sub-region, then the pixel in the sub-region is retained; otherwise, the pixel in the sub-region is removed.
[0077] In opening operations, such as Figure 4 As shown, A is the original image, and B is the convolution template. Image A is convolved using convolution template B. This involves performing a convolution calculation between image A and template B to obtain the minimum pixel value within the area covered by template B. This minimum value is then used to replace the pixel value of the reference point in the original image A, thus processing the original image A into... Figure 4 The image on the right is AB. The principle is to compare the center point of the convolution template B with the pixels of the original image A one by one. If all the points on the convolution template B are within the range of the original image A, then that point in the original image A is retained; otherwise, that point in the original image A is removed, resulting in image AB. It can be seen that image AB is still within the range of the original image A, but contains fewer points than the original image A.
[0078] like Figure 5 As shown, after filtering out noise points, it can be Figure 3 The foreground area in the image shown is roughly divided into two parts, C and D.
[0079] Step 104: Obtain the centroid of each of the sub-regions after dilation;
[0080] The image after step 102 is converted into a binary image. A binary image is an image with only two gray levels; that is, the gray value of any pixel in the image is either 0 or 255, representing black and white, respectively. All connected regions are found in the binary image using a connected component labeling algorithm. The gray value of pixels in a connected region can all be 255. The centroid is obtained by applying a computational geometric distance algorithm to each connected region. Connected regions and centroids can be drawn using different colors, and the processed image is output.
[0081] The center of mass is the point through which a region achieves a state of mass equilibrium. For regular geometric objects, the center of mass is the center of the region. The mathematical formula for the center of mass can be expressed as follows:
[0082]
[0083] Where a1 and b1 are negative infinity, a2 and b2 are positive infinity, and p and q are moments of order p+q. M pq lower order M 00 M 01 M 10 It can be used to calculate the centroid.
[0084] Step 105: Merge the multiple sub-regions according to the position of the centroid to obtain the segmented result.
[0085] If the centroids of two adjacent sub-regions are close in the column direction (less than M pixels), they can be considered the same region and merged. M is a positive integer. The value of M can be set as needed; for example, M can be 50. This means that when the centroids of two adjacent sub-regions are located in the same column, or their column coordinates differ by less than 50 pixels in the column direction, they are considered to be merged. The sub-regions can be merged by finding their convex hulls. Figure 5 Regions C and D in the image are merged to output the final image segmentation result.
[0086] In this embodiment, feature extraction is performed on the image to be processed. Based on the feature extraction results, the image to be processed is divided into multiple sub-regions. Then, the segmented sub-regions are dilated. Based on the centroid of the dilated sub-regions, the multiple sub-regions are merged. This can merge unevenly segmented regions into more accurate regions, so that images that might have been discarded due to uneven segmentation can meet the requirements of the training set samples. This reduces the amount of training samples used and can improve the accuracy and stability of region segmentation when there are few image samples.
[0087] For screen images, this embodiment can extract the region of interest within the image, discarding interference from lighting and dirt in other areas, thereby improving the stability and robustness of target recognition and reducing the time consumption of subsequent detection algorithms.
[0088] In the field of industrial imaging, the sample size of industrial images is relatively small due to the limitations of the acquisition equipment. Existing technologies often require a large number of samples to achieve satisfactory segmentation results. This embodiment can obtain stable and accurate region segmentation results with fewer samples and can be applied to the field of industrial imaging.
[0089] Embodiments of the present invention also provide an image segmentation apparatus, such as... Figure 6 As shown, it includes:
[0090] Acquisition module 21 is used to acquire the image to be processed;
[0091] The segmentation module 22 is used to extract features from the image to be processed and to segment the image to be processed into multiple sub-regions based on the feature extraction results.
[0092] In this embodiment, the image to be processed can be a screen image, such as... Figure 2 As shown, the screen image is the image obtained by photographing the display screen of the display device. In the industrial production process, a camera device can be used to take pictures of the display screen to obtain industrial images. By segmenting the industrial images into regions, they can be divided into different regions for inspection, which can improve the defect recognition rate when performing defect identification later.
[0093] The image to be processed can be a color image or a black and white image.
[0094] In some embodiments, if the image to be processed is a color image, the apparatus further includes:
[0095] The conversion module is used to convert the image to be processed into a grayscale image. Converting the image to grayscale from a color image can save image processing time and reduce the computational load of image processing.
[0096] Specifically, features can be extracted from the image to be processed using a deep convolutional neural network. Since screen images have a large resolution and require high accuracy, ResNet34 or ResNet18 networks can be used to extract features from the screen image.
[0097] Taking the ResNet18 network as an example, during feature extraction, the ResNet18 network is first used to extract features from the image to be processed, resulting in a 512x32x32 3D feature map. Then, the ASPP network layer is used to further convolve the 512x32x32 3D feature map from 5 branches to extract features, resulting in 5 256x32x32 feature maps. These 5 256x32x32 feature maps are then concatenated to obtain a 1280x32x32 feature map, which is then convolved twice to obtain a 20x32x32 feature map. The 20x32x32 feature map is then upsampled to obtain a 20x256x256 feature map. The cross-entropy loss is calculated for each pixel of the image to be processed using the loss function. Considering the class imbalance problem, corresponding weights can be added to the loss for each class. The calculation result is used as the feature value of each pixel of the image to be processed.
[0098] For each pixel, its feature value is input into a sigmoid classification network to obtain a score. This score determines whether the pixel belongs to the foreground or background region. For example, using 0.5 as the dividing line, a pixel with a score greater than 0.5 is considered to belong to the foreground region, while a pixel with a score less than 0.5 is considered to belong to the background region. This process divides multiple pixels in the image into foreground and background regions, separating them and obtaining preliminary region segmentation results.
[0099] The formula for sigmoid can be shown below, where x is the feature value of the pixel:
[0100]
[0101] The feature value of a pixel is input into the sigmoid formula, and a score is output. The score is between [0,1]. If the score is greater than the preset probability threshold, the pixel is considered to belong to a certain category; otherwise, it does not belong to a certain category.
[0102] The expansion module 23 is used to expand the segmented sub-regions;
[0103] like Figure 3 As shown, the area within the rectangle represents noise points generated during segmentation. These noise points can lead to multiple split regions after segmentation, resulting in inaccurate image segmentation. Dilation and erosion can be used to filter out noise points in these sub-regions, thereby connecting the split regions as much as possible. Specifically, opening operations can be used to filter out noise points in these sub-regions.
[0104] In some embodiments, the apparatus further includes:
[0105] The filtering module is used to filter noise points in the sub-region through opening operations. Specifically, the filtering module is used to compare the center point of the preset convolution template with the pixels of the sub-region one by one. If all the points on the preset convolution template are within the range of the sub-region, the pixel of the sub-region is retained; otherwise, the pixel of the sub-region is removed.
[0106] In opening operations, such as Figure 4 As shown, A is the original image, and B is the convolution template. Image A is convolved using convolution template B. This involves performing a convolution calculation between image A and template B to obtain the minimum pixel value within the area covered by template B. This minimum value is then used to replace the pixel value of the reference point in the original image A, thus processing the original image A into... Figure 4The image on the right is AB. The principle is to compare the center point of the convolution template B with the pixels of the original image A one by one. If all the points on the convolution template B are within the range of the original image A, then that point in the original image A is retained; otherwise, that point in the original image A is removed, resulting in image AB. It can be seen that image AB is still within the range of the original image A, but contains fewer points than the original image A.
[0107] like Figure 5 As shown, after filtering out noise points, it can be Figure 3 The foreground area in the image shown is roughly divided into two parts, C and D.
[0108] Centroid acquisition module 24 is used to acquire the centroid of each sub-region after expansion;
[0109] The image after step 102 is converted into a binary image. A binary image is an image with only two gray levels; that is, the gray value of any pixel in the image is either 0 or 255, representing black and white, respectively. All connected regions are found in the binary image using a connected component labeling algorithm. The gray value of pixels in a connected region can all be 255. The centroid is obtained by applying a computational geometric distance algorithm to each connected region. Connected regions and centroids can be drawn using different colors, and the processed image is output.
[0110] The center of mass is the point through which a region achieves a state of mass equilibrium. For regular geometric objects, the center of mass is the center of the region. The mathematical formula for the center of mass can be expressed as follows:
[0111]
[0112] Where a1 and b1 are negative infinity, a2 and b2 are positive infinity, and p and q are moments of order p+q. M pq lower order M 00 M 01 M 10 It can be used to calculate the centroid.
[0113] The merging module 25 is used to merge the multiple sub-regions according to the position of the centroid to obtain the segmented result.
[0114] If the centroids of two adjacent sub-regions are close in the column direction (less than M pixels), they can be considered the same region and merged. M is a positive integer. The value of M can be set as needed; for example, M can be 50. This means that when the centroids of two adjacent sub-regions are located in the same column, or their column coordinates differ by less than 50 pixels in the column direction, they are considered to be merged. The sub-regions can be merged by finding their convex hulls. Figure 5 Regions C and D in the image are merged to output the final image segmentation result.
[0115] In this embodiment, feature extraction is performed on the image to be processed. Based on the feature extraction results, the image to be processed is divided into multiple sub-regions. Then, the segmented sub-regions are dilated. Based on the centroid of the dilated sub-regions, the multiple sub-regions are merged. This can merge unevenly segmented regions into more accurate regions, so that images that might have been discarded due to uneven segmentation can meet the requirements of the training set samples. This reduces the amount of training samples used and can improve the accuracy and stability of region segmentation when there are few image samples.
[0116] In some embodiments, the segmentation module 21 includes:
[0117] The feature extraction unit is used to extract features from the image to be processed through a convolutional neural network to obtain a three-dimensional feature map;
[0118] The feature value calculation unit is used to perform cross-entropy loss calculation on each pixel of the image to be processed based on the three-dimensional feature map and its corresponding label to obtain the feature value of each pixel.
[0119] A classification unit is used to input the feature value of each pixel into a classification network to obtain a classification result, which indicates whether each pixel belongs to the foreground region or the background region.
[0120] The segmentation unit is used to segment the image to be processed into multiple sub-regions based on the classification result, each of the sub-regions belonging to either the foreground region or the background region.
[0121] In some embodiments, the centroid acquisition module 23 includes:
[0122] A conversion unit is used to convert the sub-region into a binary image;
[0123] The first computational unit is used to find all connected regions in the binary image using a connected component labeling algorithm;
[0124] The second calculation unit is used to obtain the centroid of each connected region by applying the geometric distance calculation algorithm.
[0125] Embodiments of the present invention also provide an image segmentation apparatus, including a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and the program or instructions, when executed by the processor, implement the steps of the image segmentation method as described above.
[0126] Embodiments of the present invention also provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the image segmentation method described above.
[0127] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage, or any other non-transferable medium that can be used to store information accessible to the computer-readable terminal device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0128] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0129] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0130] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
Claims
1. An image segmentation method, characterized in that, include: Obtain the image to be processed; The image to be processed is a screen image; The image to be processed is subjected to feature extraction, and the image to be processed is segmented into multiple sub-regions based on the feature extraction results; including: extracting features from the image to be processed using a deep convolutional neural network to obtain a three-dimensional feature map; Based on the three-dimensional feature map and its corresponding label, cross-entropy loss is calculated for each pixel of the image to be processed to obtain the feature value of each pixel. The feature value of each pixel is input into the classification network to obtain the classification result, which indicates whether each pixel belongs to the foreground region or the background region. Based on the classification result, the image to be processed is divided into multiple sub-regions, each of which belongs to either a foreground region or a background region. The segmented sub-regions are then dilated. Obtain the centroid of each sub-region after dilation; The multiple sub-regions are merged based on the position of the centroid to obtain the segmented result.
2. The image segmentation method according to claim 1, characterized in that, If the image to be processed is a color image, before performing feature extraction on the image to be processed, the method further includes: The image to be processed is converted into a grayscale image.
3. The image segmentation method according to claim 1, characterized in that, The dilation process performed on the segmented sub-regions includes: Noise points in the sub-region are filtered out by opening operations.
4. The image segmentation method according to claim 3, characterized in that, The step of filtering noise points in the sub-region through opening operation includes: The center point of the preset convolution template is compared with each pixel in the sub-region. If all points on the preset convolution template are within the range of the sub-region, the pixel in the sub-region is retained; otherwise, the pixel in the sub-region is removed.
5. The image segmentation method according to claim 1, characterized in that, The process of obtaining the centroid of each of the expanded sub-regions includes: Convert the sub-region into a binary image; All connected regions in the binary image are found using a connected component labeling algorithm; The centroid is obtained by applying the geometric distance calculation algorithm to each of the connected regions.
6. The image segmentation method according to claim 1, characterized in that, The step of merging the multiple sub-regions based on the position of the centroid includes: If the distance between the centroids of two adjacent sub-regions in the column direction is less than M pixels, the two sub-regions are merged, where M is a positive integer.
7. An image segmentation apparatus, characterized in that, include: The acquisition module is used to acquire the image to be processed; the image to be processed is a screen image. The segmentation module is used to extract features from the image to be processed and segment the image to be processed into multiple sub-regions based on the feature extraction results. The segmentation module includes: The feature extraction unit is used to extract features from the image to be processed through a convolutional neural network to obtain a three-dimensional feature map; The feature value calculation unit is used to perform cross-entropy loss calculation on each pixel of the image to be processed based on the three-dimensional feature map and its corresponding label to obtain the feature value of each pixel. A classification unit is used to input the feature value of each pixel into a classification network to obtain a classification result, which indicates whether each pixel belongs to the foreground region or the background region. A segmentation unit is used to segment the image to be processed into multiple sub-regions according to the classification result, each of the sub-regions belonging to a foreground region or a background region; An expansion module is used to expand the segmented sub-regions; The centroid acquisition module is used to acquire the centroid of each of the sub-regions after expansion. The merging module is used to merge the multiple sub-regions according to the position of the centroid to obtain the segmented result.
8. The image segmentation apparatus according to claim 7, characterized in that, If the image to be processed is a color image, the device further includes: A conversion module is used to convert the image to be processed into a grayscale image.
9. The image segmentation apparatus according to claim 7, characterized in that, The device further includes: A filtering module is used to filter noise points in the sub-region through an opening operation.
10. The image segmentation apparatus according to claim 9, characterized in that, The filtering module is specifically used to compare the center point of the preset convolution template with the pixels of the sub-region one by one. If all the points on the preset convolution template are within the range of the sub-region, the pixel in the sub-region is retained; otherwise, the pixel in the sub-region is removed.
11. The image segmentation apparatus according to claim 7, characterized in that, The centroid acquisition module includes: A conversion unit is used to convert the sub-region into a binary image; The first computational unit is used to find all connected regions in the binary image using a connected component labeling algorithm; The second calculation unit is used to obtain the centroid of each connected region by applying the geometric distance calculation algorithm.
12. The image segmentation apparatus according to claim 7, characterized in that, The merging module is specifically used to merge two adjacent sub-regions if the distance between their centroids in the column direction is less than M pixels, where M is a positive integer.
13. An image segmentation apparatus, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the image segmentation method as described in any one of claims 1 to 6.
14. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the image segmentation method as described in any one of claims 1 to 6.