Image segmentation method, device and electronic device

Abstract

The present application provides an image segmentation method, device and electronic device, and relates to the technical field of image segmentation. The method comprises: first acquisition, modeling, image segmentation, first judgment, addition and iteration steps; the device comprises: a first acquisition module, a modeling module, an image segmentation module, a first judgment module, an addition module and an iteration module. The present application can increase the accuracy of image segmentation.

Description

Technical Field

[0001] The present application relates to the technical field of image segmentation, and in particular to an image segmentation method, device and electronic device. Background Art

[0002] In today's digital image processing field, image segmentation is a crucial technology that aims to divide an image into multiple meaningful regions or objects for subsequent image analysis, understanding, and recognition. With the rapid development of computer vision and deep learning technology, image segmentation technology has been widely used in medical diagnosis, remote sensing image processing, autonomous driving, security monitoring and other fields.

[0003] The Chinese invention patent with the publication date of May 12, 2023 and the application publication number of CN116109652A is named: Image segmentation method, system, and device based on improved region growing algorithm, which uses the target detection algorithm to identify the bolt area of ​​the tower, and then determines the threshold range of the region growth, and then inputs the seed point into the region growing algorithm to obtain the region growth map, and then performs edge detection on the tower image and the region growth map to obtain the image segmentation result. However, the image content in reality is often complex and changeable, containing objects of different shapes, sizes, and textures, and the objects in the images acquired at different times will be affected by interference factors such as lighting and shadows, which increases the difficulty of segmentation and the accuracy of image segmentation is low. Summary of the invention

[0004] In order to improve the accuracy of image segmentation, the present application provides an image segmentation method, device and electronic device.

[0005] In the first aspect, the present application provides an image segmentation method, which adopts the following technical solution:

[0006] The image segmentation method comprises the following steps:

[0007] First acquisition: acquiring the image to be segmented;

[0008] Modeling: establishing an image segmentation model, wherein the image segmentation model includes: a CNN sub-model and a region growing sub-model;

[0009] Image segmentation: including first segmentation, setting seed points and second segmentation;

[0010] First segmentation: Use the CNN sub-model to segment the image to be segmented and obtain n segmentation blocks;

[0011] Set the seed point: take an edge pixel of the i-th segment block as the seed point;

[0012] Second segmentation: The region growth sub-model processes the j-th pixel point adjacent to the seed point using a preset growth criterion to obtain a processing result;

[0013] First judgment: based on the processing result, judge whether the j-th pixel point can be merged into the i-th segmentation block. If so, perform the increasing step; if not, perform the iterative step;

[0014] Add: add the jth pixel to the i-th segmentation block;

[0015] Iteration: includes the first traversal and the second traversal;

[0016] First traversal: take the j+1th pixel point as the new jth pixel point and perform the second segmentation step until the first stop condition is met;

[0017] Second traversal: take the i+1th segmentation block as the new ith segmentation block and perform the second segmentation step until the second stop condition is met.

[0018] By adopting the above technical solution, the image segmentation model of the present application includes: a CNN sub-model and a region growing sub-model. The CNN sub-model is based on a deep learning algorithm and can learn the feature representation in the image. It is suitable for different image segmentation tasks. Even when segmenting complex images, different regions in the image can be quickly identified, which improves the generalization ability of the image segmentation model. The present application adopts a two-step segmentation strategy (first segmentation and second segmentation). The first segmentation uses the CNN sub-model for preliminary segmentation, which can quickly obtain a rough segmentation area and provide a basis for subsequent regional growth. The second segmentation refines the segmentation result through the region growing sub-model, which helps to deal with the ambiguity and irregularity of the segmentation boundary. Before the second segmentation step, an edge pixel point of the i-th segmentation block is selected as the seed point, which makes it easier to distinguish different regions and increase the accuracy of the segmentation of the current segmentation block. After that, the segmentation of the entire image is completed by setting the iterative steps.

[0019] Optionally, after performing the first acquisition step and before performing the modeling step, the method further includes:

[0020] Third segmentation: along the length direction or width direction of the image to be segmented, the image to be segmented is equally divided into m sub-images;

[0021] Set pixel edge: locate the edge between the xth sub-image and the x+1th sub-image, record the edge of the xth sub-image as the first pixel edge, and record the edge of the x+1th sub-image as the second pixel edge;

[0022] First calculation: calculate the difference between the grayscale value of the yth pixel point and the grayscale value of the y+1th pixel point in the first pixel edge, and record it as the first data;

[0023] Second judgment: judging whether the first data is greater than a preset gray value threshold, if so, executing the marking step; if not, executing the second calculation step;

[0024] Second calculation: calculate the difference between the grayscale value of the yth pixel point and the grayscale value of the y+1th pixel point in the second pixel edge, and record it as the second data;

[0025] Third judgment: judging whether the second data is greater than a preset gray value threshold, if so, executing the marking step; if not, executing the third traversal step;

[0026] Mark: mark the yth pixel or the y+1th pixel as the first pixel;

[0027] Third traversal: take the x+1th sub-image as the new xth sub-image, and execute the step of setting the pixel edge until the third stop condition is met.

[0028] By adopting the above technical solution, the present application divides the image to be segmented into m sub-images, which helps to simplify subsequent processing, especially when processing large images, the image is segmented into smaller parts (sub-images), and then by calculating the gray value difference of the edge pixels of adjacent sub-images, the edge or feature change area in the sub-image can be identified. The change area is often the key point of image segmentation, which helps to distinguish different image parts. The present application converts the search for seed points in the entire image into the search for seed points in the pixel edge of each sub-image, which reduces computing resources and improves computing efficiency. By setting a gray value threshold, the present application can screen out pixels with significant gray value changes. The pixels with significant changes are likely to be edge points or feature points in a certain area of ​​the image. By marking the pixels with significant gray value changes, it is prompted that in the image to be segmented, the pixel is likely to be the edge point of the segmentation block, which helps to quickly locate the edge of the segmentation block that needs to be segmented and improve the speed of image segmentation.

[0029] Optionally, in the step of setting the seed point, the first pixel point is used to replace the edge pixel point.

[0030] By adopting the above technical solution, the present application uses the first pixel point (i.e., the previously marked edge or feature point) instead of the original edge pixel point in the step of setting the seed point, which helps to more accurately guide the region growing process, reduce the possibility of mis-segmentation, and improve segmentation efficiency. Since the first pixel point more accurately locates the edge or feature point, the region growing will converge to the correct segmentation result more quickly, and using the first pixel point as the seed point can optimize the region growing process. The region growing algorithm usually relies on the selection of seed points and growth criteria to gradually expand the segmented area. Using more accurate seed points can guide the algorithm to more quickly identify different image parts and reduce unnecessary calculations. And because the selection of seed points is more accurate, the final image segmentation result will be more refined and accurate.

[0031] Optionally, the image segmentation step further includes:

[0032] First acquisition: obtain the pixel points adjacent to the seed point in the i-th segmentation block, recorded as the first pixel point set, and obtain the pixel points adjacent to the seed point in the i+1-th segmentation block, recorded as the second pixel point set;

[0033] Third calculation: calculating the similarity between the seed point and all the pixels in the first pixel point set, and calculating the sum of all similarities as the first similarity;

[0034] Fourth calculation: calculating the similarity between the seed point and all the pixels in the second pixel point set, and calculating the sum of all the similarities as the second similarity;

[0035] Fourth judgment: judging whether the first similarity is greater than the second similarity, if so, executing the first processing step; if not, executing the second processing step;

[0036] First processing: the region growth sub-model processes the pixels in the second pixel point set using a preset growth criterion;

[0037] Second processing: the region growing sub-model processes the pixels in the first pixel set using a preset growth criterion.

[0038] By adopting the above technical solution, the present application can quantify the degree of association between the seed point and the surrounding pixels by calculating the total similarity (first similarity and second similarity) between the seed point and the pixels in the adjacent pixel set. The present application determines the direction of region growth based on the comparison results of the two similarities: if the first similarity is greater than the second similarity, the second pixel set is processed first; otherwise, the first pixel set is processed, which helps to ensure that the region growth process is carried out in the opposite direction of the most similar pixel point. In other words, the seed point is located at the edge of the segmentation block. If the seed point is more similar to the characteristics of the current pixel set, it proves that the seed point belongs to the segmentation block. If the segmentation block is to be segmented in the image to be segmented, the region growth should be performed in another pixel set to obtain the edge pixel points of the segmentation block. This step reduces the calculation content of the region growth algorithm and improves the calculation efficiency.

[0039] Optionally, after executing the fourth calculation step and before executing the fourth judgment step, the method further includes:

[0040] Sorting: in the first pixel point set and / or the second pixel point set, all the pixels are sorted in descending order according to the similarity to obtain a first pixel point sequence and / or a second pixel point sequence;

[0041] Deletion: Delete pixels whose similarity is less than a preset similarity threshold to obtain a new first pixel set and / or second pixel set.

[0042] By adopting the above technical solution, by sorting the pixels in the first pixel set and the second pixel set according to the similarity, the pixels most similar to the seed point are prioritized during the processing, which helps to reduce unnecessary calculations, because once the region growing process starts, the pixels that are highly similar to the seed point are more likely to belong to the same region. By deleting pixels whose similarity is lower than a preset threshold, the amount of calculation for subsequent processing can be further reduced. Since the pixels with low similarity in the calculated similarity are likely not to belong to the same region as the seed point, removing them from the processing list can improve the overall efficiency. By prioritizing the pixels with high similarity and deleting the pixels with low similarity, the region growing process is carried out more accurately in the correct direction, which helps to reduce the situation where pixels that do not belong to the same region are mistakenly classified into the same segmented region.

[0043] Optionally, in the first processing and / or second processing steps, the region growing sub-model is based on the first pixel point sequence and / or the second pixel point sequence, and processes the pixels in the first pixel point set and / or the second pixel point set using a preset growth criterion.

[0044] By adopting the above technical solution, the pixels in the deleted first pixel point set and the second pixel point set are processed according to the order of the pixels in the first pixel point sequence and / or the second pixel point sequence through the region growing sub-model. During the processing, once a pixel in the first pixel point sequence and / or the second pixel point sequence no longer meets the growth criterion, there is no need to process the pixels after the pixel, thereby reducing the amount of calculation and improving the calculation speed and the speed of image segmentation.

[0045] Optionally, the image segmentation model further includes a BI-LSTM sub-model, and after performing the modeling step and before performing the image segmentation step, further includes:

[0046] Second collection: collecting historical images and segmentation data of historical images to obtain k segmentation blocks after the historical images are segmented;

[0047] First determination: determine the central pixel point of the pth segmentation block in the historical image, recorded as the third pixel point;

[0048] Second determination: obtaining q segmentation blocks adjacent to the pth segmentation block in the historical image, and determining the central pixel point of the q segmentation blocks, which is recorded as the fourth pixel point;

[0049] Third acquisition: acquiring a pixel point between the third pixel point and the fourth pixel point, recorded as the fifth pixel point;

[0050] Fourth acquisition: obtaining the grayscale values ​​of all fifth pixels;

[0051] Model training: The grayscale value of the fifth pixel is used to train the BI-LSTM sub-model to obtain the trained BI-LSTM sub-model.

[0052] By adopting the above technical solution, by collecting the grayscale values ​​of the pixel points (fifth pixel point) between the third pixel point (the central pixel point of the pth segmentation block in the historical image) and the fourth pixel point (the central pixel point of the q segments adjacent to the pth segmentation block), and using these grayscale values ​​to train the BI-LSTM sub-model, the BI-LSTM sub-model can learn the spatial relationship and grayscale change pattern between the central pixel points of the pixel blocks in the image.

[0053] Optionally, after performing the first segmentation step and before performing the seed point setting step, the method further includes:

[0054] Fifth acquisition: acquiring the central pixel points of the i-th segmentation block and the i+1-th segmentation block;

[0055] Prediction: Input the central pixels of the i-th segmentation block and the i+1-th segmentation block into the trained BI-LSTM sub-model, and predict the grayscale value of the pixel between the two central pixels, which is recorded as the first grayscale value;

[0056] Fifth calculation: calculating the difference between adjacent first grayscale values, recorded as the first difference;

[0057] Fifth judgment: judging whether the first difference is greater than a preset difference threshold, if so, executing the replacing step; if not, executing the setting seed point step;

[0058] Replacement: Get the two pixels corresponding to the first difference and use one of the pixels as the new seed point.

[0059] By adopting the above technical solution, this step utilizes the powerful modeling ability of BI-LSTM on sequence data. The grayscale value of the pixel between the central pixels of two adjacent segmented blocks is predicted by the trained BI-LSTM sub-model, which can capture the potential relationship between the pixels and predict the grayscale value more accurately. For grayscale images, the pixels inside the segmented blocks generally have grayscale similarity, and the grayscale discontinuity is generally present at the boundary of the segmented blocks. Therefore, the present application calculates the difference in the predicted grayscale values ​​of adjacent pixels and compares it with the preset difference threshold. If the first difference is greater than the preset difference threshold, it indicates that the grayscale is discontinuous here and there are edge points. One of the pixels is used as a seed point and the second segmentation step is continued; otherwise, a seed point is set and the second segmentation step is performed. This step helps to quickly locate the seed point, improve computational efficiency, and optimize the segmentation boundary.

[0060] In a second aspect, the present invention provides an image segmentation device, which adopts the following technical solution:

[0061] An image segmentation device, comprising:

[0062] A first acquisition module, used for acquiring an image to be segmented;

[0063] A modeling module, used to establish an image segmentation model, wherein the image segmentation model includes: a CNN sub-model and a region growing sub-model;

[0064] An image segmentation module, which is in communication with the first acquisition module and the modeling module, comprises a first segmentation unit, a seed point setting unit and a second segmentation unit;

[0065] The first segmentation unit is used to segment the image to be segmented by using a CNN sub-model to obtain n segmentation blocks;

[0066] A seed point unit is set to use an edge pixel point of the i-th segmentation block as a seed point;

[0067] The second segmentation unit is used for the region growth sub-model to process the j-th pixel point adjacent to the seed point using a preset growth criterion to obtain a processing result;

[0068] A first judgment module is connected to the image segmentation module for judging whether the j-th pixel point can be merged into the i-th segmentation block based on the processing result;

[0069] An adding module is used to add the j-th pixel point to the i-th segmentation block;

[0070] An iteration module, which is in communication connection with the image segmentation module, comprises a first traversal unit and a second traversal unit;

[0071] The first traversal unit is used to take the j+1th pixel point as the new jth pixel point;

[0072] The second traversal unit is used to take the i+1th segmentation block as the new ith segmentation block.

[0073] By adopting the above technical solution, the device realizes the preliminary segmentation and further refinement of the image by combining the CNN sub-model and the region growth sub-model. The first segmentation unit uses the CNN sub-model to quickly identify different regions (segmentation blocks) in the image, while the second segmentation unit refines the segmentation results through the region growth sub-model. The first judgment module and the increase module ensure the accuracy and continuity of the segmentation results, while the iteration module realizes the iterative segmentation process by traversing all pixels and segmentation blocks. The present application combines the deep learning method and the region growth strategy to improve the accuracy and efficiency of image segmentation, while reducing the dependence on manual annotation and improving the generalization of the image segmentation model.

[0074] In a third aspect, the present invention provides an electronic device, which adopts the following technical solution:

[0075] An electronic device comprises: a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and the program or instruction implements the method when executed by the processor.

[0076] In summary, the present application includes at least one of the following beneficial technical effects:

[0077] 1. The image segmentation model of the present application includes: a CNN sub-model and a region growing sub-model. The CNN sub-model is based on a deep learning algorithm and can learn feature representations in an image. It is suitable for different image segmentation tasks. Even when segmenting complex images, different regions in the image can be quickly identified, which improves the generalization ability of the image segmentation model. The present application adopts a two-step segmentation strategy (first segmentation and second segmentation). The first segmentation uses the CNN sub-model for preliminary segmentation, which can quickly obtain a rough segmentation area and provide a basis for subsequent region growth. The second segmentation refines the segmentation result through the region growing sub-model, which helps to deal with the ambiguity and irregularity of the segmentation boundary. Before the second segmentation step, an edge pixel point of the i-th segmentation block is selected as the seed point, which makes it easier to distinguish different regions and increases the accuracy of the segmentation of the current segmentation block. After that, the segmentation of the entire image is completed by setting iterative steps.

[0078] 2. The present application can quantify the degree of association between the seed point and the surrounding pixels by calculating the total similarity (first similarity and second similarity) between the seed point and the pixels in the adjacent pixel set. The present application determines the direction of region growth based on the comparison results of the two similarities: if the first similarity is greater than the second similarity, the second pixel set is processed first; otherwise, the first pixel set is processed, which helps to ensure that the region growth process is carried out in the opposite direction of the most similar pixel point. In other words, if the seed point is located at the edge of the segmentation block, if the seed point is more similar to the characteristics of the current pixel set, it proves that the seed point belongs to the segmentation block. If the segmentation block is to be segmented in the image to be segmented, region growth should be performed in another pixel set to obtain the edge pixel points of the segmentation block. This step reduces the calculation content of the region growth algorithm and improves the calculation efficiency.

[0079] 3. This step utilizes the powerful modeling ability of BI-LSTM on sequence data. The trained BI-LSTM sub-model predicts the grayscale value of the pixel points between the central pixels of two adjacent segmented blocks, which can capture the potential relationship between the pixels and predict the grayscale value more accurately. For grayscale images, the pixels inside the segmented blocks generally have grayscale similarity, and the grayscale discontinuity is generally present at the boundary of the segmented blocks. Therefore, this application calculates the difference in the predicted grayscale values ​​of adjacent pixels and compares it with the preset difference threshold. If the first difference is greater than the preset difference threshold, it indicates that the grayscale is discontinuous here and there are edge points. One of the pixels is used as a seed point and the second segmentation step is continued; otherwise, a seed point is set and the second segmentation step is performed. This step helps to quickly locate the seed point, improve computational efficiency, and optimize the segmentation boundary. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 is a flow chart of Example 1 of the present application;

[0081] Figure 2 This is a flow chart from S1 first acquisition to S2 modeling in Example 2 of the present application;

[0082] Figure 3 is a schematic diagram of equally dividing the image to be segmented in Example 2 of the present application;

[0083] Figure 4 is a schematic diagram of setting pixel edges in S72 of Example 2 of the present application;

[0084] Figure 5 is a flow chart of Example 3 of the present application;

[0085] Figure 6 This is a flow chart from S2 modeling to S3 image segmentation in Example 4 of the present application;

[0086] Figure 7 This is a flowchart of S31 first segmentation to S32 setting seed points in Example 4 of the present application.

[0087] Figure numerals: 1 - first pixel side; 2 - second pixel side. DETAILED DESCRIPTION

[0088] The following combination Figures 1 to 7 This application is described in further detail.

[0089] Embodiment 1: This embodiment discloses an image segmentation method, referring to Figure 1 The method includes: S1 first acquisition, S2 modeling, S3 image segmentation, S4 first judgment, S5 addition and S6 iteration. First, the image to be segmented is acquired, then an image segmentation model is established, and then the image to be segmented is segmented and further processed using the image segmentation model. Then, based on the processing result, it is judged whether the current pixel point can be added to the segmentation block, and then segmentation and judgment are continuously and iteratively performed to achieve segmentation of the entire image. This embodiment includes the following steps:

[0090] S1 is the first acquisition, which is to acquire the image to be segmented.

[0091] S2 modeling, establishing an image segmentation model, wherein the image segmentation model includes: a CNN sub-model and a region growing sub-model.

[0092] The CNN sub-model uses deep learning technology. After being trained with a large number of historical images and corresponding historical image segmentation data, it can learn the feature representation of pixels in the image and acquire the ability to segment the image. The trained CNN sub-model is then used to perform preliminary segmentation on the image to be segmented.

[0093] Region growing sub-model: Based on the preliminary segmentation results (provided by the CNN sub-model), the segmentation results are refined using a preset growing criterion. The region growing sub-model relies on local information in the image, such as color, texture, etc., to decide which pixels should be merged into the same segmentation block.

[0094] The preset growth criterion may be one of an adjacent region grayscale difference criterion, a regional grayscale distribution statistics criterion, and a color criterion.

[0095] Adjacent region grayscale difference criterion: Region growth is performed based on the similarity of pixel grayscale values. By setting a grayscale difference threshold, if the grayscale difference of adjacent pixels is less than the threshold, they are considered similar and can be merged into the same region.

[0096] Regional grayscale distribution statistical criteria: Calculate the grayscale distribution statistics (such as mean, variance, etc.) of the pixels that have been determined as the target area, and use these statistics to evaluate whether the adjacent pixels should be merged into the target area.

[0097] Color criterion: If the color values ​​of adjacent pixels are similar within a certain range, they can be merged into the same region.

[0098] S3 image segmentation includes S31 first segmentation, S32 setting seed points and S33 second segmentation.

[0099] S31 is the first segmentation, in which the CNN sub-model is used to segment the image to be segmented to obtain n segmentation blocks.

[0100] S32 sets a seed point, randomly selects an edge pixel point of the i-th segmentation block, and uses it as the seed point.

[0101] S33: Second segmentation: the region growth sub-model processes the j-th pixel point adjacent to the seed point according to a preset growth criterion (such as a color criterion, an adjacent region grayscale difference criterion, etc.), and outputs the processing result.

[0102] S4 is the first judgment. Based on the processing results, it is determined whether the j-th pixel point can be merged into the i-th segmentation block. If so, it means that the j-th pixel point and the seed point meet the growth criteria, and S5 is executed to increase; if not, it means that the j-th pixel point and the seed point do not meet the growth criteria, and S6 is executed to iterate.

[0103] S5 increases, adding the j-th pixel to the i-th segmentation block.

[0104] S6 iteration includes S61 first traversal and S62 second traversal.

[0105] S61 is the first traversal, taking the j+1th pixel point as the new jth pixel point, and executing S33 the second segmentation until the first stop condition is met. The first stop condition is: the jth pixel point added in S5 is the last pixel point adjacent to the seed point, that is, the jth pixel point is the last pixel point to be processed.

[0106] This step is to traverse and process the pixel points in each segmentation block (each image block after image segmentation is called a segmentation block, for example: in an image, including the background and the target object, after the image is segmented, two segmentation blocks of the background and the target object are obtained), by continuously taking the next adjacent pixel point as a new processing object, and repeatedly executing the second segmentation of S33 and the first judgment of S4 until the first stop condition is met (that is, all the pixel points adjacent to the seed point are processed).

[0107] S62 is a second traversal, taking the i+1th segment block as a new i-th segment block, and performing the second segmentation in S33 until a second stop condition is met, wherein the second stop condition is: i+1>n.

[0108] This step is to traverse and process all the segmented blocks, by continuously taking the next segmented block as a new processing object and repeatedly executing the second segmentation in S33 until the second stop condition is met (that is, all the segmented blocks are processed).

[0109] This embodiment realizes the preliminary segmentation and further refinement of the image by combining the CNN sub-model and the region growing sub-model. By iteratively processing the pixels in each segmentation block and traversing all segmentation blocks, the segmentation result can be gradually optimized to improve the accuracy and efficiency of the segmentation.

[0110] Example 2: Reference Figure 2 The difference between this embodiment and embodiment 1 is that after executing S1 first acquisition and before executing S2 modeling, it also includes:

[0111] S71: third segmentation: dividing the image to be segmented into m sub-images along the length direction or the width direction of the image to be segmented.

[0112] S72 sets the pixel edge. For each pair of adjacent sub-images (the x-th sub-image and the x+1-th sub-image), find the boundary between them, and record all adjacent boundary pixel points between the x-th sub-image and the x+1-th sub-image as the first pixel edge, and record all adjacent boundary pixel points between the x+1-th sub-image and the x-th sub-image as the second pixel edge.

[0113] S73: First calculation: for the yth pixel point in the first pixel side, calculate the difference between its grayscale value and the grayscale value of the y+1th pixel point to obtain first data.

[0114] S74 is the second judgment, judging whether the first data is greater than the preset gray value threshold. If so, it means that the y-th pixel point is significantly different from the y+1-th pixel point, corresponding to the edge feature in the image, and S77 marking is executed; if not, S75 is executed for the second calculation.

[0115] S75: second calculation, calculating the difference between the grayscale value of the yth pixel point and the grayscale value of the y+1th pixel point in the second pixel edge, and recording it as the second data.

[0116] S76 is the third judgment, judging whether the second data is greater than the preset gray value threshold, if so, executing S77 marking; if not, executing S78 the third traversal.

[0117] S77 marking: in the image to be segmented, marking the yth pixel point or the y+1th pixel point as the first pixel point.

[0118] S78 is a third traversal, taking the x+1th sub-image as a new xth sub-image, and executing S72 to set pixel edges until a third stop condition is met, and then executing S2 to build a model. The third stop condition is: x+1>m.

[0119] In setting the seed point in S32, the first pixel point is used to replace the edge pixel point.

[0120] For example: Refer to Figure 3 and Figure 4 ,Suppose there is a grayscale image containing multiple objects.,The goal is to identify and mark certain boundary points between objects through the above steps,so as to use these boundary points in the subsequent image segmentation task.

[0121] S1 first acquires an image to be segmented using a camera or an image scanning device, and the image contains multiple different objects.

[0122] S71: third segmentation: converting the acquired image to be segmented into a grayscale image, and evenly segmenting it into 4 sub-images (m=4) along its length direction.

[0123] S72 sets the pixel edge. For the first pair of adjacent sub-images (the first sub-image and the second sub-image), locate the boundary between them, and record the boundary pixels of the first sub-image as the first pixel edge, and the boundary pixels of the second sub-image as the second pixel edge.

[0124] S73 first calculation, traversing each pixel point in the first pixel edge, and calculating the difference between its grayscale value and the grayscale value of the next pixel point.

[0125] In this embodiment, the grayscale value of the first pixel is 120, and the grayscale value of the next pixel is 150, so the first data is 30.

[0126] S74 is the second judgment. A grayscale value threshold of 20 is preset. Since the first data is greater than the threshold, it is considered that the grayscale value difference of the pixel point is significant and may be the boundary between the segmented blocks.

[0127] S77 marking, marking the pixel point with significant difference in gray value in the second judgment of S74 as the first pixel point in the image to be segmented, and recording its position.

[0128] S75: second calculation, S76: third judgment: if the gray value difference of the first pixel side is not significant (ie, less than the threshold), the gray value difference of adjacent pixel points in the second pixel side is calculated, and similar judgment is performed.

[0129] S78: third traversal, updating the currently processed sub-image pair to the next pair of adjacent sub-images (the second sub-image and the third sub-image), and repeating steps S72 to S77. This process continues until all sub-image pairs are processed (i.e., the third stop condition x+1>m is satisfied).

[0130] After the above steps, a series of marked first pixel points are obtained, where the first pixel points correspond to boundary pixel points between segmented blocks on the edge of image pixels.

[0131] This embodiment segments the image to be processed, obtains the pixel edges of the two sub-images at the segmentation point, obtains the pixel points whose grayscale value changes exceed a preset threshold within the pixel edge and marks them, and uses the marked pixel points instead of the edge pixel points to perform subsequent steps, which helps to quickly locate the edge points between the segmented blocks and improve the accuracy and efficiency of image segmentation.

[0132] Example 3: Reference Figure 5 The difference between this embodiment and embodiment 1 is that the S3 image segmentation further includes the following steps:

[0133] S81: First acquisition, obtaining pixel points adjacent to the seed point in the i-th segmentation block, recorded as the first pixel point set, and obtaining pixel points adjacent to the seed point in the i+1-th segmentation block, recorded as the second pixel point set.

[0134] S82 is a third calculation, which is to calculate the similarity between the seed point and all the pixel points in the first pixel point set, and calculate the sum of all the similarities as the first similarity.

[0135] S83 is a fourth calculation, which is to calculate the similarity between the seed point and all the pixel points in the second pixel point set, and calculate the sum of all the similarities as the second similarity.

[0136] In the third calculation in S82 and the fourth calculation in S83, the similarity can be calculated based on multiple dimensions such as color, brightness, texture, etc.

[0137] Take color as an example: set the color of the seed point to , the color of the i-th pixel in the first pixel set is , and there are M pixels in the first pixel set, and the color of the j-th pixel in the second pixel set is , and there are N pixels in the second pixel set, the model for calculating similarity based on Euclidean distance is as follows:

[0138] ;

[0139] .

[0140] is the value of the first similarity in this example; is the value of the second similarity in this example.

[0141] Take texture as an example: first, extract the texture features of each pixel through the local binary pattern (LBP) algorithm; traverse each pixel in the first pixel set, define a 3×3 neighborhood with it as the center, compare the pixel values ​​of the 8 pixels at the edge of the neighborhood (referred to as the neighborhood pixel value) with the pixel value of the center pixel (the center pixel in this example refers to the pixel at the center of the 3×3 neighborhood) (referred to as the center pixel value), if the neighborhood pixel value is greater than or equal to the center pixel value, then mark the position as 1, otherwise mark it as 0, arrange the 8 binary values ​​in the neighborhood into an 8-bit binary number in a clockwise or counterclockwise direction, with the binary value in the upper right corner as the first binary number, and then convert it into a decimal number, that is, the LBP feature value of the pixel is obtained. Similarly, calculate the LBP feature value of each pixel in the second pixel set and the LBP feature value of the seed point.

[0142] Create a matrix with the same total size as the two segmented blocks to store the LBP eigenvalues ​​of the first pixel set, the seed point, and the second pixel set, traverse the LBP eigenvalues ​​of each pixel in the first pixel set and the second pixel set, store its LBP eigenvalue in the corresponding matrix position, and also store the LBP eigenvalue of the seed point in the corresponding position of the matrix, and obtain the LBP eigenvector of each pixel in the first pixel set by calculation. Similarly, calculate and obtain the LBP eigenvector of each pixel in the second pixel set, as well as the LBP eigenvector corresponding to the seed point.

[0143] Based on the above LBP feature vector, the similarity between the seed point and the pixel points in the first pixel point set or the second pixel point set is calculated:

[0144] ;

[0145] ;

[0146] is the value of the first similarity in this example; is the value of the second similarity in this example; is the i-th element in the feature vector of the seed point; is the i-th element in the feature vector of the first pixel set; is the jth element in the feature vector of the seed point; is the jth element in the feature vector of the second pixel set; is the dimension of the feature vector of the seed point or the first pixel point set; is the dimension of the feature vector of the seed point or the second pixel point set; because in this example the feature vectors of the seed point, the first pixel point set, and the second pixel point set are all extracted from the same matrix, .

[0147] In other embodiments, the first similarity and the second similarity are calculated from multiple perspectives such as color, brightness, and texture. For example, the similarity is calculated from two perspectives, color and texture. Different weights are assigned to color and texture to obtain the first similarity and the second similarity:

[0148] ;

[0149] ;

[0150] is the value of the first similarity in this example; is the value of the second similarity in this example; , , and The calculation method of is as above; is the weight of the color; is the weight of the texture.

[0151] S84 sorting: in the first pixel point set, all the pixel points are sorted in descending order according to the similarities calculated in the third calculation in S82 to obtain a first pixel point sequence.

[0152] In the second pixel point set, all the pixel points are sorted in descending order according to the similarities calculated in the fourth calculation in S83 to obtain a second pixel point sequence.

[0153] S85 is deletion, in which pixel points whose similarity is less than a preset similarity threshold are deleted from the first pixel point set to obtain a new first pixel point set.

[0154] Pixels whose second similarity is less than a preset similarity threshold are deleted from the second pixel point set to obtain a new second pixel point set.

[0155] S86 is a fourth determination, determining whether the first similarity is greater than the second similarity, if so, executing S87 the first process; if not, executing S88 the second process.

[0156] S87: First processing: the region growth sub-model processes the pixel points in the new second pixel point set in S85 based on the second pixel point sequence and using a preset growth criterion.

[0157] S88 second processing, the region growth sub-model processes the pixel points in the new first pixel point set in S85 deletion based on the first pixel point sequence using a preset growth criterion.

[0158] The first processing S87 and the second processing S88 described in this embodiment are sub-steps of the second segmentation S33.

[0159] This embodiment first obtains the pixel point sets (first and second pixel point sets) adjacent to the seed point in the i-th and i+1-th segmentation blocks, calculates the sum of the similarities between the seed point and the pixels in these two sets (first and second similarities), and sorts and deletes the pixels whose similarities are lower than the threshold. Subsequently, based on the comparison between the first similarity and the second similarity, it is determined whether the regional growth sub-model processes the second pixel point set (if the first similarity is large) or the first pixel point set (if the second similarity is large) according to the preset growth criterion, and the sorted pixel point sequence will be referred to during the processing. Since the color, texture and other features in the same segmentation block should be the same or similar, through the above steps, it is only necessary to use the regional growth sub-module to process the pixel points of the segmentation block with less similarity to the pixel points where the seed point is located, which reduces the amount of calculation and improves the calculation efficiency.

[0160] Example 4: Reference Figure 6 The difference between this embodiment and embodiment 1 is that the image segmentation model further includes a BI-LSTM sub-model, and after executing S2 modeling and before executing S3 image segmentation, it also includes:

[0161] S21 is the second acquisition, collecting historical images and segmentation data of historical images, and obtaining k segmentation blocks after the historical images are segmented. The historical images come from past experiments, observations or records, and the segmentation data is obtained after preprocessing or annotating these images, indicating that the images are segmented into different segments. Through this process, k segments are obtained, which are the results of the historical images being segmented, and each segmentation block represents a specific area or object in the image.

[0162] S22 first determines, determining the center pixel point of the pth segmentation block in the historical image, recorded as the third pixel point. The center pixel point represents the core position of the segmentation block, which can be a geometric center, a centroid or other representative point.

[0163] S23 second determination, obtaining q segment blocks adjacent to the p-th segment block in the historical image, and determining the central pixel point of the q segment blocks, which is recorded as the fourth pixel point.

[0164] S24 is a third acquisition, in which pixel points between the third pixel point and the fourth pixel point are obtained, which are recorded as fifth pixel points. All fifth pixel points constitute a path or region connecting central pixel points of adjacent segmented blocks.

[0165] S25 fourth acquisition, obtaining the grayscale values ​​of all fifth pixels.

[0166] S26 model training, using the gray value of the fifth pixel to train the BI-LSTM sub-model to obtain the trained BI-LSTM sub-model.

[0167] By training the BI-LSTM sub-model, the temporal or spatial dependencies between the grayscale values ​​of the fifth pixel and the potential relationship between these grayscale values ​​and image segmentation or recognition tasks can be learned.

[0168] Reference Figure 7 In other embodiments, after performing the first segmentation in S31 and before performing S32 to set the seed point, the method further includes:

[0169] S91 fifth acquisition, acquiring the central pixel points of the i-th segmentation block and the (i+1)-th segmentation block.

[0170] S92 predicts that the central pixel points of the i-th segmentation block and the i+1-th segmentation block are input into the trained BI-LSTM sub-model, and the grayscale value of the pixel points between the two central pixel points is predicted, which is recorded as the first grayscale value.

[0171] S93 fifth calculation, calculating the difference of adjacent first gray values, recorded as the first difference. The purpose of calculating the difference of adjacent gray values ​​is to detect the change of gray values, which can reflect the change of features such as edges, textures or object boundaries in the image. By comparing these differences, it can be determined whether the gray values ​​of the pixels between the two center pixels have changed significantly, and then the boundaries of the two segmented blocks can be located.

[0172] S94 is the fifth judgment, judging whether the first difference is greater than the preset difference threshold. If so, it means that the grayscale value of the pixel between the two center pixels has changed significantly, which may be due to the existence of edges, textures or object boundaries, and it is necessary to execute S95 replacement; if not, it means that the grayscale value change is not significant, and it is necessary to execute S32 to set the seed point.

[0173] S95 is replaced by obtaining two pixel points corresponding to the first difference value, and taking one of the pixel points as a new seed point.

[0174] This embodiment identifies features or object boundaries in an image by predicting and analyzing the grayscale values ​​of pixels between the center pixels of adjacent segments, and replaces seed points as needed for further analysis or processing. This embodiment locates seed points through the change in grayscale values ​​of pixels passed by a line connecting the center pixels of two segments, thereby improving the accuracy and efficiency of subsequent processing.

[0175] Embodiment 5: This embodiment discloses an image segmentation device, the device comprising:

[0176] The first acquisition module is used to acquire the image to be segmented.

[0177] The modeling module is used to establish an image segmentation model, which includes a CNN sub-model and a region growing sub-model. The CNN sub-model can automatically learn the features in the image and perform preliminary image segmentation based on these features. The region growing sub-model uses the preliminary segmentation results of the CNN sub-model to gradually merge adjacent pixels in an iterative manner to form a more accurate segmentation result.

[0178] The image segmentation module is connected to the first acquisition module and the modeling module in communication, and includes a first segmentation unit, a seed point setting unit and a second segmentation unit.

[0179] The first segmentation unit is used to segment the image to be segmented by using a CNN sub-model to obtain n segmentation blocks.

[0180] A seed point unit is set to use an edge pixel point of the ith segment block as a seed point. This seed point will serve as the starting point for the region growing sub-model processing.

[0181] The second segmentation unit is used for the region growth sub-model to process the j-th pixel point adjacent to the seed point by using a preset growth criterion.

[0182] The first judgment module is connected to the image segmentation module for judging whether the j-th pixel point can be merged into the i-th segmentation block based on the processing result.

[0183] The adding module is used to add the j-th pixel point to the i-th segmentation block.

[0184] The iteration module is connected to the image segmentation module in communication, and includes a first traversal unit and a second traversal unit.

[0185] The first traversal unit is used to take the j+1th pixel point as the new jth pixel point.

[0186] The second traversal unit is used to take the i+1th segmentation block as the new ith segmentation block.

[0187] In this embodiment, the image to be segmented is acquired through the first acquisition module, and the CNN sub-model in the modeling module is used for preliminary segmentation. Then, the region growing sub-model is used in combination with the set seed point and the growth criterion for fine segmentation. The first judgment module and the adding module are used to decide whether to merge adjacent pixels. The iteration module is responsible for traversing all pixels and segmentation blocks to complete the task of accurate segmentation of the entire image.

[0188] Embodiment 6: This embodiment provides an electronic device, which includes: a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the described method are implemented.

[0189] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. An image segmentation method, characterized in that: include: First acquisition: acquiring the image to be segmented; Third segmentation: along the length direction or width direction of the image to be segmented, the image to be segmented is equally divided into m sub-images; Set pixel edge: locate the boundary between the x-th sub-image and the x+1-th sub-image, and record the pixel edge formed by all adjacent boundary pixels between the x-th sub-image and the x+1-th sub-image as the first pixel edge, and record the pixel edge formed by all adjacent boundary pixels between the x+1-th sub-image and the x-th sub-image as the second pixel edge; First calculation: calculate the difference between the grayscale value of the yth pixel point and the grayscale value of the y+1th pixel point in the first pixel edge, and record it as the first data; Second judgment: judging whether the first data is greater than a preset gray value threshold, if so, executing the marking step; if not, executing the second calculation step; Second calculation: calculate the difference between the grayscale value of the yth pixel point and the grayscale value of the y+1th pixel point in the second pixel edge, and record it as the second data; Third judgment: judging whether the second data is greater than a preset gray value threshold, if so, executing the marking step; if not, executing the third traversal step; Mark: mark the yth pixel or the y+1th pixel as the first pixel; Third traversal: take the x+1th sub-image as the new xth sub-image, and execute the step of setting the pixel edge until the third stop condition is met; In the step of setting the seed point, the first pixel point is used to replace the edge pixel point; Modeling: establishing an image segmentation model, wherein the image segmentation model includes: a CNN sub-model, a region growing sub-model and a BI-LSTM sub-model; Second collection: collecting historical images and segmentation data of historical images to obtain k segmentation blocks after the historical images are segmented; First determination: determine the central pixel point of the pth segmentation block in the historical image, recorded as the third pixel point; Second determination: obtaining q segmentation blocks adjacent to the pth segmentation block in the historical image, and determining the central pixel point of the q segmentation blocks, which is recorded as the fourth pixel point; Third acquisition: acquiring a pixel point between the third pixel point and the fourth pixel point, recorded as a fifth pixel point, and all fifth pixel points constitute a path or area connecting central pixel points of adjacent segmented blocks; Fourth acquisition: obtaining the grayscale values ​​of all fifth pixels; Model training: The grayscale value of the fifth pixel is used to train the BI-LSTM sub-model to obtain the trained BI-LSTM sub-model; Image segmentation: including first segmentation, fifth acquisition, prediction, fifth calculation, fifth judgment, replacement, setting seed point and second segmentation; First segmentation: Use the CNN sub-model to segment the image to be segmented and obtain n segmentation blocks; Fifth acquisition: acquiring the central pixel points of the i-th segmentation block and the i+1-th segmentation block; Prediction: Input the central pixels of the i-th segmentation block and the i+1-th segmentation block into the trained BI-LSTM sub-model, and predict the grayscale value of the pixel between the two central pixels, which is recorded as the first grayscale value; Fifth calculation: calculating the difference between the first grayscale values ​​of adjacent pixels, recorded as the first difference; Fifth judgment: judging whether the first difference is greater than a preset difference threshold, if so, executing the replacing step; if not, executing the setting seed point step; Replacement: Get two pixel points corresponding to the first difference, and use one of the pixel points as a new seed point; Set the seed point: take an edge pixel of the i-th segment block as the seed point; Second segmentation: The region growth sub-model processes the j-th pixel point adjacent to the seed point using a preset growth criterion to obtain a processing result; First judgment: based on the processing result, judge whether the j-th pixel point can be merged into the i-th segmentation block. If so, perform the increasing step; if not, perform the iterative step; Add: add the jth pixel to the i-th segmentation block; Iteration: includes the first traversal and the second traversal; First traversal: take the j+1th pixel point as the new jth pixel point and perform the second segmentation step until the first stop condition is met; Second traversal: take the i+1th segmentation block as the new ith segmentation block and perform the second segmentation step until the second stop condition is met.

2. The image segmentation method according to claim 1, characterized in that: The step of image segmentation further includes: First acquisition: obtain the pixel points adjacent to the seed point in the i-th segmentation block, recorded as the first pixel point set, and obtain the pixel points adjacent to the seed point in the i+1-th segmentation block, recorded as the second pixel point set; Third calculation: calculating the similarity between the seed point and all the pixels in the first pixel set, and calculating the sum of all similarities as the first similarity; Fourth calculation: calculating the similarity between the seed point and all the pixels in the second pixel point set, and calculating the sum of all the similarities as the second similarity; Fourth judgment: judging whether the first similarity is greater than the second similarity, if so, executing the first processing step; if not, executing the second processing step; First processing: the region growth sub-model processes the pixels in the second pixel point set using a preset growth criterion; Second processing: the region growing sub-model processes the pixels in the first pixel set using a preset growth criterion.

3. The image segmentation method according to claim 2, characterized in that: After executing the fourth calculation step and before executing the fourth judgment step, the method further includes: Sorting: in the first pixel point set and / or the second pixel point set, all the pixels are sorted in descending order according to the similarity to obtain a first pixel point sequence and / or a second pixel point sequence; Deletion: Delete pixels whose similarity is less than a preset similarity threshold to obtain a new first pixel set and / or second pixel set.

4. The image segmentation method according to claim 3, characterized in that: In the first processing and / or second processing steps, the region growing sub-model is based on the first pixel point sequence and / or the second pixel point sequence, and processes the pixels in the first pixel point set and / or the second pixel point set using a preset growth criterion.

5. An image segmentation device, characterized in that: include: A first acquisition module, used for acquiring an image to be segmented; A third segmentation module is used to divide the image to be segmented into m sub-images along the length direction or the width direction of the image to be segmented; A pixel edge module is set to locate the boundary between the x-th sub-image and the x+1-th sub-image, and the pixel edge formed by all adjacent boundary pixels between the x-th sub-image and the x+1-th sub-image is recorded as the first pixel edge, and the pixel edge formed by all adjacent boundary pixels between the x+1-th sub-image and the x-th sub-image is recorded as the second pixel edge; A first calculation module is used to calculate the difference between the grayscale value of the yth pixel point and the grayscale value of the y+1th pixel point in the first pixel edge, which is recorded as the first data; A second judgment module is used to judge whether the first data is greater than a preset gray value threshold, and if the first data is greater than the preset gray value threshold, the marking module is triggered; otherwise, the second calculation module is triggered; A second calculation module is used to calculate the difference between the grayscale value of the yth pixel point and the grayscale value of the y+1th pixel point in the second pixel edge, and record it as the second data; A third judgment module is used to judge whether the second data is greater than a preset gray value threshold, and if the second data is greater than the preset gray value threshold, the marking module is triggered; otherwise, the third traversal module is triggered; A marking module, used for marking the y-th pixel point or the y+1-th pixel point as the first pixel point; The third traversal module is used to take the x+1th sub-image as the new xth sub-image and trigger the pixel edge setting module; A modeling module, used to establish an image segmentation model, wherein the image segmentation model includes: a CNN sub-model, a region growth sub-model and a BI-LSTM sub-model; The second acquisition module is used to acquire historical images and segmentation data of historical images to obtain k segmentation blocks after the historical images are segmented; The first determination module is used to determine the central pixel point of the p-th segmentation block in the historical image, which is recorded as the third pixel point; The second determination module is used to obtain q segmentation blocks adjacent to the p-th segmentation block in the historical image, and determine the central pixel point of the q segmentation blocks, which is recorded as the fourth pixel point; A third acquisition module is used to obtain a pixel point between the third pixel point and the fourth pixel point, which is recorded as a fifth pixel point, and all fifth pixel points constitute a path or area connecting central pixel points of adjacent segmented blocks; A fourth acquisition module, used for acquiring the grayscale values ​​of all fifth pixels; A model training module, used to train the BI-LSTM sub-model using the gray value of the fifth pixel to obtain a trained BI-LSTM sub-model; An image segmentation module, which is in communication with the first acquisition module and the modeling module, comprises a first segmentation unit, a fifth acquisition unit, a prediction unit, a fifth calculation unit, a fifth judgment unit, a replacement unit, a seed point setting unit, and a second segmentation unit; The first segmentation unit is used to segment the image to be segmented by using a CNN sub-model to obtain n segmentation blocks; A fifth acquisition unit, used for acquiring central pixel points of the ith segmentation block and the (i+1)th segmentation block; A prediction unit is used to input the central pixel points of the ith segmentation block and the i+1th segmentation block into the trained BI-LSTM sub-model, and predict the grayscale value of the pixel points between the two central pixel points, which is recorded as the first grayscale value; A fifth calculation unit, used for calculating a difference between first grayscale values ​​of adjacent pixels, recorded as a first difference; a fifth judgment unit, configured to judge whether the first difference is greater than a preset difference threshold, and if the first difference is greater than the preset difference threshold, trigger the replacement unit; otherwise, trigger the seed point setting unit; A replacement unit, used to obtain two pixel points corresponding to the first difference value, and use one of the pixel points as a new seed point; A seed point unit is set, which is used to use an edge pixel point of the i-th segmentation block as a seed point, and is also used to use the first pixel point to replace the edge pixel point; The second segmentation unit is used for the region growth sub-model to process the j-th pixel point adjacent to the seed point using a preset growth criterion to obtain a processing result; A first judgment module is connected to the image segmentation module for judging whether the j-th pixel point can be merged into the i-th segmentation block based on the processing result; An adding module is used to add the j-th pixel point to the i-th segmentation block; An iteration module, which is in communication connection with the image segmentation module, comprises a first traversal unit and a second traversal unit; A first traversal unit is used to take the j+1th pixel point as a new jth pixel point and execute the steps of the second segmentation unit until a first stop condition is met; The second traversal unit is used to take the (i+1)th segmentation block as the new (i)th segmentation block and execute the steps of the second segmentation unit until the second stop condition is met.

6. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images