A level set image segmentation method, storage medium and image processing terminal

Abstract

The present invention belongs to the field of image processing technology, and discloses a level set image segmentation method, storage medium and image processing terminal, inputting an image to be segmented; extracting multiple image features; initializing an image evolution curve; using the RPCA method to reduce the dimension of the internal and external regions of the image evolution curve to extract effective representation feature vectors; then reconstructing the internal and external regions of the curve of the current image according to the effective representation feature vectors; calculating the reconstructed data of each pixel in the image, using the Euclidean distance to measure the error distance between the real image and the reconstructed data and using it as the data energy term of the energy functional; constructing an energy functional according to the above information and the inherent properties of the curve; using the variational method to solve the functional, driving the curve to evolve to the boundary of the target object, and obtaining the final segmentation result. The present invention is less affected by large and sparse noise, and in addition to removing low-level features, it also incorporates high-level features, and the segmentation results obtained by other traditional methods are more accurate.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a level set image segmentation method, a storage medium and an image processing terminal. Background Art

[0002] The concept of image segmentation appeared in the 1950s. In the course of several decades of development, thousands of image segmentation methods have been applied to multiple fields and are a popular research direction. Among them, image segmentation based on active contours uses curve evolution to detect targets in a given image. First, the initial curve is defined, and then an energy function is constructed through image data. By minimizing the energy function, the curve gradually approaches the edge of the object to be detected, and finally the target is segmented. The geometric active contour level set function in the active contour has the advantage of flexible topological transformation, and the invented method is based on this. However, in the case of complex backgrounds, it is impossible to cross the local extreme value, resulting in poor segmentation effect in complex backgrounds. Therefore, high-level features containing semantic information are introduced into the model to improve the shortcomings and enhance the segmentation effect.

[0003] At present, the commonly used image segmentation methods based on active contours in the industry are: (1) Edge-based active contour model: In an image, the edge of the target object is an important feature to characterize the target, and it is also an important basis for segmenting the object. The edge-based ACM model uses the gradient jump of the boundary in the image as an energy information constructor to drive the iterative evolution of the curve to detect the target. The Snake model proposed by Kass et al. is one of its representatives. However, this Snake model is based on the gradient features of the image to find the boundary of the target. When the background of the image is complex or the edge of the target is blurred, it cannot accurately find the contour of the target object, resulting in poor segmentation results.

[0004] (2) Region-based active contour model: In order to get rid of the constraints of image gradient information and improve the segmentation effect of fuzzy edge images, Chan and Vese proposed an ACM model based on region information, the CV model. The CV model uses the grayscale of image pixels as an energy information constructor. By solving the minimum value of the function, the edge of the target object can be found. As a region-based active contour model, the CV model has better robustness to the initial contour, and at the same time gets rid of the limitations of image gradients, overcoming the shortcomings of the traditional Snake model in processing images with blurred boundaries. Under the premise that the background and foreground are uniform, the above model can achieve good segmentation results. However, in practical applications, the background and foreground in the image are often complex, and sometimes the contrast is not high. The foreground and background are easily confused, resulting in errors in segmenting the target boundary. An effective image segmentation model is proposed, and the image information is characterized by combining high-level semantic information, which is integrated into the active contour to segment the target object when the foreground and background are complex or similar.

[0005] Through the above analysis, the problems and defects of the existing technology are: when the target edge is blurred or the foreground and background are non-uniform (the difference between the foreground and background is small or the difference in the foreground area is large), the existing active contour model is difficult to accurately segment the target object. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention provides a level set image segmentation method, a storage medium and an image processing terminal.

[0007] The present invention is implemented as follows: a level set image segmentation method, wherein the level set image segmentation method first uses the RPCA reconstruction technology to respectively extract effective characterization feature vectors of the internal and external areas of the initial contour of the image; then reconstructs the data of each pixel in the internal and external areas of the image evolution curve through the characterization feature vectors; respectively calculates the error between the reconstructed data and the real data of each pixel point in the internal and external areas of the curve; and measures the data error with Euclidean distance, accumulates the error of each pixel point to construct an energy functional, minimizes the energy functional, drives the curve evolution, and continuously iterates until it evolves to the boundary of the target object to obtain a segmentation result.

[0008] Furthermore, the level set image segmentation method comprises the following steps:

[0009] Step 1: Input the image I to be segmented with a size of M×N;

[0010] Step 2: For each pixel i of the input image, extract the color feature, saliency feature, and depth feature, etc., a total of 39 dimensions, represented by x i ;

[0011] Step 3, initialize the evolution curve and use the level represented as φ0;

[0012] Step 4: The evolution curve divides the image into the internal area A in and external area A out , inner area A in Each pixel feature x i Represented as A in =[x1,...,x m ], where A in is a 39×m matrix, where m is the total number of pixels in the inner area and the outer area A out Each pixel feature x i Indicated as A out =[x1,...,x n ], where A out It is a 39×n matrix, where n is the total number of pixels in the external area;

[0013] Step 5: For the inner area A in and external area A out Perform RPCA data feature dimension reduction to obtain the representation vector R F and R B ;

[0014] Step 6: According to the characterization vector R F Reconstruct A in Each pixel in; Similarly, according to the representation vector R B Reconstruct A out Each pixel in

[0015] Step 7: For each pixel point on the image, its data is reconstructed as the Euclidean distance between the original pixel feature and the reconstructed pixel feature, and the data error of each pixel is accumulated to construct a data energy term;

[0016] Step 8: Construct the energy functional based on the inherent properties of the curve and the data energy term in step 7:

[0017]

[0018] Where φ is the level set function of the image, Ω in is the inner area of ​​the evolution curve, Ω out is the outer region of the evolution curve, H(·) is the Heaviside function, ε i (x) is the data reconstruction function. The first term in the above formula is the inherent energy term of the curve, which represents the length of the curve. The second and third terms are data-driven, which represent the data error of the pixel points. λ1 and λ2 are weight coefficients.

[0019] Step nine: Use the variational method to minimize the functional, drive the curve evolution, and iterate continuously to obtain the segmentation result.

[0020] Furthermore, the saliency features and depth features of step 2 are calculated as follows:

[0021] Salient features: Use the FT algorithm to quickly extract the salient features of the input image;

[0022] Deep features: Use the deep learning toolbox provided by MATLAB to extract the deep features of the image.

[0023] Furthermore, the reconstruction process of step six is ​​divided into two steps:

[0024] The first step is to calculate A in The reconstruction coefficient of each pixel i in is: in Yes A in The characteristic mean of outThe reconstruction coefficient of each pixel i in is: in Yes A out The characteristic mean of

[0025] The second step is to calculate the reconstructed data of each pixel inside the curve using the reconstruction coefficient: Similarly, the reconstruction data of each pixel outside the curve is calculated by the reconstruction coefficient:

[0026] Furthermore, the steps of minimizing the energy function in step nine and driving the curve to evolve to the boundary of the target object and obtaining the segmentation result are as follows:

[0027] In the first step, the evolution equation corresponding to the energy functional is:

[0028]

[0029] Among them, K(x) is the curvature term,

[0030] In the second step, the update equation of the level set function is:

[0031]

[0032] The third step is to iteratively update the level set function φ until it converges to the target boundary or reaches the maximum number of iterations, and obtain the final segmentation result φ * .

[0033] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the level set image segmentation method.

[0034] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the level set image segmentation method.

[0035] Another object of the present invention is to provide an information data processing terminal, which is used to execute the steps of the level set image segmentation method.

[0036] Another object of the present invention is to provide an image processing terminal, which is used to execute the steps of the level set image segmentation method.

[0037] Another object of the present invention is to provide a computer vision processing terminal, which is used to execute the steps of the level set image segmentation method.

[0038] In combination with the above technical solutions and the technical problems solved, please analyze the advantages and positive effects of the technical solutions to be protected by the present invention from the following aspects:

[0039] First, in view of the technical problems existing in the above-mentioned prior art and the difficulty of solving the problems, the present invention closely combines the technical solutions to be protected and the results and data in the research and development process, and analyzes in detail and deeply how the technical solutions of the present invention solve the technical problems, and some creative technical effects brought about after solving the problems. The specific description is as follows: The present invention is based on the image segmentation method of RPCA (Robust Principal Component Analysis) data reconstruction level set.

[0040] Second, considering the technical solution as a whole or from the perspective of the product, the technical effects and advantages of the technical solution to be protected by the present invention are specifically described as follows: The model of the present invention integrates low-level features and high-level features of the image, and can segment the target object in the cases of complex foreground and background or similar foreground and background.

[0041] The present invention creatively applies the RPCA reconstruction concept to image segmentation based on active contours, effectively combines the bottom layer and high layer feature information of the image, and can effectively segment complex targets in non-uniform images. The present invention uses RPCA reconstruction technology. Compared with PCA technology, which must assume that the image information satisfies the Gaussian distribution, RPCA does not have this assumption and will not be affected when the image contains large noise or outliers. This allows the method of the present invention to effectively segment targets even when the target object is non-uniform. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flow chart of the level set image segmentation method provided by an embodiment of the present invention.

[0043] Figure 2 It is a flow chart of implementing the level set image segmentation method provided by an embodiment of the present invention.

[0044] Figure 3 It is a comparison diagram of the segmentation effects of the present invention and the traditional method on a natural non-uniform image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0046] 1. Explanatory Examples In order to enable those skilled in the art to fully understand how to implement the present invention, this section provides an illustrative example that expands and describes the technical solution of the claims.

[0047] The image segmentation method based on RPCA data reconstruction level set of the present invention applies the RPCA reconstruction concept to image segmentation. Compared with PCA, which needs to potentially assume that image information satisfies Gaussian distribution, RPCA technology can process a large amount of non-Gaussian distribution data and is not affected by large noise points and outliers. Therefore, the present invention can obtain better experimental results.

[0048] like Figure 1 As shown, the level set image segmentation method provided by the present invention comprises the following steps:

[0049] S101: inputting an image to be segmented; extracting multiple image features; initializing an image evolution curve; using the RPCA method to reduce the dimension of the inner and outer regions of the image evolution curve to extract effective representation feature vectors;

[0050] S102: then reconstructing the inner and outer regions of the curve of the current image according to the effective characterization feature vector; calculating the reconstructed data of each pixel in the image, using the Euclidean distance to measure the error distance between the real image and the reconstructed data and using it as the data energy item of the energy functional;

[0051] S103: construct an energy functional according to the above information and the inherent properties of the curve; use the variational method to solve the functional, drive the curve to evolve to the boundary of the target object, and obtain the final segmentation result.

[0052] like Figure 2 As shown, the image segmentation method based on RPCA data reconstruction level set provided by the embodiment of the present invention includes the following steps:

[0053] Step 1: Input the image I to be segmented with a size of M x N;

[0054] Step 2: For each pixel i of the input image, extract the color feature, saliency feature, and depth feature, etc., a total of 39 dimensions, represented by x i ;

[0055] Step 3, initialize the evolution curve and use the level represented as φ0;

[0056] Step 4: The evolution curve divides the image into the internal area A in and external area A out , inner area A in Each pixel feature x i Indicated as A in =[x1,...,x m ], where Ain is a 39×m matrix, where m is the total number of pixels in the inner area and the outer area A out Each pixel feature x i Indicated as A out =[x1,...,x n ], where A out It is a 39×n matrix, where n is the total number of pixels in the external area;

[0057] Step 5: For the inner area A in and external area A out Perform RPCA data feature dimension reduction to obtain the representation vector R F and R B ;

[0058] Step 6: According to the characterization vector R F Reconstruct A in Each pixel in; Similarly, according to the representation vector R B Reconstruct A out Each pixel in

[0059] Step 7: For each pixel point on the image, its data is reconstructed as the Euclidean distance between the original pixel feature and the reconstructed pixel feature, and the data error of each pixel is accumulated to construct a data energy term;

[0060] Step 8: Construct the energy functional based on the inherent properties of the curve and the data energy term in step 7:

[0061]

[0062] Where φ is the level set function of the image, Ω in is the inner area of ​​the evolution curve, Ω out is the outer region of the evolution curve, H(·) is the Heaviside function, ε i (x) is the data reconstruction function. The first term in the above formula is the inherent energy term of the curve, which represents the length of the curve. The second and third terms are data-driven, which represent the data error of the pixel points. λ1 and λ2 are weight coefficients.

[0063] Step nine: Use the variational method to minimize the functional, drive the curve evolution, and iterate continuously to obtain the segmentation result:

[0064] In the first step, the evolution equation corresponding to the energy functional is:

[0065]

[0066] In the second step, the update equation of the level set function is:

[0067]

[0068] The third step is to iteratively update the level set function φ until it converges to the target boundary or reaches the maximum number of iterations, and obtain the final segmentation result φ * .

[0069] 2. Evidence of the effects of the embodiments The embodiments of the present invention have achieved some positive effects during the development or use process, and indeed have great advantages over the prior art. The following content is described in conjunction with the data, charts, etc. of the test process.

[0070] 1. Simulation conditions

[0071] The present invention is a simulation performed by using MATLAB software on an Intel(R) Core i7-7700 3.60GHZ CPU, 16G memory, and WINDOWS 10 operating system.

[0072] 2. Simulation content

[0073] The effects of image segmentation of the present invention and traditional methods on natural non-uniform images are compared. Eight natural images with relatively complex backgrounds are selected to illustrate the advantages of the present invention. Figure 3 In FIG. 8 , the first row is 8 original images to be segmented, the second row is the ground truth, the third row is the segmentation result corresponding to the present invention, and the fourth row is the segmentation result of the comparison method.

[0074] The segmentation indexes of the present invention and the traditional method for image segmentation on natural non-uniform complex images are compared. Two segmentation indexes are used: Error Rate (ERR) and Intersection over Union (IoU). The more accurate the segmentation result, the lower the error rate and the higher the IoU. As shown in Table 1, the first line is the four images to be segmented, the second line is the evaluation index of the present method, and the third line is the evaluation index of the traditional method.

[0075] 3. Simulation effect analysis:

[0076] The present invention can segment natural images with large noise and large internal differences in the foreground and achieve good results. Figure 3 It can be seen that when the image background is complex and the internal differences of the target are large, the present invention can achieve better segmentation results, which illustrates the effectiveness of RPCA technology in reconstructing level set data. Compared with the commonly used segmentation method based on active contours in the industry, the method of the present invention can better characterize the common features of the image background, reduce the interference of large noise and complex background on segmentation, and finally obtain better segmentation results.

[0077] It can be seen from Table 1 that, whether in subjective visual observation or objective data evaluation, the RPCA technology used in the present invention has a higher segmentation accuracy than the traditional method.

[0078] Table 1

[0079]

[0080]

[0081] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0082] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A level set image segmentation method, characterized in that: The level set image segmentation method first uses the RPCA reconstruction technology to extract effective characterization feature vectors of the internal and external regions of the initial contour of the image respectively; then reconstructs the data of each pixel in the internal and external regions of the image evolution curve through the characterization feature vectors; calculates the error between the reconstructed data and the real data of each pixel point in the internal and external regions of the curve respectively; and uses the Euclidean distance to measure the data error, accumulates the error of each pixel point to construct an energy functional, minimizes the energy functional, drives the curve to evolve, and iterates continuously until it evolves to the boundary of the target object to obtain the segmentation result; The level set image segmentation method comprises the following steps: Step 1: Input the image I to be segmented with a size of M×N; Step 2: For each pixel i of the input image, extract the color feature, saliency feature, and depth feature, etc., a total of 39 dimensions, represented by x i ; Step 3, initialize the evolution curve and use the level represented as φ0; Step 4: The evolution curve divides the image into the internal area A in and external area A out , inner area A in Each pixel feature x i Represented as A in =[x1,...,x m ], where A in is a 39×m matrix, where m is the total number of pixels in the inner area and the outer area A out Each pixel feature x i Represented as A out =[x1,...,x n ], where A out It is a 39×n matrix, where n is the total number of pixels in the external area; Step 5: For the inner area A in and external area A out Perform RPCA data feature dimension reduction to obtain the representation vector R F and R B ; Step 6: According to the characterization vector R F Reconstruct A in Each pixel in; Similarly, according to the representation vector R B Reconstruct A out Each pixel in Step 7: For each pixel point on the image, its data is reconstructed as the Euclidean distance between the original pixel feature and the reconstructed pixel feature, and the data error of each pixel is accumulated to construct a data energy term; Step 8: Construct the energy functional based on the inherent properties of the curve and the data energy term in step 7: Where φ is the level set function of the image, Ω in is the inner area of ​​the evolution curve, Ω out is the outer region of the evolution curve, H(·) is the Heaviside function, ε i (x) is the data reconstruction function. The first term in the above formula is the inherent energy term of the curve, which represents the length of the curve. The second and third terms are data-driven, which represent the data error of the pixel points. λ1 and λ2 are weight coefficients. Step nine, use the variational method to minimize the functional, drive the curve evolution, and iterate continuously to obtain the segmentation result; The salient features and deep features of step 2 are calculated as follows: Salient features: Use the FT algorithm to quickly extract the salient features of the input image; Deep features: Use the deep learning toolbox provided by MATLAB to extract the deep features of the image.

2. The level set image segmentation method according to claim 1, characterized in that: The reconstruction process of step 6 is divided into two steps: The first step is to calculate A in The reconstruction coefficient for each pixel i in is: in Yes A in The characteristic mean of out The reconstruction coefficient for each pixel i in is: in Yes A out The characteristic mean of The second step is to calculate the reconstructed data of each pixel inside the curve using the reconstruction coefficient: Similarly, the reconstruction data of each pixel outside the curve is calculated by the reconstruction coefficient:

3. The level set image segmentation method according to claim 1, characterized in that: The steps of minimizing the energy function in step nine and driving the curve to evolve to the boundary of the target object and obtaining the segmentation result are as follows: In the first step, the evolution equation corresponding to the energy functional is: Among them, K(x) is the curvature term, In the second step, the update equation of the level set function is: The third step is to iteratively update the level set function φ until it converges to the target boundary or reaches the maximum number of iterations, and obtain the final segmentation result φ * .

4. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the level set image segmentation method according to any one of claims 1 to 3.

5. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the level set image segmentation method according to any one of claims 1 to 3.

6. An information data processing terminal, characterized in that: The information data processing terminal is used to execute the steps of the level set image segmentation method according to any one of claims 1 to 3.

7. An image processing terminal, characterized in that: The image processing terminal is used to execute the steps of the level set image segmentation method according to any one of claims 1 to 3.

8. A computer vision processing terminal, characterized in that: The computer vision processing terminal is used to execute the steps of the level set image segmentation method according to any one of claims 1 to 3.

Images