Laser stripe segmentation method based on multi-scale salient features

Abstract

The present invention relates to the field of laser stripe segmentation technology, and in particular to a laser stripe segmentation method based on multi-scale saliency features. The method comprises: obtaining an initial brightness saliency map through a saliency detection model; segmenting the laser stripe grayscale image using a serialization threshold to obtain a Boolean map; obtaining a regional stability image by calculating the weighted sum of different binary images; solving the weighting coefficient through principal component analysis, fusing the brightness saliency map and the regional stability saliency map to obtain a final fused saliency map; performing adaptive maximum entropy segmentation on the fused saliency map, and then obtaining the final segmentation result based on the stability metric. The present invention is aimed at different scenes and low-resolution scenes, and can not only suppress the interference of background noise, but also highlight the contrast of laser stripes, and has strong robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of laser stripe segmentation, and in particular to a laser stripe segmentation method based on multi-scale significant features. Background Art

[0002] With the development of structured light active vision imaging technology, for visual inspection systems working in structured or unstructured environments, how to accurately segment line structured light under uneven lighting and background noise interference is an important research direction and one of the research hotspots for improving the level of intelligent visual inspection. Although deep learning models can provide higher accuracy than other methods, they require larger data sets and more computational overhead. Saliency detection can eliminate redundant information in images and identify areas that can attract attention. Together with other methods, it is widely used in the field of image segmentation.

[0003] Human vision has the ability to quickly search for areas of interest. In fact, the basic features of objects in the scene, such as color information, brightness, texture and other information, are usually the first to attract the attention of the human eye. The use of saliency detection can highlight important targets in the image, thereby enabling rapid search for image targets and promoting image segmentation tasks. At present, there are three types of saliency detection methods: spatial domain feature method, statistical characteristic method and atlas method. The spatial feature method refers to extracting corresponding features from the image, such as pixel color, brightness, method and texture features, and integrating them into the saliency map. The statistical feature method refers to obtaining the saliency features of the image based on the statistical characteristics of the pixel values ​​of the image. The atlas method refers to a method that measures the saliency of the region based on the differences in color, direction and other features between pixel blocks after dividing the image into blocks.

[0004] Practice shows that although many researchers have conducted extensive research on the accurate segmentation algorithm of structured light stripes, most algorithms are based on uniform and stable light quality under uniform illumination. In view of uneven illumination and background noise interference, this paper studies the structured light stripe segmentation method. By constructing a saliency detection model and obtaining the saliency features of the laser stripe area, the influence of illumination and background noise on the stripe feature area can be effectively avoided. Finally, the adaptive maximum entropy is used to segment the saliency extraction results.

[0005] Therefore, a laser stripe segmentation method that can address the effects of uneven illumination and background noise has been developed, and can also serve line laser 3D imaging systems and provide technical support for real-time visual detection, which has huge market potential. The combination of multi-scale saliency feature fusion and maximum entropy segmentation is the core technology of the entire invention. Summary of the invention

[0006] The purpose of the present invention is to provide a laser stripe segmentation method based on multi-scale significant features to solve the following existing problems: most algorithms are based on uniform and stable light quality under a uniform lighting environment, cannot effectively address uneven lighting and background noise interference, and cannot effectively avoid the influence of lighting and background noise on the stripe feature area.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] The laser stripe segmentation method based on multi-scale salient features includes the following steps:

[0009] The initial brightness saliency map is obtained through the saliency detection model to reduce the interference of complex background, uneven illumination and noise on the laser stripes, and highlight the distinguishability of the laser stripes from the background.

[0010] The laser stripe grayscale image is segmented using the serialized threshold to obtain a Boolean map;

[0011] The regional stability image is obtained by calculating the weighted sum of different binary images to highlight the difference between the laser stripes and the background and expose the image features at different threshold levels;

[0012] The weighted coefficients are solved by principal component analysis, and the brightness saliency map and the regional stability image are fused to obtain the final fused saliency map.

[0013] The fused saliency map is adaptively segmented using maximum entropy, and the final segmentation result is obtained based on the stability metric.

[0014] Further preferably, the initial brightness saliency map is obtained by using a saliency detection model, mainly comprising:

[0015] The saliency detection module is introduced to distinguish the target from the background. The specific method is as follows:

[0016] Convert the RGB color space to the Lab color space, calculate the mean of the Lab channel image, subtract the Lab channel image from the Lab channel mean and normalize them to get the initial brightness saliency map; the function expression is:

[0017]

[0018] Among them I c (x,y) is the input image, is the average value of the input image, C represents the color channel of the input image, c∈{L,a,b}.

[0019] Further preferably, the regional stability image is obtained by calculating the weighted sum of different binary images, mainly comprising:

[0020] By calculating the Boolean map, the stable and significant areas of the laser stripe image are extracted, and the Boolean map under different segmentation thresholds is defined as BM = {BM 1 ,…,BM n}, its function expression is:

[0021] BM=Thr(I,θ)

[0022] Where Thr(·) represents the threshold function, I represents the feature map of the input image, θ=δ / 255 is the segmentation threshold, δ increases in steps of 16 and δ∈[δ / 2:δ:255-δ / 2];

[0023] After obtaining a series of Boolean maps, the stable saliency map of the laser stripe area is obtained by calculating the sum of the weights of each Boolean map, and its function expression is:

[0024]

[0025] where θ i are different segmentation thresholds normalized to [0, 1], and BM i It is the Boolean map under different segmentation thresholds.

[0026] Further preferably, the weighted coefficient is solved by principal component analysis, and the brightness saliency map and the regional stability image are fused to obtain the final fused saliency map, which is expressed as follows:

[0027]

[0028] Here is the matrix constructed from the original image. 1 and c 2 be integrated;

[0029] Calculate the covariance matrix The matrix X has the following formula:

[0030]

[0031] in, represents the covariance of the image, Represents the grayscale mean of the i-th original image;

[0032] Calculate the eigenvalues ​​and eigenvectors of the covariance matrix, and use the weight coefficient calculated by the maximum eigenvalue to express it as:

[0033]

[0034] The fused image is represented as:

[0035] F=w 1 S L +w2 S R

[0036] Among them, F is the final fused image, S L and S R is the image to be fused.

[0037] Further preferably, the fused saliency map is adaptively segmented by maximum entropy, and then the final segmentation result is obtained based on the stability metric, and the expression is as follows:

[0038] According to Shannon's theory, entropy is expressed as follows:

[0039]

[0040] Where p(x) is the probability of event x occurring;

[0041] Using an image to describe the above formula, x is a gray level of the image, p(x) is the probability that the gray value is x. If the image has N gray levels, the above formula can be expressed as:

[0042]

[0043] Assume that T is the threshold, the gray level less than T is the target area, and the gray level greater than T is the background area. The probability of the gray level of the target area and the background area is expressed as follows:

[0044]

[0045] The entropy of the target area and the background area is defined as:

[0046]

[0047] The entropy function of an image is defined as:

[0048] H(t)=H 0 (t)+H b (t)

[0049] The threshold can be expressed as:

[0050] T = argmaxH(t)

[0051] After performing adaptive maximum entropy segmentation on the fused saliency image, 8 connected regions are constructed and the area of ​​each connected region is obtained;

[0052] The final segmented image is obtained by calculating the area of ​​the connected region as a stability measure.

[0053] The present invention has at least the following beneficial effects:

[0054] The present invention adopts a saliency detection model to obtain an initial brightness saliency map. By converting the RGB color space to the Lab space, the mean difference between the original image and each color channel of Lab is calculated to obtain the initial brightness saliency feature. The saliency detection module can suppress the influence of uneven illumination and noise on laser stripes under complex backgrounds, and can effectively improve the quality of subsequent image segmentation.

[0055] The present invention uses serialized thresholds to segment the laser stripe grayscale image, and further highlights the contrast of the laser stripes by exposing the laser stripe features at different thresholds. The Boolean map is used to extract the laser stripe image features at different threshold levels, and each Boolean map is weighted and calculated to obtain a regional stability image, which further highlights the difference between the laser stripes and the background and enhances the segmentation effect.

[0056] The present invention can be used for target feature extraction and segmentation in low-light environments. Through the saliency detection model, it can effectively locate the area of ​​interest in the image and improve the image retrieval speed. At the same time, based on multi-scale saliency features, it can complement each other to complete the segmentation tasks in the corresponding image processing field.

[0057] In addition, in addition to being applied to non-destructive testing of industrial products on production lines, the present invention can also be used in robot vision guidance, three-dimensional target reconstruction and military fields, as well as other real-time production and processing fields based on structured light active imaging visual assisted measurement technology. And according to the algorithm simulation results of the present invention, for different scenes and low-resolution scenes, the algorithm of the present invention can not only suppress the interference of background noise, but also highlight the contrast of laser stripes, and has strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0059] Figure 1 Schematic diagram of adaptive stripe segmentation based on multi-scale saliency features;

[0060] Figure 2 Schematic diagram of laser stripe segmentation in different scenarios. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0062] In the process of accurate segmentation of laser stripes, the brightness saliency map and the regional stability image are fused, and on the basis of complementary advantages, the fused saliency map is adaptively segmented by maximum entropy, and the final segmentation result is obtained based on the stability metric, which is the core content of the patent of this invention. Its purpose is to improve the accuracy of laser stripe image segmentation under the influence of uneven illumination and noise by using the segmentation model that integrates saliency features and maximum entropy.

[0063] To achieve the above object, the present invention adopts the following design scheme:

[0064] First, in view of the influence of background noise, the present invention introduces a saliency detection model to obtain an initial brightness saliency map to reduce the interference of complex background, uneven illumination and noise on laser stripes. Highlight the distinguishability of laser stripes from the background; second, the laser stripe grayscale image is segmented using a serialized threshold to obtain a Boolean map. After obtaining the Boolean map, the purpose is to expose the image features at different threshold levels, and obtain a regional stability image by calculating the weighted sum of different binary images, thereby highlighting the difference between laser stripes and the background; finally, the weighted coefficient is solved by principal component analysis (PCA), and the brightness saliency map and the regional stability image are fused to obtain the final fused saliency map. On the basis of complementary advantages, the fused saliency map is adaptively segmented with maximum entropy, and then the final segmentation result is obtained based on the stability metric.

[0065] The flow chart of the stripe adaptive segmentation method integrating salient features is as follows: Figure 1 shown.

[0066] The stripe adaptive segmentation method integrating salient features is described as follows ( Figure 1 shown):

[0067] ①Introduce a saliency detection module to distinguish targets from backgrounds.

[0068] It mainly includes two parts: converting the RGB color space to the Lab color space, calculating the mean of the Lab channel image, subtracting the Lab channel image from the Lab channel mean and normalizing it to obtain the initial brightness saliency map. The function expression is:

[0069]

[0070] Among them I c (x,y) is the input image, is the average value of the input image, C represents the color channel of the input image, c∈{L,a,b}.

[0071] ② By calculating the Boolean map, the stable and significant area of ​​the laser stripe image is extracted. BM={BM 1 ,…,BMn The function expression of} is:

[0072] BM=Thr(I,θ) (2)

[0073] where Thr(·) represents the threshold function, I represents the feature map of the input image, θ=δ / 255 is the segmentation threshold, δ increases in steps of 16 and δ∈[δ / 2:δ:255-δ / 2].

[0074] After obtaining a series of Boolean maps, the stable saliency map of the laser stripe area is obtained by calculating the sum of the weights of each Boolean map. The calculation formula is as follows:

[0075]

[0076] where θ i are different segmentation thresholds normalized to [0, 1], and BM i It is the Boolean map under different segmentation thresholds.

[0077] ③ Multi-scale image fusion technology is an important method to obtain key features of the image. On the basis of complementary advantages, it avoids the influence of background noise and uneven illumination on the laser stripes, and enhances the contrast characteristics between the laser stripes and the background. The weighting coefficients of the brightness saliency map and the stable area saliency map are determined based on principal component analysis (PCA), and the optimal weighting coefficient is determined by calculating the eigenvector and corresponding eigenvalue of the input image. The expression is:

[0078]

[0079] Here is the matrix constructed from the original image. 1 and c 2 Calculate the covariance matrix Matrix X:

[0080]

[0081] represents the covariance of the image, and Represents the grayscale mean of the i-th original image.

[0082] Calculate the eigenvalues ​​and eigenvectors of the covariance matrix, and the weight coefficient calculated using the maximum eigenvalue can be expressed as:

[0083]

[0084] The fused image is represented as:

[0085] F=w 1 S L +w 2S R (7)

[0086] F is the final fused image, and S L and S R is the image to be fused.

[0087] Based on the fused image, the maximum entropy is used to segment the stripes. According to Shannon's theory, entropy is expressed as follows:

[0088]

[0089] where p(x) is the probability of event x occurring.

[0090] Using an image to describe formula (8), x is a gray level of the image, p(x) is the probability of the gray value being x. If the image has N gray levels, formula (8) can be expressed as:

[0091]

[0092] Assume that T is the threshold, the gray level less than T is the target area, and the gray level greater than T is the background area. The probability of the gray level of the target area and the background area is expressed as follows:

[0093]

[0094] The entropy of the target area and the background area is defined as:

[0095]

[0096] The entropy function of an image is defined as:

[0097] H(t)=H 0 (t)+H b (t) (14)

[0098] The threshold can be expressed as:

[0099] T=argmaxH(t) (15)

[0100] After performing adaptive maximum entropy segmentation on the fused saliency image, 8 connected regions are constructed and the area of ​​each connected region is obtained. Finally, the final segmented image is obtained by calculating the area of ​​the connected region as a stability measure. The use of stability measures can effectively improve segmentation accuracy. Figure 2 The figure shows the segmentation test of the laser stripes in different scenes and low-resolution scenes. According to the algorithm simulation results of the patent of the present invention, the algorithm of the present invention can not only suppress the interference of background noise in different scenes and low-resolution scenes, but also highlight the contrast of laser stripes, and has strong robustness.

[0101] From the above, we can know that:

[0102] The biggest advantage of the present invention is that it uses multi-scale significant features and maximum entropy segmentation model to accurately segment laser stripes in response to the interference of uneven illumination and background noise on laser stripes. The quality of laser stripe segmentation is improved. Experimental results show that this method can effectively suppress stripe image noise, and at the same time, it can effectively segment laser stripes for low-resolution laser stripe images. The laser stripe segmentation algorithm based on multi-scale significant features has good engineering application prospects. It is specifically reflected in the following points:

[0103] 1. Use the saliency detection model to obtain the initial brightness saliency map. By converting the RGB color space to the Lab space, the mean difference between the original image and each color channel of Lab is calculated to obtain the initial brightness saliency feature. This saliency detection module can suppress the influence of uneven illumination and noise on laser stripes under complex backgrounds, and can effectively improve the quality of subsequent image segmentation.

[0104] 2. Use serialized thresholds to segment the laser stripe grayscale image, and further highlight the contrast of the laser stripes by exposing the laser stripe features at different thresholds. Use Boolean maps to extract the laser stripe image features at different threshold levels, and weight and calculate each Boolean map to obtain a regional stability image, further highlighting the difference between the laser stripes and the background, and enhancing the segmentation effect.

[0105] 3. The method of the present invention can be used for target feature extraction and segmentation in low-light environments. Through the saliency detection model, it can effectively locate the area of ​​interest in the image and improve the image retrieval speed. At the same time, based on multi-scale saliency features, it can complement each other to complete the segmentation tasks in the corresponding image processing field.

[0106] 4. In addition to being used in nondestructive testing of industrial products on production lines, the present invention can also be used in robot vision guidance, three-dimensional target reconstruction and military fields, as well as other real-time production and processing fields based on structured light active imaging vision-assisted measurement technology.

[0107] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A laser stripe segmentation method based on multi-scale saliency features, characterized in that: The following steps are involved: The initial brightness saliency map is obtained through the saliency detection model to reduce the interference of complex background, uneven illumination and noise on the laser stripes, and highlight the distinguishability of the laser stripes from the background. The laser stripe grayscale image is segmented using the serialized threshold to obtain a Boolean map; The regional stability image is obtained by calculating the weighted sum of different binary images to highlight the difference between the laser stripes and the background and expose the image features at different threshold levels; The weighted coefficients are solved by principal component analysis, and the brightness saliency map and the regional stability image are fused to obtain the final fused saliency map. The fused saliency map is adaptively segmented using maximum entropy, and the final segmentation result is obtained based on the stability metric.

2. The laser stripe segmentation method based on multi-scale saliency features according to claim 1, characterized in that: in, The initial brightness saliency map is obtained through the saliency detection model, which mainly includes: The saliency detection module is introduced to distinguish the target from the background. The specific method is as follows: Convert the RGB color space to the Lab color space, calculate the mean of the Lab channel image, subtract the Lab channel image from the Lab channel mean and normalize them to get the initial brightness saliency map; the function expression is: Among them I c (x,y) is the input image, is the average value of the input image, C represents the color channel of the input image, c∈{L,a,b}, L represents the brightness of the pixel, a represents the component from green to red, and b represents the component from blue to yellow.

3. The laser stripe segmentation method based on multi-scale saliency features according to claim 1, characterized in that: in, The regional stability image is obtained by calculating the weighted sum of different binary images, which mainly includes: By calculating the Boolean map, the stable and significant area of ​​the laser stripe image is extracted, and the Boolean map under different segmentation thresholds is defined as BM = {BM1,…,BM n }, its function expression is: BM=Thr(I,θ) Where Thr(·) represents the threshold function, I represents the feature map of the input image, θ=δ / 255 is the segmentation threshold, δ increases in steps of 16 and δ∈[δ / 2:δ:255-δ / 2]; After obtaining a series of Boolean maps, the stable saliency map of the laser stripe area is obtained by calculating the sum of the weights of each Boolean map, and its function expression is: where θ i are different segmentation thresholds normalized to [0, 1], and BM i It is the Boolean map under different segmentation thresholds.

4. The laser stripe segmentation method based on multi-scale saliency features according to claim 1, characterized in that: in, The weighted coefficient is solved by principal component analysis, and the brightness saliency map and the regional stability image are fused to obtain the final fused saliency map. The expression is as follows: Here is the matrix constructed from the original image, c1 and c2 are fused; Calculate the covariance matrix The matrix X has the following formula: in, represents the covariance of the image, Represents the grayscale mean of the i-th original image; Calculate the eigenvalues ​​and eigenvectors of the covariance matrix, and use the weight coefficient calculated by the maximum eigenvalue to express it as: The fused image is represented as: F=w1S L +w2S R Among them, F is the final fused image, S L and S R is the image to be fused.

5. The laser stripe segmentation method based on multi-scale saliency features according to claim 1, characterized in that: in, The fused saliency map is adaptively segmented by maximum entropy, and the final segmentation result is obtained based on the stability metric. The expression is as follows: According to Shannon's theory, entropy is expressed as follows: Where p(x) is the probability of event x occurring; Using an image to describe the above formula, x is a gray level of the image, p(x) is the probability that the gray value is x. If the image has N gray levels, the above formula can be expressed as: Assuming T as the threshold, the gray level less than T is the target area, and the gray level greater than T is the background area. The probability of the gray level of the target area and the background area is expressed as follows: The entropy of the target area and the background area is defined as: The entropy function of an image is defined as: H(t)=H0(t)+H b (t) The threshold is expressed as: T = argmaxH(t) After performing adaptive maximum entropy segmentation on the fused saliency image, 8 connected regions are constructed and the area of ​​each connected region is obtained; The final segmented image is obtained by calculating the area of ​​the connected region as a stability measure.

Images