Method and Device for Locating and Segmenting Detection Regions of a Chassis Based on Bessel Curves

Abstract

The present invention relates to the field of image segmentation, and provides a method and device for positioning and segmenting the detection area of a casing based on a Bezier curve. The method includes annotating and generating a Bezier curve template, and selecting a detection area according to the Bezier curve template; performing mean filtering and bilateral filtering on the casing image to be detected to obtain image I c ; obtaining the upper and lower boundaries of the casing image according to image I c ; rotating image I c so that the sum of the angles between the upper boundary, the lower boundary of the rotated image I c and the horizontal line is 0, and denoting the rotated image as image I R ; obtaining the weighted edge image of image I R by using a multi-threshold Canny edge detection algorithm; and obtaining the detection area of the casing image to be detected by matching the Bezier curve template with the weighted edge image. The present invention can reduce the workload of annotation, improve the segmentation accuracy, and be applicable to various casing images.

Description

Technical Field

[0001] The present invention relates to the field of image segmentation, and particularly to a method and device for positioning and segmenting a detection area of a casing based on a Bezier curve. Background Art

[0002] In the manufacturing process of consumer electronic products such as mobile phones, the appearance quality of the casing is a very prominent and important part of the mobile phone. Therefore, the quality inspection of the casing is an indispensable part of the production and manufacturing process. In the work of the production line, this work mainly relies on manual inspection, which has problems such as high labor costs, easy fatigue of personnel, and low efficiency. Therefore, an automatic inspection method is urgently needed.

[0003] Although the detection algorithm mainly based on deep learning technology has a certain robustness to misdetecting local images of product structures as defects, when there is a large amount of defect-like information in the non-detection area, such as various assembly structures on the inner surface of the casing that are inevitably captured during the shooting process, it will still have a huge impact on the algorithm, resulting in a large number of missed detections and misdetections. Therefore, it is necessary to first locate and segment the detection area, and limit the detection area on the product surface through area planning.

[0004] Since there are numerous casing models, it is almost impossible to apply one or a limited number of automatic segmentation algorithms to all casing product images. Therefore, manual annotation is required for auxiliary positioning. Currently, manual annotation mainly uses polygon annotation, which requires more annotation points for arc structures and is difficult to accurately achieve annotation. Summary of the Invention

[0005] To solve the technical problems existing in the prior art, a method and device for positioning and segmenting a detection area of a casing based on a Bezier curve are provided. Based on the annotation with a Bezier curve, this method combines image processing technology with Bezier curve annotation and matching to complete the positioning and segmentation of the detection area, which can reduce the workload of annotation, improve the segmentation accuracy, and be applicable to various casing images.

[0006] The first object of the present invention is to provide a method for positioning and segmenting a detection area of a casing based on a Bezier curve.

[0007] The second object of the present invention is to provide a computer device.

[0008] The first object of the present invention can be achieved by adopting the following technical solutions:

[0009] A method for positioning and segmenting a detection area of a casing based on a Bezier curve, the method comprising:

[0010] S1. Annotate and generate a Bezier curve template B, and select a detection area according to the Bezier curve template B;

[0011] S2. Perform mean filtering and bilateral filtering on the image of the housing to be detected to obtain image I c ;

[0012] S3. Obtain the upper and lower boundaries of the housing image according to image I c ;

[0013] S4. Rotate image I c so that the sum of the angles between the upper boundary and the lower boundary of the rotated image I c and the horizontal line is 0. Denote the rotated image as image I R . Use the multi-threshold Canny edge detection algorithm to obtain the weighted edge image E R of image I Q ;

[0014] S5. Obtain the detection area of the housing image to be detected by matching the Ser curve template B with the weighted edge image E Q ;

[0015] In the preferred technical solution, step S1 includes:

[0016] S11. Collect N sample images of the same model and the same angle of the housing;

[0017] S12. Use multiple closed multi-segment Bezier curves to annotate the first sample image to obtain the annotated Bezier curve. Copy the annotated Bezier curve to the remaining sample images and modify the positions of the annotation points; each segment of the Bezier curve needs to be annotated with at least 4 points, including a starting point, an ending point and two control points;

[0018] S13. Average the Bezier curves of the N sample images to generate the final Bezier curve annotation;

[0019] S14. Use the straight line segment that is located at the top and has a length exceeding the preset value T1×W of the final Bezier curve annotation as its upper boundary, and use the straight line segment that is located at the bottom and has a length exceeding the preset value T1×W as the lower boundary;

[0020] S15. Rotate the final Bezier curve annotation so that the sum of the angles between its upper and lower boundaries and the horizontal line is 0 to obtain the Bezier curve template B;

[0021] Step S4 includes:

[0022] S41. Set the high threshold T2 and the low threshold T3 in the Canny edge detection algorithm, and obtain the edge image E1 of image I R ;

[0023] S42. Respectively use 2×T2 and T3 as the high and low thresholds in the Canny edge detection algorithm, and obtain the edge image E2 of image I R ;

[0024] S43. Respectively take 4×T2 and T3 as the high and low thresholds in the Canny edge detection algorithm, and obtain the edge image E3 of the image I R ;

[0025] S44. Perform weighted summation on the edge image E1, the edge image E2, and the edge image E3 to obtain the weighted edge image E Q .

[0026] The second object of the present invention can be achieved by adopting the following technical solutions:

[0027] A computer device includes a processor and a memory for storing programs executable by the processor. When the processor executes the programs stored in the memory, the above-mentioned method for positioning and segmenting the casing detection area based on the Bezier curve is implemented.

[0028] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0029] For the method and device for positioning and segmenting the casing detection area based on the Bezier curve of the present invention, on the basis of using the Bezier curve for annotation, through image processing technology combined with Bezier curve annotation and matching, the positioning and segmentation of the detection area are completed, which can reduce the workload of annotation, improve the segmentation accuracy, and be applicable to various casing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the structures shown in these drawings without creative efforts.

[0031] Figure 1 is a flowchart of the method for positioning and segmenting the casing detection area based on the Bezier curve in the embodiment of the present invention;

[0032] Figure 2 is an effect diagram of the annotation result of using multiple closed multi-segment Bezier curves to annotate the sample image in the embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of the effect of the longitudinal edge map in the embodiment of the present invention;

[0034] Figure 4 is a schematic diagram of the straight line found by the Hough line transformation in the embodiment of the present invention;

[0035] Figure 5Schematic diagram of the edge image E1 of the embodiment of the present invention;

[0036] Figure 6 Schematic diagram of the edge image E2 of the embodiment of the present invention;

[0037] Figure 7 Schematic diagram of the edge image E3 of the embodiment of the present invention;

[0038] Figure 8 Schematic diagram of the detection area effect of the image to be inspected in the embodiment of the present invention. Detailed implementation manners

[0039] Next, the technical solution of the present invention will be further described in detail with reference to the drawings and embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. The implementation manners of the present invention are not limited thereto. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0040] Embodiment 1:

[0041] As Figure 1 shown, the flowchart of the method for positioning and segmenting the detection area of the casing based on the Bessel curve. The method for positioning and segmenting the detection area of the casing based on the Bessel curve according to the present invention includes the steps:

[0042] S1. Mark and generate the Bessel curve template B, and select the detection area according to the Bessel curve template B. The specific steps are as follows:

[0043] S11. Collect N sample images of the same model and the same angle of the casing. In this embodiment, N = 3.

[0044] S12. Mark the first sample image with multiple closed multi-segment Bessel curves to obtain the marked curves, copy the marked curves to the remaining sample images and modify the positions of the marked points to make the curves fit the new casing image better. Among them, each segment of the Bessel curve needs to be marked with at least 4 points, including a starting point, an ending point, and two control points. The remaining sampling points on the curve can be obtained by calculating the Bernstein polynomial. For a closed multi-segment Bessel curve, the ending point of the previous curve segment should be used as the starting point of the next curve segment, and the ending point of the last curve segment should be equal to the starting point of the first curve segment. Only 4 points are needed for the Bessel curve to better represent a curve segment, while polygon marking requires more marked points. Therefore, using the Bessel curve requires less marking work than using polygon marking. The original design curve can be directly transformed into the captured image by methods such as mapping transformation and manual modification, so that the marking work is even less.

[0045] As Figure 2As shown, the annotation result effect diagram of the sample image is obtained by using multiple closed poly-segment Bezier curves.

[0046] The calculation method of the Bezier curve based on the Bernstein form is as follows:

[0047]

[0048] Among them, B(t) is the coordinate of the sampling point at position t, P0 is the starting point coordinate, P1 and P2 are the control point coordinates, P3 is the ending point coordinate, t ∈ [0, 1] is the normalized position of the sampling point in the line segment, 0 represents the position where the starting point is located, and 1 represents the position where the ending point is located.

[0049] S13. Average the Bezier curves of N sample images to generate the final Bezier curve annotation. Specifically, for the same segment of Bezier curve in all images, taking its respective starting point as the origin, obtain the coordinate averages of its respective control points and the ending point. Finally, the averaged control points and the ending point together with the starting point form the final Bezier curve annotation.

[0050] S14. Use the straight line segment that is at the topmost and has a length exceeding the preset value T1×W of the final Bezier curve annotation as its upper boundary, and the straight line segment that is at the bottommost and has a length exceeding the preset value T1×W as the lower boundary. Where W represents the width of the image, and T1 ∈ [0, 1] is the preset threshold. In this embodiment, T1 = 0.125.

[0051] S15. Rotate the final Bezier curve annotation so that the sum of the angles between its upper and lower boundaries and the horizontal line is 0, to obtain the Bezier curve template B. The rotated curve is the Bezier curve template B, and the detection area is selected according to the Bezier curve template B.

[0052] S2. Perform mean filtering and bilateral filtering on the image of the casing to be detected to reduce noise and obtain the preprocessed image I c .

[0053] First, use mean filtering with a kernel of (3, 3) to remove the overall noise of the image, and then use bilateral filtering to remove more noise while preserving the edges. In this embodiment, the bilateral filtering kernel size is 5, the gray variance is 15, and the spatial variance is 15.

[0054] S3. Obtain the upper and lower boundaries of the casing image, and the specific steps are as follows:

[0055] S31. Use the Sobel operator in the vertical direction (3×3) to convolve with the image I c to obtain the longitudinal gradient map I y .

[0056] Using the Sobel operator to calculate the gradient is a classic method in image processing. The Sobel operator is:

[0057]

[0058] S32. Use the threshold T y to binarize the vertical gradient map I y to obtain the vertical edge map I B , as Figure 3 shown, the schematic diagram of the vertical edge map effect. Specifically, the pixel value of the binary image I B at the coordinate (x, y) is:

[0059]

[0060] S33. Use the Hough line transform to find the lines in the vertical edge map I B whose included angle with the horizontal line is in the range of [-α, α] and the cumulative length passing through the pixel points with a value of 1 exceeds the preset value T1×W. As Figure 4 shown, the schematic diagram of the lines found by the Hough line transform. In this embodiment, α = 3°. The Hough line transform is a classic method in image processing. Since the lines in the edge map may be disconnected, here it actually refers to the lines whose cumulative length exceeds the preset value T1×W.

[0061] S34. Traverse all the found lines, and find the ordinate y of the intersection point of each line with the left edge of the image l，i and the ordinate y of the intersection point of each line with the right edge of the image r，i , initialize the upper boundary list L t , and put the line with the minimum y l，i into the list L t . Where i represents the line number.

[0062] S35. Traverse the remaining lines. If the ordinate y of the intersection point of a line with the right edge of the image r，i is less than the ordinate y r，S , then put this line into the list L t . Calculate the average of all the ordinates y t and the ordinates y l，i in the list L l，i respectively to obtain the average intersection points of these lines with the left and right edges of the image, and use the line passing through these two average intersection points as the upper boundary of the casing image; where the ordinate y r，S represents the ordinate of the intersection point of the line numbered S with the right edge, and S represents the number of the line with the minimum y l，i .

[0063] S36. Initialize the lower boundary list L b , and put the line with the maximum y l，i into the list L b . Traverse the remaining lines. If y r，i> y r，X If so, put this straight line into the list L b In the list L b All the y in it l，i And y r，i Find the average respectively to obtain the average intersection points of these straight lines with the left and right edges of the image, and take the straight line passing through these two average intersection points as the lower boundary of the chassis image; among them, y r，X Represents the ordinate of the intersection point of the straight line numbered X with the right edge, and X represents y l，i The number of the straight line with the largest value.

[0064] S4. Rotate the image I c So that the sum of the angles between its upper and lower boundaries and the horizontal line is 0, and denote the rotated image as I R Use the multi-threshold Canny edge detection algorithm to obtain the weighted edge image E of the image I R Q .

[0065] Simply put, it is to perform Canny edge detection with different thresholds to obtain different edges, and then obtain the final edge through weighted summation. The specific steps are as follows:

[0066] S41. Set the high threshold T2 and low threshold T3 in the Canny edge detection algorithm, and obtain the edge image E1 of the image I R As shown in Figure 5 The schematic diagram of the edge image E1. In this embodiment, T2 = 10 and T3 = 4.

[0067] S42. Respectively use 2×T2 and T3 as the high and low thresholds in the Canny edge detection algorithm, and obtain the edge image E2 of the image I R As shown in Figure 6 The schematic diagram of the edge image E2.

[0068] S43. Respectively use 4×T2 and T3 as the high and low thresholds in the Canny edge detection algorithm, and obtain the edge image E3 of the image I R As shown in Figure 7 The schematic diagram of the edge image E3.

[0069] S44. Perform weighted summation on the edge image E1, the edge image E2, and the edge image E3 to obtain the weighted edge image E Q . Preferably, the formula for weighted summation is:

[0070] E Q = 0.1E1 + 0.3E2 + 0.6E3

[0071] E Q (x, y) is the weighted edge image E Q ​The value at coordinates (x, y), where x is the abscissa and y is the ordinate.

[0072] The lower the high threshold, the more false edges (edges caused by noise) are detected. Therefore, its weight should be set slightly smaller.

[0073] S5. By matching the Bezier curve template B with the weighted edge image E Q to obtain the detection area of the housing image to be detected, the specific steps are as follows:

[0074] S51. Scale the Bezier curve template B so that the straight lines of the upper and lower boundaries of the Bezier curve template B are aligned with the upper and lower boundary straight lines of the image I R ;

[0075] S52. Slide the aligned Bezier curve template B along the horizontal axis in the image E Q for search and matching, and calculate the correlation function value R. The specific method is as follows: Align the left boundary of the Bezier curve template B with the left edge of the image I R , and gradually move the Bezier curve template B to the right from this position until the right boundary of the Bezier curve template B reaches the right edge of the image I R . At each position, perform horizontal scaling at the scaling scales of [0.95, 0.975, 1, 1.025, 1.05], calculate the weighted weights of all positions where the Bezier curve template B coincides, and the correlation function value R is equal to the sum of the weighted weights e(x, y) of all positions where the Bezier curve template B coincides. The larger this value, the higher the fitting degree of the Bezier template B to the image to be detected. The calculation formula of the correlation function value R is as follows:

[0076]

[0077]

[0078] where e(x, y) represents the weighted weight of the weighted edge image E Q at coordinates (x, y). In the formula, ⊙ represents element multiplication at the corresponding positions, max represents finding the maximum value of the matrix, and E Q (,) is the value of the weighted edge image E Q at coordinates (,).

[0079] S53. Take the position and scaling coefficient corresponding to the maximum correlation function value R as the optimal position of the Bezier template B, and determine the detection area of the image to be detected according to the detection area at the optimal position of the Bezier template B. As Figure 8 shown, it is a schematic diagram of the detection area effect of the image to be detected in the embodiment of the present invention.

[0080] In summary, based on the use of Bessel curves for annotation, the present invention combines image processing technology with Bessel curve annotation and matching to complete the positioning and segmentation of the detection area, which can reduce the annotation workload, improve the segmentation accuracy, and be applicable to various chassis images. A Bessel curve can better represent a curve with only 4 points, while polygon annotation requires more annotation points. Therefore, the annotation workload required for using Bessel curves is less than that for using polygon annotation. The original design curve can be directly transformed into the captured image by methods such as mapping transformation and manual modification, resulting in an even smaller annotation workload. The present invention performs segmentation through the method of manual annotation plus registration, has no requirements for the appearance features of the chassis, and can be applicable to various chassis images.

[0081] Embodiment 2:

[0082] This embodiment provides a computer device, which can be a server, a computer, etc. It includes a processor, a memory, an input device, a display, and a network interface connected through a system bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the processor executes the computer program stored in the memory, it implements the method for positioning and segmenting the chassis detection area based on Bessel curves in Embodiment 1 above, as follows:

[0083] Annotate and generate a Bessel curve template B, and select the detection area according to the Bessel curve template B;

[0084] Perform mean filtering and bilateral filtering on the chassis image to be detected to obtain image I c ;

[0085] According to image I c Obtain the upper and lower boundaries of the chassis image;

[0086] Rotate image I c So that the sum of the angles between the upper boundary and the lower boundary of the rotated image I c and the horizontal line is 0. Denote the rotated image as image I R , and use the multi-threshold Canny edge detection algorithm to obtain the weighted edge image E R of image I Q ;

[0087] By matching the Bessel curve template B and the weighted edge image E Q , obtain the detection area of the chassis image to be detected.

[0088] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A method for positioning and segmenting the detection area of a casing based on Bessel curves, characterized in that, It includes the following steps: S1. Mark and generate a B-spline curve template B, and select a detection area according to the B-spline curve template B; The step S1 includes: S11. Collect N sample images of the same model and the same angle of the casing; S12. Use multiple closed polyline B-spline curves to mark the first sample image to obtain the marked B-spline curve, copy the marked B-spline curve to the remaining sample images and modify the position of the marked points; at least 4 points need to be marked for each segment of the B-spline curve, including a starting point, an ending point and two control points; S13. Average the B-spline curves of the N sample images to generate a final B-spline curve annotation; S14. Use the straight line segment at the uppermost position and with a length exceeding the preset value T1×W in the final B-spline curve annotation as its upper boundary, and use the straight line segment at the lowermost position and with a length exceeding the preset value T1×W as the lower boundary; S15. Rotate the final B-spline curve annotation so that the sum of the angles between its upper and lower boundaries and the horizontal line is 0 to obtain the B-spline curve template B; S2. Perform mean filtering and bilateral filtering on the image of the casing to be detected to obtain image I c ; S3. Obtain the upper and lower boundaries of the chassis image according to the image I c Obtain the upper and lower boundaries of the chassis image; S4. Rotate the image I c such that the sum of the angles between the upper boundary and the horizontal line and the lower boundary and the horizontal line of the rotated image I c is 0, and denote the rotated image as image I R , and obtain the weighted edge image E of image I R using the multi-threshold Canny edge detection algorithm Q ; S5. By matching the Serret curve template B with the weighted edge image E Q to obtain the detection area of the housing image to be detected; The step S5 includes: S51. Scale the B-spline curve template B to align the straight lines of the upper and lower boundaries of the B-spline curve template B with the straight lines of the upper and lower boundaries of the image I R ; S52. Slide the aligned B-spline curve template B along the horizontal axis in the weighted edge image E Q for search and matching, and calculate the correlation function value R. The calculation formula of the correlation function value R is as follows: R = ∑ (x,y)∈B e(x, y), where, e(x, y) represents the weighted edge image E Q is the weighted weight at the coordinate (x, y), where ⊙ represents element-by-element multiplication at the corresponding positions, and max represents finding the maximum value of the matrix, E Q (,) is the weighted edge image E Q at the value at the coordinate (,); S53. Use the position and scaling factor corresponding to the maximum relevant function value R as the optimal position of the B-spline template B, and determine the detection area of the image to be inspected according to the detection area of the optimal position of the B-spline template B; The aligned Bessel curve template B in the image E Q is slid along the horizontal axis for search and matching, including the steps: Align the left boundary of the Bessel curve template B with the image I R left edge and gradually move the Bessel curve template B to the right until the right boundary of the Bessel curve template B reaches the image I R right edge, perform horizontal scaling at a preset scaling scale at each position, and calculate the weighted weights at all positions where the Bessel curve template B coincides.

2. The method for positioning and segmenting the casing detection area based on the Bessel curve according to claim 1, wherein The step S3 includes: S31. Convolve the image I with a vertical 3×3 Sobel operator c to obtain a vertical gradient map I y ; S32. Use the threshold T y to binarize the vertical gradient map I y to obtain the vertical edge map I B ; S33. Use the Hough line transform to find the vertical edge map I B A straight line in which the included angle with the horizontal line in is within the preset angle range and the cumulative length passing through the pixel points with a value of 1 exceeds the preset value T1×W; S34. Traverse all the lines found in step S33, and calculate the ordinate y of the intersection point of the line and the left edge of the image l,i , and the ordinate y of the intersection point of the line and the right edge of the image r,i . Initialize the upper boundary list L t . Put the line with the minimum ordinate y l,i into the list L t ; S35. Traverse the remaining lines. When the ordinate y of the intersection point of the line and the right edge of the image r,i is less than the ordinate y r,S , then put this line into the list L t . Calculate the average of all the ordinates y t in the list L l,i and the ordinate y l,i respectively, to obtain the average intersection points of these lines with the left and right edges of the image. The line passing through these two average intersection points is used as the upper boundary of the casing image. Among them, the ordinate y r,S represents the ordinate of the intersection point of the line numbered S and the right edge, and S represents the number of the line with the smallest ordinate y l,i . S36. Initialize the lower boundary list L b , put the ordinate y l,i of the maximum straight line into the list L b ; traverse the remaining straight lines. When the ordinate y r,i is greater than the ordinate y r,X , then put this straight line into the list L b . Calculate the average of all the ordinates y b in the list L l,i and the ordinate y r,i respectively, to obtain the average intersections of these straight lines with the left and right edges of the image. The straight line passing through these two average intersections is used as the lower boundary of the chassis image; where y r,X represents the ordinate of the intersection of the straight line numbered X with the right edge, and X represents the number of the straight line with the largest ordinate y l,i .

3. The method for positioning and segmenting the casing detection area based on the Bessel curve according to claim 2, wherein The preset angle is [-3°, 3°].

4. The method for positioning and segmenting the casing detection area based on the Bessel curve according to claim 1 or 2, characterized in that, In the preset value T1×W in the step, T1∈[0, 1], and W represents the width of the image.

5. The method for positioning and segmenting the casing detection area based on the Bessel curve according to claim 1, wherein The step S4 includes: S41. Set the high threshold T2 and low threshold T3 in the Canny edge detection algorithm, and obtain the edge image E1 of the image I R ; S42. Respectively take 2×T2 and T3 as the high and low thresholds in the Canny edge detection algorithm, and obtain the edge image E2 of the image I R ; S43. Respectively take 4×T2 and T3 as the high and low thresholds in the Canny edge detection algorithm, and obtain the edge image E3 of the image I R ; S44. Perform weighted summation on the edge images E1, E2, and E3 to obtain a weighted edge image E Q .

6. The method for positioning and segmenting the detection area of the casing based on the B-spline curve according to claim 5, wherein, The high threshold T2 = 10, and the low threshold T3 = 4; The formula for weighted summation of the edge image E1, the edge image E2, and the edge image E3 is: E Q = 0.1E1 + 0.3E2 + 0.6E3.

7. A computer device, comprising a processor and a memory for storing processor-executable programs, characterized in that, When the processor executes the program stored in the memory, it implements the method for positioning and segmenting the detection area of the casing based on the B-spline curve according to any one of claims 1-6.

Images