Method for detecting line card on surface of pavement mechanical structural member

Abstract

The invention discloses a pavement mechanical structural member surface line card detection method, which comprises the following steps of: 1, acquiring an image; a second step of target detection; 3, image analysis: detecting all the line cards by adopting a YOLOv8 model to obtain line card ROI regions; the image corresponding to the ROI area of the line card is processed by the following specific operation steps: step 31, OTSU binarization processing is carried out on the image corresponding to the ROI area of the line card; a 32nd step of filling the narrower discontinuities using a morphological expansion operation; step 33, extracting a line card skeleton of a single pixel edge by using morphological refining operation; step 34, performing vertex detection; step 35, performing closed region detection; and 36, carrying out accurate fitting on the line card. The method is based on the computer vision and deep learning technology, and the number, the position and the orientation of the line cards on the surface of the structural part are accurately detected.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and deep learning, and in particular to a method for detecting surface line cards of pavement mechanical structural parts. Background Art

[0002] See Figure 1 Structural parts are important parts of road machinery, such as bases, pallets and brackets that play a supporting role. A certain number of wire clips are usually manually welded on the surface of structural parts, and the wire clips are used to fix the wire harness. The number, position and direction of the wire clips have an important impact on the fixing effect of the wire harness. When manually welding wire clips, there may be leakage, wrong position or direction welding, etc. Therefore, in the incoming material inspection of structural parts, the number, position and direction of the wire clips are an important inspection content. Traditional manual-based wire card quality inspection has problems such as low efficiency and easy errors. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art.

[0004] To this end, the present invention proposes a method for detecting line cards on the surface of pavement mechanical structural parts, which accurately detects the number, position and orientation of line cards on the surface of structural parts based on computer vision and deep learning technology.

[0005] The method for detecting line cards on the surface of a pavement mechanical structure according to an embodiment of the present invention comprises the following steps: Step 1, image acquisition: using an image acquisition module to obtain a surface image of the structure to be inspected; Step 2, target detection: use the image processing module to denoise the image; Step 3: Image analysis: Use the YOLOv8 model to detect all line cards. For each detected line card rectangular area, expand the top, bottom, left, and right sides. pixels, is greater than or equal to A positive integer is obtained to obtain the line card ROI area; the image corresponding to the line card ROI area is processed as follows. The specific operation steps are: Step 31: First, perform OTSU binarization processing on the image corresponding to the line card ROI area to obtain an OTSU binarization image. The outline of the line card after binarization has discontinuities. Step 32: Then, a morphological dilation operation is used to fill the narrower gaps; Step 33: Next, a morphological thinning operation is used to extract a wire card skeleton of a single pixel edge, wherein the wire card skeleton includes a wire card body formed by a closed curve and a wire card end formed by an extended straight line; Step 34: Next, vertex detection is performed, and the line card skeleton is analyzed for connected domains to locate the vertices; Step 35: Next, perform closed area detection. The closed area is the line card body. Starting from the vertex at the end of the line card, traverse along the connectivity relationship. According to the number of neighbors of each pixel, all pixels between the vertex and the closed point are obtained. The closed point is the intersection of the line card body and the end of the line card. All pixels between the vertex and the closed point are deleted, and the closed point is retained, so as to obtain a closed curve with the closed point as the starting point. Step 36: Finally, the line card body is accurately fitted, and the minimum circumscribed rectangle is fitted to the closed area to accurately obtain the line card body area, wherein the center point of the rectangle represents the position of the line card body, and the direction represented by the long side of the rectangle is the direction of the line card body.

[0006] The beneficial effect of the present invention is that the number, position and orientation of the wire clips on the surface of the structural part are accurately detected by using computer vision technology and deep learning technology, so as to achieve accurate fitting of the wire clips.

[0007] According to one embodiment of the present invention, in the first step, the image acquisition module includes a lighting system and an industrial camera, and the surface image of the structure to be inspected is obtained through the cooperation of the light source and the camera.

[0008] According to one embodiment of the present invention, a lighting system is used to supplement light on the surface of the structural part, a CCD camera is used to photograph the surface of the structural part, and the photographed surface image of the structural part is transmitted to a line card detection system of a computer.

[0009] According to an embodiment of the present invention, in the second step, bilateral filtering is used to perform denoising on the image.

[0010] According to one embodiment of the present invention, the calculation formula of the bilateral filtering operation is:

[0011]

[0012] The symbols in the above formula mean: Indicates the coordinate point of the current convolved pixel; Indicates the range pixel coordinate points; Represents the value calculated by two Gaussian functions; represents the smoothing parameter.

[0013] According to one embodiment of the present invention, in step 32, a kernel size of A morphological dilation operation is used to fill narrower gaps.

[0014] According to one embodiment of the present invention, in the step 34, the line card skeleton is Connected domain analysis of neighborhoods.

[0015] According to an embodiment of the present invention, in the third step, for each detected line card rectangular area, the upper, lower, left and right areas are expanded. pixels to obtain the line card ROI area.

[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a schematic diagram of the structure of the structural parts and line cards; Figure 2 is a flow chart of the present invention; Figure 3 This is a schematic diagram of the YOLOv8 model detecting all line cards; Figure 4 It is a schematic diagram of the line card ROI area; Figure 5 It is the image corresponding to the line card ROI area; Figure 6 It is the OTSU binary image; Figure 7 It is a schematic diagram of morphological expansion; Figure 8 is a schematic diagram of morphological refinement; Fig. 9 It is a schematic diagram of vertex detection; Fig.10 is a schematic diagram of closed area detection; Fig.11 It is a schematic diagram of the precise fitting of the line card; Fig.12 It is an abstract graph of the line card skeleton for connected domain analysis; Fig.13 It is a schematic diagram of the structure of the line card skeleton. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] The following describes in detail the surface wire card detection method of a pavement mechanical structure component according to an embodiment of the present invention with reference to the accompanying drawings.

[0022] See Figure 2 The surface line card detection method of the pavement mechanical structure of the present invention comprises the following steps: Step 1, image acquisition: Use the image acquisition module to obtain the surface image of the structure to be inspected; the image acquisition module includes a lighting system and an industrial camera, and the light source and the camera work together to obtain the surface image of the structure to be inspected. Use the lighting system to fill in the light on the surface of the structure, use the CCD camera to shoot the surface of the structure, and transfer the captured surface image of the structure to the computer's line card detection system.

[0023] Step 2, target detection: Use the image processing module to denoise the image; the image processing module mainly performs image denoising. Affected by the complex light environment on site, the image of the structural parts has certain noise interference, and image preprocessing operations are required. Bilateral filtering considers both spatial information and grayscale similarity to achieve edge-preserving denoising. Compared with mean filtering, median filtering and Gaussian filtering, bilateral filtering retains the edge characteristics of the image while denoising.

[0024] Preferably, bilateral filtering is used to denoise the image. The calculation formula for the bilateral filtering operation is:

[0025]

[0026] The symbols in the above formula mean: Indicates the coordinate point of the current convolved pixel; Indicates the range pixel coordinate points; Represents the value calculated by two Gaussian functions; Represents the smoothing parameter. In the flat area of ​​the image, the pixel value changes very little, and Gaussian blur is performed; in the edge area of ​​the image, the pixel value changes greatly, and the edge information is retained.

[0027] Step 3, Image Analysis: Use the YOLOv8 model to detect all line cards, such as Figure 3 As shown in the figure. Since the line card has an irregular shape and different orientations, and YOLOv8 uses a rectangular frame to locate the line card, it is impossible to accurately fit the position and orientation of the line card. Therefore, it is necessary to perform further image processing and analysis on each line card to accurately obtain the position and orientation of the line card. It should be noted that ① the YOLOv8 model uses a rectangular frame to detect a single line card. There are several line cards corresponding to several rectangular frames. ② The size of the rectangular area of ​​each line card may be different, generally at least If the camera resolution is increased, the size of the line card will also increase.

[0028] For each line card rectangular area detected by the YOLOv8 model, expand it up, down, left, and right pixels, is greater than or equal to A positive integer is obtained to obtain the line card ROI (Region of Interest) area; preferably, for each line card rectangular area detected by the YOLOv8 model, the top, bottom, left and right are expanded pixels, and get the line card ROI area, such as Figure 4 shown.

[0029] like Figure 5 As shown, the image corresponding to the line card ROI area is processed as follows. The specific steps are: Step 31: Figure 6 As shown, first, the image corresponding to the line card ROI area is subjected to OTSU (Otsu method) binarization processing to obtain an OTSU binarization image. The outline of the line card after binarization has discontinuities; Step 32: Figure 7 As shown, a morphological dilation operation is then used to fill the narrower discontinuities; preferably, a kernel size of A morphological dilation operation is used to fill narrower gaps.

[0030] Step 33: Figure 8 As shown, the morphological thinning operation is then used to extract the line card skeleton of the single pixel edge (Zhang-Suen skeleton extraction algorithm). The line card skeleton includes a line card body composed of a closed curve and a line card end composed of an extended straight line (see Fig.13 ); That is, the line card skeleton has a closed curve (such as Fig.13 In addition to the green closed curve in the figure, there are also redundant line card ends (such as Fig.13 The blue straight line in the figure needs to be removed to obtain the real line card body area so as to accurately fit the line card.

[0031] Step 34: Fig. 9 As shown, then, vertex detection is performed, and the line card skeleton is analyzed for connected domains to locate the vertices; preferably, the line card skeleton is analyzed for The connected domain analysis of the neighborhood can be abstracted as Fig.12 The shape shown in the figure, where the dots represent skeleton pixels and the lines between the dots represent connectivity. Obviously, the vertices represented by the red dots are only neighborhood pixels, and other pixels have at least There are neighborhood pixels, so the vertex can be located by traversing the connected domain of the line card skeleton.

[0032] Step 35: Fig.10 As shown, next, the closed area detection is performed. The closed area is the line card body. Starting from the vertex at the end of the line card, traverse along the connectivity relationship. According to the number of neighbors of each pixel point, all the pixels between the vertex and the closed point are obtained. The closed point is the intersection of the line card body and the end of the line card. All the pixels between the vertex and the closed point are deleted, and the closed point is retained, so a closed curve with the closed point as the starting point is obtained; specifically, as Fig.12 As shown, the closed points represented by blue are Neighborhood pixels, the vertices represented by red have Neighborhood pixels, pixels between vertices and closed points have Therefore, starting from the vertex and traversing along the connectivity relationship, all the pixels between the vertex and the closed point can be obtained according to the number of neighbors of each pixel. All the pixels between the vertex and the closed point are deleted, and the closed point is retained, then a closed curve with the closed point as the starting point can be obtained.

[0033] Step 36: Fig.11 As shown, finally, the line card body is accurately fitted, and the minimum circumscribed rectangle is fitted to the closed area, so that the line card body area can be accurately obtained, where the center point of the rectangle represents the position of the line card body, and the direction represented by the long side of the rectangle is the direction of the line card body (from the short side of the rectangle closest to the vertex to the short side of another rectangle).

[0034] In addition, it is necessary to add that: Fig. 9Vertex detection is to determine the starting point of the end of the line card shown in red. Starting from the vertex, traverse the pixels along the connectivity relationship until a closed point is encountered. The pixels between the vertex and the closed point are the ends of the line card that need to be deleted. After deleting the end of the line card, a closed curve is obtained, and then the minimum circumscribed rectangle is fitted to the closed area. The reason why the end of the line card is deleted here is because it will interfere with the fitting of the minimum circumscribed rectangle, resulting in inaccurate position and direction of the line card. Another function of vertex detection is to help determine the direction of the line card. The circumscribed rectangle fitted by the line card, the long side represents the direction, but a straight line can have two directions, and the direction of the line card is from the end (the end of the straight line) to the top (the end of the curve). See Fig.11 , the red rectangle is the minimum circumscribed rectangle fitting of the closed area (white curve), the blue dot is the center point of the rectangle, representing the position of the line card, and the red arrow represents the direction of the line card (you only need to know the direction of the line card body). The line card is used to fix the wire harness. The direction of the line card needs to meet the requirements of the wire harness routing and cannot be welded at will. This system only detects the number and direction of the line card and returns it to the upper-level application. In the upper-level application, it is also necessary to determine whether the direction of the line card is appropriate based on the specific requirements of the routing.

[0035] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for detecting line cards on the surface of a pavement mechanical structure, characterized in that: The following steps are involved: Step 1, image acquisition: using an image acquisition module to obtain a surface image of the structure to be inspected; Step 2: Target detection: Use the image processing module to denoise the image; Step 3: Image analysis: Use the YOLOv8 model to detect all line cards. For each detected line card rectangular area, expand the top, bottom, left, and right sides. pixels, is greater than or equal to A positive integer is obtained to obtain the line card ROI area; the image corresponding to the line card ROI area is processed as follows. The specific operation steps are: Step 31: First, perform OTSU binarization processing on the image corresponding to the line card ROI area to obtain an OTSU binarization image. The outline of the line card after binarization has discontinuities. Step 32: Then, a morphological dilation operation is used to fill the narrower gaps; Step 33: Next, a morphological thinning operation is used to extract a wire card skeleton of a single pixel edge, where the wire card skeleton includes a wire card body formed by a closed curve and a wire card end formed by an extended straight line; Step 34: Next, vertex detection is performed, and the line card skeleton is analyzed for connected domains to locate the vertices; Step 35: Next, perform closed area detection. The closed area is the line card body. Starting from the vertex at the end of the line card, traverse along the connectivity relationship. According to the number of neighbors of each pixel, all pixels between the vertex and the closed point are obtained. The closed point is the intersection of the line card body and the end of the line card. All pixels between the vertex and the closed point are deleted, and the closed point is retained, so as to obtain a closed curve with the closed point as the starting point. Step 36: Finally, the line card body is accurately fitted, and the minimum circumscribed rectangle is fitted to the closed area to accurately obtain the line card body area, wherein the center point of the rectangle represents the position of the line card body, and the direction represented by the long side of the rectangle is the direction of the line card body.

2. The surface line card detection method of road mechanical structural parts according to claim 1 is characterized in that: In the first step, the image acquisition module includes a lighting system and an industrial camera, and the surface image of the structure to be inspected is obtained through the cooperation of the light source and the camera.

3. The surface line card detection method of road mechanical structural parts according to claim 2 is characterized in that: The surface of the structural part is supplemented with light by a lighting system, and the surface of the structural part is photographed by a CCD camera, and the photographed surface image of the structural part is transmitted to the line card detection system of the computer.

4. The surface line card detection method of road mechanical structural parts according to claim 1 is characterized in that: In the second step, bilateral filtering is used to perform denoising on the image.

5. The method for detecting line cards on the surface of road mechanical structural parts according to claim 4, characterized in that: The calculation formula for the bilateral filtering operation is: The symbols in the above formula mean: Indicates the coordinate point of the currently convolved pixel; Indicates the range pixel coordinate points; Represents the value calculated by two Gaussian functions; represents the smoothing parameter.

6. The surface line card detection method of road mechanical structure parts according to claim 1 is characterized in that: In the step 32, the kernel size is A morphological dilation operation is used to fill narrower gaps.

7. The method for detecting line cards on the surface of road mechanical structural parts according to claim 1, characterized in that: In the 34th step, the line card skeleton is Connected domain analysis of neighborhoods.

8. The method for detecting line cards on the surface of road mechanical structural parts according to claim 1, characterized in that: In the third step, for each detected line card rectangular area, the upper, lower, left and right areas are expanded pixels to obtain the line card ROI area.

Images