A plate nail position detection method based on cluster detection and feature screening

Abstract

The application provides a plate nail position detection method based on cluster detection and feature screening, and relates to the technical field of image data processing. The method takes a plate nail position detection algorithm in a PC terminal as a core, realizes bidirectional communication of a PLC and the PC terminal through an RS-485 serial port, and transmits relevant parameters of plate processing to the PC terminal in a starting stage to complete initialization of an internal program of the PC terminal. When the detection system starts, an image data of a plate processed on a flow line is captured by a camera and transmitted to the PC terminal, and is detected by the plate nail position detection algorithm, and a result is returned to the PLC. The application can replace a complicated manual detection link, is high in work efficiency and detection precision, and can effectively screen out production defective products with missing nails or missed nails according to plate image information.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for detecting nail positions on board material based on cluster detection and feature screening. Background Technology

[0002] As we all know, panel processing is very common in daily life, and various panel processing plants cannot function without the process of joining and nailing panels together. In today's fully automated production lines, the ability to quickly and accurately inspect finished products has become particularly important. However, due to the small size of individual nails and the varying number in different processing methods, automated inspection faces significant challenges. Currently, many panel processing plants still rely on traditional manual inspection, requiring staff to continuously monitor and inspect finished products. This is not only extremely time-consuming and labor-intensive, but also very inefficient.

[0003] Therefore, it is necessary to design a board nail position detection method based on cluster detection and feature screening to solve the above problems. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method for detecting nail positions in sheet metal based on cluster detection and feature screening.

[0005] A method for detecting nail positions in board materials based on cluster detection and feature screening includes the following steps:

[0006] Step 1: Connect the PLC to the PC via serial port, and use the camera to connect the camera to the PC port via power cable, data cable, and control cable.

[0007] Step 2: Initialize the serial port program on the PC; configure the port used for serial communication, set the port number, and set the baud rate, data format, and parity bit for serial communication;

[0008] Step 3: Use PLC to set the processing plan for the sheet metal on the production line, including the positions where the sheet metal needs to be processed, and the number and shape of nails to be processed at each position; at the same time, transmit the set parameters to the PC through the serial port. After receiving the parameters, the PC will assign them to the variables. If no new parameters are passed in, the previous parameters will be used by default.

[0009] Step 4: After the board processing begins, the boards will be sent to the nail guns one by one via a conveyor belt under the control of the PLC. The finished products will be sent to the camera by the conveyor belt, where the camera will capture images of the finished boards and transmit the captured image data to the PC for inspection.

[0010] Step 5: After receiving the image, the PC will detect it using a board nail position detection algorithm. Based on the detection results, it will determine whether the board is missing nails. The detection results will be transmitted to the PLC via serial port. The PLC will then make a decision on the board's destination. If nails are missing, the board will be transported to the nail replacement area for replacement. If the board is normal, it will be transported to the finished product area.

[0011] The steps of the board nail position detection algorithm are as follows:

[0012] Step S1: Perform grayscale and binarization image preprocessing on the board image, changing all pixel values ​​in the image to 0 and 255, where 0 is black and 255 is white; Step S2: Obtain all connected component information in the binarized image through a connected component extraction algorithm;

[0013] The connected component extraction algorithm is a function from the OpenCV library in Python. The obtained connected component information includes the number of connected components, the length and width of the minimum bounding rectangle corresponding to the connected component (i.e., the maximum length and width of the connected component), and the area of ​​the connected component (i.e., the number of pixels in the connected component).

[0014] Step S3: A cluster detection algorithm is set up to infer the processing position on the board surface based on the extracted connected component information;

[0015] The settlement detection algorithm first stores the x-coordinates and y-coordinates of the centers of all connected regions into two corresponding arrays, arranged in ascending order. It then iterates through the two arrays; if the difference between any two adjacent x-coordinates or y-coordinates is less than a set value, they are considered continuous, and the continuous value is increased; otherwise, the continuous value is reset to zero. When the continuous value exceeds the set threshold, the continuous regions constitute a settlement. The average of all x-coordinates and y-coordinates constituting a settlement is taken as the x-axis and y-axis values ​​of the detection center for that settlement. The recorded x-coordinates and y-coordinates reflect the clustering of anchors on the x-axis and y-axis. The center coordinates of all settlements are calculated by combining the x-coordinates and y-coordinates. The detection area is then divided into a rectangular region centered on the settlement center coordinates.

[0016] Step S4: The feature filtering method is set.

[0017] The feature selection method includes both color constraints and shape constraints.

[0018] The color constraint uses HSV color channels to screen for pin colors. The filtering range of HSV parameters and the image traversal interval are set. Then, the RGB channels of the original image are converted to HSV channels, and pixels that conform to the HSV parameter range are traversed and screened. Pixels that conform to the HSV parameter range will return their corresponding connected component indices and save them to the corresponding HSV array for later use. If the connected component has already been added to the array, it will be skipped to avoid duplicate addition.

[0019] The shape constraints set the number of pixels in the connected components, the maximum length and width of the connected components, and the difference between them. Then, all connected components in the entire image are traversed, and the indices of the connected components that meet the requirements are added to the shape constraint array for later use.

[0020] Step S5: Perform the pin marking algorithm to obtain and mark the pin detection results;

[0021] The nail location marking algorithm, after image preprocessing and cluster detection, yields several rectangular detection regions. Through color and shape constraints in feature filtering, color constraint arrays and shape constraint arrays are also obtained, storing the indices of connected components that meet the color and shape requirements, respectively. Within each rectangular inspection region, this method saves and counts the connected components within the region that simultaneously belong to both the color and shape constraint arrays, serving as the final nail location detection result. Simultaneously, all rectangular inspection regions are marked with red rectangles, and all nails detected within the rectangles are marked with blue circles.

[0022] Step S6: Sending back detection results. After the pin detection results are obtained, the PC will continuously send the detection results to the PLC via the serial port. When the PC receives a greeting message from the PLC, it indicates that the PLC has started receiving data. The PC will then continuously send multiple data frames containing the detection results, and then enter a new detection phase, continuing to wait for parameters and start signals.

[0023] The beneficial effects of adopting the above technical solution are as follows:

[0024] This invention provides a method for nail position detection in sheet metal based on cluster detection and feature selection. This invention eliminates the need for time-consuming algorithms such as edge extraction to perform nail position detection on different types of sheet metal under three-nail gun and five-nail gun conditions. By taking pictures of the processed sheet metal, the method automatically detects the number of nails on both sides of the sheet metal and compares them with preset values ​​to determine if any nails are missing. No manual operation is required during operation, replacing the traditional tedious manual inspection process, saving workers' time and effort, and greatly improving work efficiency. Furthermore, since cluster detection provides a detection area, only this area needs to be inspected, enabling the detection of different sheet metal and different processing methods, greatly improving compatibility, versatility, and detection speed. Attached Figure Description

[0025] Figure 1 This is a flowchart of a board nail position detection method based on cluster detection and feature screening in an example of the present invention.

[0026] Figure 2 This is a diagram of the data frame structure for serial communication between the PLC and PC in an example of the present invention.

[0027] Figure 3 This is a schematic diagram of the PLC's processing position and processing scheme for the sheet metal in an example of the present invention.

[0028] Figure 4 This is a flowchart illustrating the process of the board nail position detection algorithm in an example of the present invention.

[0029] Figure 5 This is a connected component information graph obtained by the connected component extraction algorithm in an example of the present invention.

[0030] Figure 6 This is a flowchart illustrating the process of the plate cluster detection algorithm in this invention.

[0031] Figure 7 This is a flowchart illustrating the process of constraining the color of the board material in an example of the present invention.

[0032] Figure 8 This is a flowchart illustrating the process of constraining the shape of the sheet metal in an example of the present invention.

[0033] Figure 9 This is a flowchart illustrating the process of marking the nail positions on the board in an example of the present invention. Detailed Implementation

[0034] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0035] A method for detecting nail positions in board materials based on cluster detection and feature screening, such as... Figure 1 As shown, it includes the following steps:

[0036] Step 1: Connect the PLC to the PC via serial port, and use the camera to connect the camera to the PC port via power cable, data cable, and control cable.

[0037] In this embodiment, the hardware consists of an S7-1500 PLC, a camera, and a PC. The S7-1500 PLC is connected to the PC via an RS-485 serial port. The camera is connected to the PC port via a power cable, data cable, and control cable. The camera is located above the output port of the production line. After the detection system starts, it is responsible for capturing image information of the finished sheet metal and sending it to the PC via the data cable. This is used for the transmission of sheet metal images.

[0038] Step 2: Initialize the serial port program on the PC; configure the port used for serial communication, set the port number, and set the baud rate, data format, and parity bit for serial communication;

[0039] In this embodiment, a subroutine for serial port data transmission and reception is defined for direct data transmission. A bidirectional communication program is used to handle data exchange between the S7-1500 PLC and the PC, mainly including the reception and transmission of internal data on the PC. When the PLC sets parameters to be transmitted to the PC via the RS-485 serial port, the PLC first sends a start bit to the serial port. When the PC receives the start signal, if no higher-priority program is executing, it executes an interrupt subroutine via a serial port interrupt response to receive data from the serial port. The data frame format conforms to the Modbus protocol, such as... Figure 2 As shown in the image. In the frame format, bit 0 is the station number, bit 1 is the function number, and the last two bits are the high and low bits of the CRC checksum. Specifically, in the transmitting frame, bits 2-3 are the register address of the sending PLC, and bits 4-5 are the data bits; in the receiving frame, bit 2 is the number of data bytes, and bits 3-4 are the transmitted data. The high-order bits represent the number of nails on the upper surface of the board, and the low-order bits represent the number of nails on the lower surface. If both are 0, it is considered a start signal. When the end flag is received, data transmission is complete. The PC verifies the data according to the parity bit set during initialization to ensure error-free reception.

[0040] Step 3: Use PLC to set the processing plan for the sheet metal on the production line, including the positions where the sheet metal needs to be processed, and the number and shape of nails to be processed at each position; at the same time, transmit the set parameters to the PC through the serial port. After receiving the parameters, the PC will assign them to the variables. If no new parameters are passed in, the previous parameters will be used by default.

[0041] In this embodiment, an S7-1500 PLC is used to set the sheet metal processing plan on the production line. Operators can set the distance to be processed on the sheet metal by inputting information on the screen. By clicking on the pin distribution at each location on the screen, the number of pins to be processed and the pin distribution can be selected. The operation interface is as follows: Figure 3 As shown. After the processing parameters are set, they will be loaded into various variables inside the PLC to control the entire sheet metal processing flow. If no new parameters are passed in, the previous parameters will be used by default.

[0042] Step 4: After the board processing begins, the boards will be sent to the nail guns one by one via a conveyor belt under the control of the PLC. The finished products will be sent to the camera by the conveyor belt, where the camera will capture images of the finished boards and transmit the captured image data to the PC for inspection.

[0043] After the PLC sets the processing parameters on the operation interface and sends them to the PC, the sheet metal processing flow begins. First, the sheet metal placed on the pallet is transported to a designated location and stopped. A nail gun above then nails the sheet metal at the designated location according to the processing position parameters; the number and style of nails match the settings on the operation interface. Once that location on the sheet metal surface is processed, the conveyor belt moves the sheet metal so that the unprocessed area is below the nail gun, and the nail gun continues processing the sheet metal. This process is repeated to complete the processing of the entire sheet metal. The finished product is then conveyed to a camera, which captures images of the finished sheet metal and transmits the captured image data to the PC for inspection.

[0044] Step 5: After receiving the image, the PC will detect it using the board nail position detection algorithm. Based on the detection results, it will determine whether the board is missing nails. The detection results will be transmitted to the PLC via RS-485 serial port. The PLC will then make a decision on the board's destination. If nails are missing, the board will be transported to the nail replacement area for replacement. If the board is normal, it will be transported to the finished product area.

[0045] The plate nail position detection algorithm is as follows: Figure 4 As shown, the steps are as follows:

[0046] Step S1: Perform grayscale and binarization image preprocessing on the board image, changing all pixel values ​​in the image to 0 and 255, where 0 is black and 255 is white, to facilitate subsequent feature detection and edge extraction. The threshold for binarization is set to the global adaptive threshold of Otsu's method. This threshold is automatically calculated based on the histogram of the bimodal image, which can significantly reduce binarization differences caused by image factors and achieve better binarization results.

[0047] Step S2: Obtain all connected component information in the binarized image using a connected component extraction algorithm;

[0048] The connected component extraction algorithm is a function from the OpenCV library in Python, which can be called directly. The obtained connected component information includes the number of connected components, the length and width of the minimum bounding rectangle corresponding to the connected component (i.e., the maximum length and width of the connected component), and the area of ​​the connected component (i.e., the number of pixels in the connected component).

[0049] The nail is considered a connected component with a pixel count less than a certain value, which depends on the camera's focal length, fixed position, and the nail's size. Connected component extraction algorithms can then be used to obtain relevant information about all connected components in the binarized image, such as... Figure 5 As shown. This includes the number of connected components, the length and width of the minimum bounding rectangle corresponding to each connected component (i.e., the maximum length and width of the connected component), and the area of ​​the connected component (i.e., the number of pixels in the connected component). Connected component extraction transforms the target of pin detection from all pixels in the image to specific closed regions within the binary image, significantly improving detection efficiency.

[0050] Step S3: A cluster detection algorithm is set up to infer the processing position on the board surface based on the extracted connected component information;

[0051] The settlement detection algorithm is as follows: Figure 6 As shown, firstly, the x-coordinates and y-coordinates of the centers of all connected regions are stored in two corresponding arrays, arranged in ascending order. The two arrays are then iterated through. If the difference between any two adjacent x-coordinates or y-coordinates is less than a set value, they are considered continuous, and the continuous value is increased; otherwise, the continuous value is reset to zero. When the continuous value exceeds a set threshold, the continuous regions form clusters. The average of all x-coordinates and y-coordinates constituting a cluster is taken as the x-axis and y-axis values ​​of the detection center for that cluster. The recorded x-coordinates and y-coordinates reflect the clustering of anchors on the x-axis and y-axis. The center coordinates of all clusters are calculated by combining the x-coordinates and y-coordinates. The detection area is then divided into rectangular regions centered on the cluster center coordinates.

[0052] In subsequent detection, the coordinates of the detection center will be used as a reference to perform feature filtering on a rectangular area of ​​fixed length and width around it. This can effectively avoid interference from similar points in irrelevant areas and greatly reduce the program's running time.

[0053] Step S4: The feature filtering method is set.

[0054] The feature selection method includes both color constraints and shape constraints.

[0055] The color constraint uses the HSV color channel to screen the pin colors. For example... Figure 7 As shown, HSV is a color space created by A.S. Smith in 1978 based on the intuitive characteristics of color. Also known as the hexagonal pyramid model, it is widely used in image processing. The three color parameters H, S, and V represent the hue, saturation, and lightness of an object's color, respectively. In the HSV color space, it is closer to people's perceptual experience of color than RGB, and it is easier to track objects of a certain color. It can very intuitively express the hue, vibrancy, and brightness of a color, facilitating color comparison. The process involves setting the HSV parameter filtering range and the image traversal interval, then converting the original image's RGB channels to HSV channels, and then traversing and screening pixels that conform to the HSV parameter range. Pixels that conform to the HSV parameter range will return their corresponding connected component indices and save them to the corresponding HSV array for later use. If the connected component has already been added to the array, it will be skipped to avoid duplicate addition.

[0056] Shape constraints such as Figure 8As shown, nails on the board surface have a fixed size. When the camera model and location are determined, the pixel size of the nails in the acquired board image is also fixed, which can serve as an effective basis for nail location detection. This method obtains the connected component information of the entire image during connected component extraction, including the number of connected components, the maximum length and width of each connected component, and the number of pixels in each connected component. To efficiently detect nails and filter out noise and irrelevant points that do not meet nail characteristics, it is necessary to limit the number of pixels and the maximum length and width of each connected component: if the number of pixels in a connected component on the board is less than a set threshold, and the maximum length and width and the difference between the maximum length and width of the connected component are both within the specified range, then the connected component is considered a nail; conversely, if any of the pixel count, maximum length and width, or their difference does not meet the requirements, it is considered an irrelevant point on the board. The number of pixels, maximum length and width, and their difference are set. Then, all connected components in the entire image are traversed, and the serial numbers of the connected components that meet the requirements are added to the shape constraint array for subsequent calls.

[0057] Step S5: Perform the pin marking algorithm to obtain and mark the pin detection results;

[0058] The pin marking algorithm, after image preprocessing and cluster detection, can obtain several rectangular detection regions. Through color and shape constraints in feature filtering, we can also obtain color constraint arrays and shape constraint arrays, which store the connected component indices that meet the color and shape requirements, respectively. For example... Figure 9 As shown, within each rectangular inspection area, this method saves and counts the connected components within the inspection area that simultaneously belong to both the color constraint array and the shape constraint array, using this count as the final nail position detection result for the board material. Simultaneously, all rectangular inspection areas are marked with red rectangles, and all nails detected within the rectangles are marked with blue circles.

[0059] Step S6: Sending back detection results. After the pin detection results are obtained, the PC will continuously send the detection results to the PLC via the serial port. When the PC receives a greeting message from the PLC, it indicates that the PLC has started receiving data. The PC will then continuously send multiple data frames containing the detection results, and then enter a new detection phase, continuing to wait for parameters and start signals.

[0060] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A plate nail position detection method based on colony detection and feature screening, characterized in that, The method comprises the following steps: Step 1: connecting the PLC and the PC end through a serial port, and connecting the camera with the PC end port through a power line, a data line and a control line by using a shooting device; Step 2: initializing the serial port program in the PC end, configuring the port used for serial communication, setting the port number, and setting the baud rate, data format and parity bit of the serial communication; Step 3: setting the board processing scheme on the production line by using the PLC, including the positions of the board to be processed and the number and shape of nails to be processed at each position; meanwhile, the set parameters are transmitted to the PC end through the serial port, and the PC end assigns the parameters to variables after receiving the parameters, and the previous parameters are used by default if no new parameters are transmitted; Step 4: after the board processing starts, the board is sent to the nail gun under the control of the PLC through the conveying belt for processing, and the finished product is sent to the camera by the conveying belt for image acquisition, and the acquired image data are transmitted to the PC end for detection; Step 5: after the PC end receives the image, the board nail position detection algorithm is used for detection, and whether the board is missing nails or not is obtained according to the detection result, and the detection result is transmitted to the PLC through the serial port, and the PLC makes a decision on the direction of the board, and if the board is missing nails, the board is transported to the nail supplement area for nail supplement, and if the board is normal, the board is transported to the finished product area; The board nail position detection algorithm in step 5 comprises the following steps: Step S1: the board image is subjected to image preprocessing of gray scale and binary, and all pixel values in the image are changed to 0 and 255, wherein 0 is black and 255 is white; Step S2: all connected domain information in the binary image is obtained through a connected domain extraction algorithm; Step S3: a cluster detection algorithm is set, and the surface processing position of the board is inferred according to the extracted connected domain information; The cluster detection algorithm in step S3 first stores the horizontal coordinates and vertical coordinates of the centers of all connected domains into two corresponding arrays, and arranges the arrays from small to large; two arrays are traversed respectively, if the difference between two adjacent horizontal coordinates or vertical coordinates is less than a set value, it is considered to be continuous, and the continuous value is increased, otherwise the continuous value is cleared; when the continuous value is greater than a set threshold, the continuous region constitutes a cluster, the average value of all horizontal coordinates and vertical coordinates constituting the cluster is taken as the detection center horizontal axis value and the detection center vertical axis value of the cluster, and the recorded horizontal and vertical coordinates reflect the aggregation of the nail position in the horizontal and vertical axes, and the center coordinates of all clusters are calculated by cross combination of the horizontal and vertical coordinates, and the detection region is divided in the rectangular range with the cluster center coordinates as the center; Step S4: a feature screening method is set; Step S5: a nail position marking algorithm is run to obtain and mark the nail position detection result; Step S6: the detection result is returned, after the nail position detection result is obtained, the PC end continuously sends the detection result to the PLC through the serial port; when the PC end receives the greeting information sent by the PLC, it indicates that the PLC has started to receive, and the PC continuously sends multiple data frames containing the detection result on this basis, and then enters new detection and continues to wait for the parameter and start signal.

2. The method according to claim 1, wherein, The connected domain extraction algorithm in step S2 is a function in the opencv library in Python. The obtained connected domain information includes the number of connected domains, the length and width of the minimum bounding rectangle corresponding to the connected domain, i.e., the maximum length and width of the connected domain, and the area of the connected domain, i.e., the number of pixels of the connected domain.

3. The method according to claim 1, wherein, The feature screening method in step S4 is as follows: the feature screening includes color constraint and shape constraint. The color constraint adopts the HSV color channel to screen the nail color. The screening range of the HSV parameter and the traversal interval of the picture are set, then the RGB channel of the original picture is converted into the HSV channel, and the pixel points meeting the HSV parameter range are traversed and screened on this basis. The pixel points meeting the HSV parameter range return the corresponding connected domain serial number and are saved into the HSV corresponding array for calling. If the connected domain has been added to the array, it will be directly skipped to avoid repeated addition. The shape constraint sets the pixel number of the connected domain, the maximum length and width of the connected domain and the difference value. Then, all the connected domains of the whole picture are traversed, and the connected domain serial numbers meeting the requirements are added to the shape constraint array for subsequent calling.

4. The method according to claim 1, wherein, The nail marking algorithm in step S5 is as follows: after image preprocessing and settlement detection, a plurality of rectangular detection regions can be obtained. Through the color constraint and shape constraint in the feature screening, the color constraint array and the shape constraint array can also be obtained, which respectively store the connected domain serial numbers meeting the color requirements and the shape requirements. In each rectangular detection region, the connected domains belonging to both the color constraint array and the shape constraint array are saved and counted as the final board nail detection result. At the same time, all the rectangular detection regions are marked with red rectangles, and all the detected nails in the rectangular regions are marked with blue circles.

Images