A relay surface detection system based on machine vision technology

Abstract

The present invention belongs to the technical field of component detection, and specifically discloses a relay surface detection system based on machine vision technology. The system includes a light source information detection module, a relay preliminary detection module, a detection adjustment judgment module, a detection adjustment confirmation module, a detection adjustment execution terminal, a relay detection analysis module, and a relay detection feedback terminal. By setting up the light source information detection module to monitor the light intensity in real time, the present invention ensures a stable detection environment. The relay preliminary detection module collects and stitches images to provide comprehensive data. The detection adjustment judgment module determines the adjustment requirements based on the images and light data. The detection adjustment confirmation module clarifies the adjustment items and indicators to achieve intelligent and accurate adjustment. The relay detection analysis module deeply analyzes the collected images and outputs surface feature indicators. The relay detection feedback terminal feeds back the indicators to production management personnel to assist in quality control and improve the detection accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of component detection, and more specifically, relates to a relay surface detection system based on machine vision technology. Background Art

[0002] As a commonly used key component in an electrical control system, the surface quality of a relay directly affects the performance and reliability of the product. Traditional relay surface detection methods mainly rely on manual visual inspection, which is inefficient, and the detection accuracy is easily affected by human factors, making it difficult to meet the high requirements for quality inspection in modern industrial production. With the continuous development of machine vision technology, applying it to relay surface detection has become an efficient and accurate detection method.

[0003] An existing technology, such as a real-time detection system for surface components of a PCB disclosed in a Chinese invention patent application with the application number 201911046408.1, includes: a PCB conveyor belt, an infrared sensor, a camera, an industrial control computer, a relay, and an alarm device. Among them, the infrared sensor is configured to detect the PCB board on the PCB conveyor belt and send a detection signal to the industrial control computer. The industrial control computer is configured to control the PCB board conveyor belt to stop conveying based on the detection signal, send an instruction to the camera to collect a PCB image, receive and process the collected image information, and in response to the processing result indicating that there is a problem with the surface components of the PCB, control the alarm device to alarm through the relay. Thus, compared with traditional detection methods, it has many advantages such as accurate fault location accuracy, high detection efficiency, and the ability to give a real-time alarm, thereby realizing the efficient detection of the PCB board.

[0004] Another existing technology, such as a visual detection system and its detection method for micro-defects on the surface of a ring-shaped precision part disclosed in a Chinese invention patent application with the application number 201910160191.0, includes a light-shielding box and a conveying platform for placing the part to be detected. The conveying platform penetrates through the light-shielding box. Inside the light-shielding box, there are a lighting module, a visual detection module, and a control module. The lighting module includes more than one light source. The visual detection module includes an image acquisition device, a polarizing lens, detection software, and a processor. The control module includes a single-chip microcomputer, a relay, and a motor. The relay is connected to the light source, and the motor is connected to the polarizing lens. Thus, the automatic detection, classification, and estimation of the depth of micro-defects on the surface of the ring-shaped precision part are realized, effectively reducing the adverse effects of the part's own texture and micro-scratches on defect detection, improving the detection efficiency, reducing the defective rate of precision parts, and reducing the working intensity.

[0005] In view of the above technical solutions, obviously, there are still the following deficiencies in the surface detection of components such as relays: 1. The detection pertinence is insufficient. Currently, the influence of various information such as component shape and surface characteristics on the detection results is not considered, and a targeted detection strategy is not designed, resulting in certain deficiencies in the effectiveness and reliability of the detection results.

[0006] 2. The light source is a key factor in the image acquisition process. In the actual production environment, the light intensity, angle, and uniformity may change. Currently, the environmental considerations are few, which affects the accurate extraction of surface features by image processing and analysis algorithms, ultimately resulting in certain deviations in the accuracy and effectiveness of the detection results, and also affecting the detection efficiency. Summary of the Invention

[0007] In view of this, to solve the problems raised in the above background technology, a relay surface detection system based on machine vision technology is proposed.

[0008] The object of the present invention can be achieved by the following technical solutions: The present invention provides a relay surface detection system based on machine vision technology, and the system includes: a light source information detection module, which is used to start the light intensity sensors arranged at each light source detection point in the corresponding quality inspection area on the production line to perform real-time detection of the light intensity, and obtain the real-time detected light intensity of each light source detection point.

[0009] A relay preliminary detection module, which is used to start each camera on the current relay production line to perform image acquisition, and splice the acquired images to output a panoramic image.

[0010] A detection adjustment judgment module, which is used to judge the detection adjustment requirements based on the panoramic image and the real-time detected light source intensity of each light source detection point.

[0011] A detection adjustment confirmation module, which is used to confirm the preliminary adjustment items and the adjustment indicators of the preliminary adjustment items when it is judged as a requirement.

[0012] A detection adjustment execution terminal, which is used to start the corresponding adjustment terminal for detection adjustment based on the preliminary adjustment items and the adjustment indicators of the preliminary adjustment items.

[0013] A relay detection and analysis module, which is used to extract the positions of each relay from the panoramic image when the detection adjustment is completed, start the camera group at the corresponding position to perform image acquisition, analyze the acquired images, and output a set of relay surface features.

[0014] A relay detection feedback terminal, which is used to feedback the set of relay surface features to the relay production management personnel.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) By setting up a light source information detection module to monitor the light intensity in real time, the present invention ensures a stable detection environment. The relay preliminary detection module collects and stitches panoramic images to provide comprehensive data. The detection adjustment judgment module determines the adjustment requirements based on the images and light data. The detection adjustment confirmation module clarifies the adjustment items and indicators to achieve intelligent and precise adjustment. The relay detection and analysis module collects multi-view images for in-depth analysis after adjustment and outputs surface feature indicators. The relay detection feedback terminal feeds back the indicators to production management personnel to assist in quality control, improving the detection accuracy, flexibility, and efficiency, and constituting a complete and efficient collaborative detection process.

[0016] (2) By analyzing the light source interference degree from two dimensions of time and position, the present invention can comprehensively evaluate the light stability of the light source at different times and different positions, solves the problem of less consideration of environmental factors such as the light source currently, can effectively reduce the influence of subsequent light factors on the accurate extraction of surface features by image processing and analysis algorithms, reduces the error sources, further ensures the accuracy and effectiveness of the detection results, and at the same time reduces the complexity of later image processing, thereby improving the detection efficiency.

[0017] (3) By analyzing the shape regularity and contour similarity ratio based on the three-dimensional contour of the relay, and then setting the light source interference degree, the present invention effectively solves the problem of insufficient pertinence in current detection, fully considers the influence of various information such as the shape and surface features of the relay on the detection results, and is convenient for the subsequent design of targeted detection strategies, and at the same time further ensures the effectiveness and reliability of the subsequent detection results.

[0018] (4) By confirming the preliminary adjustment items and the adjustment indicators of the preliminary adjustment items, the present invention provides a quantitative standard for the adjustment work, making the adjustment process more targeted and controllable. At the same time, through precise adjustment, it can significantly improve the stability of the detection environment and the accuracy of detection data, avoid detection errors caused by unreasonable environmental factors or detection parameters, and can also reduce unnecessary repeated operations and resource waste, ensuring that the detection work can be completed efficiently and with high quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a schematic diagram of the connection of each module of the system of the present invention.

[0021] Figure 2 This is a schematic diagram of the overall implementation steps of the present invention.

[0022] Figure 3 This is a schematic diagram of the detection adjustment requirement judgment process of the present invention. Detailed implementation manners

[0023] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] Please refer to Figure 1 and Figure 2 As shown, the present invention provides a relay surface detection system based on machine vision technology, and the system includes: a light source information detection module, a relay preliminary detection module, a detection adjustment judgment module, a detection adjustment confirmation module, a detection adjustment execution terminal, a relay detection analysis module, and a relay detection feedback terminal.

[0025] Among the above, the relay preliminary detection module is respectively connected to the light source information detection module and the detection adjustment judgment module, the detection adjustment confirmation module is respectively connected to the detection adjustment judgment module and the detection adjustment execution terminal, and the relay detection analysis module is respectively connected to the detection adjustment execution terminal and the relay detection feedback terminal.

[0026] The light source information detection module is used to start the light intensity sensors arranged at each light source detection point in the corresponding quality inspection area on the production line to perform real-time light intensity detection, and obtain the real-time detected light intensity of each light source detection point.

[0027] The relay preliminary detection module is used to start each camera on the current relay production line to perform image acquisition, and splice the acquired images to output a panoramic image.

[0028] It should be added that the image splicing can be performed through image splicing technology, where the image splicing technology is a relatively mature existing technology. The image splicing is not the inventive point of the present invention and is only achieved by using the existing technology, and its specific splicing process will not be described here.

[0029] The detection adjustment judgment module is used to judge the detection adjustment requirements based on the panoramic image and the real-time detected light source intensity of each light source detection point.

[0030] Specifically, please refer to Figure 3As shown in the figure, detect and adjust the demand judgment, including: W1. Analyze the light source interference degree based on the real-time detected light intensity of each light source detection point. .

[0031] W2. Taking the conveying direction of the production line as the positive direction, construct the center line of the production line along the positive direction. Use the target detection technology to extract the number of relays and the distances between the center points of each relay and the corresponding center line of the production line from the panoramic image, and extract the distances between adjacent relays. At the same time, connect the center points of each relay in sequence along the positive direction to obtain the connection line of the relay center points, and analyze the deviation degree of the relay position distribution. .

[0032] W3. Use the feature point detection algorithm to obtain the feature points of each relay from the panoramic image, match the feature points of each relay with the feature points in the relay image in the standard placement state, and calculate the tilt angle of each relay through the pose estimation algorithm.

[0033] W4. Obtain the three-dimensional contour of the relay from the panoramic image and set the bearing light source interference degree.

[0034] W5. Define that the light source interference degree exceeding the bearing light source interference degree is condition 1, define that the relay position distribution deviation degree is greater than the set position distribution deviation degree as condition 2, and define that there is a relay tilt angle exceeding the set allowable tilt angle as condition 3.

[0035] W6. Judge whether there are any established conditions among conditions 1, 2, and 3. If so, take the demand as the judgment result; otherwise, take the non-demand as the judgment result.

[0036] In a specific embodiment, the set position distribution deviation degree can take a value of 0.2, and the set position distribution deviation degree can take a value .

[0037] Furthermore, regarding the analysis of the light source interference degree in step W1, it includes: W11. Calculate the mean value of the real-time detected light intensity of each light source detection point to obtain the average detected light intensity of each light source detection point.

[0038] W12. If the average detected light intensity of a certain light source monitoring point is not within the set appropriate detected light intensity interval, assign the light source interference degree as 1. Otherwise, for the same time, calculate the standard deviation of the corresponding detected light intensity of each light source detection point to obtain the light intensity difference degree of different light source detection points corresponding to each time. Calculate the average light intensity difference degree of different light source monitoring points through the mean value, denoted as .

[0039] W13. For the same light source detection point, calculate the standard deviation of the detected light intensities at different times to obtain the light intensity difference degrees corresponding to different times for each light source detection point. Calculate the average light intensity difference degree at different times through averaging, denoted as ;

[0040] W14. Calculate the light source interference degree , , where \(e\) is the natural constant, and respectively represent the light intensity difference degrees of different set light source monitoring points and the light intensity difference degrees of different set times, and respectively represent the proportion coefficients of the light intensity difference degrees of different light source monitoring points and the light intensity difference degrees of different times.

[0041] Understandably, by analyzing the light intensity difference degrees at different times, the laws and trends of light changes can be determined. For example, if it is found that the light intensity fluctuates greatly within a certain time period, the reasons can be further investigated, such as whether it is due to the periodic changes of the power supply system or the influence of natural light. For these problems, corresponding measures can be taken, such as adjusting the brightness of artificial light sources, adding voltage stabilizing equipment or choosing to conduct detections during time periods with relatively stable light conditions, so as to effectively reduce the interference of time factors on light. After understanding the light intensity difference degrees at different positions, the positions, angles of light sources can be adjusted targeted or additional auxiliary light sources can be added to improve the uneven light situation. For example, if the light intensity at a certain detection point is significantly lower than that of other points, the light source angle near this position can be adjusted or the number of light sources can be increased to increase the light intensity. In addition, according to the distribution of the light intensity difference degrees, the positions of detection equipment can be reasonably arranged to ensure that stable and uniform light can be obtained during the detection process.

[0042] It should be added that in the high-precision detection of relay surface defects, tiny light intensity differences may lead to deviations in detection results. At this time, the influence of the light intensity differences of different light source monitoring points on the detection results is more critical, so relatively large weights are given to highlight the importance of the light intensity difference degree in the position dimension, that is and can be respectively set to 0.6 and 0.4.

[0043] It should also be added that and can be respectively set to 5 and 3, where the unit is based on the actual measurement unit of light intensity, such as lux.

[0044] The embodiments of the present invention analyze the interference of light sources from two dimensions, time and position, and can comprehensively evaluate the lighting stability of the light source at different times and positions, thereby solving the problem that the current environmental factors such as light sources are less considered. It can effectively reduce the impact of subsequent lighting factors on image processing and analysis algorithms on the accurate extraction of surface features, reduce the source of errors, further ensure the accuracy and effectiveness of the detection results, and at the same time reduce the tediousness of subsequent image processing, thereby improving detection efficiency.

[0045] Further, regarding the analysis of the relay position distribution deviation in step W2, it includes: W21, taking the distance between the center point position of each relay and the corresponding center line of the production line as the center offset of each relay, and selecting the maximum center offset from them, recorded as .

[0046] W22, calculate the standard deviation of the center offset of each relay, recorded as ,right and Normalization is performed and the results after normalization are used as the evaluation variables of each center deviation, which are recorded as and ,Will As the center offset difference, denoted as .

[0047] W23, extract the shortest distance from the distances between adjacent relays , and calculate the standard deviation of the distance between each adjacent relay to obtain the distance interval difference, which is recorded as ,right and Normalization is performed, and then the statistical method of the center offset difference is used to calculate the position spacing difference, which is recorded as .

[0048] W24. Extract the number of peak points and valley points from the center point line of the relay, sum them up, and divide them by the number of relays. The ratio is recorded as ,Will As the position placement difference, denoted as .

[0049] W25. Set the weights of center offset difference, position spacing difference and position placement difference, and calculate the relay position distribution deviation by weighted average.

[0050] It should be added that if the distance between the relays is too close or the distribution is relatively disordered, they may block each other during image acquisition, resulting in incomplete and unclear imaging of the surfaces of some relays. Even if there is no complete blockage, it may also cause the boundaries of the relays in the image to be blurred, affecting the quality and feature extraction of the image. For example, when using machine vision for detection, it is difficult for the camera to accurately capture the independent images of each relay, making it difficult to accurately perform subsequent tasks such as defect recognition and size measurement, ultimately affecting the detection accuracy and reliability.

[0051] It should be added that for The specific normalization formula for normalization processing is: , is the preset allowable placement center offset. The specific normalization formula for normalizing is: , is the standard deviation of the center offset of the set reference.

[0052] It also should be added that the results of normalizing and are used as the position evaluation variables, denoted as and respectively. The specific normalization formula for normalizing is: , is the preset allowable placement spacing. The specific normalization formula for normalizing is: , is the distance interval difference degree of the set reference.

[0053] In a specific embodiment, and can be manually imported. and can be 2 cm and 3 cm respectively.

[0054] In a specific embodiment, the position distribution of the relays directly determines the subsequent light conditions. Especially when the spacing is too short, it will affect the integrity and detailed presentation effect of the image acquisition of the two relays. Therefore, the weight coefficient of the position spacing difference degree is set to be the largest, with a value of 0.4, the weight coefficient of the center offset difference degree is the second largest, with a value of 0.35, and the weight coefficient of the center offset difference degree is the smallest, with a value of 0.25.

[0055] Furthermore, in a specific embodiment of the object detection technology described in step W2, a deep learning-based object detection algorithm can be specifically adopted, that is, by utilizing the powerful feature extraction ability of the convolutional neural network, the panoramic image is learned and trained to identify the relay objects in the image. Common algorithms include Faster R-CNN, YOLO, SSD, etc. Taking Faster R-CNN as an example, it first generates candidate regions that may contain the object through the Region Proposal Network, and then classifies and regresses the bounding boxes of these candidate regions to accurately locate the position of the relay in the image.

[0056] Furthermore, in a specific embodiment of the feature point detection algorithm described in step W3, it can be specifically selected from three algorithms: SIFT, SURF, and ORB. During the detection process, the algorithm will search for points with unique features in the image, such as corner points, prominent points on the edges, etc. At the same time, descriptors are calculated for each detected feature point. The descriptor is a set of vectors that can characterize the local image features around the feature point. For example, the descriptor of the ORB algorithm is based on BRIEF, which generates a binary string by comparing the gray values of the pixels around the feature point for subsequent feature point matching.

[0057] It should be added that the matching of the feature points described in step W3 is achieved through a feature point matching algorithm. Among them, the feature point matching algorithms include BFMatcher and FLANN. BFMatcher calculates the distance between the descriptors of two feature points, such as Hamming distance, Euclidean distance, etc., to find the most matching point pairs. The descriptors of each feature point in the standard image are calculated with the descriptors of all feature points in the panoramic image, and the point with the smallest distance is selected as the matching point. All the obtained matching points are sorted in ascending order of distance. The smaller the distance, the higher the similarity of the two feature points and the greater the reliability of the matching. Usually, only the first several best matching points are retained for subsequent calculations to reduce the influence of noise and incorrect matches. In an actual scenario, in order to improve the accuracy of the matching, some screening conditions can be set, such as only retaining the matching points with a distance less than a certain threshold, or using the cross-validation method to ensure the uniqueness of the matching.

[0058] It should also be added that when matching the feature points of each relay with the feature points of the relay image in the pre-set standard placement state in step W3, it also includes extracting the coordinates of the feature points in the standard image from the sorted matching points as the source points and the coordinates of the corresponding matching feature points in the panoramic image as the target points. These coordinates will be used to calculate the homography matrix that describes the projective transformation relationship between the two images.

[0059] It should be noted that since there may be some erroneous matching points in the feature point matching process, directly using all matching points to calculate the homography matrix will lead to inaccurate results. The RANSAC algorithm is an iterative method for estimating mathematical model parameters from noisy data. When calculating the homography matrix, the RANSAC algorithm randomly selects a set of matching points, assuming that this set of points is the correct matching point, and calculates a homography matrix. Then, this matrix is used to verify all other matching points, and the number of matching points that satisfy the matrix, that is, the number of inliers, is counted. After multiple iterations, the homography matrix with the largest number of inliers is selected as the final result. This can effectively eliminate the interference of erroneous matching points and obtain a more accurate homography matrix.

[0060] It should be added that the posture estimation algorithm described in step W3 calculates the tilt angle of each relay, including: decomposing the calculated homography matrix into a rotation matrix, a translation vector, and a scale factor. The rotation matrix describes the rotation change of the image in three-dimensional space, the translation vector represents the translation amount of the image in three-dimensional space, and the scale factor reflects the scaling of the image. The rotation matrix contains the rotation information of the image. Assuming that the image plane is the XY plane, we are concerned about the rotation angle around the Z axis. According to the element relationship of the rotation matrix, the rotation angle around the Z axis can be calculated by the following formula: , and They are the elements in the 1st row and 0th column and the 0th row and 0th column in the rotation matrix respectively. The resulting rotation angle of the Z axis is the tilt angle of the corresponding relay.

[0061] The calculation of the tilt angle of each relay in step W3 is to evaluate the placement posture of the relay. The uneven placement of the relay will change the relative angle and distance between its surface and the light source. For example, when one end of the relay is tilted, the light intensity received by the tilted part and the lower part will be significantly different. This will cause uneven brightness on the surface of the relay in the collected image. Some areas are too bright and may cover the details of the defect, while the darker areas may not be able to clearly display the surface features due to insufficient light, which seriously affects the subsequent identification and analysis of defects. Unevenly placed relays will cause geometric distortion in the image captured by the camera. The originally regular edges may become curved and the size ratio may also deviate. This distortion will interfere with the image analysis algorithm's accurate measurement of the relay's shape, size and other features, resulting in errors in the detection results. In addition, the edge information on the relay surface is an important basis for feature extraction. Uneven placement will make the edge appear irregular in the image, increasing the difficulty of the edge detection algorithm. Traditional edge detection algorithms may not be able to accurately extract complete and clear edges, resulting in deviations in the subsequent recognition of the relay contour, affecting the judgment of its position and posture, and thus interfering with the positioning of surface defects. Therefore, it is necessary to detect and analyze the placement status of the relay.

[0062] Further, regarding the setting of the bearing light source interference degree in step W4, it includes: W41. Based on the three-dimensional contour of the relay, obtain the volume of the relay, and at the same time calculate the volume of the minimum circumscribed sphere of the relay. Take the ratio of the volume of the relay to the volume of the minimum circumscribed sphere as the shape regularity, denoted as .

[0063] W42. Divide the three-dimensional contour of the relay into each cutting surface in the same direction, and sort each cutting surface according to the order of division. Denote the cutting surface with the first sorting position as the reference cutting surface, and denote the other cutting surfaces as each analysis cutting surface.

[0064] W43. Compare the coincidence of each analysis cutting surface with the reference cutting surface to obtain the coincidence area, and divide it by the contour area of the corresponding analysis cutting surface. Denote the ratio as the contour coincidence ratio.

[0065] W44. Count the number of analysis cutting surfaces with a contour coincidence ratio greater than the set reference contour coincidence ratio, and divide it by the total number of analysis cutting surfaces. Denote the ratio as the contour similarity ratio .

[0066] W45. Set the bearing light source interference degree , , is the set reference allowable light source interference degree, and respectively represent the proportion coefficients corresponding to the set shape regularity and contour similarity ratio, is the set reference contour similarity ratio.

[0067] In a specific embodiment, can take a value of 0.3, can take a value of 0.6.

[0068] In another specific embodiment, the shape regularity is analyzed from the overall shape level, and the contour similarity ratio is analyzed and specified from the detail level. The larger the contour similarity ratio, the more regular the contour. Exemplarily, and can respectively take values of 0.45 and 0.55.

[0069] In the embodiment of the present invention, by analyzing the shape regularity and contour similarity ratio based on the three-dimensional contour of the relay, and then setting the bearing light source interference degree, the problem of insufficient pertinence in current detection is effectively solved, and the influence of various information such as the shape and surface characteristics of the relay on the detection result is fully considered, which is convenient for the subsequent design of targeted detection strategies, and at the same time further ensures the effectiveness and reliability of the subsequent detection result.

[0070] The detection, adjustment, and confirmation module is used to confirm the preliminary adjustment items and the adjustment indicators of the preliminary adjustment items when it is determined as a requirement.

[0071] Specifically, to confirm the preliminary adjustment items, including: if only condition 1 is satisfied, the light source is taken as the preliminary adjustment item.

[0072] If condition 1 is not satisfied and condition 2 or condition 3 is satisfied, the relay attitude is taken as the preliminary adjustment item.

[0073] Another specifically, to confirm the adjustment indicators of the preliminary adjustment items, including: B1. If the preliminary adjustment item is the light source, extract the current light intensity from the real-time detected light intensities at each light source detection point, and take the difference between the appropriate light intensity for imaging and the current light intensity at each light source detection point as the light intensity adjustment value, which is used as the adjustment indicator.

[0074] B2. If the preliminary adjustment item is the relay attitude, confirm the adjustment methods, including robotic arm adjustment and vibration adjustment.

[0075] B3. If the adjustment method is robotic arm adjustment, if the center offset of a certain relay is greater than the set allowable placement center offset or the distance between a certain relay and its adjacent relay exceeds the set allowable placement spacing , mark this relay as the relay to be adjusted.

[0076] B3. Record the center offset of the relay to be adjusted and the distance between it and its adjacent relay as and , respectively. Take as the center offset adjustment value, take as the adjacent distance adjustment value, and further take the center offset adjustment value and the adjacent distance adjustment value of the relay to be adjusted as the adjustment indicators.

[0077] B4. If the adjustment method is vibration adjustment, confirm the adjustment vibration frequency and the adjustment vibration amplitude, which are used as the adjustment indicators.

[0078] Furthermore, the confirmation of the adjustment method in step B2 includes: B21. Count the number of relays with a center offset exceeding the allowable placement center offset and divide it by the total number of relays to obtain the center offset exceeding ratio.

[0079] B22. Count the number of pairs of adjacent relays with a distance lower than the allowable placement spacing and divide it by the total number of pairs of adjacent relays to obtain the position deviation exceeding ratio.

[0080] B23. If there is a center offset exceeding ratio or a position deviation exceeding ratio greater than its set reference value, take vibration adjustment as the adjustment method; otherwise, take robotic arm adjustment as the adjustment method.

[0081] Understandably, when there are large position deviations in a large number of relays, the adjustment efficiency by adjusting them one by one with a manipulator is relatively low, while vibration adjustment can adjust multiple relays at one time. Therefore, when the center offset ratio or the position deviation ratio exceeds its set reference value, vibration adjustment is used as the adjustment method; otherwise, manipulator adjustment is used as the adjustment method.

[0082] Further, in step B4, confirming the adjustment vibration frequency and the adjustment vibration amplitude includes: B41. Setting the weights of the center offset ratio and the position deviation ratio, and calculating the relay distribution deviation degree through weighted average .

[0083] B42. Calculating the adjustment vibration frequency , , where is the set reference relay distribution deviation degree, , and respectively represent the set first reference vibration frequency, the second reference vibration frequency and the reference vibration frequency, .

[0084] B43. Calculating the adjustment vibration amplitude , , where is the set reference reference vibration amplitude.

[0085] It should be added that generally, for relatively small position deviations, a relatively high vibration frequency can be selected, such as 50 - 100 Hz. That is, in this way, the relay can be finely adjusted in a relatively short time to gradually return to the correct position. For relatively large deviations, a relatively low vibration frequency may need to be selected, such as 10 - 30 Hz, in order to give the relay enough time and displacement space to adjust its position. That is, exemplarily, can take a value of 0.4, , and can take values of 100 Hz, 30 Hz and 10 Hz respectively.

[0086] It also should be added that for slight position deviations, the vibration amplitude can be set within a relatively small range, such as ±0.5 mm, to avoid damage to the relay caused by excessive adjustment. For relatively serious deviations, the vibration amplitude may need to be increased to ±2 mm or more. When determining the vibration amplitude, the fixing method and the bearing capacity of the relay also need to be considered to ensure that the vibration amplitude will not cause the relay to loosen or be damaged. Exemplarily, can take a value of 0.5 mm.

[0087] The detection and adjustment execution terminal is used to start the corresponding adjustment terminal for detection and adjustment based on the preliminary adjustment item and the adjustment index of the preliminary adjustment item.

[0088] It should be added that the specific adjustment for detection and adjustment is as follows: When the preliminary adjustment item is the light source, pulse width modulation controls the light emission intensity of the LED by adjusting the pulse width of the LED drive current. That is, according to the light emission intensity to be adjusted, by changing the time ratio of the high level, the lighting time of the LED is changed, so as to achieve dimming.

[0089] When the preliminary adjustment item is the relay attitude and the adjustment method is the manipulator, start the machine installed on the production line to make adjustments according to the corresponding adjustment index.

[0090] When the preliminary adjustment item is the relay attitude and the adjustment method is vibration adjustment, start the vibration terminal installed at the corresponding position on the production line and make corresponding adjustments according to the adjustment index.

[0091] In a specific embodiment, the vibration terminal can be a vibrating disk, which is a commonly used device in the process of feeding relays. By adjusting the vibration frequency and vibration amplitude of the vibrating disk, the relays can be arranged more orderly on the feeding track, reducing the situations of uneven placement and inconsistent distances.

[0092] The relay detection and analysis module is used to, after the detection and adjustment are completed, extract the positions of each relay from the panoramic image, start the camera group at the corresponding position for image acquisition, analyze the acquired images, and output the set of relay surface features.

[0093] Specifically, analyzing the acquired images includes: U1. Using the edge detection algorithm to extract the contours of each relay in the acquired image, and then calculating the length, width and height of each relay to construct a list of size features of each relay.

[0094] U2. Using the gray-level co-occurrence matrix to extract each texture feature on the surface of each relay to construct a list of texture features of each relay.

[0095] U3. Locating the number of scratch areas on the surface of each relay, the scratch area of each scratch area, the number of deformation areas and the deformation volume of each deformation area from the acquired image, and constructing a list of defect features of each relay.

[0096] U4. Integrating the list of size features, the list of texture features and the list of defect features of each relay to obtain the set of relay surface features.

[0097] In a specific embodiment, the edge detection algorithm can adopt the Canny algorithm. Among them, the Canny algorithm can accurately detect the edge information in the image through steps such as Gaussian filtering to smooth the image, calculating the gradient magnitude and direction, non-maximum suppression, and double-threshold detection and connecting edges, so as to obtain the contour of the relay.

[0098] It should be added that the length, width, and height of the calculated contour can be obtained by counting the pixel points on the contour or using specific geometric algorithms.

[0099] In a specific embodiment, the texture features include contrast, energy, and entropy, and the deformation includes depressions and protrusions. That is, the number of deformation points is the sum of the number of depression points and the number of protrusion points. Moreover, the number of scratch points on the surface of each relay, the scratch area of each scratch point, the number of deformation points, and the deformation volume of each deformation point can also be realized by the object detection algorithm.

[0100] The relay detection feedback terminal is used to feedback the set of relay surface features to the relay production management personnel.

[0101] In the embodiment of the present invention, the light source information detection module is set to monitor the light intensity in real time, ensuring a stable detection environment. The relay preliminary detection module collects and stitches panoramic images, providing comprehensive data. The detection adjustment judgment module judges the adjustment requirements according to the image and light data. The detection adjustment confirmation module clarifies the adjustment items and indicators to achieve intelligent and precise adjustment. The relay detection analysis module collects multi-view images for in-depth analysis after adjustment and outputs surface feature indicators. The relay detection feedback terminal feeds back the indicators to the production management personnel to assist in quality control, improving the detection accuracy, detection flexibility, and detection efficiency, and constituting a complete and efficient collaborative detection process.

[0102] The above content is only an example and explanation of the concept of the present invention. Those skilled in the art of this technology make various modifications or supplements to the described specific embodiments or use similar methods to replace them. As long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, they should all belong to the protection scope of the present invention.

Claims

1. A relay surface detection system based on machine vision technology, characterized in that, The system includes: A light source information detection module, which starts the light intensity sensors installed at each light source detection point in the corresponding quality inspection area on the production line to perform real-time detection of the light intensity, and obtains the real-time detected light intensity of each light source detection point; A relay preliminary detection module, which starts each camera on the current relay production line to collect images, splices the collected images, and outputs a panoramic image; A detection adjustment judgment module, which judges the detection adjustment requirements based on the panoramic image and the real-time detected light source intensity of each light source detection point; A detection adjustment confirmation module, which, when the judgment is a requirement, confirms the preliminary adjustment items and the adjustment indicators of the preliminary adjustment items; A detection adjustment execution terminal, which, based on the preliminary adjustment items and the adjustment indicators of the preliminary adjustment items, starts the corresponding adjustment terminal to perform detection and adjustment; A relay detection and analysis module, which, after the detection and adjustment are completed, extracts the positions of each relay from the panoramic image, starts the camera group at the corresponding position to collect images, analyzes the collected images, and outputs a set of relay surface features; A relay detection feedback terminal, which feeds back the set of relay surface features to the relay production management personnel; Performing detection adjustment requirement judgment, including: Analyze the light source interference degree based on the real-time detected light intensity at each light source detection point ; Analysis , including: Calculating the mean value of the real-time detected light intensity of each light source detection point to obtain the average detected light intensity of each light source detection point; If the average detected light intensity of a certain light source monitoring point is not within the set appropriate detected light intensity range, assign a light source interference degree of 1. Otherwise, for the same time, calculate the standard deviation of the detected light intensities corresponding to each light source detection point to obtain the light intensity difference degree of different light source detection points corresponding to each time. Calculate the average light intensity difference degree of different light source monitoring points through averaging, denoted as ; For the same light source detection point, calculate the standard deviation of the detected light intensities at different times to obtain the light intensity difference degrees corresponding to different times for each light source detection point, and obtain the average light intensity difference degree at different times through mean calculation, denoted as ; , is the natural constant, and respectively represent the light intensity difference degrees of different light source monitoring points and the light intensity difference degrees at different times, and respectively represent the proportion coefficients of the light intensity difference degrees of different light source monitoring points and the light intensity difference degrees at different times; Taking the conveying direction of the production line as the positive direction, construct the center line of the production line along the positive direction. Using object detection technology, extract the number of relays, the distance between the center points of each relay and the corresponding center line of the production line from the panoramic image, and extract the distance between adjacent relays. At the same time, connect the center points of each relay in sequence along the positive direction to obtain the connection line of the relay center points, and analyze the deviation degree of the relay position distribution ; Using a feature point detection algorithm to obtain each feature point of each relay from the panoramic image, matching each feature point of each relay with each feature point in the relay image in the standard placement state, and calculating the tilt angle of each relay through a pose estimation algorithm; Obtain the three-dimensional contour of the relay from the panoramic image and set the interference degree of the bearing light source ; , is the interference degree of the set reference light source, and respectively represent the proportion coefficients corresponding to the set shape regularity and contour similarity ratio, is the contour similarity ratio of the set reference, is the shape regularity, is the contour similarity ratio; Based on the three-dimensional contour of the relay, the volume of the relay is obtained. At the same time, the volume of the minimum circumscribed sphere of the relay is calculated, and the ratio of the volume of the relay to the volume of the minimum circumscribed sphere is used as the shape regularity, denoted as ; Dividing the three-dimensional contour of the relay into each cutting surface in the same direction, sorting each cutting surface according to the order of division, recording the cutting surface with the first sorting position as the reference cutting surface, and recording the other cutting surfaces as each analysis cutting surface; Performing coincidence comparison between each analysis cutting surface and the reference cutting surface to obtain the coincidence area, dividing it by the contour area of the corresponding analysis cutting surface, and recording the ratio as the contour coincidence ratio; Count the number of analysis segmentation planes where the statistical contour overlap ratio is greater than the set reference contour overlap ratio, divide it by the total number of analysis segmentation planes, and record the ratio as the contour similarity ratio ; Defining that the light source interference degree exceeding the bearing light source interference degree is condition 1, defining that the relay position distribution deviation degree is greater than the set position distribution deviation degree as condition 2, and defining that there is a relay tilt angle exceeding the set allowable tilt angle as condition 3; Judging whether there is a condition that holds among conditions 1, 2, and 3. If there is, taking the requirement as the judgment result, otherwise taking the non-requirement as the judgment result.

2. The surface detection system of a relay based on machine vision technology according to claim 1, wherein: Analyzing the relay position distribution deviation degree includes: Take the distance between the center point position of each relay and the corresponding center line of the production line as the center offset of each relay, and screen out the maximum center offset from them, denoted as ; Calculate the standard deviation of the center offset of each relay, denoted as , for and perform normalization processing, and use the results after normalization processing as the evaluation variables of each center offset, denoted as and , take as the center offset difference degree, denoted as ; Extract the shortest distance from the distances between adjacent relays , and calculate the standard deviation of the distances between adjacent relays to obtain the distance interval difference degree, denoted as . For and , perform normalization processing, and then, in the same way as the statistical method of the center offset difference degree, calculate the position interval difference degree, denoted as ; Extract the number of peak points and the number of valley points from the center point connection lines of the relays, sum the two, divide by the number of relays, and record the ratio as , and use as the position placement difference degree, denoted as ; Setting the weights of the center offset difference degree, the position spacing difference degree, and the position placement difference degree, and calculating the relay position distribution deviation degree through weighted average.

3. The surface detection system of a relay based on machine vision technology according to claim 2, wherein: Confirming the preliminary adjustment items includes: If only condition 1 holds, taking the light source as the preliminary adjustment item; If condition 1 does not hold and condition 2 or condition 3 holds, taking the relay posture as the preliminary adjustment item.

4. The surface detection system of a relay based on machine vision technology according to claim 3, wherein: Confirming the adjustment indicators of the preliminary adjustment items includes: If the preliminary adjustment item is the light source, extracting the current light intensity from the real-time detected light intensity of each light source detection point, and taking the difference between the camera-appropriate light intensity and the current light intensity of each light source detection point as the light intensity adjustment value and using it as the adjustment indicator; If the preliminary adjustment item is the attitude of the relay, confirm the adjustment method, including robot adjustment and vibration adjustment; If the adjustment item is the manipulator adjustment, if the center offset of a certain relay is greater than the permitted placement center offset set or the distance between a certain relay and its adjacent relay exceeds the permitted placement spacing set below , mark this relay as the adjustment relay; The center offset of the adjustment relay and the distance between it and the adjacent relay are respectively denoted as and . Let be the center offset adjustment value, and let be the adjacent distance adjustment value. Furthermore, the center offset adjustment value and the adjacent distance adjustment value of the adjustment relay are used as adjustment indicators; If the adjustment item is vibration adjustment, confirm the adjustment vibration frequency and adjustment vibration amplitude, and use them as adjustment indicators.

5. The surface detection system of a relay based on machine vision technology according to claim 4, characterized in that: The confirmation of the adjustment method includes: Count the number of relays with a center offset exceeding the permitted placement center offset, and divide it by the total number of relays to obtain the center offset exceeding ratio; Count the number of adjacent relay pairs with a distance lower than the permitted placement spacing, and divide it by the total number of adjacent relay pairs to obtain the position deviation exceeding ratio; If there is a center offset exceeding ratio or a position deviation exceeding ratio greater than its set reference value, use vibration adjustment as the adjustment method; otherwise, use robot adjustment as the adjustment method.

6. The surface detection system of a relay based on machine vision technology according to claim 5, characterized in that: The confirmation of the adjustment vibration frequency and adjustment vibration amplitude includes: Set the weights of the center offset exceedance ratio and the position deviation exceedance ratio, and calculate the relay distribution deviation degree through weighted average ; Calculate and adjust the vibration frequency , , is to set the reference relay distribution deviation degree, , and respectively represent the set first reference vibration frequency, second reference vibration frequency and reference vibration frequency, ; Calculate the adjusted vibration amplitude , , is the set reference vibration amplitude.

7. The surface detection system of a relay based on machine vision technology according to claim 1, wherein: The analysis of the collected images includes: Use the edge detection algorithm to extract the contours of each relay in the collected image, and then calculate the length, width and height of each relay to construct a list of size features of each relay; Use the gray-level co-occurrence matrix to extract each texture feature on the surface of each relay to construct a list of texture features of each relay; Locate the number of scratch areas, the scratch area of each scratch area, the number of deformation areas, and the deformation volume of each deformation area on the surface of each relay from the collected image to construct a list of defect features of each relay; Integrate the list of size features, texture features and defect features of each relay to obtain a set of surface features of the relay.

Images