Relay surface detection system based on machine vision technology

Abstract

The invention belongs to the technical field of component detection, and particularly discloses a relay surface detection system based on a machine vision technology. The system comprises a light source information detection module, a relay preliminary detection module, a detection adjustment judgment module, a detection adjustment confirmation module, a detection adjustment execution terminal, a relay detection analysis module and a relay detection feedback terminal. The light source information detection module is arranged to monitor illumination intensity in real time, the stability of a detection environment is ensured, the relay preliminary detection module collects and splices images, comprehensive data is provided, the detection adjustment judgment module judges adjustment requirements according to the images and illumination data, and the detection adjustment confirmation module determines adjustment items and indexes. Intelligent and accurate adjustment is realized, and the relay detection and analysis module deeply analyzes acquired images and outputs surface characteristic indexes; and the relay detection feedback terminal feeds back the indexes to production management personnel, so that the quality control is facilitated, and the detection accuracy is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of component detection, and in particular relates to a relay surface detection system based on machine vision technology. Background Art

[0002] Relays are key components commonly used in electrical control systems, and their surface quality directly affects product performance and reliability. Traditional relay surface inspection methods mainly rely on manual visual inspection, which is inefficient, and the inspection accuracy is easily affected by human factors, making it difficult to meet the high quality inspection requirements of modern industrial production. With the continuous development of machine vision technology, its application in relay surface inspection has become an efficient and accurate inspection method.

[0003] Prior art, such as a Chinese invention patent application with application number 201911046408.1, discloses a real-time detection system for PCB surface components, which includes: a PCB conveyor belt, an infrared sensor, a camera, an industrial computer, a relay, and an alarm device, wherein the infrared sensor is configured to detect the PCB board on the PCB conveyor belt and send a detection signal to the industrial computer. The industrial computer is configured to control the PCB board conveyor belt to stop transmission based on the detection signal, and to send an instruction to the camera to collect PCB images, receive and process the collected image information, and control the alarm device through the relay to alarm in response to the processing result showing that there is a problem with the PCB surface components. Compared with traditional detection methods, it has many advantages such as accurate fault location accuracy, high detection efficiency, and real-time alarm, thereby realizing efficient detection of PCB boards.

[0004] Another example of the prior art is the visual inspection system and method for micro-defects on the surface of annular precision parts disclosed in the Chinese invention patent application with application number 201910160191.0, which includes a light shielding box and a conveying platform for placing the parts to be tested, wherein the conveying platform passes through the light shielding box, and a lighting module, a visual inspection module and a control module are arranged in the light shielding box. The lighting module includes one or more light sources. The visual inspection module includes an image acquisition device, a polarizing lens, inspection software and a processor. The control module includes a single-chip microcomputer, a relay and a motor, the relay is connected to the light source, and the motor is connected to the polarizing lens. In this way, the automatic detection, classification and estimation of defect depth of micro-defects on the surface of annular precision parts are realized, which effectively reduces the adverse effects of the part's own texture and tiny scratches on defect detection, improves the inspection efficiency, reduces the defect rate of precision parts, and reduces the work intensity.

[0005] With regard to the above technical solutions, it is obvious that the surface detection of components such as relays still has the following deficiencies: 1. The detection is not targeted enough. Currently, the impact of various information such as component shape and surface characteristics on the detection results has not been considered, and targeted detection strategies have not been designed, resulting in certain deficiencies in the effectiveness and reliability of the detection results.

[0006] 2. Light source is a key factor in the image acquisition process. In the actual production environment, the light intensity, angle and uniformity may change. The current environment takes little consideration, which affects the image processing and analysis algorithms to accurately extract surface features, and ultimately leads to a certain deviation in the accuracy and effectiveness of the detection results, and also affects the detection efficiency. Summary of the invention

[0007] In view of this, in order to solve the problems raised in the above background technology, a relay surface detection system based on machine vision technology is proposed.

[0008] The purpose of the present invention can be achieved through the following technical solutions: The present invention provides a relay surface detection system based on machine vision technology, the system comprising: a light source information detection module, used to start the light intensity sensors installed in each light source detection point in the corresponding quality inspection area on the production line to perform real-time detection of light intensity, and obtain the real-time detection light intensity of each light source detection point.

[0009] The relay preliminary detection module is used to start each camera on the current relay production line to collect images, stitch the collected images, and output a panoramic image.

[0010] The detection adjustment judgment module is used to judge the detection adjustment needs based on the panoramic image and the real-time detection of the light source intensity at each light source detection point.

[0011] The detection adjustment confirmation module is used to confirm the preliminary adjustment items and the adjustment indicators of the preliminary adjustment items when it is determined to be a demand.

[0012] The detection and adjustment execution terminal is used to start the corresponding adjustment terminal to perform detection and adjustment based on the preliminary adjustment items and the adjustment indicators of the preliminary adjustment items.

[0013] The relay detection and analysis module is used to extract the position of each relay from the panoramic image after the detection and adjustment are completed, start the camera group at the corresponding position to collect images, analyze the collected images, and output the relay surface feature set.

[0014] The relay detection feedback terminal is used to feed back the relay surface feature set to the relay production management personnel.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention ensures the stability of the detection environment by setting a light source information detection module to monitor the light intensity in real time. The relay preliminary detection module collects and splices panoramic images to provide comprehensive data. The detection adjustment judgment module determines the adjustment requirements based on the image and light data. The detection adjustment confirmation module clarifies the adjustment items and indicators to achieve intelligent and precise adjustment. After adjustment, the relay detection and analysis module collects multi-view images for in-depth analysis and outputs surface feature indicators. The relay detection feedback terminal feeds back the indicators to production management personnel, which helps quality control, improves detection accuracy, detection flexibility and detection efficiency, and constitutes a complete and efficient collaborative detection process.

[0016] (2) By analyzing the interference of light sources from two dimensions, time and position, the present invention can comprehensively evaluate the lighting stability of the light source at different times and positions, solving the problem that the current environmental factors such as light sources are less considered. It can effectively reduce the impact of subsequent lighting factors on image processing and analysis algorithms on the accurate extraction of surface features, reduce the source of errors, further ensure the accuracy and effectiveness of the detection results, and at the same time reduce the tediousness of subsequent image processing, thereby improving detection efficiency.

[0017] (3) The present invention performs shape regularity and contour similarity analysis based on the three-dimensional contour of the relay, and then sets the interference degree of the carrying light source, thereby effectively solving the problem of insufficient targeted detection in the current situation. It fully considers the influence of various information such as the shape and surface characteristics of the relay on the detection results, thereby facilitating the subsequent design of targeted detection strategies, and at the same time further ensuring the effectiveness and reliability of subsequent detection results.

[0018] (4) The present invention provides a quantitative standard for the adjustment work by confirming the preliminary adjustment items and confirming the adjustment indicators of the preliminary adjustment items, making the adjustment process more targeted and controllable. At the same time, through precise adjustment, the stability of the detection environment and the accuracy of the detection data can be significantly improved, avoiding detection errors caused by environmental factors or unreasonable detection parameters. It can also reduce unnecessary repeated operations and waste of resources, ensuring that the detection work can be completed efficiently and with high quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0020] Figure 1 It is a schematic diagram of the connection of each module of the system of the present invention.

[0021] Figure 2 It is a schematic flow chart of the overall implementation steps of the present invention.

[0022] Figure 3 This is a schematic diagram of the detection and adjustment demand determination process of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] See also Figure 1 and Figure 2 As shown, the present invention provides a relay surface detection system based on machine vision technology, the system comprising: a light source information detection module, a relay preliminary detection module, a detection adjustment judgment module, a detection adjustment confirmation module, a detection adjustment execution terminal, a relay detection analysis module and a relay detection feedback terminal.

[0025] In the above, the relay preliminary detection module is respectively connected to the light source information detection module and the detection adjustment judgment module, the detection adjustment confirmation module is respectively connected to the detection adjustment judgment module and the detection adjustment execution terminal, and the relay detection analysis module is respectively connected to the detection adjustment execution terminal and the relay detection feedback terminal.

[0026] The light source information detection module is used to start the light intensity sensors installed in each light source detection point in the corresponding quality inspection area on the production line to perform real-time detection of light intensity, and obtain the real-time detected light intensity of each light source detection point.

[0027] The relay preliminary detection module is used to start each camera on the current relay production line to collect images, splice the collected images, and output a panoramic image.

[0028] It should be added that the image stitching can be performed by image stitching technology, wherein the image stitching technology is a relatively mature technology available. Image stitching is not the inventive point described in the present invention, and is only achieved by using existing technology. The specific stitching process will not be described here.

[0029] The detection adjustment judgment module is used to judge the detection adjustment demand based on the panoramic image and the real-time detection light source intensity of each light source detection point.

[0030] Specifically, see Figure 3As shown, the detection and adjustment needs are judged, including: W1, based on the real-time detection of light intensity at each light source detection point, analyzing the light source interference .

[0031] W2. Take the transmission direction of the production line as the positive direction, construct the center line of the production line along the positive direction, use the target detection technology to extract the number of relays and the distance between the center point of each relay and the corresponding center line of the production line from the panoramic image, and extract the distance between each adjacent relay. At the same time, connect the center point of each relay in turn along the positive direction to obtain the relay center point connection line, and analyze the relay position distribution deviation .

[0032] W3. Use feature point detection algorithm to obtain feature points of each relay from the panoramic image, match the feature points of each relay with the feature points in the relay image under the standard placement state, and calculate the tilt angle of each relay through the posture estimation algorithm.

[0033] W4. Obtain the three-dimensional outline of the relay from the panoramic image and set the interference degree of the carrying light source.

[0034] W5. The light source interference exceeding the load light source interference is defined as condition 1, the relay position distribution deviation being greater than the set position distribution deviation is defined as condition 2, and the inclination angle of the relay exceeding the set allowable inclination angle is defined as condition 3.

[0035] W6. Determine whether any of the conditions 1, 2, and 3 are met. If so, the requirement is used as the judgment result. Otherwise, the non-requirement is used as the judgment result.

[0036] In a specific embodiment, the position distribution deviation can be set to 0.2, and the position distribution deviation can be set to .

[0037] Furthermore, the analysis of the light source interference degree in step W1 includes: W11, calculating the average of the real-time detection light intensity of each light source detection point to obtain the average detection light intensity of each light source detection point.

[0038] W12. If the average detected light intensity of a certain light source monitoring point is not within the set appropriate detection light intensity range, the light source interference is assigned a value of 1. Otherwise, for the same time, the standard deviation of the corresponding detected light intensity of each light source detection point is calculated to obtain the light difference of different light source detection points at each time. The average light difference of different light source monitoring points is calculated by the mean value, which is recorded as .

[0039] W13. For the same light source detection point, the standard deviation of the detected light intensity at different times is calculated to obtain the light intensity difference of each light source detection point at different times. The average light intensity difference at different times is obtained by mean calculation, which is recorded as .

[0040] W14. Calculate the interference of light sources , , is a natural constant, and They represent the illumination difference between different light source monitoring points and the illumination difference between different time periods. and They respectively represent the proportion coefficients of illumination difference at different light source monitoring points and illumination difference at different times.

[0041] Understandably, by analyzing the difference in light intensity at different times, the law and trend of light changes can be determined. For example, if it is found that the light intensity fluctuates greatly within a certain period of time, the cause can be further investigated, such as whether it is due to periodic changes in the power supply system or the influence of natural light. In response to these problems, corresponding measures can be taken, such as adjusting the brightness of artificial light sources, adding voltage stabilizing equipment, or choosing to conduct detection in a time period when the light conditions are relatively stable, thereby effectively reducing the interference of time factors on light. After understanding the difference in light at different locations, the position and angle of the light source can be adjusted in a targeted manner, or auxiliary light sources can be added to improve the uneven light conditions. For example, if the light intensity at a certain detection point is significantly lower than that at other points, the light intensity can be increased by adjusting the angle of the light source near the position or increasing the number of light sources. In addition, the position of the detection equipment can be reasonably arranged according to the distribution of light differences to ensure that stable and uniform light can be obtained during the detection process.

[0042] It should be added that in high-precision relay surface defect detection, small differences in illumination may lead to deviations in the detection results. At this time, the difference in illumination at different light source monitoring points has a more critical impact on the detection results, so a larger weight is given to highlight the importance of the illumination difference in the position dimension, that is, and The values ​​can be 0.6 and 0.4 respectively.

[0043] It is also necessary to add that and The values ​​can be 5 and 3 respectively, where the unit is based on the actual measurement unit of light intensity, such as lux.

[0044] The embodiments of the present invention analyze the interference of light sources from two dimensions, time and position, and can comprehensively evaluate the lighting stability of the light source at different times and positions, solving the problem that the current environmental factors such as light sources are less considered. It can effectively reduce the impact of subsequent lighting factors on image processing and analysis algorithms on the accurate extraction of surface features, reduce the source of errors, further ensure the accuracy and effectiveness of the detection results, and at the same time reduce the tediousness of subsequent image processing, thereby improving detection efficiency.

[0045] Further, regarding the analysis of the relay position distribution deviation in step W2, it includes: W21, taking the distance between the center point position of each relay and the corresponding center line of the production line as the center offset of each relay, and selecting the maximum center offset from them, recorded as .

[0046] W22. Calculate the standard deviation of the center offset of each relay and obtain the center offset difference, which is recorded as ,right and Normalization is performed and the results after normalization are used as the evaluation variables of each center deviation, which are recorded as and ,Will As the center offset difference, denoted as .

[0047] W23, extract the shortest distance from the distances between adjacent relays , and calculate the standard deviation of the distance between each adjacent relay to obtain the distance interval difference, which is recorded as ,right and Normalization is performed, and then the position spacing difference is calculated in the same way as the center offset difference, which is recorded as .

[0048] W24. Extract the number of peak points and valley points from the center point line of the relay, sum them up, and divide them by the number of relays. The ratio is recorded as ,Will As the position placement difference, denoted as .

[0049] W25, set the weights of center offset difference, position spacing difference and position placement difference, and calculate the relay position distribution deviation by weighted average.

[0050] It should be added that if the distance between relays is too close or the distribution is disordered, they may block each other during image acquisition, resulting in the inability to fully and clearly image the surface of some relays. Even if there is no complete occlusion, the relay boundary in the image may be blurred, affecting the image quality and feature extraction. For example, when using machine vision for inspection, it is difficult for the camera to accurately capture the independent image of each relay, making it difficult to accurately perform subsequent defect identification, size measurement and other tasks, ultimately affecting the detection accuracy and reliability.

[0051] It should be added that The specific normalization formula for normalization is: , The preset allowable placement center offset is The specific normalization formula for normalization is: , To set the center offset difference of the reference.

[0052] It is also necessary to add that and The normalized results are used as the evaluation variables for each position and are recorded as and ,right The specific normalization formula for normalization is: , For the preset allowable placement distance, The specific normalization formula for normalization is: , To set the reference distance interval difference.

[0053] In a specific embodiment, and The specific value of can be manually imported. and The values ​​can be 2cm and 3cm respectively.

[0054] In a specific embodiment, the position distribution of the relays directly determines the subsequent lighting conditions. In particular, a too short spacing will affect the integrity of the image acquisition of the two relays and the detailed presentation effect. Therefore, the weight coefficient of the position spacing difference is set to the largest value, which is 0.4. The weight coefficient of the center offset difference is second, which is 0.35. The weight coefficient of the center offset difference is the smallest, which is 0.25.

[0055] Furthermore, regarding the target detection technology described in step W2, in a specific embodiment, a target detection algorithm based on deep learning can be specifically used, that is, the powerful feature extraction capability of the convolutional neural network is utilized to learn and train the panoramic image to identify the relay target in the image. Common algorithms include FasterR-CNN, YOLO, SSD, etc. Taking FasterR-CNN as an example, it first generates candidate regions that may contain targets through a region proposal network, and then classifies and regresses these candidate regions by bounding boxes, thereby accurately locating the position of the relay in the image.

[0056] Furthermore, in a specific embodiment, the feature point detection algorithm described in step W3 can be selected from the three algorithms of SIFT, SURF and ORB. During the detection process, the algorithm will look for points with unique features in the image, such as corner points, protruding points on the edge, etc. At the same time, a descriptor is calculated for each detected feature point. The descriptor is a set of vectors that can characterize the local image features around the feature point. For example, the descriptor of the ORB algorithm is based on BRIEF, which generates a binary string by comparing the grayscale values ​​of the pixels around the feature point for subsequent feature point matching.

[0057] It should be added that the matching of the feature points described in step W3 is achieved through a feature point matching algorithm, wherein the feature point matching algorithms include BFMatcher and FLANN. BFMatcher finds the best matching point pair by calculating the distance between two feature point descriptors, such as Hamming distance, Euclidean distance, etc. The distance between the descriptor of each feature point in the standard image and the descriptors of all feature points in the panoramic image is calculated, and the point with the closest distance is selected as the matching point. All the obtained matching points are sorted from small to large according to the distance. The smaller the distance, the higher the similarity between the two feature points, and the greater the reliability of the matching. Usually only the first several best matching points are retained for subsequent calculations to reduce the impact of noise and false matching. In a practical scenario, in order to improve the accuracy of matching, some screening conditions can be set, such as only retaining matching points with a distance less than a certain threshold, or using a cross-validation method to ensure the uniqueness of the match.

[0058] It should also be added that in step W3, when matching each feature point of each relay with each feature point of the relay image in the preset standard placement state, the coordinates of the feature points in the standard image are extracted from the sorted matching points as the source points and the coordinates of the corresponding matching feature points in the panoramic image as the target points. These coordinates will be used to calculate the homography matrix describing the projection transformation relationship between the two images.

[0059] It should be noted that since there may be some erroneous matching points in the feature point matching process, directly using all matching points to calculate the homography matrix will lead to inaccurate results. The RANSAC algorithm is an iterative method for estimating mathematical model parameters from noisy data. When calculating the homography matrix, the RANSAC algorithm randomly selects a set of matching points, assuming that this set of points is the correct matching point, and calculates a homography matrix. Then, this matrix is ​​used to verify all other matching points, and the number of matching points that satisfy the matrix, that is, the number of inliers, is counted. After multiple iterations, the homography matrix with the largest number of inliers is selected as the final result. This can effectively eliminate the interference of erroneous matching points and obtain a more accurate homography matrix.

[0060] It should be added that the posture estimation algorithm described in step W3 calculates the tilt angle of each relay, including: decomposing the calculated homography matrix into a rotation matrix, a translation vector, and a scale factor. The rotation matrix describes the rotation change of the image in three-dimensional space, the translation vector represents the translation amount of the image in three-dimensional space, and the scale factor reflects the scaling of the image. The rotation matrix contains the rotation information of the image. Assuming that the image plane is the XY plane, we are concerned about the rotation angle around the Z axis. According to the element relationship of the rotation matrix, the rotation angle around the Z axis can be calculated by the following formula: , and They are the elements in the 1st row and 0th column and the 0th row and 0th column in the rotation matrix respectively. The resulting rotation angle of the Z axis is the tilt angle of the corresponding relay.

[0061] The calculation of the tilt angle of each relay in step W3 is to evaluate the placement posture of the relay. The uneven placement of the relay will change the relative angle and distance between its surface and the light source. For example, when one end of the relay is tilted, the light intensity received by the tilted part and the lower part will be significantly different. This will cause uneven brightness on the surface of the relay in the collected image. Some areas are too bright and may cover the details of the defect, while the darker areas may not be able to clearly display the surface features due to insufficient light, which seriously affects the subsequent identification and analysis of defects. Unevenly placed relays will cause geometric distortion in the image captured by the camera. The originally regular edges may become curved and the size ratio may also deviate. This distortion will interfere with the image analysis algorithm's accurate measurement of the relay's shape, size and other features, resulting in errors in the detection results. In addition, the edge information on the relay surface is an important basis for feature extraction. Uneven placement will make the edge appear irregular in the image, increasing the difficulty of the edge detection algorithm. Traditional edge detection algorithms may not be able to accurately extract complete and clear edges, resulting in deviations in the subsequent recognition of the relay contour, affecting the judgment of its position and posture, and thus interfering with the positioning of surface defects. Therefore, it is necessary to detect and analyze the placement status of the relay.

[0062] Further, regarding setting the interference degree of the light source in step W4, it includes: W41, based on the three-dimensional contour of the relay, obtaining the volume of the relay, and calculating the minimum circumscribed sphere volume of the relay, and taking the ratio of the volume of the relay to the volume of the minimum circumscribed sphere volume as the shape regularity, recorded as .

[0063] W42. Divide the three-dimensional contour of the relay into various cutting planes in the same direction, and sort the various cutting planes according to the order of division. The cutting plane with the first sorting position is recorded as the reference cutting plane, and the other cutting planes are recorded as various analysis cutting planes.

[0064] W43. Compare and overlap each analysis segmentation surface with the reference segmentation surface to obtain the overlap area, and divide it by the contour area of ​​the corresponding analysis segmentation surface. The ratio is recorded as the contour overlap ratio.

[0065] W44, count the number of analysis segmentation faces whose contour coincidence ratio is greater than the set reference contour coincidence ratio, and divide it by the total number of analysis segmentation faces, and record the ratio as the contour similarity ratio .

[0066] W45, set the interference degree of the light source , , The interference level of the permitted light source is set as the benchmark. and They are respectively expressed as the proportion coefficients corresponding to the set shape regularity and contour similarity ratio, To set the reference contour similarity ratio.

[0067] In a specific embodiment, The value can be 0.3. The value can be 0.6.

[0068] In another specific embodiment, the shape regularity is analyzed from the overall shape level, and the contour similarity ratio is analyzed from the detail level. The larger the contour similarity ratio, the more standardized the contour. For example, and The values ​​can be 0.45 and 0.55 respectively.

[0069] The embodiment of the present invention performs shape regularity and contour similarity analysis based on the three-dimensional contour of the relay, and then sets the interference degree of the carrying light source, thereby effectively solving the problem of insufficient targeted detection in the current detection. It fully considers the influence of various information such as the relay shape and surface characteristics on the detection results, thereby facilitating the subsequent design of targeted detection strategies, and further ensuring the effectiveness and reliability of subsequent detection results.

[0070] The detection adjustment confirmation module is used to confirm the preliminary adjustment items and the adjustment indicators of the preliminary adjustment items when it is determined that there is a need.

[0071] Specifically, confirming the preliminary adjustment item includes: if only condition 1 is met, taking the light source as the preliminary adjustment item.

[0072] If condition 1 is not met, and condition 2 or condition 3 is met, the relay attitude is used as a preliminary adjustment item.

[0073] Specifically, confirming the adjustment index of the prepared adjustment item includes: B1. If the prepared adjustment item is a light source, extracting the current light intensity from the real-time detected light intensity of each light source detection point, and taking the difference between the suitable light intensity for the camera and the current light intensity of each light source detection point as the light intensity adjustment value, and as the adjustment index.

[0074] B2. If the prepared adjustment item is relay posture, confirm the adjustment method, including manipulator adjustment and vibration adjustment.

[0075] B3. If the adjustment item is robot adjustment, if the center offset of a relay is greater than the set allowable placement center offset Or the distance between a relay and its adjacent relay exceeds or falls below the set allowable placement spacing , and record the relay as the adjustment relay.

[0076] B3. The center offset of the adjustment relay and the distance between it and its adjacent relay are recorded as and ,Will As the center offset adjustment value, As the proximity distance adjustment value, the center offset adjustment value and the proximity distance adjustment value of the adjustment relay are then used as adjustment indicators.

[0077] B4. If the adjustment item is vibration adjustment, confirm the adjustment vibration frequency and adjustment vibration amplitude, and use them as adjustment indicators.

[0078] Furthermore, the confirmation adjustment method in step B2 includes: B21, counting the number of relays whose center offset exceeds the permitted placement center offset, and dividing the number by the total number of relays to obtain a center offset excess ratio.

[0079] B22. Count the number of adjacent relay pairs whose distance is less than the allowable placement spacing and divide it by the total number of adjacent relay pairs to obtain the position deviation excess ratio.

[0080] B23. If the center offset excess ratio or position deviation excess ratio is greater than the set reference value, vibration adjustment is used as the adjustment method, otherwise, manipulator adjustment is used as the adjustment method.

[0081] Understandably, when a large number of relays have position deviations, the efficiency of adjusting them one by one through the manipulator is low, while vibration adjustment can adjust multiple relays at a time. Therefore, when there is a center offset excess ratio or a position deviation excess ratio greater than its set reference value, vibration adjustment is used as the adjustment method, otherwise, manipulator adjustment is used as the adjustment method.

[0082] Further, in step B4, confirming to adjust the vibration frequency and the vibration amplitude includes: B41, setting the weights of the center deviation excess ratio and the position deviation excess ratio, and obtaining the relay distribution deviation degree by weighted average calculation .

[0083] B42. Calculate and adjust vibration frequency , , To set the reference relay distribution deviation, , and represent the set first reference vibration frequency, second reference vibration frequency and base vibration frequency respectively, .

[0084] B43. Calculate and adjust the vibration amplitude , , is the set reference vibration amplitude.

[0085] It should be added that, generally speaking, for smaller position deviations, a higher vibration frequency, such as 50-100 Hz, can be selected, that is, the relay can be fine-tuned in a shorter time so that it gradually returns to the correct position. For larger deviations, a lower vibration frequency, such as 10-30 Hz, may need to be selected to give the relay enough time and displacement space to adjust its position. For example, The value can be 0.4. , and The possible values ​​are 100Hz, 30Hz and 10Hz respectively.

[0086] It should also be added that for slight position deviations, the vibration amplitude can be set within a smaller range, such as ±0.5 mm, to avoid damage to the relay due to over-adjustment. For more serious deviations, the vibration amplitude may need to be increased to ±2 mm or more. When determining the vibration amplitude, it is also necessary to consider the fixing method and bearing capacity of the relay to ensure that the vibration amplitude does not cause the relay to loosen or be damaged. For example, The value can be 0.5 mm.

[0087] The detection and adjustment execution terminal is used to start the corresponding adjustment terminal to perform detection and adjustment based on the preliminary adjustment item and the adjustment index of the preliminary adjustment item.

[0088] It should be added that the specific adjustment for detection adjustment is as follows: when the preparatory adjustment item is the light source, the pulse width modulation controls the luminous intensity of the LED by adjusting the pulse width of the LED driving current. That is, the lighting time of the LED is changed by changing the time proportion of the high level according to the luminous intensity to be adjusted, thereby achieving dimming.

[0089] When the preparatory adjustment item is relay posture and the adjustment method is robot, start the machinery placed on the production line to adjust according to the corresponding adjustment index.

[0090] When the preparatory adjustment item is relay posture and the adjustment method is vibration adjustment, start the vibration terminal placed in the corresponding position of the production line and make corresponding adjustments according to the adjustment index.

[0091] In a specific embodiment, the vibration terminal can be a vibration disk, which is a commonly used device in the relay feeding process. By adjusting the vibration frequency and vibration amplitude of the vibration disk, the relays can be arranged more orderly on the feeding track, reducing uneven placement and inconsistent distances.

[0092] The relay detection and analysis module is used to extract the position of each relay from the panoramic image after completing the detection and adjustment, start the camera group at the corresponding position to collect images, analyze the collected images, and output the relay surface feature set.

[0093] Specifically, analyzing the collected image includes: U1, using an edge detection algorithm to extract the contour of each relay in the collected image, and then calculating the length, width and height of each relay, and constructing a size feature list of each relay.

[0094] U2. Use gray-level co-occurrence matrix to extract the texture features of each relay surface and construct a texture feature list of each relay.

[0095] U3. Locate the number of scratches on the surface of each relay, the scratch area of ​​each scratch, the number of deformations, and the deformation volume of each deformation from the collected images, and construct a defect feature list of each relay.

[0096] U4. Integrate the size feature list, texture feature list and defect feature list of each relay to obtain a relay surface feature set.

[0097] In a specific embodiment, the edge detection algorithm may adopt the Canny algorithm, wherein the Canny algorithm can accurately detect the edge information in the image through steps such as smoothing the image by Gaussian filtering, calculating the gradient amplitude and direction, non-maximum suppression, dual threshold detection and connecting edges, thereby obtaining the outline of the relay.

[0098] It should be added that the length, width and height of the contour can be calculated by counting the pixel points on the contour or using a specific geometric algorithm.

[0099] In a specific embodiment, the texture features include contrast, energy and entropy, and the deformation includes depressions and protrusions, that is, the number of deformations is the sum of the number of depressions and the number of protrusions, and the number of scratches on the surface of each relay, the scratch area of ​​each scratch, the number of deformations, and the deformation volume of each deformation can also be achieved using a target detection algorithm.

[0100] The relay detection feedback terminal is used to feed back the relay surface feature set to the relay production management personnel.

[0101] The embodiment of the present invention ensures the stability of the detection environment by setting a light source information detection module to monitor the light intensity in real time. The relay preliminary detection module collects and splices panoramic images to provide comprehensive data. The detection adjustment judgment module determines the adjustment requirements based on the image and light data. The detection adjustment confirmation module clarifies the adjustment items and indicators to achieve intelligent and precise adjustment. After adjustment, the relay detection and analysis module collects multi-view images for in-depth analysis and outputs surface feature indicators. The relay detection feedback terminal feeds back the indicators to production management personnel to assist quality control, improve detection accuracy, detection flexibility and detection efficiency, and constitute a complete and efficient collaborative detection process.

[0102] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, they shall all fall within the protection scope of the present invention.

Claims

1. A relay surface detection system based on machine vision technology, characterized in that: The system includes: The light source information detection module is used to start the light intensity sensors installed in each light source detection point in the corresponding quality inspection area on the production line to perform real-time detection of light intensity, and obtain the real-time detection light intensity of each light source detection point; The relay preliminary detection module is used to start each camera on the current relay production line to collect images, stitch the collected images, and output a panoramic image; A detection adjustment judgment module is used to judge the detection adjustment needs based on the panoramic image and the real-time detection of the light source intensity at each light source detection point; A detection adjustment confirmation module is used to confirm the prepared adjustment items and the adjustment indicators of the prepared adjustment items when it is determined to be a demand; The detection and adjustment execution terminal is used to start the corresponding adjustment terminal to perform detection and adjustment based on the preliminary adjustment items and the adjustment indicators of the preliminary adjustment items; The relay detection and analysis module is used to extract the position of each relay from the panoramic image after the detection and adjustment is completed, start the camera group at the corresponding position to collect images, analyze the collected images, and output the relay surface feature set; The relay detection feedback terminal is used to feed back the relay surface feature set to the relay production management personnel.

2. A relay surface detection system based on machine vision technology as claimed in claim 1, characterized in that: The determination of the detection and adjustment requirements includes: Analyze the light source interference based on real-time detection of light intensity at each light source detection point ; Taking the transmission direction of the production line as the positive direction, the center line of the production line is constructed along the positive direction. The number of relays and the distance between the center point of each relay and the corresponding center line of the production line are extracted from the panoramic image using target detection technology. The distance between adjacent relays is also extracted. At the same time, the center point positions of each relay are connected in sequence along the positive direction to obtain the relay center point connection line, and the relay position distribution deviation is analyzed. ; The feature points of each relay are obtained from the panoramic image using a feature point detection algorithm, the feature points of each relay are matched with the feature points in the relay image in the standard placement state, and the tilt angle of each relay is calculated using a posture estimation algorithm; Get the three-dimensional outline of the relay from the panoramic image and set the interference degree of the load light source; The light source interference degree exceeding the load light source interference degree is defined as condition 1, the relay position distribution deviation degree is greater than the set position distribution deviation degree is defined as condition 2, and the existence of the relay tilt angle exceeding the set allowable tilt angle is defined as condition 3; Determine whether any of the conditions 1, 2, and 3 are met. If so, the requirement is used as the determination result. Otherwise, the non-requirement is used as the determination result.

3. A relay surface detection system based on machine vision technology as claimed in claim 2, characterized in that: The analyzing the light source interference degree comprises: Calculate the average of the real-time detection light intensity of each light source detection point to obtain the average detection light intensity of each light source detection point; If the average detected light intensity of a certain light source monitoring point is not within the set appropriate detection light intensity range, the light source interference is assigned a value of 1. Otherwise, for the same time, the standard deviation of the corresponding detected light intensity of each light source detection point is calculated to obtain the light difference of different light source detection points at each time. The average light difference of different light source monitoring points is calculated by the mean value, which is recorded as ; For the same light source detection point, the standard deviation of the detected light intensity at different times is calculated to obtain the light intensity difference of each light source detection point at different times. The average light intensity difference at different times is obtained by mean calculation, which is recorded as ; Calculate the interference of light sources , , is a natural constant, and They represent the illumination difference between different light source monitoring points and the illumination difference between different time periods. and They respectively represent the proportion coefficients of illumination difference at different light source monitoring points and illumination difference at different times.

4. A relay surface detection system based on machine vision technology as claimed in claim 3, characterized in that: The analyzing relay position distribution deviation comprises: The distance between the center point of each relay and the corresponding center line of the production line is taken as the center offset of each relay, and the maximum center offset is selected and recorded as ; Calculate the standard deviation of the center offset of each relay and obtain the center offset difference, which is recorded as ,right and Normalization is performed and the results after normalization are used as the evaluation variables of each center deviation, which are recorded as and ,Will As the center offset difference, denoted as ; Extract the shortest distance from the distances between adjacent relays , and calculate the standard deviation of the distance between each adjacent relay to obtain the distance interval difference, recorded as ,right and Normalization is performed, and then the position spacing difference is calculated in the same way as the center offset difference, which is recorded as ; Extract the number of peak points and valley points from the center point line of the relay, sum them up, and divide them by the number of relays. The ratio is recorded as ,Will As the position placement difference, denoted as ; The weights of center offset difference, position spacing difference and position placement difference are set, and the relay position distribution deviation is calculated by weighted average.

5. A relay surface detection system based on machine vision technology as claimed in claim 2, characterized in that: The setting of the interference degree of the light source includes: Based on the three-dimensional contour of the relay, the volume of the relay is obtained, and the minimum circumscribed sphere volume of the relay is calculated. The ratio of the volume of the relay to the volume of the minimum circumscribed sphere is taken as the shape regularity, which is recorded as ; The three-dimensional contour of the relay is divided into various cutting planes in the same direction, and the various cutting planes are sorted according to the order of the cutting, the cutting plane with the first sorting position is recorded as the reference cutting plane, and the other cutting planes are recorded as the analysis cutting planes; Compare and overlap each analysis segmentation surface with the reference segmentation surface to obtain the overlap area, and divide it by the contour area of ​​the corresponding analysis segmentation surface, and record the ratio as the contour overlap ratio; Count the number of analysis segmentation faces whose contour overlap ratio is greater than the set reference contour overlap ratio, and divide it by the total number of analysis segmentation faces. The ratio is recorded as the contour similarity ratio. ; Set the interference level of the light source , , The interference level of the permitted light source is set as the benchmark. and They are respectively expressed as the proportion coefficients corresponding to the set shape regularity and contour similarity ratio, To set the reference contour similarity ratio.

6. A relay surface detection system based on machine vision technology as claimed in claim 4, characterized in that: The confirmation of the preliminary adjustment items includes: If only condition 1 is met, the light source is used as a preliminary adjustment item; If condition 1 is not met, and condition 2 or condition 3 is met, the relay attitude is used as a preliminary adjustment item.

7. A relay surface detection system based on machine vision technology as claimed in claim 6, characterized in that: The adjustment indicators for confirming the preliminary adjustment items include: If the prepared adjustment item is a light source, the current light intensity is extracted from the real-time detected light intensity of each light source detection point, and the difference between the appropriate light intensity for the camera and the current light intensity of each light source detection point is used as the light intensity adjustment value and as the adjustment index; If the prepared adjustment item is relay posture, confirm the adjustment method, including manipulator adjustment and vibration adjustment; If the adjustment item is robot adjustment, if the center offset of a relay is greater than the set allowable placement center offset Or the distance between a relay and its adjacent relay exceeds or falls below the set allowable placement spacing , the relay is recorded as the adjustment relay; The center offset of the adjustment relay and the distance between it and its adjacent relay are respectively recorded as and ,Will As the center offset adjustment value, As the proximity distance adjustment value, the center offset adjustment value and the proximity distance adjustment value of the adjustment relay are then used as adjustment indicators; If the adjustment item is vibration adjustment, confirm the adjustment vibration frequency and adjustment vibration amplitude, and use them as adjustment indicators.

8. A relay surface detection system based on machine vision technology as claimed in claim 7, characterized in that: The confirmation and adjustment method includes: Count the number of relays whose center offset exceeds the permitted center offset and divide it by the total number of relays to obtain the center offset excess ratio; The number of adjacent relay pairs whose distance is less than the allowable placement spacing is counted and divided by the total number of adjacent relay pairs to obtain the position deviation excess ratio; If the center offset excess ratio or the position deviation excess ratio is greater than the set reference value, vibration adjustment is used as the adjustment method, otherwise the manipulator adjustment is used as the adjustment method.

9. A relay surface detection system based on machine vision technology as claimed in claim 8, characterized in that: The step of confirming and adjusting the vibration frequency and the vibration amplitude includes: Set the weights of the center deviation excess ratio and the position deviation excess ratio, and calculate the relay distribution deviation by weighted average. ; Calculate and adjust vibration frequency , , To set the reference relay distribution deviation, , and represent the set first reference vibration frequency, second reference vibration frequency and base vibration frequency respectively, ; Calculate and adjust the vibration amplitude , , is the set reference vibration amplitude.

10. The relay surface detection system based on machine vision technology as claimed in claim 1, characterized in that: The analysis of the collected images includes: The edge detection algorithm is used to extract the outline of each relay in the collected image, and then the length, width and height of each relay are calculated to construct a list of size features of each relay; The gray-level co-occurrence matrix is ​​used to extract the texture features of each relay surface and construct a texture feature list of each relay; The number of scratches on the surface of each relay, the scratch area of ​​each scratch, the number of deformations, and the deformation volume of each deformation are located from the collected images, and a defect feature list of each relay is constructed; The size feature list, texture feature list and defect feature list of each relay are integrated to obtain the relay surface feature set.

Images