Laser repair method, device, equipment and storage medium for multi-workpiece table

Abstract

The present invention relates to the field of laser repair technology, and in particular to a laser repair method, device, equipment and storage medium for a multi-workpiece table. The method comprises: acquiring calibration image data and performing coordinate extraction processing, completing coordinate calibration between a workstation and a camera device based on the extracted feature coordinate information, and obtaining a detection determination posture matrix, a repair posture matrix and a re-inspection posture matrix; when the detection field magnification of the camera device is less than a preset magnification threshold, determining a first position to be repaired based on the acquired image data to be detected and the detection determination posture matrix, and then confirming a first position to be operated based on the first position to be repaired and the repair posture matrix, so as to perform laser repair of a defective panel, and finally, re-inspecting the defective panel based on the acquired first re-inspection image data, the first position to be repaired and the re-inspection posture matrix; the method disclosed in the present application can realize automatic detection, repair and re-inspection of defective panels, and effectively improve the overall repair efficiency.

Description

Technical Field

[0001] The present invention relates to the field of laser repair technology, and in particular to a laser repair method, device, equipment and storage medium for a multi-workpiece table. Background Art

[0002] With the rapid development of materials science, microelectronics technology, and semiconductor manufacturing and applications, the application areas of display panels are expanding; despite the continuous optimization and improvement of manufacturing equipment and production processes, the complex processes in the production process still inevitably lead to product defects, such as bright spots, flash points, and broken bright spots. These defects have a negative impact on display quality and thus reduce product yield; in addition, as the size of display panels continues to increase, it has become more economical to repair these defective substrates than to discard defective substrates during the manufacturing process; therefore, using laser technology to repair bad panels has become a common panel repair method.

[0003] The laser repair process usually includes five main steps: loading, alignment, inspection and evaluation, repair, and re-inspection of unloading. In existing production workshops, the repair process is a semi-automatic process, which requires manual intervention to determine the repair plan and specific location. At present, the bottleneck of panel repair efficiency lies in the manual inspection and judgment links. The repair work can only be started after the manual judgment is completed, and manual re-inspection is required after the repair is completed. This efficiency limitation makes it difficult for panel repair equipment to be synchronized with assembly line production. Furthermore, since the number of panels that need to be repaired is uncertain and the time required for different repair methods varies, it becomes complicated to integrate the repair work into assembly line production.

[0004] Currently, production workshops generally have multiple repair equipment, which cooperate with handling robotic arms and scheduling software to deliver panels to be repaired to idle equipment in real time. In other words, the bottleneck problem of production efficiency is solved by increasing the number of repair equipment. However, this method has disadvantages such as large footprint, high risk of equipment redundancy, and potential waste of resources.

[0005] It can be seen that the existing technology still needs to be improved and enhanced. Summary of the invention

[0006] In order to overcome the deficiencies of the prior art, an object of the present invention is to provide a laser repair method for a multi-workpiece table, which can realize automatic detection, repair and re-inspection of defective panels, and effectively improve the overall repair efficiency.

[0007] The first aspect of the present invention provides a laser repair method for a multi-workpiece table, comprising: obtaining calibration image data fed back by a camera device, in which a calibration plate is located at a detection and determination station, a repair station, and a re-inspection station, performing coordinate extraction processing on the calibration image data to obtain feature coordinate information; completing coordinate calibration between the detection and determination station, the repair station, and the re-inspection station and the camera device based on the feature coordinate information to obtain a detection and determination posture matrix, a repair posture matrix, and a re-inspection posture matrix; judging whether the detection field magnification of the camera device is less than a preset magnification threshold, and if so, obtaining the workpiece The workbench is located at the detection and judgment station, and the image data to be detected is obtained. Based on the image data to be detected and the detection and judgment posture matrix, the first position to be repaired is determined; when the workbench is located at the repair station, the first position to be worked on is confirmed based on the first position to be repaired and the repair posture matrix, and the defective panel on the workbench is laser repaired based on the confirmed first position to be worked on; the first re-inspection image data of the workbench when the workbench is located at the re-inspection station is obtained, and based on the first re-inspection image data, the first position to be repaired and the re-inspection posture matrix, the defective panel on the workbench is re-inspected to obtain a first re-inspection result.

[0008] Optionally, in a first embodiment of the first aspect of the present invention, the determination of whether the detection field of view magnification of the camera device is less than a preset magnification threshold value further includes: if not, obtaining calibration image data fed back by the camera device and the calibration plate located at the calibration alignment station, performing coordinate extraction processing on the calibration image data to obtain calibration coordinate information; completing coordinate calibration between the calibration alignment station and the camera device based on the calibration coordinate information to obtain a calibration alignment pose matrix; obtaining image data to be calibrated when the workbench is located at the calibration alignment station, and determining a second position to be repaired based on the image data to be calibrated and the calibration alignment pose matrix; when the workbench is located at the repair station, confirming a second position to be repaired based on the second position to be repaired, the detection determination pose matrix and the repair pose matrix, and performing laser repair on the defective panel on the workbench based on the confirmed second position to be repaired; obtaining second re-inspection image data when the workbench is located at the re-inspection station, and re-inspecting the defective panel on the workbench based on the second re-inspection image data, the second position to be repaired and the re-inspection pose matrix to obtain a second re-inspection result.

[0009] Optionally, in a second implementation of the first aspect of the present invention, the calibration image data fed back by the camera device, in which the calibration plate is located at the detection and determination station, the repair station and the re-inspection station, is obtained, and coordinate extraction processing is performed on the calibration image data to obtain feature coordinate information, specifically including: obtaining calibration image data fed back by the camera device, in which the calibration plate is located at the detection and determination station, the repair station and the re-inspection station, the calibration image data including calibration images of multiple angles in which the calibration plate is respectively located at the detection and determination station, the repair station and the re-inspection station; performing grayscale conversion processing and filtering and denoising processing on the multiple calibration images one by one using the OpenCV library to obtain preprocessed calibration image data; performing feature extraction processing on the preprocessed calibration image data using the Harris corner detection algorithm to obtain feature coordinate information, and saving the feature coordinate information as a data file in CSV format.

[0010] Optionally, in a third embodiment of the first aspect of the present invention, the coordinate calibration between the detection and judgment station, the repair station and the re-inspection station and the camera device is completed based on the feature coordinate information to obtain the detection and judgment pose matrix, the repair pose matrix and the re-inspection pose matrix, which specifically includes: obtaining the actual feature coordinate information of the calibration plate, and calculating the external parameter matrix of the camera device using Zhang Zhengyou calibration algorithm based on the actual feature coordinate information and the feature coordinate information; obtaining the three-dimensional position information of the detection and judgment station, the three-dimensional position information of the repair station and the three-dimensional position information of the re-inspection station, and respectively calculating the detection and judgment pose matrix, the repair pose matrix and the re-inspection pose matrix based on the acquired three-dimensional position information and the calculated external parameter matrix.

[0011] Optionally, in a fourth implementation manner of the first aspect of the present invention, the acquiring of image data to be detected on a workbench located at a detection and determination station, and determining a first position to be repaired based on the image data to be detected and a detection and determination posture matrix, specifically comprises: acquiring image data to be detected on a workbench located at a detection and determination station, the image data to be detected comprising a defective panel image of a defective panel located on the workbench; inputting the defective panel image into a defect detection model pre-built based on a YOLOv3 tiny network to obtain a defect marking image, wherein defect position information is recorded on the defect marking image, and assuming that the defect position information is ; Assume that the detection determination pose matrix is , based on the defect position information and the detection and determination posture matrix, the first position to be repaired is determined. .

[0012] Optionally, in a fifth implementation of the first aspect of the present invention, when the workbench is located on the repair station, the first position to be repaired is confirmed based on the first position to be repaired and the repair posture matrix, and laser repair is performed on the defective panel on the workbench based on the confirmed first position to be repaired, specifically including: judging whether there is a vacant repair station, if not, moving the workbench to the waiting station; when a vacant repair station appears, moving the workbench to the repair station; assuming that the repair posture matrix is , based on the first position to be repaired and the repair posture matrix, the first position to be worked on is determined, and the first position to be worked on is / ; Perform laser repair on the defective panel on the workbench based on the confirmed first position to be operated.

[0013] Optionally, in a sixth embodiment of the first aspect of the present invention, the first re-inspection image data of the workbench being located at the re-inspection station is obtained, and based on the first re-inspection image data, the first position to be repaired and the re-inspection posture matrix, the defective panel on the workbench is re-inspected to obtain a first re-inspection result, specifically including: obtaining first re-inspection position information based on the first position to be repaired and the re-inspection posture matrix; obtaining first re-inspection image data of the workbench being located at the re-inspection station, and performing grayscale conversion and filtering denoising on the first re-inspection image data using the OpenCV library to obtain first pre-processed re-inspection image data; and based on the first pre-processed re-inspection image data and the first re-inspection position information, using Mura evaluation technology to re-inspect the defective panel on the workbench to obtain a first re-inspection result.

[0014] The second aspect of the present invention provides a laser repair device for a multi-workpiece table, including: an extraction module, which is used to obtain calibration image data fed back by a camera device, in which a calibration plate is located at a detection and determination station, a repair station, and a re-inspection station, and to perform coordinate extraction processing on the calibration image data to obtain feature coordinate information; a calibration module, which is used to complete the coordinate calibration between the detection and determination station, the repair station, and the re-inspection station and the camera device based on the feature coordinate information, and obtain a detection and determination posture matrix, a repair posture matrix, and a re-inspection posture matrix; a determination module, which determines whether the detection field magnification of the camera device is less than a preset magnification threshold, and if so, obtains The image data to be inspected when the workbench is located at the inspection and judgment station is obtained, and the first position to be repaired is determined based on the image data to be inspected and the inspection and judgment posture matrix; the repair module is used to confirm the first position to be worked on based on the first position to be repaired and the repair posture matrix when the workbench is located at the repair station, and perform laser repair on the defective panel on the workbench based on the confirmed first position to be worked on; the re-inspection module is used to obtain the first re-inspection image data when the workbench is located at the re-inspection station, and re-inspect the defective panel on the workbench based on the first re-inspection image data, the first position to be repaired and the re-inspection posture matrix to obtain a first re-inspection result.

[0015] The third aspect of the present invention provides a laser repair device for a multi-workpiece table, the laser repair device for a multi-workpiece table comprising: a memory and at least one processor, the memory storing instructions; at least one of the processors calls the instructions in the memory to enable the laser repair device for a multi-workpiece table to perform each step of the laser repair method for a multi-workpiece table described in any one of the above-mentioned items.

[0016] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, and when the instructions are executed by a processor, the various steps of the laser repair method for a multi-workpiece table described in any one of the above items are implemented.

[0017] In the technical solution of the present invention, by acquiring calibration image data and performing coordinate extraction processing, the coordinate calibration between the workstation and the camera device is completed based on the extracted feature coordinate information, and the detection and determination posture matrix, the repair posture matrix and the re-inspection posture matrix are obtained; the first position to be repaired is determined based on the acquired image data to be detected, where the workbench is located at the detection and determination station, and the detection and determination posture matrix, and then the first position to be operated is confirmed based on the first position to be repaired and the repair posture matrix, so as to perform laser repair of the defective panel at the repair station, and finally, the defective panel is re-inspected based on the acquired first re-inspection image data, the first position to be repaired and the re-inspection posture matrix, where the workbench is located at the re-inspection station; the method disclosed in the present application can realize automatic detection, repair and re-inspection of defective panels, and effectively improve the overall repair efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A first flow chart of a laser repair method for a multi-workpiece table provided in an embodiment of the present invention;

[0019] Figure 2 A second flow chart of the laser repair method for a multi-workpiece table provided in an embodiment of the present invention;

[0020] Figure 3 A third flow chart of the laser repair method for a multi-workpiece table provided in an embodiment of the present invention;

[0021] Figure 4 A fourth flow chart of a laser repair method for a multi-workpiece table provided in an embodiment of the present invention;

[0022] Figure 5 A fifth flow chart of the laser repair method for a multi-workpiece table provided in an embodiment of the present invention;

[0023] Figure 6 A sixth flow chart of a laser repair method for a multi-workpiece table provided in an embodiment of the present invention;

[0024] Figure 7 A seventh flow chart of a laser repair method for a multi-workpiece table provided in an embodiment of the present invention;

[0025] Figure 8 A schematic structural diagram of a laser repair device for a multi-workpiece table provided in an embodiment of the present invention;

[0026] Fig. 9 A schematic structural diagram of a laser repair device for a multi-workpiece table provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention provides a laser repair method, device, equipment and storage medium for a multi-workpiece table. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the laser repair method for a multi-workpiece table in an embodiment of the present invention includes:

[0029] 101. Acquire calibration image data fed back by the camera device, in which the calibration plate is located at the detection and determination station, the repair station, and the re-inspection station, and perform coordinate extraction processing on the calibration image data to obtain feature coordinate information;

[0030] In this embodiment, a calibration plate is usually used in a machine vision system. It has known geometric features and dimensions and can be used to calibrate the parameters of a camera and a lens to ensure measurement accuracy. The calibration plate can be a checkerboard calibration plate, on which black and white square grids are printed to form a checkerboard pattern. The calibration plate is placed at the detection and determination station, the repair station, and the re-inspection station of the camera device to obtain feedback image data of the camera device. By performing coordinate extraction processing on the obtained calibration image data, characteristic coordinate information used for calibration of the camera device can be obtained. The characteristic coordinate information is crucial for correcting parameters such as the viewing angle and distortion of the camera device to ensure the accuracy of image processing and measurement.

[0031] 102. Complete the coordinate calibration between the detection and determination station, the repair station, the re-inspection station and the camera device based on the feature coordinate information to obtain the detection and determination posture matrix, the repair posture matrix and the re-inspection posture matrix;

[0032] In this embodiment, the detection and determination posture matrix, the repair posture matrix and the re-inspection posture matrix are 4x4 homogeneous matrices, which are used to describe the position and direction of the detection and determination station, the repair station and the re-inspection station in three-dimensional space. Specifically, they are used to describe the relative position relationship between the detection and determination station, the repair station and the re-inspection station and the camera device; the posture matrix, the repair posture matrix and the re-inspection posture matrix are constructed based on the rotation matrix R and the translation vector T, the rotation matrix R is a 3x3 rotation matrix, which describes the rotation relationship between the camera device and the station, and the translation vector T describes the translation relationship between the camera device and the station.

[0033] In this embodiment, the coordinate extraction process and the coordinate calibration process ensure the precise alignment between various types of workstations and the camera device, provide a reliable basis for subsequent detection, repair and re-inspection, and ensure the accuracy and reliability of subsequent processing.

[0034] 103. Determine whether the detection field magnification of the camera device is less than a preset magnification threshold. If so, obtain the image data to be detected on the workbench at the detection and determination station, and determine the first position to be repaired based on the image data to be detected and the detection and determination posture matrix;

[0035] In this embodiment, the defect problem of the defective panel on the workbench and the defect position corresponding to the defect problem are confirmed based on the image data to be detected, and then the association between the defect position and the camera coordinate system is determined based on the detection posture matrix, which points out the direction for subsequent repair work and improves the accuracy and efficiency of detection.

[0036] 104. When the workbench is located on the repair station, a first position to be repaired is confirmed based on the first position to be repaired and the repair posture matrix, and laser repair is performed on the defective panel on the workbench based on the confirmed first position to be repaired;

[0037] In this embodiment, the first position to be repaired is accurately located based on the repair posture matrix and the first position to be repaired, and laser technology is used to perform precise repair, thereby ensuring the accuracy and efficiency of the repair work.

[0038] 105. Acquire first re-inspection image data of a workbench located at a re-inspection station, and re-inspect the defective panel on the workbench based on the first re-inspection image data, the first position to be repaired, and the re-inspection posture matrix to obtain a first re-inspection result;

[0039] In this embodiment, the position to be re-inspected is confirmed based on the first position to be repaired and the re-inspection posture matrix, and the defective panel is re-inspected based on the first re-inspection image data and the position to be re-inspected, so that comprehensive and detailed autonomous re-inspection work can be achieved, ensuring the reliability of the quality of the repaired panel.

[0040] The present application discloses a laser repair method for multiple workpiece tables, which obtains calibration image data and performs coordinate extraction processing, completes the coordinate calibration between the workstation and the camera device based on the extracted feature coordinate information, and obtains the detection and determination posture matrix, the repair posture matrix and the re-inspection posture matrix; determines the first position to be repaired based on the acquired image data to be detected, where the worktable is located at the detection and determination station, and the detection and determination posture matrix, and then confirms the first position to be repaired based on the first position to be repaired and the repair posture matrix, so as to perform laser repair of the defective panel at the repair station, and finally, re-inspects the defective panel based on the acquired first re-inspection image data, the first position to be repaired and the re-inspection posture matrix, where the worktable is located at the re-inspection station; the method disclosed in the present application can realize automatic detection, repair and re-inspection of defective panels, and effectively improve the overall repair efficiency; further, by calibrating the coordinates of various types of workstations, there is no need to repeatedly adjust the position of the worktable multiple times in the future, and the association and conversion between coordinates can be realized through the obtained posture matrix, further improving the overall repair efficiency.

[0041] See also Figure 2 The second embodiment of the laser repair method for a multi-workpiece table in the embodiment of the present invention includes:

[0042] 201. If not, obtain calibration image data fed back by the camera device, in which the calibration plate is located on the calibration alignment station, and perform coordinate extraction processing on the calibration image data to obtain calibration coordinate information;

[0043] 202. Complete coordinate calibration between the calibration alignment station and the camera device based on the calibration coordinate information to obtain a calibration alignment pose matrix;

[0044] In this embodiment, the calibration alignment pose matrix is ​​calculated based on the extrinsic parameter matrix of the camera device, the second characteristic coordinate information and the pre-acquired three-dimensional position information of the calibration alignment station; the calibration alignment pose matrix is ​​a 4x4 homogeneous matrix, which is used to describe the position and direction of the calibration alignment station in the three-dimensional space, and specifically, to describe the relative position relationship between the calibration alignment station and the camera device; the calibration alignment pose matrix is ​​constructed based on the rotation matrix R and the translation vector T, the rotation matrix R is a 3x3 rotation matrix, which describes the rotation relationship between the camera device and the station, and the translation vector T describes the translation relationship between the camera device and the station.

[0045] 203. Acquire the image data to be calibrated of the workbench located at the calibration and alignment station, and determine the second position to be repaired based on the image data to be calibrated and the calibration and alignment pose matrix;

[0046] 204. When the workbench is located on the repair station, a second position to be repaired is confirmed based on the second position to be repaired, the detection and determination posture matrix, and the repair posture matrix, and laser repair is performed on the defective panel on the workbench based on the confirmed second position to be repaired;

[0047] 205. Obtain second re-inspection image data of the workbench at the re-inspection station, and re-inspect the defective panel on the workbench based on the second re-inspection image data, the second position to be repaired, and the re-inspection posture matrix to obtain a second re-inspection result.

[0048] In the actual working process, the position error of each type of workstation is generally small, and because the detection field of view of the camera device is large, the above steps 201 and 205 may not be performed in actual production; however, when a large error or a large detection field of view magnification occurs in actual production, if the defect position on the defective panel is not calibrated, it is easy to exceed the field of view and cause the defect position to be unable to be captured; when calibration is required, the workbench is placed on the calibration and alignment station, and after preprocessing and defect detection of the image to be calibrated taken by the camera device, the second position to be repaired when the workbench is located on the calibration and alignment station is obtained, which is recorded as , and then by calculating , find the defect center value ; After the workbench has been calibrated and aligned, the second position to be repaired can be identified. Therefore, there is no need to transport the workbench to the detection and judgment station, and the workbench can be directly transferred to the repair station. After the workbench is transferred to the repair station, the calculation formula , and the position of the second waiting job is , ensuring that the defects are within the image field of view to ensure that subsequent repair tasks are accurate and reliable.

[0049] In this embodiment, the repairing step and re-inspecting step based on the second position to be repaired may refer to the repairing step and re-inspecting step based on the first position to be repaired, and the working principles of the two are the same.

[0050] See also Figure 3 A third embodiment of the laser repair method for a multi-workpiece table in the embodiment of the present invention includes:

[0051] 301. Acquire calibration image data fed back by a camera device, in which the calibration plate is located at a detection and determination station, a repair station, and a re-inspection station, wherein the calibration image data includes calibration images at multiple angles when the calibration plate is respectively located at the detection and determination station, the repair station, and the re-inspection station;

[0052] In this embodiment, calibration images of the calibration plate at different types of workstations and at different angles are collected to provide more comprehensive visual information, thereby improving the accuracy and reliability of detection. Furthermore, multi-angle calibration images help to calibrate the camera device more accurately, ensuring that in the actual working environment, accurate detection results can be obtained regardless of the position or angle of the calibration plate.

[0053] 302. Use the OpenCV library to perform grayscale conversion and filtering denoising on multiple calibration images one by one to obtain pre-processed calibration image data;

[0054] In this embodiment, the OpenCV library is used to perform grayscale conversion and filtering denoising on multiple calibration images one by one, so as to obtain preprocessed calibration image data; the grayscale conversion process converts the color image into a grayscale image, which simplifies the complexity of data processing while retaining the brightness information of the image; the filtering denoising process reduces the noise in the image and improves the image quality by applying the Gaussian filtering algorithm; and the preprocessing of the calibration image data facilitates the subsequent image analysis and feature extraction, thereby improving the accuracy and robustness of the feature extraction algorithm.

[0055] 303. Perform feature extraction processing on the pre-processed calibration image data using the Harris corner detection algorithm to obtain feature coordinate information, and save the feature coordinate information as a data file in CSV format;

[0056] In this embodiment, the Harris corner detection algorithm is applied to identify corner points in an image. The Harris corner detection algorithm determines the corner points by calculating the Harris response function value of each pixel point. A point with a high response function value indicates that the point is likely to be a corner point. Local maximum points are extracted from the Harris response graph as corner points, and the coordinate information of these corner points is recorded, and then the extracted corner point coordinate information is saved as a data file in CSV format. The Harris corner detection algorithm can accurately identify the corner point features in an image, and these features are very useful for tasks such as image matching and target recognition.

[0057] See also Figure 4 The fourth embodiment of the laser repair method for a multi-workpiece table in the embodiment of the present invention includes:

[0058] 401. Acquire actual feature coordinate information of the calibration plate, and calculate the extrinsic parameter matrix of the camera device using the Zhang Zhengyou calibration algorithm based on the actual feature coordinate information and the feature coordinate information;

[0059] 402. Acquire the three-dimensional position information of the detection and determination station, the three-dimensional position information of the repair station, and the three-dimensional position information of the re-inspection station, and respectively calculate the detection and determination pose matrix, the repair pose matrix, and the re-inspection pose matrix based on the acquired three-dimensional position information and the calculated external parameter matrix;

[0060] In this embodiment, first, the key point coordinates are extracted according to the feature coordinate information, and the extracted key point coordinates are converted from the camera coordinate system to the world coordinate system using the known extrinsic parameter matrix. Then, according to the known directions and positions of the detection and judgment station, the repair station and the re-inspection station in the three-dimensional space and based on the converted coordinates, the detection and judgment pose matrix, the repair pose matrix and the re-inspection pose matrix are calculated.

[0061] See also Figure 5 The fifth embodiment of the laser repair method for a multi-workpiece table in the embodiment of the present invention includes:

[0062] 501. Acquire image data to be detected on a workbench located at a detection and determination station, wherein the image data to be detected includes a defective panel image of a defective panel located on the workbench;

[0063] In this embodiment, after acquiring the defective panel image, a median filter is used to perform denoising processing on the defective panel image, and then the filtered defective panel image is sequentially subjected to image enhancement processing and image binarization processing to convert the defective panel image into a grayscale image; then the Canny edge detection algorithm is used to identify the edges in the grayscale image to prepare for defect location; finally, the grayscale image after edge detection is subjected to image segmentation processing to divide the grayscale image into multiple regions, so as to facilitate the subsequent separate analysis of the defect situation of each region.

[0064] 502. Input the defect panel image into the defect detection model pre-built based on the YOLOv3 tiny network to obtain a defect mark image, on which the defect position information is recorded. Suppose the defect position information is ;

[0065] In this embodiment, a dataset of labeled defective panel images is collected, and it is ensured that the collected dataset of labeled defective panel images contains various types of defects; then the YOLOv3 tiny network is selected as the infrastructure, and the labeled dataset is used to train the YOLOv3 tiny network, and the network parameters are adjusted by the Bayesian optimization algorithm to optimize the defect detection performance; finally, the cross-validation method is used to evaluate the impact of parameter adjustment on the generalization ability of the model, and the defect detection model that passes the evaluation is deployed to the repair system; compared with other complex deep learning models, YOLOv3 tiny can process images quickly and realize real-time defect detection, and while ensuring performance, it has low computing resource requirements, which helps to reduce deployment costs.

[0066] 503. Assume that the detection and determination posture matrix is , based on the defect position information and the detection and determination posture matrix, the first position to be repaired is determined. .

[0067] See also Figure 6 The sixth embodiment of the laser repair method for a multi-workpiece table in the embodiment of the present invention includes:

[0068] 601. Determine whether there is an empty repair station. If not, move the workbench to the waiting station.

[0069] In this embodiment, it is possible to determine whether there are any vacant repair stations by using the in-place sensors on the repair stations, or to determine whether there are any vacant repair stations by using the real-time images obtained by the camera device; further, by introducing waiting stations, the bottleneck problem in the production process can be effectively alleviated, the interruption of the production process can be avoided, and the continuous operation of the production line can be ensured; further, the existence of waiting stations also provides greater flexibility for production scheduling, which helps to improve overall production efficiency and equipment utilization.

[0070] Furthermore, in this embodiment, when the waiting station includes multiple workbenches, the White Shark optimization algorithm can be used to generate a scheduling plan for the workbenches according to the defect quantity and defect type of the defective panel to confirm the transfer order of the workbenches from the waiting station to the repair station; specifically, the laser repair system will collect the defect quantity information and defect type of each defective panel, and confirm the estimated repair time based on the defect quantity information and defect type, and then the White Shark optimization algorithm will calculate the priority of each workbench according to the system's estimated repair time, and arrange the transfer order of the workbenches from the waiting station to the repair station according to the priority; generating the transfer order through the White Shark optimization algorithm can optimize resource allocation, ensure that the workbenches are allocated to where they are most needed, avoid resource waste, and improve the repair efficiency of defective panels, reducing the panel's stay time at the waiting station.

[0071] 602. When a vacant repair station appears, move the workbench to the repair station;

[0072] In this embodiment, the workbench can be transferred between different types of workstations by a robot arm or by a production conveyor line.

[0073] 603. Assume that the repair pose matrix is , based on the first position to be repaired and the repair pose matrix, the first position to be worked on is determined, and the first position to be worked on is set to , the first waiting operation position, repair posture matrix, defect position information and detection determination posture matrix satisfy the calculation formula: ;

[0074] In this embodiment, the constructed repair pose matrix and detection determination pose matrix are both constructed based on the rotation matrix R and the translation vector T, that is, the coordinate association between the repair pose matrix and the detection determination pose matrix is ​​realized through the camera device; therefore, the relationship between the first position to be worked on, the repair pose matrix, the defect position information and the detection determination pose matrix follows the law of coordinate transformation; according to the theory of coordinate transformation, the defect position is converted to the first position to be repaired through the detection determination pose matrix, and the repair pose matrix converts the first position to be repaired to the first position to be worked on; therefore, the product of the detection determination pose matrix and the defect position information should be equal to the product between the first position to be worked on and the repair pose matrix.

[0075] 604. Perform laser repair on the defective panel on the workbench based on the confirmed first waiting position for operation;

[0076] In this embodiment, the calculated first position data to be operated is transmitted to the laser repair equipment. The laser repair equipment adjusts the position of the laser head according to the first position data to be operated so as to align the defective area for repair. Parameters such as the intensity, pulse width and frequency of the laser beam need to be adjusted according to the type and severity of the defect.

[0077] See also Figure 7 The seventh embodiment of the laser repair method for a multi-workpiece table in the embodiment of the present invention includes:

[0078] 701. Acquire first re-inspection position information based on the first position to be repaired and the re-inspection posture matrix;

[0079] In this embodiment, the rechecked posture matrix is ​​assumed to be , let the first recheck position information be The first re-inspection position information, the re-inspection posture matrix and the first position to be repaired satisfy the calculation formula: .

[0080] 702. Acquire first re-inspection image data of a workbench located at a re-inspection station, and use the OpenCV library to perform grayscale conversion and filtering denoising on the first re-inspection image data to obtain first pre-processed re-inspection image data;

[0081] 703. Based on the first pre-processed re-inspection image data and the first re-inspection position information, re-inspect the defective panel on the workbench using a Mura evaluation technology to obtain a first re-inspection result.

[0082] In this embodiment, the defective panel on the workbench is re-inspected by using the Mura evaluation technology. The Mura evaluation technology is a technology used to detect defects on the display panel. It can identify defects such as uneven brightness, color difference, scratches, and stains on the panel, and compare them with theoretical values ​​to confirm the defect repair effect, thereby confirming the defect repair rate. The defect repair rate = (the number of defects solved) / the total number of defects. The first re-inspection result is the defect repair rate. The defect repair rate is compared with the preset repair pass rate to determine whether the repair is completed. When the repair is completed, the material is unloaded to the finished product area. When the repair is not completed and can no longer be repaired, the material is unloaded to the waste area. By applying the Mura evaluation technology, the extracted defect position can be analyzed in detail to independently determine whether the defective panel is repaired. It has the advantages of high detection efficiency, short detection cycle and high detection accuracy. In addition, the automated re-inspection process reduces the dependence on manual operation, thereby reducing labor costs.

[0083] The above describes the laser repair method for a multi-workpiece table in an embodiment of the present invention. The following describes the laser repair device for a multi-workpiece table in an embodiment of the present invention. Figure 8, an embodiment of the laser repair device for a multi-workpiece table in the embodiment of the present invention includes:

[0084] The extraction module 801 is used to obtain the calibration image data fed back by the camera device, in which the calibration plate is located at the detection and determination station, the repair station and the re-inspection station, and perform coordinate extraction processing on the calibration image data to obtain feature coordinate information;

[0085] The calibration module 802 is used to complete the coordinate calibration between the detection and determination station, the repair station, and the re-inspection station and the camera device based on the feature coordinate information to obtain the detection and determination posture matrix, the repair posture matrix, and the re-inspection posture matrix;

[0086] The determination module 803 determines whether the detection field magnification of the camera device is less than a preset magnification threshold. If so, the image data to be detected on the workbench at the detection and determination station is obtained, and the first position to be repaired is determined based on the image data to be detected and the detection and determination posture matrix;

[0087] The repair module 804 is used for, when the workbench is located at the repair station, confirming the first position to be repaired based on the first position to be repaired and the repair posture matrix, and performing laser repair on the defective panel on the workbench based on the confirmed first position to be repaired;

[0088] The re-inspection module 805 is used to obtain the first re-inspection image data of the workbench at the re-inspection station, and re-inspect the defective panel on the workbench based on the first re-inspection image data, the first position to be repaired and the re-inspection posture matrix to obtain a first re-inspection result.

[0089] Based on the same idea as the method in the above embodiment, the device provided by the present application can implement the method in the above embodiment.

[0090] above Figure 8 The laser repair device for a multiple workpiece table in an embodiment of the present invention is described in detail from the perspective of modular functional entities. The laser repair device for a multiple workpiece table in an embodiment of the present invention is described in detail from the perspective of hardware processing.

[0091] Fig. 9: is a schematic diagram of the structure of a laser repair device for multiple workpiece tables provided in an embodiment of the present invention. The laser repair device 900 for multiple workpiece tables may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 910 (for example, one or more processors) and a memory 920, and one or more storage media 930 (for example, one or more mass storage devices) storing application programs 933 or data 932. Among them, the memory 920 and the storage medium 930 can be short-term storage or permanent storage. The program stored in the storage medium 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the laser repair device 900 for multiple workpiece tables. Furthermore, the processor 910 can be configured to communicate with the storage medium 930, and execute a series of instruction operations in the storage medium 930 on the laser repair device 900 for multiple workpiece tables to implement the steps of the laser repair method for multiple workpiece tables provided in the above-mentioned method embodiments.

[0092] The laser repair device 900 for multiple workpiece tables may further include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input and output interfaces 960, and / or one or more operating systems 931, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art may understand that the structure of the laser repair device for multiple workpiece tables shown in the present application does not constitute a limitation on the laser repair device based on multiple workpiece tables, and may include more or less components than shown in the figure, or combine certain components, or arrange components differently.

[0093] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of a laser repair method for a multi-workpiece table.

[0094] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0095] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0096] Finally, it should be noted that the above description is only a preferred example of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A laser repair method for a multi-workpiece table, characterized in that: include: Step 1: obtain calibration image data fed back by the camera device when the calibration plate is located at the detection and determination station, the repair station, and the re-inspection station, perform coordinate extraction processing on the calibration image data, and obtain feature coordinate information; Step 2: Based on the feature coordinate information, complete the coordinate calibration between the detection and determination station, the repair station, the re-inspection station and the camera device to obtain the detection and determination pose matrix, the repair pose matrix and the re-inspection pose matrix; Step 3: Determine whether the detection field magnification of the camera device is less than a preset magnification threshold; if so, obtain the image data to be detected on the workbench at the detection and determination station, and determine the first position to be repaired based on the image data to be detected and the detection and determination posture matrix, specifically including: Acquire image data to be detected on a workbench located at a detection and determination station, wherein the image data to be detected includes a defective panel image of a defective panel located on the workbench; The defective panel image is input into the defect detection model pre-built based on the YOLOv3 tiny network to obtain a defect mark image, on which the defect location information is recorded. Suppose the defect location information is ; Assume that the detection determination pose matrix is , based on the defect position information and the detection and determination posture matrix, the first position to be repaired is determined, and the first position to be repaired is ; When the workbench is located on the repair station, the first position to be repaired is confirmed based on the first position to be repaired and the repair posture matrix, and the defective panel on the workbench is laser repaired based on the confirmed first position to be repaired, specifically including: Determine whether there is an available repair station. If not, move the workbench to the waiting station; When a vacant repair station appears, move the workbench to the repair station; Assume the repair pose matrix is , based on the first position to be repaired and the repair posture matrix, the first position to be worked on is determined, and the first position to be worked on is / ; Performing laser repair on the defective panel on the workbench based on the confirmed first position to be operated; Acquire first re-inspection image data of a workbench located at a re-inspection station, and re-inspect the defective panel on the workbench based on the first re-inspection image data, the first position to be repaired, and the re-inspection posture matrix to obtain a first re-inspection result, specifically including: Based on the first position to be repaired and the re-inspection posture matrix, obtaining first re-inspection position information; Acquire first re-inspection image data where the workbench is located at the re-inspection station, and use the OpenCV library to perform grayscale conversion processing and filtering and denoising processing on the first re-inspection image data to obtain first pre-processed re-inspection image data; Based on the first pre-processed re-inspection image data and the first re-inspection position information, the defective panel on the workbench is re-inspected using the Mura evaluation technology to obtain a first re-inspection result; In step 3, if the detection field magnification of the camera device is not less than the preset magnification threshold, the calibration image data fed back by the camera device and the calibration plate located on the calibration alignment station is obtained, and the coordinate extraction process is performed on the calibration image data to obtain the calibration coordinate information; Based on the calibration coordinate information, coordinate calibration between the calibration alignment station and the camera device is completed to obtain a calibration alignment pose matrix; Acquire the image data to be calibrated on the calibration and alignment station of the workbench, and determine the second position to be repaired based on the image data to be calibrated and the calibration and alignment pose matrix; When the workbench is located on the repair station, the second position to be repaired is confirmed based on the second position to be repaired, the detection and determination posture matrix and the repair posture matrix, and the defective panel on the workbench is laser repaired based on the confirmed second position to be repaired; Acquire second re-inspection image data of the workbench at the re-inspection station, and re-inspect the defective panel on the workbench based on the second re-inspection image data, the second position to be repaired, and the re-inspection posture matrix to obtain a second re-inspection result.

2. A laser repair method for a multi-workpiece table according to claim 1, characterized in that: The step 1 of obtaining calibration image data fed back by the camera device when the calibration plate is located at the detection and determination station, the repair station, and the re-inspection station, and performing coordinate extraction processing on the calibration image data to obtain feature coordinate information specifically includes: Acquire calibration image data fed back by a camera device when the calibration plate is located at a detection and determination station, a repair station, and a re-inspection station, wherein the calibration image data includes calibration images at multiple angles when the calibration plate is located at the detection and determination station, the repair station, and the re-inspection station respectively; Use the OpenCV library to perform grayscale conversion and filtering denoising on multiple calibration images one by one to obtain pre-processed calibration image data; Harris corner detection algorithm is used to perform feature extraction on the preprocessed calibration image data to obtain feature coordinate information, which is then saved as a data file in CSV format.

3. The laser repair method for a multi-workpiece table according to claim 1, characterized in that: The coordinate calibration between the detection and determination station, the repair station, and the re-inspection station and the camera device is completed based on the feature coordinate information in step 2 to obtain the detection and determination pose matrix, the repair pose matrix, and the re-inspection pose matrix, specifically including: Acquire actual feature coordinate information of the calibration plate, and calculate the extrinsic parameter matrix of the camera device using the Zhang Zhengyou calibration algorithm based on the actual feature coordinate information and the feature coordinate information; The three-dimensional position information of the detection and determination station, the three-dimensional position information of the repair station and the three-dimensional position information of the re-inspection station are obtained, and based on the three three-dimensional position information obtained and the calculated external parameter matrix, the detection and determination pose matrix, the repair pose matrix and the re-inspection pose matrix are calculated respectively.

4. A laser repair device for a multi-workpiece table, characterized in that: The laser repair device for a multi-workpiece table is used to implement each step of the laser repair method for a multi-workpiece table as described in any one of claims 1 to 3, and the laser repair device for a multi-workpiece table includes: An extraction module is used to obtain calibration image data fed back by a camera device when the calibration plate is located at a detection and determination station, a repair station, and a re-inspection station, and to perform coordinate extraction processing on the calibration image data to obtain feature coordinate information; A calibration module is used to complete the coordinate calibration between the detection and determination station, the repair station, and the re-inspection station and the camera device based on the feature coordinate information to obtain the detection and determination posture matrix, the repair posture matrix, and the re-inspection posture matrix; A determination module determines whether the detection field magnification of the camera device is less than a preset magnification threshold. If so, the image data to be detected on the workbench at the detection and determination station is obtained, and the first position to be repaired is determined based on the image data to be detected and the detection and determination posture matrix; A repair module, used for, when the workbench is located at the repair station, confirming a first position to be repaired based on the first position to be repaired and the repair posture matrix, and performing laser repair on the defective panel on the workbench based on the confirmed first position to be repaired; The re-inspection module is used to obtain first re-inspection image data of the workbench at the re-inspection station, and re-inspect the defective panel on the workbench based on the first re-inspection image data, the first position to be repaired and the re-inspection posture matrix to obtain a first re-inspection result.

5. A laser repair device for a multi-workpiece table, characterized in that: The laser repair device for a multi-workpiece table comprises: a memory and at least one processor, wherein instructions are stored in the memory; At least one of the processors calls the instructions in the memory to enable the laser repair equipment for multiple workpiece stages to execute the various steps of the laser repair method for multiple workpiece stages according to any one of claims 1 to 3.

6. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the laser repair method for a multi-workpiece table as described in any one of claims 1 to 3 are implemented.

Images