Tray lens inclination detection method, device and equipment and medium

Abstract

The invention discloses a charging tray lens inclination detection method, device and equipment and a medium, and the method comprises the steps: obtaining a charging tray image, and enabling a charging tray to be provided with a plurality of module hole sites; performing image segmentation on the tray image to obtain a plurality of connected domains; the breakpoints of the edges of the connected domains are connected through a preset algorithm, a plurality of closed contours are obtained, and one closed contour corresponds to one module hole site; and calculating the offset of each closed contour, and calibrating an error area in the tray image based on the offset. The technical problems that in the automatic feeding and discharging process of traditional vehicle-mounted camera module production equipment, when the module inclines, detection is inaccurate, and efficiency is low are solved, and the position and the inclination state of the module in the tray are accurately recognized and processed.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to a method, device, equipment and medium for detecting the tilt of a material tray lens. Background Art

[0002] With the increasing demand for automotive lenses, the automation level and production efficiency requirements for automotive camera module production equipment are also constantly improving. In particular, in the automatic loading and unloading process, module tilt problems frequently occur, causing equipment to crash and affecting production efficiency. Therefore, it is very important to detect lens tilt during the automatic loading and unloading process of the equipment.

[0003] However, most existing equipment does not use visual inspection technology during automatic loading and unloading, or only performs photo inspection on a single module. In the process of picking and unloading, the equipment mainly relies on manually preset positions and gantry accuracy. When the module is tilted, the equipment lacking visual assistance cannot accurately perceive the abnormality, resulting in the inability of the gripper to grasp accurately, increasing the risk of equipment collision. In addition, even if a device uses single-module tilt detection technology, it can only judge a single module and cannot fully monitor all modules in the tray. Especially when the modules in the tray are densely arranged, the single-module detection method is inefficient, which seriously affects the loading and unloading speed.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a method, device, equipment and medium for detecting the tilt of a material tray lens, aiming to solve the technical problems of inaccurate and low efficiency detection when the module is tilted during the automatic loading and unloading process of traditional vehicle-mounted camera module production equipment.

[0006] To achieve the above-mentioned purpose, the present application provides a method for detecting the tilt of a material tray lens, comprising: acquiring a material tray image, wherein the material tray has a plurality of module hole positions; performing image segmentation on the material tray image to obtain a plurality of connected domains; connecting the breakpoints at the edges of the connected domains using a preset algorithm to obtain a plurality of closed contours, wherein one closed contour corresponds to one module hole position; calculating the offset of each of the closed contours, and calibrating the error area in the material tray image based on the offset.

[0007] Optionally, the connecting the breakpoints on the edge of the connected domain using a preset algorithm to obtain a plurality of closed contours includes: for any connected domain, obtaining all the breakpoints on the edge of the connected domain; and connecting adjacent breakpoints in pairs using a preset algorithm to obtain a closed contour.

[0008] Optionally, before calculating the offset of each closed contour, the method also includes: determining the serial number of each closed contour based on a preset labeling strategy and obtaining the error position coordinates; calculating the offset of each closed contour and calibrating the error area in the tray image based on the offset includes: calculating the offset of each closed contour and determining the offset position coordinates based on the offset; calibrating the error area in the tray image based on the offset position coordinates and the error position coordinates.

[0009] Optionally, the preset labeling strategy includes: dividing all closed contours in the tray image into rows and columns to obtain the number of closed contours in each row and the number of closed contours in each column in the tray image; obtaining the maximum number of row modules and the maximum number of column modules of the template image corresponding to the tray; respectively calculating whether the number of closed contours in each row is the same as the maximum number of row modules and whether the number of closed contours in each column is the same as the maximum number of column modules; if the number of closed contours in each row is the same as the maximum number of row modules and the number of closed contours in each column is the same as the maximum number of column modules, then labeling all closed contours in a preset order; otherwise, inserting virtual modules into rows that are different from the maximum number of row modules, or inserting virtual modules into columns that are different from the maximum number of column modules, and labeling all closed contours after the insertion of the virtual modules in a preset order.

[0010] Optionally, the calculating the offset of each closed contour includes: calculating the centroid coordinates of each closed contour and obtaining the ideal centroid coordinates of each closed contour, the ideal centroid coordinates being the centroid coordinates of the corresponding position of each closed contour in the template image, and the template image is an image of the material tray without lens tilt; determining the offset of each closed contour based on the centroid coordinates and the ideal centroid coordinates.

[0011] Optionally, before performing image segmentation on the tray image, the method further includes: preprocessing the tray image; wherein the preprocessing at least includes: converting the tray image into a grayscale image, denoising the tray image, and enhancing the tray image.

[0012] Optionally, acquiring the tray image includes: acquiring a plurality of initial images of the tray using an acquisition device; and correcting and splicing the plurality of initial images based on a calibration result of the acquisition device to obtain the tray image.

[0013] In addition, to achieve the above-mentioned purpose, the present application also provides a material tray lens tilt detection device, including: an image acquisition module, used to obtain a material tray image, wherein the material tray has a plurality of module hole positions; an image segmentation module, used to perform image segmentation on the material tray image to obtain a plurality of connected domains; a breakpoint connection module, used to connect the breakpoints at the edge of the connected domain using a preset algorithm to obtain a plurality of closed contours, wherein one closed contour corresponds to one module hole position; a position calibration module, used to calculate the offset of each of the closed contours, and calibrate the error area in the material tray image based on the offset.

[0014] The present application also provides a material tray lens tilt detection device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned material tray lens tilt detection method.

[0015] The present application also provides a computer-readable storage medium, comprising: a computer program stored therein, wherein when the computer program is executed by a processor, the above-mentioned material tray lens tilt detection method is implemented.

[0016] The present application proposes a method, device, equipment and medium for detecting the tilt of a material tray lens. First, a material tray image containing multiple module hole positions is obtained, and the image is segmented to obtain multiple connected domains. Then, the breakpoints at the edges of the connected domains are connected through a preset algorithm to form multiple closed contours, each of which corresponds to a module hole position. Finally, the offset of these closed contours is calculated to calibrate the error area in the material tray image. This solves the technical problem of inaccurate and inefficient detection when the module is tilted during the automatic loading and unloading process of traditional vehicle-mounted camera module production equipment, realizes accurate identification and processing of the position and tilt state of the module in the material tray, and improves the efficiency and accuracy of automatic loading and unloading of the equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of a method for detecting a tilt of a lens of a material tray according to an embodiment of the present application;

[0018] Figure 2 It is a schematic diagram of the error area;

[0019] Figure 3 is a schematic diagram of another embodiment of an error region;

[0020] Figure 4 It is a flow chart of a method for detecting a tilt of a lens of a material tray according to another embodiment of the present application;

[0021] Figure 5It is a structural block diagram of a device for detecting a tilt of a lens of a material tray according to an embodiment of the present application;

[0022] Figure 6 It is a structural schematic diagram of a tray lens tilt detection device according to an embodiment of the present application.

[0023] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0024] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0025] Traditional automotive camera module production equipment does not use visual inspection technology during the automatic loading and unloading process or only performs shooting inspection on a single module. When the module in the tray is tilted, the gripper cannot obtain information about the tilted lens, which will cause serious collision accidents during the loading and unloading process. In addition, the single-sequence method of detecting lens tilt is used, which makes it impossible to determine the lens conditions around the current lens and the position of the current lens in the tray. At the same time, when there are too many modules in a tray, this method will also slow down the loading and unloading speed.

[0026] In order to solve the above problems, the present application provides a method for detecting the tilt of a material tray lens, and the present application scheme is introduced in detail below.

[0027] Figure 1 Flow chart of a tray lens tilt detection method according to an embodiment of the present application, the tray lens tilt detection method can be applied to vehicle camera module production equipment, refer to Figure 1 The method for detecting the tilt of the lens of the material tray may include the following steps:

[0028] Step S11, obtaining a material tray image, wherein the material tray has a plurality of module holes.

[0029] In the specific implementation process, firstly, a collection device is used to obtain a tray image, wherein the tray has a plurality of module holes, which are empty spaces or holes on the tray for placing or fixing the vehicle camera module, and are used to ensure that the module can be correctly and stably placed during the loading and unloading process. It can be understood that by collecting the tray image and identifying the module holes therein, the status and position information of the module can be further analyzed.

[0030] In one embodiment, in step S11, obtaining the tray image may specifically include:

[0031] S111, using a collection device to collect multiple initial images of the material tray;

[0032] S112, correcting and splicing the multiple initial images based on the calibration results of the acquisition device to obtain a tray image.

[0033] In a specific implementation process, the acquisition device may be a camera, which is used to capture images of the calibration plate from different angles, and the optical parameters of the camera are calculated using a calibration algorithm, wherein the optical parameters include an intrinsic parameter matrix and a distortion coefficient.

[0034] It is understandable that the calibration plate can be a checkerboard calibration plate, a dot calibration plate or other known calibration plates. In this embodiment, the calibration plate images are captured from different angles and the camera lens used is calibrated in order to calculate the optical parameters of the shooting camera.

[0035] Furthermore, the hardware parameters corresponding to the camera and lens models and the working height are obtained to calculate the actual field of view of the captured image.

[0036] Furthermore, the size of the tray and the size of each module hole are obtained, the number of camera movements and the physical movement distance required to shoot the complete tray are calculated, the captured image is subjected to anti-distortion correction, and the multiple captured images are then stitched into a complete tray image. It is understandable that the size of the tray and the size of each module hole can be multiplied and divided by the actual field of view size of the captured image to obtain the number of times the entire tray needs to be shot in the XY direction, that is, the number of camera movements, and the corresponding physical movement distance of the camera during shooting.

[0037] It should be noted that this exemplary embodiment can first perform anti-distortion processing on each captured image based on the distortion coefficient, and then stitch the images of each field of view based on the actual field of view of the captured image, the number of camera movements required to capture a complete tray, and the physical movement distance and using a feature matching algorithm. Finally, ROI cropping is used to remove useless edge areas or overlapping parts in the stitching result, thereby obtaining a more accurate and complete tray image.

[0038] Step S12, performing image segmentation on the tray image to obtain multiple connected domains;

[0039] Step S13, using a preset algorithm to connect the breakpoints on the edge of the connected domain to obtain a plurality of closed contours, wherein one closed contour corresponds to one module hole position.

[0040] It should be noted that since the light source usually used by the equipment is a ring light source, there are usually dark edges at the four corners of the captured image. Direct image segmentation will cause the pixels of the lens part and the dark edges caused by the ring light source to be recognized as the same subject, resulting in a serious change in the shape of the module hole. Therefore, after obtaining a complete tray image, the image needs to be pre-processed to reduce the impact of noise caused by uneven lighting on subsequent image segmentation.

[0041] In one embodiment, before step S12, the embodiment further includes: preprocessing the tray image; wherein the preprocessing at least includes: converting the tray image into a grayscale image, denoising the tray image, and enhancing the tray image.

[0042] In the specific implementation process, the tray image is first converted into a grayscale image, and then the tray image is filtered using an image denoising algorithm to reduce the impact of dark edges caused by uneven lighting on image segmentation.

[0043] Furthermore, the image of the tray is enhanced using an image enhancement algorithm to increase the grayscale value of the brighter area in the tray image and reduce the grayscale value of the darker area, thereby expanding the contrast between the module area and the tray background area and further reducing the impact of light noise.

[0044] It should be noted that after the tray image is preprocessed, the tray image will show a transition similar to black-gray-white, where the darkest part, that is, the area with the lowest grayscale value, represents the lens module, the white part, that is, the area with a higher grayscale value, represents the background area in the tray, and the gray part represents the noise caused by uneven lighting such as dark edges. If the binarization process is performed directly, the black and gray areas will be identified as one category. Based on this, in the specific implementation process, a multi-threshold segmentation algorithm can be used to segment the tray image. Exemplarily, the multi-threshold segmentation algorithm can be an improved multi-threshold segmentation algorithm based on a genetic algorithm. In other embodiments, the multi-threshold segmentation algorithm can also be a K-means algorithm. It should be noted that if the K-means algorithm is used, the K value must be no less than 3. The multi-threshold segmentation algorithm can be set by the staff according to actual needs, and the multi-threshold segmentation algorithm is not specifically limited here.

[0045] Furthermore, the segmented tray image is subjected to morphological processing and connected domain extraction to obtain multiple connected domains. It can be understood that using morphology to process the segmentation result can remove noise, fill holes and smooth the segmentation boundaries so as to extract the shapes corresponding to the connected domains, i.e., the holes of each module.

[0046] It should be noted that in the process of finding the closed contours of each shape, that is, the holes of each module, the module contours are not closed due to incomplete images of the lens part during splicing. At the same time, after the segmented images are processed using the morphological algorithm, there are still some breakpoints. Therefore, it is necessary to connect the breakpoints to obtain the closed contours in order to identify the tilt state of each module.

[0047] In one embodiment, in step S13, using a preset algorithm to connect the breakpoints on the edge of the connected domain to obtain multiple closed contours may specifically include:

[0048] S131, for any connected domain, obtaining all breakpoints on the edge of the connected domain;

[0049] S132. Use a preset algorithm to connect adjacent breakpoints in pairs to obtain a closed contour.

[0050] The preset algorithm may be a broken line connection algorithm, which may restore a complete structure or path by filling or connecting broken parts in the image.

[0051] In the specific implementation process, all the breakpoints on the edge of each connected domain are first obtained, and then the adjacent breakpoints are connected in pairs using the broken line connection algorithm to obtain a closed contour. It can be understood that since the shape of the module hole is approximately circular, the breakpoints can be connected based on the curvature and the continuation direction to obtain a complete closed contour.

[0052] S14, calculating the offset of each closed contour, and calibrating the error area in the tray image based on the offset.

[0053] The offset can represent the offset degree and tilt direction of the module, and the error area is the area where the module tilts.

[0054] In one embodiment, in step S14, calculating the offset of each closed contour may include:

[0055] S141, calculating the centroid coordinates of each closed contour and obtaining the ideal centroid coordinates of each closed contour, wherein the ideal centroid coordinates are the centroid coordinates of the corresponding position of each closed contour in the template image, and the template image is an image of the material tray without lens tilt;

[0056] S142. Determine the offset of each closed contour based on the centroid coordinates and the ideal centroid coordinates.

[0057] In a specific implementation process, the centroid coordinates of each closed contour are first calculated, and then a template image is obtained. The template image is an image of a material tray without lens tilt. Then, the centroid coordinates of each closed contour in the template image are calculated.

[0058] Furthermore, the offset of each closed contour is determined based on the centroid coordinates in the template image and the centroid coordinates in the tray image, wherein the offset includes a tilt direction and a degree of offset, and the degree of offset is a specific tilt value.

[0059] It is understandable that when a module is severely tilted, other modules located in the tilted direction of the module will also be affected. Therefore, it is necessary to extract not only the coordinates of the severely tilted module, but also the coordinates of other modules affected by the module.

[0060] In the specific implementation process, first determine whether each closed contour is tilted based on the offset of each closed contour. If tilted, determine the coordinates of the disturbed closed contour based on the offset of the closed contour, and add the coordinates of the tilted closed contour and the coordinates of the disturbed closed contour as offset position coordinates to the ungraspable array.

[0061] Furthermore, the ungraspable array is marked as an error area in the tray image and is framed with a red rectangle to facilitate monitoring by the staff. Figure 2 , Figure 2 is a schematic diagram of the error area in this embodiment, Figure 3 is a schematic diagram of another embodiment of an error area, Figure 2 and Figure 3 The red rectangle is the error area.

[0062] The present application proposes a method for detecting the tilt of a material tray lens. First, a material tray image containing multiple module hole positions is obtained, and the image is segmented to obtain multiple connected domains. Then, the breakpoints at the edges of the connected domains are connected by a preset algorithm to form multiple closed contours, each of which corresponds to a module hole position. Finally, the offset of these closed contours is calculated to calibrate the error area in the material tray image. This solves the technical problem of inaccurate and inefficient detection when the module is tilted during the automatic loading and unloading process of traditional vehicle-mounted camera module production equipment, realizes accurate identification and processing of the position and tilt state of the module in the material tray, and improves the efficiency and accuracy of automatic loading and unloading of the equipment.

[0063] Based on the above embodiments, Figure 4 Flow chart of a method for detecting a tilt of a lens of a material tray according to another embodiment of the present application. Figure 4 Based on Figure 1 The preferred embodiment of the corresponding tray lens tilt detection method is shown in FIG. Figure 4 The method for detecting the tilt of the lens of the material tray may include the following steps:

[0064] S41, acquiring a material tray image, wherein the material tray has a plurality of module holes;

[0065] S42, performing image segmentation on the tray image to obtain multiple connected domains;

[0066] S43, using a preset algorithm to connect the breakpoints on the edge of the connected domain to obtain a plurality of closed contours, wherein one closed contour corresponds to one module hole position;

[0067] S44, determining the serial number of each closed contour based on a preset labeling strategy and obtaining the error position coordinates;

[0068] S45, calculating the offset of each closed contour, and determining the offset position coordinates based on the offset;

[0069] S46. Marking an error area in the tray image based on the offset position coordinates and the error position coordinates.

[0070] It should be noted that during the sorting and grabbing of trays, the trays are usually designed according to the maximum number of modules in each row and column. However, in actual situations, there may be missing holes. This is not only because the modules are offset in the holes, but also because some holes themselves are not filled with modules, resulting in vacancies. If this phenomenon of missing holes is not detected and handled, it may cause abnormalities in the subsequent grabbing process and even affect the stability of the entire loading and unloading system.

[0071] In one embodiment, in step S44, the preset labeling strategy may include:

[0072] S441, dividing all closed contours in the tray image into rows and columns to obtain the number of closed contours in each row and the number of closed contours in each column in the tray image;

[0073] S442, obtaining the maximum number of row modules and the maximum number of column modules of the template image corresponding to the material tray;

[0074] S443, respectively calculating whether the number of closed contours in each row is the same as the maximum number of row modules and whether the number of closed contours in each column is the same as the maximum number of column modules;

[0075] If the number of closed contours in each row is the same as the maximum number of row modules and the number of closed contours in each column is the same as the maximum number of column modules, then all closed contours are numbered in a preset order;

[0076] Otherwise, virtual modules are inserted into rows that are different from the maximum number of row modules, or virtual modules are inserted into columns that are different from the maximum number of column modules, and all closed contours after the virtual modules are inserted are numbered in a preset order.

[0077] In a specific implementation process, all closed contours in the tray image are divided into rows and columns and the maximum number of row modules and the maximum number of column modules of the tray image are obtained, wherein the maximum number of row modules is the maximum number of module holes that can be accommodated in each row when the tray is designed, and the maximum number of column modules is the maximum number of module holes that can be accommodated in each column when the tray is designed.

[0078] Further, the number of module holes in each row and the number of module holes in each column in the tray image are obtained, and the number of module holes in each row and the maximum number of modules in a row are calculated to determine whether the number of module holes in each column is the same as the maximum number of modules in a column;

[0079] If the number of rows and the number of columns are the same, all closed contours are numbered in a preset order, wherein the preset order may start from the upper left corner of the tray image, and number the closed contours from top to bottom and from left to right in sequence;

[0080] If the number of rows or columns is different, virtual modules are inserted into rows that are different from the maximum number of row modules, or virtual modules are inserted into columns that are different from the maximum number of column modules, and all closed contours after the virtual modules are inserted are numbered in a preset order.

[0081] Furthermore, the coordinates of the virtual module are used as the error position coordinates and the error position coordinates are added to the ungraspable array.

[0082] It can be understood that this embodiment determines whether there are missing holes based on the maximum number of rows and columns designed for the tray and the number of modules in each row and column actually detected. If there are missing holes, virtual module points need to be inserted to fill these gaps. The role of the virtual module points is to occupy places to ensure that the number of modules in each row and column remains consistent when the sorting algorithm is sorting, thereby avoiding sorting errors caused by missing holes.

[0083] In addition, if virtual module points are not inserted and necessary supplements are not made, the sorting system may not be able to correctly identify the actual position of the module, which may cause problems when the equipment is grasping. Therefore, virtual module points must not only be inserted when holes are missing, but also their row and column coordinates must be added to the non-graspable array to ensure that these virtual points are not treated as graspable targets. Through such processing, grasping anomalies caused by missing holes or module offsets can be effectively avoided, ensuring the smooth operation and efficient operation of the entire loading and unloading system.

[0084] It should be noted that the material tray lens tilt detection method of this embodiment effectively solves the problem of whether the landing point is correct when the gantry drives the camera to move, and can realize a wide range of lens tilt detection, greatly improving the working efficiency of the equipment. At the same time, when facing problems such as lack of material at individual acupoints and small tilt angle of the module lens, it can also give accurate judgments.

[0085] Based on the above embodiments, Figure 5 FIG. 1 is a structural block diagram of a device for detecting a tilt of a lens of a material tray according to an embodiment of the present application, as shown in FIG. Figure 5 As shown, the material tray lens tilt detection device 500 may include: an image acquisition module 510, an image segmentation module 520, a breakpoint connection module 530 and a position calibration module 540, wherein:

[0086] The image acquisition module 510 is used to acquire an image of a material tray, wherein the material tray has a plurality of module holes;

[0087] The image segmentation module 520 is used to perform image segmentation on the tray image to obtain multiple connected domains;

[0088] The breakpoint connection module 530 is used to connect the breakpoints at the edge of the connected domain using a preset algorithm to obtain a plurality of closed contours, where one closed contour corresponds to one module hole position;

[0089] The position calibration module 540 is used to calculate the offset of each closed contour and calibrate the error area in the tray image based on the offset.

[0090] In an exemplary embodiment, the breakpoint connection module 530 may also be used to obtain all breakpoints on the edge of any connected domain, and connect adjacent breakpoints in pairs using a preset algorithm to obtain a closed contour.

[0091] In an exemplary embodiment, the breakpoint connection module 530 can also be used to determine the serial number of each closed contour based on a preset labeling strategy and obtain the error position coordinates; the calculation of the offset of each closed contour and the calibration of the error area in the tray image based on the offset include: calculating the offset of each closed contour and determining the offset position coordinates based on the offset; calibrating the error area in the tray image based on the offset position coordinates and the error position coordinates.

[0092] In an exemplary embodiment, the breakpoint connection module 530 can also be used to divide all closed contours in the tray image into rows and columns to obtain the number of closed contours in each row and the number of closed contours in each column in the tray image; obtain the maximum number of row modules and the maximum number of column modules of the template image corresponding to the tray; calculate whether the number of closed contours in each row is the same as the maximum number of row modules and whether the number of closed contours in each column is the same as the maximum number of column modules; if the number of closed contours in each row is the same as the maximum number of row modules and the number of closed contours in each column is the same as the maximum number of column modules, then number all closed contours in a preset order; otherwise, insert virtual modules into rows that are different from the maximum number of row modules, or insert virtual modules into columns that are different from the maximum number of column modules, and number all closed contours after the insertion of the virtual modules in a preset order.

[0093] In an exemplary embodiment, the position calibration module 540 can also be used to calculate the centroid coordinates of each closed contour and obtain the ideal centroid coordinates of each closed contour, wherein the ideal centroid coordinates are the centroid coordinates of the corresponding position of each closed contour in the template image, and the template image is an image of the material tray without lens tilt; the offset of each closed contour is determined based on the centroid coordinates and the ideal centroid coordinates.

[0094] In an exemplary embodiment, the image segmentation module 520 can also be used to pre-process the tray image; wherein the pre-processing at least includes: converting the tray image into a grayscale image, denoising the tray image, and enhancing the tray image.

[0095] In an exemplary embodiment, the image acquisition module 510 can also be used to acquire multiple initial images of the material tray using an acquisition device; based on the calibration result of the acquisition device, the multiple initial images are corrected and spliced ​​to obtain the material tray image.

[0096] Those skilled in the art should understand that the division of the various modules in the embodiment is merely a division of logical functions, and in actual application, all or part of them can be integrated into one or more actual carriers, and these modules can be implemented entirely in the form of software called by a processing unit, or entirely in the form of hardware, or in the form of a combination of software and hardware. It should be noted that each module in a material tray lens tilt detection device in this embodiment corresponds one-to-one to each step in a material tray lens tilt detection method in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned material tray lens tilt detection method, which will not be repeated here.

[0097] Based on the above embodiments, Figure 6FIG. 1 is a schematic diagram of a device for detecting a tilt of a lens in a material tray according to an embodiment of the present application. Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute a material tray lens tilt detection method, the method comprising: obtaining a material tray image, wherein the material tray has a plurality of module hole positions; performing image segmentation on the material tray image to obtain a plurality of connected domains; connecting the breakpoints of the edge of the connected domain using a preset algorithm to obtain a plurality of closed contours, wherein one closed contour corresponds to one module hole position; calculating the offset of each closed contour, and calibrating the error area in the material tray image based on the offset.

[0098] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0099] On the basis of the above embodiments, on the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the tray lens tilt detection method provided by the above methods, the method including: acquiring a tray image, wherein the tray has a plurality of module hole positions; performing image segmentation on the tray image to obtain a plurality of connected domains; connecting the breakpoints on the edges of the connected domains using a preset algorithm to obtain a plurality of closed contours, wherein one closed contour corresponds to one module hole position; calculating the offset of each closed contour, and calibrating the error area in the tray image based on the offset.

[0100] On the basis of the above embodiments, on another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the tray lens tilt detection method provided by the above methods, the method comprising: acquiring a tray image, wherein the tray has a plurality of module hole positions; performing image segmentation on the tray image to obtain a plurality of connected domains; connecting the breakpoints on the edges of the connected domains using a preset algorithm to obtain a plurality of closed contours, wherein one closed contour corresponds to one module hole position; calculating the offset of each closed contour, and calibrating the error area in the tray image based on the offset.

[0101] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting the tilt of a tray lens, characterized in that: The method comprises: Acquire a material tray image, wherein the material tray has a plurality of module holes; Performing image segmentation on the tray image to obtain multiple connected domains; Using a preset algorithm to connect the breakpoints on the edge of the connected domain to obtain a plurality of closed contours, wherein one closed contour corresponds to one module hole position; The offset of each closed contour is calculated, and the error area is marked in the tray image based on the offset.

2. The method for detecting the tilt of a material tray lens according to claim 1, characterized in that: The method of connecting the breakpoints on the edge of the connected domain using a preset algorithm to obtain a plurality of closed contours includes: For any connected domain, obtain all breakpoints on the edge of the connected domain; Adjacent breakpoints are connected in pairs using a preset algorithm to obtain a closed contour.

3. The method for detecting the tilt of a material tray lens according to claim 1, characterized in that: Before calculating the offset of each closed contour, the method further includes: Determine the serial number of each closed contour based on a preset labeling strategy and obtain the coordinates of the error position; The calculating the offset of each closed contour and calibrating the error area in the tray image based on the offset comprises: Calculating the offset of each closed contour, and determining the offset position coordinates based on the offset; An error area is calibrated in the tray image based on the offset position coordinates and the error position coordinates.

4. The method for detecting the tilt of a tray lens according to claim 3, characterized in that: The preset labeling strategy includes: Dividing all closed contours in the tray image into rows and columns to obtain the number of closed contours in each row and the number of closed contours in each column in the tray image; Obtaining the maximum number of row modules and the maximum number of column modules of the template image corresponding to the material tray; Calculate respectively whether the number of closed contours in each row is the same as the maximum number of row modules and whether the number of closed contours in each column is the same as the maximum number of column modules; If the number of closed contours in each row is the same as the maximum number of row modules and the number of closed contours in each column is the same as the maximum number of column modules, then all closed contours are numbered in a preset order; Otherwise, virtual modules are inserted into rows that are different from the maximum number of row modules, or virtual modules are inserted into columns that are different from the maximum number of column modules, and all closed contours after the virtual modules are inserted are numbered in a preset order.

5. The method for detecting the tilt of a material tray lens according to claim 1, characterized in that: The calculating the offset of each closed contour comprises: Calculate the centroid coordinates of each closed contour and obtain the ideal centroid coordinates of each closed contour, wherein the ideal centroid coordinates are the centroid coordinates of the corresponding position of each closed contour in the template image, and the template image is a tray image without lens tilt; The offset of each closed contour is determined based on the centroid coordinates and the ideal centroid coordinates.

6. The method for detecting the tilt of a tray lens according to claim 1, characterized in that: Before performing image segmentation on the tray image, the method further includes: Preprocessing the tray image; Wherein, the preprocessing at least includes: converting the tray image into a grayscale image, denoising the tray image, and enhancing the tray image.

7. The method for detecting the tilt of a tray lens according to claim 1, characterized in that: The step of obtaining the tray image comprises: Using a collection device to collect multiple initial images of the material tray; Based on the calibration result of the acquisition device, the multiple initial images are corrected and spliced ​​to obtain a tray image.

8. A device for detecting the tilt of a material tray lens, characterized in that: include: An image acquisition module, used to acquire an image of a tray, wherein the tray has a plurality of module holes; An image segmentation module is used to perform image segmentation on the tray image to obtain multiple connected domains; A breakpoint connection module, used to connect the breakpoints at the edge of the connected domain using a preset algorithm to obtain a plurality of closed contours, wherein one closed contour corresponds to one module hole position; The position calibration module is used to calculate the offset of each closed contour and calibrate the error area in the tray image based on the offset.

9. A tray lens tilt detection device, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the tray lens tilt detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the tray lens tilt detection method according to any one of claims 1 to 7 is implemented.

Images