A tray lens tilt detection method, device, equipment and medium

Abstract

The application discloses a tray lens tilt detection method, device, equipment and medium, including: acquiring a tray image, wherein the tray has a plurality of module hole positions; image segmentation is performed on the tray image to obtain a plurality of connected domains; the breakpoints of the connected domain edges are connected by using a preset algorithm to obtain a plurality of closed contours, and one closed contour corresponds to one module hole position; the offset of each closed contour is calculated, and an error area is calibrated in the tray image based on the offset. The application solves the technical problems of traditional vehicle-mounted camera module production equipment in the automatic feeding and discharging process, that is, inaccurate detection and low efficiency when the module is tilted, and realizes accurate identification and processing of the position and tilt state of the module in the tray.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image detection, in particular to a tray lens tilt detection method, device, equipment and medium. BACKGROUND

[0002] With the increasing demand for vehicle-mounted lenses, the automation level and production efficiency of vehicle-mounted camera module production equipment are also constantly improving. In particular, in the automatic feeding and unloading link, module tilt problems occur frequently, causing equipment collisions and affecting production efficiency. Therefore, it is crucial to detect lens tilt during the automatic feeding and unloading process of the equipment.

[0003] However, most existing equipment does not use visual detection technology or only detects single modules during the automatic feeding and unloading process. During the feeding and unloading process, the equipment mainly relies on manual preset positions and gantry accuracy. When the module is tilted, the equipment without visual assistance cannot accurately perceive the abnormality, resulting in inaccurate gripping of the clamping jaw and increasing the risk of equipment collision. In addition, even if some equipment uses single-module tilt detection technology, it can only judge a single module and cannot comprehensively monitor all modules in the tray. In particular, when the modules in the tray are densely arranged, the single-module detection method is inefficient, which seriously affects the feeding and unloading speed.

[0004] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0005] The main purpose of the present application is to provide a tray lens tilt detection method, device, equipment and medium, which aims to solve the technical problems of inaccurate detection and low efficiency of traditional vehicle-mounted camera module production equipment during the automatic feeding and unloading process when the module is tilted.

[0006] To achieve the above purpose, the present application provides a tray lens tilt detection method, comprising: acquiring a tray image, wherein the tray has a plurality of module hole positions; performing image segmentation on the tray image to obtain a plurality of connected domains; connecting the breakpoints of the connected domain edges using a preset algorithm to obtain a plurality of closed contours, one closed contour corresponding to one module hole position; calculating the offset of each closed contour, and marking the error area in the tray image based on the offset.

[0007] Optionally, the step of connecting the breakpoints of the connected domain edges using a preset algorithm to obtain a plurality of closed contours comprises: for any connected domain, acquiring all breakpoints of the connected domain edge; connecting adjacent breakpoints two by two using a preset algorithm to obtain a closed contour.

[0008] Optionally, before the calculating the offset of each of the closed contours, the method further comprises: determining the serial numbers of the closed contours and obtaining the error position coordinates based on a preset labeling strategy; and the calculating the offset of each of the closed contours and labeling the error region in the tray image based on the offset comprises: calculating the offset of each of the closed contours and determining the offset position coordinates based on the offset; and labeling the error region in the tray image based on the offset position coordinates and the error position coordinates.

[0009] Optionally, the preset labeling strategy comprises: dividing all the closed contours in the tray image into rows and columns to obtain the number of closed contours in each row and the number of closed contours in each column; obtaining the maximum number of row modules and the maximum number of column modules possessed by the template image corresponding to the tray; respectively calculating whether the number of closed contours in each row and the maximum number of row modules are the same and whether the number of closed contours in each column and the maximum number of column modules are the same; if the number of closed contours in each row and the maximum number of row modules are the same and the number of closed contours in each column and the maximum number of column modules are the same, labeling all the closed contours in a preset order; otherwise, inserting a virtual module into the row with the different maximum number of row modules, or inserting a virtual module into the column with the different maximum number of column modules, and labeling all the closed contours after the insertion of the virtual module in a preset order.

[0010] Optionally, the calculating the offset of each of the closed contours comprises: calculating the centroid coordinates of each of the closed contours and obtaining ideal centroid coordinates of each of the closed contours, the ideal centroid coordinates being the centroid coordinates of the corresponding position of each of the closed contours in a template image, the template image being a tray image without lens tilt; and determining the offset of each of the closed contours based on the centroid coordinates and the ideal centroid coordinates.

[0011] Optionally, before the image segmentation of the tray image, the method further comprises: pre-processing the tray image; wherein the pre-processing at least comprises: converting the tray image into a gray-scale image, denoising the tray image, and image enhancement of the tray image.

[0012] Optionally, the obtaining the tray image comprises: using a collection device to collect a plurality of initial images of the tray; and correcting and splicing the plurality of initial images based on the calibration result of the collection device to obtain the tray image.

[0013] In addition, to achieve the above object, the application further provides a tray lens tilt detection device, comprising: an image acquisition module, configured to acquire a tray image, wherein the tray has a plurality of module hole positions; an image segmentation module, configured to perform image segmentation on the tray image to obtain a plurality of connected domains; a breakpoint connection module, configured to connect breakpoints on edges of the connected domains by using a preset algorithm to obtain a plurality of closed contours, and one closed contour corresponds to one module hole position; and a position calibration module, configured to calculate offset amounts of the closed contours and calibrate an error region in the tray image based on the offset amounts.

[0014] The application further provides a tray lens tilt detection device, comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned tray lens tilt detection method.

[0015] The application further provides a computer-readable storage medium, comprising: a computer program stored in the computer-readable storage medium, and the computer program is executed by a processor to implement the above-mentioned tray lens tilt detection method.

[0016] The application provides a tray lens tilt detection method, device, equipment and medium, which first acquires a tray image containing a plurality of module hole positions, performs image segmentation on the image to obtain a plurality of connected domains, connects breakpoints on edges of the connected domains by using a preset algorithm to form a plurality of closed contours, each closed contour corresponds to one module hole position, and finally calculates offset amounts of the closed contours to calibrate an error region in the tray image, thereby solving the technical problems of inaccurate detection and low efficiency of a traditional vehicle-mounted camera module production equipment in an automatic feeding and discharging process when a module is tilted, achieving accurate identification and processing of a position and a tilt state of the module in the tray, and improving the efficiency and accuracy of automatic feeding and discharging of the equipment. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A flowchart of a tray lens tilt detection method according to an embodiment of the application;

[0018] Figure 2 An error region schematic diagram;

[0019] Figure 3 An error region schematic diagram of another embodiment;

[0020] Figure 4 A flowchart of a tray lens tilt detection method according to another embodiment of the application;

[0021] Figure 5A structure block diagram of the tray lens tilt detection device according to an embodiment of the present application;

[0022] Figure 6 A structure schematic diagram of the tray lens tilt detection device according to an embodiment of the present application.

[0023] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0024] It should be understood that the specific embodiments described herein are merely intended to explain the present application, and are not intended to limit the present application.

[0025] The conventional vehicle-mounted camera module production equipment does not use visual detection technology in the automatic feeding and discharging process or only detects a single module, and when the modules in the tray are tilted, the gripper cannot obtain the information of the tilted lens, which may cause serious collision accidents in the taking and placing process. In addition, the single sequence detection lens tilt method cannot determine the situation of the lenses around the current lens and the position of the current lens in the tray, and when the number of modules in a tray is too large, this method will also slow down the feeding and discharging speed.

[0026] To solve the above problems, the present application provides a tray lens tilt detection method, and the present application will be described in detail below.

[0027] Figure 1 A flowchart of the tray lens tilt detection method according to an embodiment of the present application, which can be applied to a vehicle-mounted camera module production equipment, and the flowchart is shown in Figure 1 The tray lens tilt detection method can include the following steps:

[0028] Step S11, acquiring a tray image, wherein the tray has a plurality of module hole positions.

[0029] In the specific implementation process, first, the acquisition device is used to acquire a tray image, wherein the tray has a plurality of module hole positions, the module hole position is a space or a hole on the tray for placing or fixing the vehicle-mounted camera module, which is used to ensure that the module can be correctly and stably placed during feeding and discharging. It can be understood that by acquiring the tray image and identifying the module hole position therein, the state and position information of the module can be further analyzed.

[0030] In an embodiment, in step S11, acquiring a tray image can specifically include:

[0031] S111, using the acquisition device to acquire a plurality of initial images of the tray;

[0032] S112, correcting and splicing the plurality of initial images based on the calibration result of the acquisition device to obtain a tray image.

[0033] In the specific implementation process, the acquisition device can be a camera, the camera is used to shoot images of the calibration board from different angles, and the optical parameters of the camera are calculated through a calibration algorithm, wherein the optical parameters include an intrinsic matrix and distortion coefficients.

[0034] It can be understood that the calibration board can be a checkerboard calibration board, a circular dot calibration board or other calibration boards with known structures. In the embodiment, the images of the calibration board are shot from different angles to calibrate the camera lens used, so as to calculate the optical parameters of the shooting camera.

[0035] Further, the hardware parameters corresponding to the camera and lens model and the working height are obtained, and the actual field of view of the shooting image is calculated.

[0036] Further, the size of the tray and the size of each module hole are obtained, the number of camera movements and the physical movement distance required for shooting the complete tray are calculated, the de-distortion correction of the shot pictures is performed, and then the plurality of shot images are spliced into a complete tray image. It can be understood that the size of the tray and the size of each module hole can be multiplied and divided by the size of the actual field of view of the shot image to obtain the number of times of shooting the entire tray in the XY direction, that is, the number of camera movements, and the corresponding physical movement distance of the camera during shooting.

[0037] It should be noted that the exemplary embodiment can first perform de-distortion processing on each shot image based on the distortion coefficients, then splice the images of each field of view based on the actual field of view of the shot image, the number of camera movements and the physical movement distance required for shooting the complete tray and using a feature matching algorithm, and finally use ROI cropping to remove the useless edge area or overlapping part in the splicing result, thereby obtaining a more accurate and complete tray image.

[0038] Step S12, image segmentation is performed on the tray image to obtain a plurality of connected domains;

[0039] Step S13, the breakpoints of the edges of the connected domains are connected by using a preset algorithm to obtain a plurality of closed contours, and one closed contour corresponds to one module hole.

[0040] It should be noted that since the light source commonly used by the device is a ring light source, the corners of the captured picture usually have dark edges. Direct image segmentation will cause the pixels of the lens part and the dark edges caused by the ring light source to be identified as the same subject, thereby causing the shape of the module hole to change seriously. Therefore, after obtaining the complete tray image, the image needs to be preprocessed to reduce the influence of the noise caused by uneven lighting on subsequent image segmentation.

[0041] In an embodiment, before step S12, the embodiment further includes: pre-processing the tray image; wherein the pre-processing at least includes: converting the tray image to a grayscale image, denoising the tray image, and image enhancement of the tray image.

[0042] In the specific implementation process, the tray image is first converted to a grayscale image, and then an image denoising algorithm is used to filter the tray image to reduce the influence of the dark edges caused by uneven lighting on image segmentation.

[0043] Further, an image enhancement algorithm is used to enhance the tray image to increase the gray value of the brighter area in the tray image and reduce the gray value of the darker area, thereby expanding the contrast between the module area and the tray background area and further reducing the influence of lighting noise.

[0044] It should be noted that after pre-processing the tray image, the tray picture will present a transition similar to black-gray-white, wherein the darkest part, i.e., the area with the lowest gray value, represents the lens module, the white part, i.e., the area with a higher gray value, represents the background area in the tray, and the gray part represents noise caused by uneven lighting, such as dark edges. If direct binarization processing is performed, the black and gray areas will be identified as one category. Based on this, in the specific implementation process, a multi-threshold segmentation algorithm can be used to segment the tray image. Exemplarily, the multi-threshold segmentation algorithm can be a multi-threshold segmentation algorithm improved based on a genetic algorithm. In other embodiments, the multi-threshold segmentation algorithm can also be a K-means algorithm. It should be noted that if the K-means algorithm is used, the K value should not be less than 3. The multi-threshold segmentation algorithm can be set by the staff according to actual needs, and the multi-threshold segmentation algorithm is not specifically limited here.

[0045] Further, the segmented tray image is subjected to morphological processing and connected component extraction to obtain a plurality of connected components. It can be understood that using morphology to process the segmentation result can remove noise, fill holes, and smooth the segmentation boundary, so as to extract the shape corresponding to each connected component, i.e., each module hole.

[0046] It should be noted that in the process of finding the closed contour of each shape, i.e., each module hole, the image of the lens part may not be fully captured during splicing, resulting in an unclosed module contour. In addition, after processing the segmented image using a morphological algorithm, there may still be some breakpoints. Therefore, the breakpoints need to be connected to obtain a closed contour in order to identify the tilt state of each module.

[0047] In an embodiment, in step S13, connecting the breakpoints of the connected domain edge using a preset algorithm to obtain a plurality of closed contours can specifically include:

[0048] S131, for any connected domain, obtaining all breakpoints of the connected domain edge;

[0049] S132, connecting adjacent breakpoints two by two using a preset algorithm to obtain a closed contour.

[0050] The preset algorithm can be a broken line connection algorithm, which can restore the complete structure or path by filling or connecting the broken part of the image.

[0051] In the specific implementation process, first, all the breakpoints of the connected domain edge are obtained, and then the adjacent breakpoints are connected two by two using the broken line connection algorithm to obtain a closed contour. It can be understood that since the module hole shape is approximately circular, the breakpoints can be connected based on the curvature and continuation direction to obtain a complete closed contour.

[0052] S14, calculating the offset of each closed contour and marking the error area in the tray image based on the offset.

[0053] The offset can represent the degree of offset and the tilt direction of the module, and the error area is the area where the module is tilted.

[0054] In an embodiment, in step S14, calculating the offset of each closed contour can include:

[0055] S141, calculating the centroid coordinates of each closed contour and obtaining the ideal centroid coordinates of each closed contour, the ideal centroid coordinates being the centroid coordinates of the corresponding position of each closed contour in the template image, the template image being a tray image without lens tilt;

[0056] S142, determining the offset of each closed contour based on the centroid coordinates and the ideal centroid coordinates.

[0057] In the specific implementation process, first, the centroid coordinates of each closed contour are calculated, and then a template image is obtained, which is a tray image without lens tilt, and then the centroid coordinates of each closed contour in the template image are calculated.

[0058] Furthermore, the offset of each closed contour is determined based on the centroid coordinates in the template image and the centroid coordinates in the material tray image. The offset includes the tilt direction and the degree of offset, and the degree of offset is a specific tilt value.

[0059] It is understandable that when a module is severely tilted, other modules located in the tilt direction will also be affected. Therefore, it is necessary not only to extract the coordinates of the severely tilted module, but also to extract the coordinates of other modules that are affected by the tilt.

[0060] In the specific implementation process, firstly, it is determined whether each closed contour is tilted based on the offset of each closed contour. If it is tilted, the coordinates of the disturbed closed contour are determined based on the offset of the closed contour. The coordinates of the tilted closed contour and the coordinates of the disturbed closed contour are added to the non-crawling array as offset position coordinates.

[0061] Furthermore, the ungrabable arrays are marked as error areas in the tray image and selected with a red rectangle for worker monitoring. See also... Figure 2 , Figure 2 This is a schematic diagram of the error area in this embodiment. Figure 3 This is a schematic diagram of another embodiment of the error region. Figure 2 and Figure 3 The area within the red rectangle is the error area.

[0062] This application proposes a method for detecting lens tilt on a material tray. First, an image of the material tray containing multiple module holes is acquired, and the image is segmented to obtain multiple connected components. Then, a preset algorithm is used to connect the breakpoints at the edges of the connected components to form multiple closed contours, each corresponding to a module hole. Finally, by calculating the offset of these closed contours, the error area is marked in the material tray image. This method solves the technical problem of inaccurate and inefficient detection when the module tilts during the automatic loading and unloading process of traditional automotive camera module production equipment. It achieves accurate identification and processing of the module's position and tilt state in the material tray, improving the efficiency and accuracy of automatic loading and unloading.

[0063] Based on the above embodiments, Figure 4 This is a flowchart of a tray lens tilt detection method according to another embodiment of this application. Figure 4 For based on Figure 1 The preferred embodiment of the corresponding tray lens tilt detection method is described below. Figure 4 The method for detecting the tilt of the tray lens may include the following steps:

[0064] S41. Obtain an image of the material tray, wherein the material tray has multiple module holes;

[0065] S42, image segmentation is performed on the tray image to obtain a plurality of connected domains;

[0066] S43, the breakpoints of the connected domain edges are connected using a preset algorithm to obtain a plurality of closed contours, and one closed contour corresponds to one module hole position;

[0067] S44, the serial numbers of the closed contours are determined based on a preset labeling strategy, and error position coordinates are obtained;

[0068] S45, the offsets of the closed contours are calculated, and offset position coordinates are determined based on the offsets;

[0069] S46, an error region is marked in the tray image based on the offset position coordinates and the error position coordinates.

[0070] It should be noted that in the process of tray sorting and grabbing, the tray is usually designed according to the maximum number of modules in each row and column. However, in actual situations, there may be missing hole positions, which is not only because the modules have been offset in the hole positions, but also because some hole positions may not be filled with modules, resulting in vacancies. If this phenomenon of missing hole positions is not detected and processed, it may cause abnormalities in the subsequent grabbing process, and even affect the stability of the entire feeding and discharging system.

[0071] In an embodiment, in step S44, the preset labeling strategy can include:

[0072] S441, the closed contours in each row and column of the tray image are divided to obtain the number of closed contours in each row and the number of closed contours in each column of the tray image;

[0073] S442, the maximum number of row modules and the maximum number of column modules possessed by the template image corresponding to the tray are obtained;

[0074] S443, whether the number of closed contours in each row and the maximum number of row modules are the same and whether the number of closed contours in each column and the maximum number of column modules are the same are calculated respectively;

[0075] If the number of closed contours in each row and the maximum number of row modules are the same and the number of closed contours in each column and the maximum number of column modules are the same, the closed contours are labeled in a preset order;

[0076] Otherwise, virtual modules are inserted into the rows with different maximum number of row modules, or virtual modules are inserted into the columns with different maximum number of column modules, and all the closed contours after inserting the virtual modules are labeled in a preset order.

[0077] In the specific implementation process, all closed contours in the tray image are divided into rows and columns, and the maximum number of row modules and the maximum number of column modules of the tray image are obtained, wherein the maximum number of row modules is the maximum number of module hole positions that can be accommodated in each row during tray design, and the maximum number of column modules is the maximum number of module hole positions that can be accommodated in each column during tray design.

[0078] Further, the number of module hole positions in each row and the number of module hole positions in each column of the tray image are obtained, and whether the number of module hole positions in each row is the same as the maximum number of row modules and whether the number of module hole positions in each column is the same as the maximum number of column modules is calculated respectively.

[0079] If the number of rows and the number of columns are the same, all closed contours are labeled in a preset order, wherein the preset order can be from the top left corner of the tray image, from top to bottom, and from left to right to label the closed contours in sequence.

[0080] If the number of rows or the number of columns is not the same, a virtual module is inserted for the row that is not the same as the maximum number of row modules, or a virtual module is inserted for the column that is not the same as the maximum number of column modules, and all closed contours after the virtual module is inserted are labeled in a preset order.

[0081] Further, the coordinates of the virtual module are taken as error position coordinates, and the error position coordinates are added to the non-graspable array.

[0082] It can be understood that the embodiment determines whether there is a hole position missing according to the maximum number of rows and columns of the tray design and the number of modules in each row and column actually detected. If there is a hole position missing, a virtual module point needs to be inserted to fill the vacancy. The role of the virtual module point is to occupy the position, so that the number of modules in each row and column remains consistent when the sorting algorithm is sorting, thereby avoiding sorting errors caused by missing hole positions.

[0083] In addition, if a virtual module point is not inserted and necessary supplement is not performed, it may cause the sorting system to fail to correctly identify the actual position of the module, and further cause problems when the device is grasped. Therefore, the virtual module point not only needs to be inserted when the hole position is missing, but also needs to be added to the non-graspable array to ensure that these virtual points will not be regarded as graspable targets. Through such processing, it can effectively avoid the grasping abnormalities caused by missing hole positions or module offset, and ensure the smooth operation and efficient operation of the whole feeding and discharging system.

[0084] It should be noted that the tray lens tilt detection method of the embodiment effectively solves the problem of whether the landing point is correct when the gantry drives the camera to move, and can realize large-range lens tilt detection, greatly improving the working efficiency of the device. At the same time, when facing problems such as individual hole position missing and small lens tilt angle of the module, accurate judgment can also be given.

[0085] On the basis of the above embodiments, Figure 5 The structure block diagram of the tray lens tilt detection device according to an embodiment of the present application is shown in Figure 5 The tray lens tilt detection device 500 can include an image acquisition module 510, an image segmentation module 520, a breakpoint connection module 530, and a position calibration module 540, wherein,

[0086] The image acquisition module 510 is configured to acquire a tray image, wherein the tray has a plurality of module hole positions.

[0087] The image segmentation module 520 is configured to perform image segmentation on the tray image to obtain a plurality of connected domains.

[0088] The breakpoint connection module 530 is configured to connect the breakpoints of the edges of the connected domains using a preset algorithm to obtain a plurality of closed contours, and each closed contour corresponds to a module hole position.

[0089] The position calibration module 540 is configured to calculate the offset of each closed contour and calibrate an error region in the tray image based on the offset.

[0090] In an example embodiment, the breakpoint connection module 530 can also be configured to, for any connected domain, acquire all breakpoints of the edges of the connected domain; and connect adjacent breakpoints two by two using a preset algorithm to obtain a closed contour.

[0091] In an example embodiment, the breakpoint connection module 530 can also be configured to determine the serial number of each closed contour based on a preset labeling strategy and obtain error position coordinates; and the calculation of the offset of each closed contour and the calibration of the error region in the tray image based on the offset includes: calculation of the offset of each closed contour and determination of offset position coordinates based on the offset; and calibration of the error region in the tray image based on the offset position coordinates and the error position coordinates.

[0092] In the example embodiment, the breakpoint connection module 530 can also be configured to divide all the closed contours in the tray image into rows and columns to obtain the number of closed contours in each row and the number of closed contours in each column of the tray image; obtain the maximum number of row modules and the maximum number of column modules of the template image corresponding to the tray; calculate whether the number of closed contours in each row and the maximum number of row modules are the same and whether the number of closed contours in each column and the maximum number of column modules are the same; if the number of closed contours in each row and the maximum number of row modules are the same and the number of closed contours in each column and the maximum number of column modules are the same, label all the closed contours in a preset order; otherwise, insert a virtual module into the row with a different maximum number of row modules or insert a virtual module into the column with a different maximum number of column modules, and label all the closed contours after the virtual module is inserted in a preset order.

[0093] In the example embodiment, the position calibration module 540 can also be configured to calculate the centroid coordinates of each closed contour and obtain ideal centroid coordinates of each closed contour, the ideal centroid coordinates being the centroid coordinates of the corresponding position of each closed contour in a template image, the template image being a tray image without lens tilt; determine the offset of each closed contour based on the centroid coordinates and the ideal centroid coordinates.

[0094] In the example embodiment, the image segmentation module 520 can also be configured to pre-process the tray image; wherein the pre-processing at least includes: converting the tray image into a grayscale image, denoising the tray image, and image enhancement of the tray image.

[0095] In the example embodiment, the image acquisition module 510 can also be configured to acquire multiple initial images of a tray by using an acquisition device; correct and splice the multiple initial images based on the calibration result of the acquisition device to obtain a tray image.

[0096] Those skilled in the art should understand that the division of each module in the embodiment is only a logical division of functions, and all or part of the modules can be integrated onto one or more actual carriers in actual applications, and the modules can all be implemented in the form of software by a processing unit, or all be implemented in the form of hardware, or be implemented in the form of software and hardware combination. It should be noted that the modules in the tray lens tilt detection device in the embodiment correspond one by one to the steps in the tray lens tilt detection method in the foregoing embodiment, and therefore, the specific embodiments of the embodiment can refer to the embodiments of the foregoing tray lens tilt detection method, which will not be described here.

[0097] On the basis of the above-described embodiments, Figure 6This is a schematic diagram of the structure of a tray lens tilt detection device according to one embodiment of this application, as shown below. Figure 6 As shown, the electronic device may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a tray lens tilt detection method, which includes: acquiring a tray image, wherein the tray has multiple module holes; performing image segmentation on the tray image to obtain multiple connected components; connecting the breakpoints of the edges of the connected components using a preset algorithm to obtain multiple closed contours, each closed contour corresponding to a module hole; calculating the offset of each closed contour, and marking error areas in the tray image based on the offset.

[0098] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] Based on the above embodiments, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the tray lens tilt detection method provided by the above methods. The method includes: acquiring a tray image, wherein the tray has multiple module holes; performing image segmentation on the tray image to obtain multiple connected components; connecting the breakpoints of the edges of the connected components using a preset algorithm to obtain multiple closed contours, each closed contour corresponding to a module hole; calculating the offset of each closed contour, and marking an error region in the tray image based on the offset.

[0100] On the basis of the above-mentioned embodiments, in still another aspect, the application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the tray lens tilt detection method provided by the above-mentioned methods, the method comprising: acquiring a tray image, wherein the tray has a plurality of module hole positions; performing image segmentation on the tray image to obtain a plurality of connected domains; connecting breakpoints of edges of the connected domains by using a preset algorithm to obtain a plurality of closed contours, one closed contour corresponding to one module hole position; calculating offset amounts of the closed contours, and marking an error region in the tray image based on the offset amounts.

[0101] The above are only preferred embodiments of the present application, and do not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings of the present application, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting the tilt of a material tray lens, characterized in that, The method comprises: acquiring a tray image, wherein the tray has a plurality of module hole positions; performing image segmentation on the tray image by using a multi-threshold segmentation algorithm, and performing morphological processing and connected domain extraction on the segmented tray image to obtain a plurality of connected domains; connecting breakpoints of edges of the connected domains by using a preset algorithm to obtain a plurality of closed contours, and one closed contour corresponds to one module hole position; determining serial numbers of the closed contours based on a preset labeling strategy and obtaining error position coordinates; calculating offset amounts of the closed contours and determining offset position coordinates based on the offset amounts; labeling error regions in the tray image based on the offset position coordinates and the error position coordinates; wherein the preset labeling strategy comprises: dividing all the closed contours in the tray image into rows and columns to obtain the number of closed contours in each row and the number of closed contours in each column of the tray image; acquiring the maximum number of row modules and the maximum number of column modules possessed by a template image corresponding to the tray; respectively calculating whether the number of closed contours in each row and the maximum number of row modules are the same and whether the number of closed contours in each column and the maximum number of column modules are the same; if the number of closed contours in each row and the maximum number of row modules are the same and the number of closed contours in each column and the maximum number of column modules are the same, labeling all the closed contours in a preset order; otherwise, inserting a virtual module into a row with a different number of row modules or inserting a virtual module into a column with a different number of column modules, and labeling all the closed contours after inserting the virtual module in a preset order; the preset labeling strategy is used to determine the serial numbers of the closed contours and obtain the error position coordinates, which comprises taking coordinates of the virtual module as the error position coordinates.

2. The tray lens tilt detection method according to claim 1, wherein the preset algorithm is used to connect the breakpoints of the edges of the connected domains to obtain a plurality of closed contours, which comprises: for any connected domain, acquiring all the breakpoints of the edges of the connected domain; using the preset algorithm to connect adjacent breakpoints two by two to obtain a closed contour.

3. The tray lens tilt detection method according to claim 1, wherein the offset amounts of the closed contours are calculated, which comprises: calculating the centroid coordinates of each closed contour and acquiring ideal centroid coordinates of each closed contour, the ideal centroid coordinates being the centroid coordinates of the corresponding positions of each closed contour in a template image, the template image being a tray image without lens tilt; determining the offset amounts of the closed contours based on the centroid coordinates and the ideal centroid coordinates.

4. The tray lens tilt detection method according to claim 1, wherein before the image segmentation on the tray image, the method further comprises: preprocessing the tray image; wherein the preprocessing at least comprises: converting the tray image into a grayscale image, denoising the tray image, and image enhancement on the tray image.

5. The tray lens tilt detection method according to claim 1, wherein the acquisition of the tray image comprises: using a collection device to collect a plurality of initial images of the tray; based on the calibration results of the collection device, correcting and splicing processing is performed on the plurality of initial images to obtain the tray image.

6. A dolly lens tilt detection apparatus, characterized by, comprises: an image collection module for acquiring a tray image, wherein the tray has a plurality of module hole positions; An image segmentation module is configured to perform image segmentation on the tray image by using a multi-threshold segmentation algorithm, and perform morphological processing and connected domain extraction on the segmented tray image to obtain a plurality of connected domains. A breakpoint connection module is configured to connect breakpoints of edges of the connected domains by using a preset algorithm to obtain a plurality of closed contours, one of which corresponds to one module hole position. An error coordinate calibration module is configured to determine serial numbers of the closed contours and obtain error position coordinates based on a preset labeling strategy. A position calibration module is configured to calculate offset amounts of the closed contours, determine offset position coordinates based on the offset amounts, and calibrate error regions in the tray image based on the offset position coordinates and the error position coordinates. The preset labeling strategy in the error coordinate calibration module includes: dividing all the closed contours in the tray image into rows and columns to obtain the number of closed contours in each row and the number of closed contours in each column in the tray image. Obtain the maximum number of row modules and the maximum number of column modules possessed by the template image corresponding to the tray. Calculate whether the number of closed contours in each row and the maximum number of row modules are the same and whether the number of closed contours in each column and the maximum number of column modules are the same. If the number of closed contours in each row and the maximum number of row modules are the same and the number of closed contours in each column and the maximum number of column modules are the same, label all the closed contours in a preset order. Otherwise, insert a virtual module into a row with a different number of row modules, or insert a virtual module into a column with a different number of column modules, and label all the closed contours after the virtual module insertion in a preset order. The error coordinate calibration module is further configured to take coordinates of the virtual module as error position coordinates.

7. A dolly lens tilt detection apparatus characterized by comprising: The computer program is executed by the processor to implement the tray lens tilt detection method of any one of claims 1-5. The computer program is executed by the processor to implement the tray lens tilt detection method of any one of claims 1-5. ​ 8. A computer readable storage medium storing a computer program, characterized in that, ​

Images