A correction method, device, system, electronic device, storage medium and program product

Abstract

The application discloses a correction method, device, system, electronic equipment, storage medium and program product. An image of a marker surface is obtained by image acquisition of the marker surface by the image sensor; wherein the marker surface includes at least three marker blocks, the center points of the at least three marker blocks form a right triangle, and the center points are respectively first-type marker block center points and second-type marker block center points, the first-type marker block center points are located at the center of the marker surface image; the coordinates of the second-type marker block center points in the marker surface image are determined; the tilt angles are determined based on the coordinates of the second-type marker block center points; wherein the tilt angles include vertical tilt angles and horizontal tilt angles; and the AA gripper and / or the SUT platform are controlled to move based on the tilt angles. The correction method provided by the embodiment of the application can reduce the calculation amount of active alignment between a lens and an image sensor and improve the efficiency of active alignment.

Description

Technical Field

[0001] This invention relates to the field of optical camera module manufacturing technology, and in particular to a calibration method, apparatus, system, electronic device, storage medium, and program product. Background Technology

[0002] Currently, active alignment (AA) technology in the production process of optical camera modules mainly uses intelligent AA scanning (iAAscan) to determine the optimal tilt between the lens and the image sensor, as well as the optimal Z-axis position (representing the image sensor's image plane height). This process involves a sensor unit transport (SUT) platform carrying the image sensor, moving it from top to bottom along the Z-axis at a fixed distance, performing multiple scans and adjustments until two consecutive steps show a decrease in image quality evaluation metrics. Each step requires mechanical positioning for stabilization, image acquisition, and calculation of image quality evaluation metrics. This process involves frequent iterations, high computational load, and low efficiency. Especially in large-scale production, this traditional method becomes a bottleneck for production capacity. Summary of the Invention

[0003] This invention provides a calibration method, apparatus, system, electronic device, storage medium, and program product that can reduce the computational load of active alignment between the lens and the image sensor and improve the efficiency of active alignment.

[0004] In a first aspect, embodiments of the present invention provide a calibration method for active alignment (AA) between a lens and an image sensor during the assembly of an optical camera module. The lens is held by AA grippers, and the image sensor is mounted on a sensor transport unit (SUT) platform. The method includes at least two iterative calibration operations, each of which includes the following steps: A marker surface image is obtained by the image sensor acquiring an image of the marker surface; wherein the marker surface is placed on the lens, the marker surface includes at least three marker blocks, the center points of the at least three marker blocks form a right triangle, which are the center points of the first type of marker block and the second type of marker block, respectively, and the center point of the first type of marker block is located at the center of the marker surface image; Determine the coordinates of the center point of the second type of marker block in the marker surface diagram; The tilt angle is determined based on the coordinates of the center point of the second type of marker block; wherein, the tilt angle includes a vertical tilt angle and a horizontal tilt angle; The movement of the AA gripper and / or the SUT platform is controlled based on the tilt angle.

[0005] Secondly, embodiments of the present invention also provide a calibration device for active alignment (AA) between a lens and an image sensor during the assembly of an optical camera module. The lens is held by AA grippers, and the image sensor is mounted on a sensor transport SUT platform. The device includes: The marker surface image acquisition module is used to acquire a marker surface image obtained by the image sensor capturing an image of the marker surface; wherein, the marker surface is placed on the lens, and the marker surface includes at least three marker blocks, the center points of the at least three marker blocks forming a right triangle, which are the center points of a first type of marker block and a second type of marker block, respectively, and the center point of the first type of marker block is located at the center of the marker surface image. The coordinate determination module is used to determine the coordinates of the center point of the second type of marker block in the marker surface map; A tilt angle determination module is used to determine a tilt angle based on the coordinates of the center point of the second type of marker block; wherein, the tilt angle includes a vertical tilt angle and a horizontal tilt angle; The control module is used to control the movement of the AA gripper and / or the SUT platform based on the tilt angle.

[0006] Thirdly, embodiments of the present invention also provide a calibration system, the system comprising: AA grippers, SUT platform and calibration device; The AA gripper is used to hold the lens in the optical camera module, the SUT platform is used to place the image sensor in the optical camera module, and the calibration device is used to perform the calibration method described in the embodiments of the present invention.

[0007] Fourthly, embodiments of the present invention also provide an electronic device, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the correction method described in the embodiments of the present invention.

[0008] Fifthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions that are used to cause a processor to execute the correction method described in the embodiments of the present invention.

[0009] Sixthly, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the correction method as described in the embodiments of the present invention.

[0010] This invention discloses a calibration method, apparatus, system, electronic device, storage medium, and program product. The method is used for active alignment (AA) between a lens and an image sensor during the assembly of an optical camera module. The lens is held by AA grippers, and the image sensor is mounted on a sensor transport unit (SUT) platform. The method includes at least two iterative calibration operations, each of which includes the following steps: acquiring a marker surface image obtained by the image sensor from image acquisition of the marker surface; wherein the marker surface is placed on the lens, and the marker surface includes at least three marker blocks, the center points of which form a right triangle, representing the center points of a first type of marker block and a second type of marker block, respectively, with the center point of the first type of marker block located at the center of the marker surface image; determining the coordinates of the center point of the second type of marker block in the marker surface image; determining a tilt angle based on the coordinates of the center point of the second type of marker block; wherein the tilt angle includes a vertical tilt angle and a horizontal tilt angle; and controlling the movement of the AA grippers and / or the SUT platform based on the tilt angle. The calibration method provided in this invention determines the tilt angle between the lens and the image sensor based on the coordinates of the center point of the calibration block with a specific layout in the marker surface map, thereby controlling the movement of the AA gripper and / or SUT platform to compensate for the tilt angle. It does not require frequent iterative calculations, which can reduce the amount of computation for active alignment between the lens and the image sensor and improve the efficiency of active alignment. Attached Figure Description

[0011] Figure 1 This is a flowchart of a correction method according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of a marker surface according to Embodiment 1 of the present invention; Figure 3 This is a schematic diagram illustrating the imaging principle of a marker surface in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of a marker surface view according to Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of the structure of a calibration device according to Embodiment 1 of the present invention; Figure 6 This is a schematic diagram of the structure of a correction system according to Embodiment 1 of the present invention; Figure 7 This is a schematic diagram of the structure of an electronic device according to Embodiment 1 of the present invention. Detailed Implementation

[0012] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0013] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0014] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0015] Example 1 Figure 1 This is a flowchart of a calibration method provided in Embodiment 1 of the present invention. This embodiment is applicable to the active alignment between the lens and the image sensor during the assembly of an optical camera module. The method can be executed by a calibration device, which can be implemented in software and / or hardware, or optionally by an electronic device, such as a mobile terminal, a PC, or a server.

[0016] The lens is held by an AA gripper, and the image sensor is mounted on a sensor transport SUT platform. The method includes at least two iterative calibration operations, each of which includes the following steps: S110, acquire the marker surface image obtained by the image sensor acquiring the image of the marker surface.

[0017] The marker surface is placed on the lens and includes at least three marker blocks. The center points of the at least three marker blocks form a right-angled triangle, which are the center points of a first type of marker block and a second type of marker block, respectively. The center point of the first type of marker block is located at the center of the marker surface. The center point of the first type of marker block is the center point of the marker block located at the right-angle vertex of the right-angled triangle, and the center point of the second type of marker block is the center point of the marker block not located at the right-angle vertex; and the distance between the center points of the first type of marker block and the center points of the second type of marker block is less than or equal to a set multiple of the field of view. The marker blocks are rectangular or circular in shape.

[0018] For example, Figure 2 This is a schematic diagram of a marker surface in an embodiment of the present invention, such as... Figure 2 As shown, the surface of the calibration block includes three blocks, O, A, and B, where the center points of O, A, and B form a right-angled triangle. The center point of O is located at the right-angle vertex of the triangle, making it a first-type center point. The center points of A and B are not located at the right-angle vertex, making them second-type center points. Assuming the length of OA is denoted as 'a' and the length of OB as 'b', then a ≤ 0.3F and b ≤ 0.3F, where F is the field of view of the lens or sensor. The specific layout of these calibration blocks makes their imaging highly sensitive to changes in tilt angle.

[0019] In this embodiment, before image acquisition, the distance between the image sensor's image plane and the standard focal plane (also known as the ideal focal plane) needs to be initially set. Specifically, this can be achieved by adjusting the height of the AA SUT (Alternating Object Target) using auxiliary positioning methods such as laser ranging, to position the image plane near the ideal focal plane, ensuring that the distance between the image sensor's image plane and the standard focal plane is less than a set threshold. Theoretically, for typical scenarios with a feature size a ≥ 20mm, when the optical system meets the condition that the ratio of object distance to focal length h / f > 10 (this condition covers most mainstream non-macro scenarios such as smartphone camera modules, security monitoring lenses, industrial machine vision systems, and telephoto photography systems), applying the tilt angle correction method of this application can achieve a defocus tolerance exceeding 100µm while ensuring an accuracy error of less than 0.03°. This significant tolerance characteristic allows for the use of only a simple positioning device to meet the initial positioning requirements of the system, thus transforming this method into a practical technology with high robustness and low implementation cost.

[0020] After the initial setting of the image plane height is completed, the image sensor performs real-time image acquisition of the marker surface to obtain the marker surface map. If the center point of the first type of marker block is not located in the center of the marker surface map, the SUT platform is controlled to move in parallel so that the center point of the first type of marker block in the obtained marker surface map is located in the center of the marker surface map.

[0021] S120, determine the coordinates of the center point of the second type of marker block in the marker surface diagram.

[0022] Among them, the coordinates of the center point of the second type of marker block are represented based on a pre-established spatial rectangular coordinate system. The X-axis and Y-axis of the spatial rectangular coordinate system are located in the image plane of the image sensor, and the Z-axis is perpendicular to the image plane.

[0023] Optionally, the coordinates of the center point of the second type of marker block in the marker surface image can be determined by: performing distortion correction and image binarization preprocessing on the marker surface image; and determining the coordinates of the center point of the second type of marker block from the preprocessed marker surface image based on the centroid coordinate formula.

[0024] The distortion correction method for the marker surface image can be any existing method, and is not limited here. Image binarization is the process of converting the grayscale value of each pixel into 0 or 1. The process of determining the coordinates of the center point of the second type of marker block from the preprocessed marker surface image based on the centroid coordinate formula can be as follows: first, determine the pixel coordinates of the center point of the second type of marker block in the preprocessed marker surface image based on the centroid coordinate formula, and then convert the pixel coordinates into coordinates in the aforementioned spatial rectangular coordinate system.

[0025] S130, determine the tilt angle based on the coordinates of the center point of the second type of marker block.

[0026] The tilt angle includes the vertical tilt angle and the horizontal tilt angle. The tilt angle can be understood as the angle between the lens optical axis and the normal to the image plane of the image sensor. This tilt angle can be decomposed into the vertical tilt angle (represented by rotation around the X-axis of a Cartesian coordinate system). ) and the horizontal tilt angle of rotation about the Y-axis ( The spatial rectangular coordinate system is established with the image plane of the image sensor as the reference. The X-axis and Y-axis are located in the image plane, and the Z-axis is perpendicular to the image plane (i.e., the direction of the image plane normal). The tilt angle is represented based on this spatial rectangular coordinate system.

[0027] In this embodiment, the method of determining the tilt angle based on the coordinates of the center point of the second type of marker block is derived from the imaging principle. Figure 3 This is a schematic diagram illustrating the imaging principle of a marker surface in an embodiment of the present invention, such as... Figure 3 As shown, the marked surface includes three marked blocks (also referred to as black ROI boxes), whose geometric centers form the vertices of a right triangle, namely O, A, and B. The right-angle vertex O is the center point of the marker board, with coordinates O(0, 0, 0); vertex A along the X-axis has coordinates A(a, 0, 0) and a length of OA; vertex B along the Y-axis has coordinates B(0, b, 0) and a length of OB; the hypotenuse AB has a length of... The projection position of the marker block onto the image sensor's image plane must satisfy: a ≤ 0.3F and b ≤ 0.3F (where F is the field of view of the lens or image sensor) to reduce the impact of lens edge distortion on the imaging position. Given the object distance (test distance) as h and the lens focal length as f, the image distance L can be calculated using the Gaussian formula. When the lens and image sensor are not tilted, the ideal image plane is the O''A''B'' plane (enclosed by the red line segments), satisfying: O'' is the center of the image plane, with coordinates O''(0, 0, 0); A'' coordinates are... (-a·L / h, 0, 0) O''A'' length a'' = a·L / h B'' coordinates are (0, -b·L / h, 0) O''B'' length b'' = b·L / h The length of hypotenuse A''B'' In actual imaging, the geometric center pixel coordinates of the three ROIs are extracted and converted into physical coordinates of the image plane: O' (actual image plane center) coordinates (x0, y0, z1), A' coordinates (x1, y1, z1), O'A' length a', B' coordinates (x2, y2, z1), O'B' length b', and the hypotenuse A'B' length c'.

[0028] Regarding the vertical tilt angle and horizontal tilt angle The formula derivation is as follows: The relationship between the actual imaging space and the ideal imaging plane: The actual imaging plane is formed by rotating the ideal image plane around the X-axis sequentially. Rotate around the Y-axis We obtain the composite rotation matrix (where, Approximate angle for small angles: , ): .

[0029] The ideal image point is rotated to obtain the actual 3D image point: A' coordinates: B' coordinates are Given the image center S(0,0,L), the image plane is the z=0 plane, and the projection of the spatial point P(x', y',z') onto the image plane is Q(x,y,0). Then we have: Substituting the three-dimensional coordinates of A' and B', we obtain the actual image plane coordinates: the coordinates of A' are... The coordinate of B' is Calculate the offset: , Solve this problem. : , The derivation results show that the tilt angle is related to the lens focal length, object distance, the distance between the center point of the second type of marker block and the center point of the first type of marker block on the marked object surface, and the coordinates of the center point of the second type of marker block.

[0030] Optionally, the tilt angle can be determined based on the coordinates of the center point of the second type of marker block by: obtaining the lens focal length, object distance, and first distance; wherein, the first distance is the distance between the center point of the second type of marker block and the center point of the first type of marker block in the marker object surface; and determining the tilt angle based on the lens focal length, object distance, first distance, and the coordinates of the center point of the second type of marker block.

[0031] Specifically, based on the above derivation, the formula for determining the tilt angle based on the lens focal length, object distance, first distance, and the coordinates of the center point of the second type of marker block can be expressed as: ; ;in, The horizontal tilt angle is... Let be the vertical tilt angle, a and b be the first distances, h be the object distance, and L be the image distance, expressed as: f is the lens focal length. These are the coordinates of the center point of the second type of marker block.

[0032] Optionally, the tilt angle can be determined based on the lens focal length, object distance, first distance, and coordinates of the center point of the second type of marker block as follows: if the number of marker blocks is greater than 3, then any center point of the second type of marker block on the first right-angled side and any center point of the second type of marker block on the second right-angled side are grouped together to obtain multiple groups of center points of marker blocks; for each group of center points of marker blocks, a candidate tilt angle is determined based on the lens focal length, object distance, first distance, and coordinates of the center point of the second type of marker block within the group of center points of marker blocks; the average value of the candidate tilt angles corresponding to multiple groups of center points of marker blocks is determined as the final tilt angle.

[0033] In this embodiment, if the number of marker blocks is greater than 3, the center points of two second-type marker blocks that can form a right triangle are grouped together. For each group of second-type marker block center points, a set of vertical tilt angles and horizontal tilt angles are calculated according to the above formula based on the lens focal length, object distance, first distance, and the coordinates of the second-type marker block center points within the group. Finally, the average value of multiple sets of vertical tilt angles and horizontal tilt angles is determined as the final vertical tilt angle and horizontal tilt angle. For example, Figure 4 This is a schematic diagram of a marker surface view in an embodiment of the present invention, such as... Figure 4As shown, there are four marker blocks, namely O, A, B and C. A and B can be grouped together, and C and B can be grouped together to obtain the center points of two groups of second-type marker blocks. Based on the lens focal length, object distance, first distance and the coordinates of the center points of A and B, a set of vertical tilt angles and horizontal tilt angles are calculated according to the above formula. Then, based on the lens focal length, object distance, first distance and the coordinates of the center points of C and B, another set of vertical tilt angles and horizontal tilt angles are calculated according to the above formula. Finally, the average of the two sets of vertical tilt angles and horizontal tilt angles is calculated to obtain the final vertical tilt angles and horizontal tilt angles.

[0034] S140 controls the movement of the AA gripper and / or SUT platform based on tilt angle.

[0035] The AA gripper can operate in six degrees of freedom (x, y, z, ...). , , ) or five degrees of freedom (x, y, z, , The SUT operates in six degrees of freedom or three degrees of freedom (x, y, z). In this embodiment, if the AA gripper operates in six degrees of freedom and the SUT operates in six degrees of freedom, then the AA gripper and / or the SUT platform are controlled to perform corresponding rotational movements based on the tilt angle to compensate for the aforementioned vertical and horizontal tilt angles. If the AA gripper operates in five degrees of freedom and the SUT operates in three degrees of freedom, then the AA gripper is controlled to perform corresponding rotational movements based on the tilt angle to compensate for the aforementioned vertical and horizontal tilt angles.

[0036] In this embodiment, in order to improve the accuracy of calibration, a second compensation can be performed after the previous compensation is completed, that is, S110-S140 is repeated.

[0037] The technical solution of this embodiment obtains a marker surface map obtained by the image sensor acquiring an image of the marker surface. The marker surface is placed on the lens and includes at least three marker blocks. The center points of the at least three marker blocks form a right triangle, which are the center points of a first type of marker block and a second type of marker block, respectively. The center point of the first type of marker block is located at the center of the marker surface map. The coordinates of the center point of the second type of marker block in the marker surface map are determined. A tilt angle is determined based on the coordinates of the center point of the second type of marker block. The tilt angle includes a vertical tilt angle and a horizontal tilt angle. The movement of the AA gripper and / or the SUT platform is controlled based on the tilt angle. The correction method provided by this embodiment determines the tilt angle between the lens and the image sensor based on the coordinates of the center points of calibration blocks with a specific layout in the marker surface map, thereby controlling the movement of the AA gripper and / or the SUT platform to compensate for the tilt angle. This eliminates the need for frequent iterative calculations, reducing the computational load of active alignment between the lens and the image sensor and improving the efficiency of active alignment.

[0038] Example 2 Figure 5 This is a schematic diagram of a calibration device provided in Embodiment 2 of the present invention. The device is used for active alignment (AA) between the lens and the image sensor during the assembly of an optical camera module. The lens is held by AA grippers, and the image sensor is mounted on a sensor transport SUT platform. The device: The marker surface image acquisition module 510 is used to acquire a marker surface image obtained by the image sensor acquiring an image of the marker surface; wherein, the marker surface is placed on the lens, and the marker surface includes at least three marker blocks, the center points of the at least three marker blocks forming a right triangle, which are the center points of a first type of marker block and a second type of marker block, respectively, and the center point of the first type of marker block is located at the center of the marker surface image; The coordinate determination module 520 is used to determine the coordinates of the center point of the second type of marker block in the marker surface map; The tilt angle determination module 530 is used to determine the tilt angle based on the coordinates of the center point of the second type of marker block; wherein, the tilt angle includes a vertical tilt angle and a horizontal tilt angle; The control module 540 is used to control the movement of the AA gripper and / or the SUT platform based on the tilt angle.

[0039] Optionally, the coordinate determination module 520 is also used for: The surface image of the marker is preprocessed with distortion correction and image binarization; The coordinates of the center point of the second type of marker block are determined from the preprocessed marker surface map based on the centroid coordinate formula.

[0040] Optionally, the tilt angle determination module 530 is also used for: Obtain the lens focal length, object distance, and first distance; wherein, the first distance is the distance between the center point of the second type of marker block and the center point of the first type of marker block in the marked object surface; The tilt angle is determined based on the lens focal length, object distance, first distance, and the coordinates of the center point of the second type of marker block.

[0041] Optionally, the tilt angle determination module 530 is also used for: If the number of marker blocks is greater than 3, then any center point of a second type of marker block located on the first right-angled side and any center point of a second type of marker block located on the second right-angled side are grouped together to obtain multiple groups of marker block center points; For each group of marker block center points, a candidate tilt angle is determined based on the lens focal length, object distance, first distance, and the coordinates of the second type of marker block center points within the group of marker block center points. The average value of the candidate tilt angles corresponding to the center point groups of multiple marker blocks is determined as the final tilt angle.

[0042] Optionally, the tilt angle determination module 530 is also used for: ; ;in, The vertical tilt angle is... Let be the horizontal tilt angle, a and b be the first distances, h be the object distance, and L be the image distance, expressed as: f is the lens focal length. These are the coordinates of the center point of the second type of marker block.

[0043] Optionally, it also includes: a SUT platform control module, used for: If the center point of the first type of marker block is not located at the center of the marker surface map, the SUT platform is controlled to move in parallel so that the center point of the first type of marker block in the obtained marker surface map is located at the center of the marker surface map.

[0044] Optionally, the distance between the image plane of the image sensor and the standard focal plane is less than a set threshold.

[0045] Optionally, the shape of the marker block is rectangular or circular.

[0046] Optionally, the center point of the first type of marker block is the center point of the marker block located at the right-angle vertex of the right triangle, and the center point of the second type of marker block is the center point of the marker block not located at the right-angle vertex; and the distance between the center point of the first type of marker block and the center point of the second type of marker block is less than or equal to a set multiple of the field of view.

[0047] Optionally, the coordinates of the center point of the second type of marker block and the tilt angle are represented based on a pre-established spatial rectangular coordinate system, wherein the X-axis and Y-axis of the spatial rectangular coordinate system are located in the image plane of the image sensor, and the Z-axis is perpendicular to the image plane.

[0048] The above-described apparatus can execute the methods provided in all the foregoing embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the above methods. Technical details not described in detail in this embodiment can be found in the methods provided in all the foregoing embodiments of the present invention.

[0049] Example 3 Figure 6 This is a schematic diagram of the structure of a correction system provided in Embodiment 3 of the present invention, as shown below. Figure 6 The system includes an AA gripper, a SUT platform, and a calibration device. The AA gripper is used to hold the lens in the optical camera module, the SUT platform is used to place the image sensor in the optical camera module, and the calibration device is used to perform the calibration method described in the above embodiments.

[0050] The AA gripper has five degrees of freedom, while the SUT platform has three degrees of freedom.

[0051] Example 4 Figure 7 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components, connections and relationships between components, and their functions shown herein are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0052] like Figure 7As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0053] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0054] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as correction methods.

[0055] In some embodiments, the correction method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the correction method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the correction method by any other suitable means (e.g., by means of firmware).

[0056] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0057] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0058] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0059] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0060] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0061] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0062] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the correction method provided in any embodiment of this application.

[0063] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0064] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0065] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A calibration method, characterized in that, The method is used for active alignment (AA) between a lens and an image sensor during the assembly of an optical camera module. The lens is held by AA grippers, and the image sensor is mounted on a sensor transport unit (SUT) platform. The method includes at least two iterative alignment operations, each of which includes the following steps: A marker surface image is obtained by the image sensor acquiring an image of the marker surface; wherein the marker surface is placed on the lens, the marker surface includes at least three marker blocks, the center points of the at least three marker blocks form a right triangle, which are the center points of the first type of marker block and the second type of marker block, respectively, and the center point of the first type of marker block is located at the center of the marker surface image; Determine the coordinates of the center point of the second type of marker block in the marker surface diagram; The tilt angle is determined based on the coordinates of the center point of the second type of marker block; wherein, the tilt angle includes a vertical tilt angle and a horizontal tilt angle; The movement of the AA gripper and / or the SUT platform is controlled based on the tilt angle.

2. The method according to claim 1, characterized in that, Determining the coordinates of the center point of the second type of marker block in the marker surface map includes: The surface image of the marker is preprocessed with distortion correction and image binarization; The coordinates of the center point of the second type of marker block are determined from the preprocessed marker surface map based on the centroid coordinate formula.

3. The method according to claim 1, characterized in that, Determining the tilt angle based on the coordinates of the center point of the second type of marker block includes: Obtain the lens focal length, object distance, and first distance; wherein, the first distance is the distance between the center point of the second type of marker block and the center point of the first type of marker block in the marked object surface; The tilt angle is determined based on the lens focal length, object distance, first distance, and the coordinates of the center point of the second type of marker block.

4. The method according to claim 3, characterized in that, Determining the tilt angle based on the lens focal length, object distance, first distance, and the coordinates of the center point of the second type of marker block includes: If the number of marker blocks is greater than 3, then any center point of a second type of marker block located on the first right-angled side and any center point of a second type of marker block located on the second right-angled side are grouped together to obtain multiple groups of marker block center points; For each group of marker block center points, a candidate tilt angle is determined based on the lens focal length, object distance, first distance, and the coordinates of the second type of marker block center points within the group of marker block center points. The average value of the candidate tilt angles corresponding to the center point groups of multiple marker blocks is determined as the final tilt angle.

5. The method according to claim 3 or 4, characterized in that, Determining the tilt angle based on the lens focal length, object distance, first distance, and the coordinates of the center point of the second type of marker block includes: ; ;in, The horizontal tilt angle is... Let be the vertical tilt angle, a and b be the first distances, h be the object distance, and L be the image distance, expressed as: f is the lens focal length. These are the coordinates of the center point of the second type of marker block.

6. The method according to claim 1, characterized in that, After acquiring the marker surface image obtained by the image sensor capturing the marker surface, the method further includes: If the center point of the first type of marker block is not located at the center of the marker surface map, the SUT platform is controlled to move in parallel so that the center point of the first type of marker block in the obtained marker surface map is located at the center of the marker surface map.

7. The method according to claim 1, characterized in that, The distance between the image plane of the image sensor and the standard focal plane is less than a set threshold.

8. The method according to claim 1, characterized in that, The shape of the marker block is rectangular or circular.

9. The method according to claim 1, characterized in that, The center point of the first type of marker block is the center point of the marker block located at the right-angle vertex of the right triangle, and the center point of the second type of marker block is the center point of the marker block not located at the right-angle vertex; and the distance between the center point of the first type of marker block and the center point of the second type of marker block is less than or equal to a set multiple of the field of view.

10. The method according to claim 1, characterized in that, The coordinates of the center point of the second type of marker block and the tilt angle are represented based on a pre-established spatial rectangular coordinate system. The X-axis and Y-axis of the spatial rectangular coordinate system are located in the image plane of the image sensor, and the Z-axis is perpendicular to the image plane.

11. A calibration device, characterized in that, The device is used for active alignment (AA) between the lens and the image sensor during the assembly of an optical camera module. The lens is held by AA grippers, and the image sensor is mounted on a sensor transport SUT platform. The device includes: The marker surface image acquisition module is used to acquire a marker surface image obtained by the image sensor capturing an image of the marker surface; wherein, the marker surface is placed on the lens, and the marker surface includes at least three marker blocks, the center points of the at least three marker blocks forming a right triangle, which are the center points of a first type of marker block and a second type of marker block, respectively, and the center point of the first type of marker block is located at the center of the marker surface image. The coordinate determination module is used to determine the coordinates of the center point of the second type of marker block in the marker surface map; A tilt angle determination module is used to determine a tilt angle based on the coordinates of the center point of the second type of marker block; wherein, the tilt angle includes a vertical tilt angle and a horizontal tilt angle; The control module is used to control the movement of the AA gripper and / or the SUT platform based on the tilt angle.

12. A calibration system, characterized in that, The system includes: AA grippers, SUT platform, and calibration device; The AA gripper is used to hold the lens in the optical camera module, and the SUT platform is used to place the image sensor in the optical camera module; the calibration device is used to perform the calibration method according to any one of claims 1-10.

13. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the correction method according to any one of claims 1-10.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the correction method according to any one of claims 1-10.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the correction method as described in any one of claims 1-10.

Images