Camera calibration method, device, equipment and storage medium

Abstract

A camera calibration method, device, equipment and storage medium are disclosed. An infrared camera is controlled to capture a plurality of light spot images of calibration balls located at different orientations; for each calibration light spot image, a first intersection point is determined according to light spot position information in the calibration light spot image and a set initial extrinsic parameter; a first distance corresponding to the calibration light spot image is determined according to a light reflection principle based on the first intersection point; wherein the first distance is a distance between a center of the calibration ball and the first intersection point; the set initial extrinsic parameter is optimized based on the first distances of the plurality of calibration light spot images to obtain a target extrinsic parameter of the infrared camera; the target extrinsic parameter is verified according to the test light spot image, and if the verification is passed, the camera is calibrated according to the target extrinsic parameter. The camera calibration method provided in the embodiment of the present application can improve the efficiency and accuracy of camera calibration based on light spot images to calibrate the extrinsic parameter of the camera.

Description

Technical Field

[0001] The present invention relates to the field of eye-tracking technology, and in particular to a camera calibration method, apparatus, device and storage medium. Background Technology

[0002] In the field of eye tracking, determining the pose information of the optical center of an infrared camera, i.e., the camera's extrinsic parameters, is a crucial step in improving the accuracy of eye tracking algorithms. Current technologies often require the use of a scaled calibration board and a known-sized checkerboard to obtain these parameters. The extrinsic parameters are calculated by taking images of the same checkerboard at different scales. This method involves numerous coordinate transformations, introducing more systematic errors; furthermore, finding precise corner points using the checkerboard is time-consuming and inefficient. Summary of the Invention

[0003] This invention provides a camera calibration method, apparatus, device, and storage medium, which can improve the efficiency and accuracy of camera calibration.

[0004] In a first aspect, embodiments of the present invention provide a camera calibration method, including:

[0005] An infrared camera is controlled to capture images of a calibration ball located at different positions, obtaining multiple light spot images; wherein, the radius of the calibration ball is determined by the radius of curvature of the human cornea; the light spot images include calibration light spot images and test light spot images;

[0006] For each calibration spot image, the first intersection point is determined based on the spot position information in the calibration spot image and the initial extrinsic parameters; wherein, the first intersection point is the intersection of the line connecting the camera optical center and the center of the calibration ball with the infrared lamp ring surface, and the extrinsic parameters are the pose information of the infrared camera optical center;

[0007] Based on the first intersection point, the first distance corresponding to the calibration spot pattern is determined according to the principle of light reflection; wherein, the first distance is the distance between the center of the calibration ball and the first intersection point;

[0008] Based on the first distance of the multiple calibration spot images, the initial extrinsic parameters are optimized to obtain the target extrinsic parameters of the infrared camera;

[0009] The target extrinsic parameters are verified based on the test spot pattern. If the verification is successful, the camera is calibrated based on the target extrinsic parameters.

[0010] Secondly, embodiments of the present invention also provide a camera calibration device, comprising:

[0011] The light spot image acquisition module is used to control the infrared camera to take pictures of the calibration ball located at different positions and obtain multiple light spot images; wherein, the radius of the calibration ball is determined by the radius of curvature of the human cornea; the light spot images include calibration light spot images and test light spot images;

[0012] The first intersection point determination module is used to determine the first intersection point for each calibration spot image based on the spot position information in the calibration spot image and the set initial extrinsic parameters; wherein, the first intersection point is the intersection point of the line connecting the camera optical center and the center of the calibration ball and the infrared lamp ring surface, and the extrinsic parameters are the pose information of the infrared camera optical center;

[0013] The first distance determination module is used to determine the first distance corresponding to the calibration spot pattern based on the first intersection point according to the principle of light reflection; wherein, the first distance is the distance between the center of the calibration ball and the first intersection point;

[0014] The target extrinsic parameter acquisition module is used to optimize the set initial extrinsic parameters based on the first distance of the multiple calibration spot images to obtain the target extrinsic parameters of the infrared camera;

[0015] The target extrinsic parameter calibration module is used to verify the target extrinsic parameters based on the test spot image. If the verification is successful, the camera is calibrated based on the target extrinsic parameters.

[0016] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the camera calibration method described in the embodiments of the present invention.

[0020] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute and implement the camera calibration method described in the embodiments of the present invention.

[0021] This invention discloses a camera calibration method, apparatus, device, and storage medium. An infrared camera is controlled to photograph a calibration sphere located at different orientations, obtaining multiple light spot images. The radius of the calibration sphere is determined by the radius of curvature of the human cornea. The light spot images include calibration light spot images and test light spot images. For each calibration light spot image, a first intersection point is determined based on the light spot position information and initial extrinsic parameters. The first intersection point is the intersection of the line connecting the optical center of the camera and the center of the calibration sphere with the infrared lamp ring surface. The extrinsic parameter is the pose information of the optical center of the infrared camera. Based on the first intersection point, a first distance corresponding to the calibration light spot image is determined according to the principle of light reflection. The first distance is the distance between the center of the calibration sphere and the first intersection point. The initial extrinsic parameters are optimized based on the first distances of the multiple calibration light spot images to obtain the target extrinsic parameters of the infrared camera. The target extrinsic parameters are verified according to the test light spot images. If the verification is successful, the camera is calibrated based on the target extrinsic parameters. The camera calibration method provided in this embodiment of the invention calibrates camera extrinsic parameters based on spot patterns, which can improve the efficiency and accuracy of camera calibration. Attached Figure Description

[0022] Figure 1 This is a flowchart of a camera calibration method according to Embodiment 1 of the present invention;

[0023] Figure 2 This is a schematic diagram of the light spot pattern in Embodiment 1 of the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of a camera calibration device according to Embodiment 2 of the present invention;

[0025] Figure 4 This is a schematic diagram of the structure of an electronic device according to Embodiment 3 of the present invention. Detailed Implementation

[0026] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0027] Example 1

[0028] Figure 1 This is a flowchart of a camera calibration method provided in Embodiment 1 of the present invention. This embodiment can be applied to the calibration of camera extrinsic parameters in eye tracking. The method can be executed by a camera calibration device, which can be implemented in software and / or hardware. Optionally, it can be implemented by an electronic device, such as a mobile terminal, a PC, or a server.

[0029] In this embodiment, the camera's extrinsic parameters are calibrated using a pre-established Cartesian coordinate system for the light ring. This Cartesian coordinate system is defined as follows: For the right eye, the horizontal direction to the right of the light ring surface is defined as the positive x-axis, the vertical downward direction of the light ring surface is defined as the positive y-axis, and the direction perpendicular to the light ring surface forward is defined as the positive z-axis. For the left eye, the horizontal direction to the left of the light ring surface is defined as the positive x-axis, the vertical downward direction of the light ring surface is defined as the positive y-axis, and the direction perpendicular to the light ring surface forward is defined as the positive z-axis.

[0030] The lamp ring surface can be characterized by the coordinates of each infrared lamp. For example, the lamp ring surface can be represented in the following form: L = [-13.0500, 21.4300, 0; -22.1900, 5.9000, 0; -23.1100, -12.4400, 0; -9.8900, -22.1100, 0; 9.8900, -22.1100, 0; 23.1200, -12.4400, 0; 25.5900, 5.8400, 0; 15.4200, 21.2400, 0].

[0031] like Figure 1 As shown, the method specifically includes the following steps:

[0032] S110 controls the infrared camera to take pictures of the calibration ball located in different positions, and obtain multiple light spot images.

[0033] The radius of the calibration ball is determined by the radius of curvature of the human cornea; optionally, the radius of the calibration ball is equal to or similar to the radius of curvature of the human cornea. The calibration ball is also capable of reflecting infrared light; optionally, the reflectivity of the calibration ball is higher than a set value. The light spot diagram includes a calibration light spot diagram and a test light spot diagram. The calibration light spot diagram is used to determine the camera's extrinsic parameters, and the test light spot diagram is used to test and verify the determined camera extrinsic parameters.

[0034] In this embodiment, the calibration ball is continuously moved. Each time the calibration ball moves to a position corresponding to a certain orientation, the infrared camera in the eye-tracking device captures an image of the calibration ball, obtaining multiple light spot images. The light spots in these images are formed by the reflection of infrared light from the calibration ball. For example, Figure 2 This is the light spot image obtained in this embodiment, such as... Figure 2 As shown, the light spot diagram contains 8 light spots, which are generated by the calibration ball reflecting the infrared light emitted by 8 infrared light sources.

[0035] S120, for each calibration spot image, determine the first intersection point based on the spot position information in the calibration spot image and the set initial external parameters.

[0036] The first intersection point is the intersection of the line connecting the camera's optical center and the center of the calibration sphere with the infrared lamp torus. The extrinsic parameter is the pose information of the infrared camera's optical center. The pose information includes the position and rotation information of the camera's optical center. The position information can be represented by the camera's optical center's coordinates, and the rotation information can be represented by the rotation matrix or rotation angle corresponding to the camera's optical center. The initial extrinsic parameters are preset. For example, they can be set as: Tco = [0, 27.81, -2.92], Rco = [1, 0, 0; 0, 0.7071, 0.7071; 0, -0.7071, 0.7071]

[0037] In this embodiment, each calibration spot image represents the position of the calibration ball. When the camera extrinsic parameters match the actual state, the calculated positions of the calibration balls using m lights should be consistent. When the camera extrinsic parameters do not match the actual state, the calculated positions of the calibration balls deviate significantly. Therefore, the camera extrinsic parameters can be determined through an optimization algorithm.

[0038] Optionally, the method for determining the first intersection point based on the spot position information in the calibration spot diagram and the initial extrinsic parameters can be as follows: For each spot in the calibration spot diagram, determine the second intersection point between the reflection line corresponding to the spot and the infrared lamp ring surface; connect the second intersection point with the point where the corresponding infrared lamp is located to obtain the straight line corresponding to the spot; determine the maximum likelihood intersection point of multiple straight lines corresponding to the calibration spot diagram as the first intersection point.

[0039] In this embodiment, for the j-th spot g(i,j) of the i-th calibration spot image, the intersection point grp(i,j) of the reflected ray and the lamp ring surface corresponding to the spot g(i,j) and the direction grd(i,j) of the reflected ray are obtained based on the current camera extrinsic parameters Rcn and Tcn and the camera intrinsic parameters (fc, cc, kc). This reflected ray is the reflected light ray after the infrared lamp Lj is reflected at the current calibration ball position.

[0040] Optionally, the second intersection point between the reflected line corresponding to the light spot and the infrared lamp ring surface can be determined by: obtaining the extension line of the line connecting the light spot and the optical center of the infrared camera; and determining the intersection point of the extension line and the infrared lamp ring surface as the second intersection point.

[0041] Specifically, connect g(i,j) to the camera optical center O, and extend the line to intersect the infrared lamp torus z=0 to obtain the second intersection point.

[0042] In this embodiment, for m infrared lamps, m second intersection points are obtained in the manner described above. These second intersection points are then connected to the corresponding infrared lamp locations to obtain m straight lines. Based on the equations corresponding to these m straight lines, an overdetermined system of equations is formed. Solving this system yields the maximum likelihood intersection point of the m straight lines, which is then used as the first intersection point.

[0043] At this point, the reflected ray direction grd(i,j), the intersection point grp(i,j) of the reflected ray and the lamp ring, the lamp coordinates L(j), the radius R of the calibration sphere, the first intersection point Co, and the direction b from the optical center to the center of the calibration sphere are obtained. S130, based on the first intersection point, the first distance corresponding to the calibration spot pattern is determined according to the principle of light reflection.

[0044] The first distance is the distance between the center of the calibrated ball and the first intersection point.

[0045] In this embodiment, the method of determining the first distance corresponding to the calibration spot based on the principle of light reflection based on the first intersection point can be as follows: set a first initial distance, determine the center of the calibration ball and the reflection point corresponding to the spot based on the first initial distance and the first intersection point; construct an optimization function for the first initial distance based on the center of the calibration ball, the reflection point, the infrared lamp and the second intersection point through the principle of light reflection; optimize the first initial distance according to the optimization function to obtain the first distance corresponding to the spot.

[0046] The method for determining the center of the calibration sphere and the reflection point corresponding to the light spot based on the first initial distance and the first intersection point can be as follows: determine the center of the calibration sphere corresponding to the light spot based on the first initial distance, the first intersection point and the optical center of the infrared camera; determine the second distance based on the center of the calibration sphere, the radius of the calibration sphere and the second intersection point using the law of cosines; and determine the reflection point based on the second distance and the second intersection point.

[0047] Specifically, based on the direction of the line connecting the first intersection point and the optical center of the infrared camera (direction b), which points from the optical center to the center of the calibration sphere, the coordinates of the calibration sphere's center are obtained by multiplying the direction vector b by the first initial distance and adding the coordinates of the first intersection point. For example, assuming the distance between the calibration sphere's center c and the first intersection point is kc, and setting an initial value (the first initial distance), the coordinates Cn of c can be obtained, i.e., Cn = co + kc * b.

[0048] Wherein, the second distance is the distance between the second intersection point and the reflection point. Assuming the reflection point is q, and the distance between the reflection point and the second intersection point grp (the second distance) is kq, then q = grp + kq * grd. In the triangle formed by the second intersection point grp, the center of the sphere c, and the reflection point q, based on the law of cosines, we can obtain R... 2 =kq 2 +d 2 -2·cosθ·kq·d, where R is the radius of the calibration ball, d is the distance between the second intersection point grp and the center c of the ball, and cosθ is the angle between c-grp and q-grp. Since kc is a known quantity, there is only one unknown quantity, kq. Calculating the quadratic equation yields kq expressed in terms of kc, and based on kq, the coordinate expression of the reflection point q can be determined.

[0049] According to the reflection theorem, the angle of incidence equals the angle of reflection, meaning the angle formed by the reflection point q of the infrared lamp L and the center c of the sphere, and the angle formed by the second intersection point grp, the reflection point q, and the center c of the sphere are equal. Based on the equality of these two angles, an optimization function is constructed. The optimization objective is to make the difference between the angle formed by the reflection point q of the infrared lamp L and the center c of the sphere, and the angle formed by the second intersection point grp, the reflection point q, and the center c of the sphere close to 0, i.e., the difference is less than a certain set value. This value of qc is the optimal value, and the optimal qc value is determined as the first distance corresponding to the j-th light spot.

[0050] S140 optimizes the initial extrinsic parameters based on the first distance of multiple calibration spot images to obtain the target extrinsic parameters of the infrared camera.

[0051] In this embodiment, the method of optimizing the initial extrinsic parameters based on the first distance of multiple calibration spot images can be as follows: determine the first statistical value of the first distance of each calibration spot image; calculate the average value of the first statistical value of each calibration spot image to obtain the optimization parameters; iteratively optimize the initial extrinsic parameters based on the optimization parameters until the optimization termination condition is met to obtain the target extrinsic parameters of the infrared camera.

[0052] The first statistical value can be either the standard deviation or the variance. For each calibration spot image containing m spots, m first distances can be obtained. The standard deviation or variance of these m first distances is then calculated to obtain the first statistical value corresponding to each calibration spot image. Assuming there are L calibration spot images, the average of the first statistical values ​​for the L calibration spot images is calculated. This average is used as an optimization parameter, and the initial extrinsic parameters are optimized based on this parameter to obtain the optimized camera extrinsic parameters. Then, based on the optimized camera extrinsic parameters, the first statistical value corresponding to each calibration spot image is calculated according to the above embodiment. The average of the first statistical values ​​is then calculated, and this average is used as an optimization parameter. The initial extrinsic parameters are optimized based on this parameter to obtain the optimized camera extrinsic parameters. The camera extrinsic parameters are iteratively optimized in the above manner until the optimization termination condition is met, thus obtaining the target extrinsic parameters of the infrared camera.

[0053] S150 verifies the target's extrinsic parameters based on the test spot pattern. If the verification is successful, the camera is calibrated based on the target's extrinsic parameters.

[0054] Specifically, the method for verifying the target extrinsic parameters based on the test spot pattern can be as follows: determine the second statistical value of the test spot pattern based on the target extrinsic parameters; if the second statistical value meets the set conditions, the verification is successful.

[0055] The second statistical value can be the mean, minimum, or maximum value of the first statistical values ​​of N test spot images. The second statistical value is compared with a set threshold. If the comparison result meets the set conditions, the verification is successful, and the camera is calibrated according to the target extrinsic parameters.

[0056] In this embodiment, firstly, based on the target camera extrinsic parameters, the first statistical value of the first distance of each test spot image is determined according to the scheme of the above embodiment. Then, the mean, minimum or maximum value of the first statistical values ​​of N test spot images is calculated as the second statistical value of the test spot image. Finally, the second statistical value is compared with a set threshold. If the comparison result meets the set conditions, the verification is passed, and the camera is calibrated according to the target extrinsic parameters.

[0057] The technical solution of this embodiment involves controlling an infrared camera to capture images of calibration spheres located at different orientations, obtaining multiple light spot images. The radius of the calibration sphere is determined by the radius of curvature of the human cornea. The light spot images include calibration light spot images and test light spot images. For each calibration light spot image, a first intersection point is determined based on the light spot position information and initial extrinsic parameters. The first intersection point is the intersection of the line connecting the camera's optical center and the center of the calibration sphere with the infrared lamp ring surface. The extrinsic parameter is the pose information of the infrared camera's optical center. Based on the first intersection point, a first distance corresponding to the calibration light spot image is determined according to the principle of light reflection. The first distance is the distance between the center of the calibration sphere and the first intersection point. The initial extrinsic parameters are optimized based on the first distances of the multiple calibration light spot images to obtain the target extrinsic parameters of the infrared camera. The target extrinsic parameters are verified using the test light spot images. If the verification is successful, the camera is calibrated based on the target extrinsic parameters. The camera calibration method provided in this embodiment of the invention calibrates camera extrinsic parameters based on spot patterns, which can improve the efficiency and accuracy of camera calibration.

[0058] Example 2

[0059] Figure 3 This is a schematic diagram of the structure of a camera calibration device provided in Embodiment 2 of the present invention, as shown below. Figure 3 As shown, the device includes:

[0060] The light spot acquisition module 310 is used to control the infrared camera to take pictures of the calibration ball located in different positions and obtain multiple light spot images; wherein, the radius of the calibration ball is determined by the radius of curvature of the human cornea; the light spot images include calibration light spot images and test light spot images;

[0061] The first intersection point determination module 320 is used to determine the first intersection point for each calibration spot image based on the spot position information in the calibration spot image and the set initial extrinsic parameters; wherein, the first intersection point is the intersection point of the line connecting the camera optical center and the calibration ball center with the infrared lamp ring surface, and the extrinsic parameters are the pose information of the infrared camera optical center;

[0062] The first distance determination module 330 is used to determine the first distance corresponding to the calibration spot pattern based on the first intersection point according to the principle of light reflection; wherein, the first distance is the distance between the center of the calibration ball and the first intersection point;

[0063] The target extrinsic parameter acquisition module 340 is used to optimize the initial extrinsic parameters based on the first distance of multiple calibration spot images to obtain the target extrinsic parameters of the infrared camera;

[0064] The target extrinsic calibration module 350 is used to verify the target extrinsic parameters based on the test spot pattern. If the verification is successful, the camera is calibrated based on the target extrinsic parameters.

[0065] Optionally, the first intersection point determination module 320 is also used for:

[0066] For each spot in the calibration spot diagram, determine the second intersection point between the reflected line corresponding to the spot and the infrared lamp ring surface;

[0067] Connect the second intersection point with the corresponding infrared lamp location to obtain the straight line corresponding to the light spot;

[0068] Determine the maximum likelihood intersection point of the multiple straight lines corresponding to the calibrated spot pattern, and use it as the first intersection point.

[0069] Optionally, the first intersection point determination module 320 is also used for:

[0070] Obtain the extension line of the line connecting the light spot and the optical center of the infrared camera;

[0071] Determine the intersection point of the extended line and the infrared lamp ring surface as the second intersection point.

[0072] Optionally, the first distance determination module 330 is also used for:

[0073] Set a first initial distance, and determine the center of the calibration ball and the reflection point corresponding to the light spot based on the first initial distance and the first intersection point;

[0074] An optimization function for the first initial distance is constructed based on the principle of light reflection by calibrating the center of the ball, the reflection point, the infrared lamp, and the second intersection point.

[0075] The initial distance is optimized according to the optimization function to obtain the first distance corresponding to the light spot.

[0076] Optionally, the first distance determination module 330 is also used for:

[0077] The center of the calibration sphere corresponding to the light spot is determined based on the first initial distance, the first intersection point, and the optical center of the infrared camera.

[0078] The second distance is determined using the law of cosines based on the calibrated center and radius of the small ball and the second intersection point; where the second distance is the distance between the second intersection point and the reflection point.

[0079] The reflection point is determined based on the second distance and the second intersection point.

[0080] Optionally, the target extrinsic parameter acquisition module 340 is also used for:

[0081] Determine the first statistical value of the first distance for each calibrated spot pattern;

[0082] The average value of the first statistical value of each calibration spot pattern is used to obtain the optimization parameters;

[0083] The initial extrinsic parameters are iteratively optimized based on the optimization parameters until the optimization termination condition is met, thereby obtaining the target extrinsic parameters of the infrared camera.

[0084] Optionally, the target extrinsic calibration module 350 is also used for:

[0085] The second statistical value of the test spot pattern is determined based on the target extrinsic parameters;

[0086] If the second statistical value meets the set conditions, the verification is successful.

[0087] The above-described apparatus can execute the methods provided in all the foregoing embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the above methods. Technical details not described in detail in this embodiment can be found in the methods provided in all the foregoing embodiments of the present invention.

[0088] Example 3

[0089] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0090] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0091] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0092] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as camera calibration methods.

[0093] In some embodiments, the camera calibration method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the camera calibration method by any other suitable means (e.g., by means of firmware).

[0094] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0095] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0096] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0097] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0098] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0099] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0100] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0101] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A camera calibration method, characterized in that, include: An infrared camera is controlled to capture images of a calibration ball located at different positions, obtaining multiple light spot images; wherein, the radius of the calibration ball is determined by the radius of curvature of the human cornea; the light spot images include calibration light spot images and test light spot images; For each calibration spot image, the first intersection point is determined based on the spot position information in the calibration spot image and the initial extrinsic parameters; wherein, the first intersection point is the intersection of the line connecting the camera optical center and the center of the calibration ball with the infrared lamp ring surface, and the extrinsic parameters are the pose information of the infrared camera optical center; Based on the first intersection point, the first distance corresponding to the calibration spot pattern is determined according to the principle of light reflection; wherein, the first distance is the distance between the center of the calibration ball and the first intersection point; The initial extrinsic parameters of the infrared camera are optimized based on the first distance from multiple calibration spot images to obtain the target extrinsic parameters of the infrared camera. The target extrinsic parameters are verified based on the test spot pattern. If the verification is successful, the camera is calibrated based on the target extrinsic parameters. The step of determining the first intersection point based on the spot position information in the calibrated spot diagram and the set initial extrinsic parameters includes: For each spot in the calibration spot diagram, determine the second intersection point between the reflection line corresponding to the spot and the infrared lamp ring surface; Connect the second intersection point with the point where the corresponding infrared lamp is located to obtain the straight line corresponding to the light spot; Determine the maximum likelihood intersection point of the multiple straight lines corresponding to the calibration spot pattern, and use it as the first intersection point; Determining the second intersection point between the reflected line corresponding to the light spot and the infrared lamp ring surface includes: Obtain the extension line of the line connecting the light spot and the optical center of the infrared camera; The intersection point of the extension line and the infrared lamp ring surface is determined as the second intersection point; The step of determining the first distance corresponding to the calibrated light spot pattern based on the first intersection point according to the principle of light reflection includes: Set a first initial distance, and determine the center of the calibration ball and the reflection point corresponding to the light spot based on the first initial distance and the first intersection point; An optimization function for the first initial distance is constructed based on the center of the calibrated ball, the reflection point, the infrared lamp, and the second intersection point using the principle of light reflection. The first initial distance is optimized according to the optimization function to obtain the first distance corresponding to the light spot; The step of determining the center of the calibration sphere and the reflection point corresponding to the light spot based on the first initial distance and the first intersection point includes: The center of the calibration sphere corresponding to the light spot is determined based on the first initial distance, the first intersection point, and the optical center of the infrared camera. The second distance is determined using the law of cosines based on the center of the calibrated ball, the radius of the calibrated ball, and the second intersection point; wherein, the second distance is the distance between the second intersection point and the reflection point; The reflection point is determined based on the second distance and the second intersection point.

2. The method according to claim 1, characterized in that, The initial extrinsic parameters are optimized based on the first distance of the multiple calibrated spot images, including: Determine the first statistical value of the first distance for each calibrated spot pattern; The average value of the first statistical value of each calibration spot pattern is used to obtain the optimization parameters; Based on the optimization parameters, the initial extrinsic parameters are iteratively optimized until the optimization termination condition is met, thereby obtaining the target extrinsic parameters of the infrared camera.

3. The method according to claim 1, characterized in that, The target extrinsic parameters are verified based on the test spot pattern, including: Determine the second statistical value of the test spot pattern based on the target extrinsic parameters; If the second statistical value meets the set conditions, the verification is successful.

4. A camera calibration device, characterized in that, include: The light spot image acquisition module is used to control the infrared camera to take pictures of the calibration ball located at different positions and obtain multiple light spot images; wherein, the radius of the calibration ball is determined by the radius of curvature of the human cornea; the light spot images include calibration light spot images and test light spot images; The first intersection point determination module is used to determine the first intersection point for each calibration spot image based on the spot position information in the calibration spot image and the set initial extrinsic parameters; wherein, the first intersection point is the intersection point of the line connecting the camera optical center and the center of the calibration ball and the infrared lamp ring surface, and the extrinsic parameters are the pose information of the infrared camera optical center; The first distance determination module is used to determine the first distance corresponding to the calibration spot pattern based on the first intersection point according to the principle of light reflection; wherein, the first distance is the distance between the center of the calibration ball and the first intersection point; The target extrinsic parameter acquisition module is used to optimize the set initial extrinsic parameters based on the first distance of multiple calibration spot images to obtain the target extrinsic parameters of the infrared camera; The target extrinsic parameter calibration module is used to verify the target extrinsic parameters based on the test spot image. If the verification is successful, the camera is calibrated based on the target extrinsic parameters. The first intersection point determination module is further configured to: For each spot in the calibration spot diagram, determine the second intersection point between the reflected line corresponding to the spot and the infrared lamp ring surface; Connect the second intersection point with the corresponding infrared lamp location to obtain the straight line corresponding to the light spot; Determine the maximum likelihood intersection point of the multiple straight lines corresponding to the calibrated spot pattern, and use it as the first intersection point; The first intersection point determination module is also specifically used for: Obtain the extension line of the line connecting the light spot and the optical center of the infrared camera; Determine the intersection point of the extended line and the infrared lamp ring surface as the second intersection point; The first distance determination module is further configured to: Set a first initial distance, and determine the center of the calibration ball and the reflection point corresponding to the light spot based on the first initial distance and the first intersection point; An optimization function for the first initial distance is constructed based on the principle of light reflection by calibrating the center of the ball, the reflection point, the infrared lamp, and the second intersection point. The initial distance is optimized according to the optimization function to obtain the first distance corresponding to the light spot; The first distance determination module is further specifically used for: The center of the calibration sphere corresponding to the light spot is determined based on the first initial distance, the first intersection point, and the optical center of the infrared camera. The second distance is determined using the law of cosines based on the calibrated center and radius of the small ball and the second intersection point; where the second distance is the distance between the second intersection point and the reflection point. The reflection point is determined based on the second distance and the second intersection point.

5. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the camera calibration method according to any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the camera calibration method according to any one of claims 1-3.

Images