Light source calibration method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses a light source calibration method and device, electronic equipment and a computer readable storage medium. The method comprises the steps of obtaining a calibration image set of a preset calibration plate based on a target camera and a target light source under multiple postures of the preset calibration plate; determining camera parameters of a target camera based on the calibration image set, and calculating incidence point coordinates of the target light source on a preset calibration plate under each attitude; based on the camera parameters, the incidence point coordinates and a surface normal vector of a preset calibration plate, incident ray vectors of the target light source under the at least two different postures are calculated; and determining the position of the target light source based on the at least two different incident ray vectors. Therefore, while the accuracy of target light source calibration is improved, the operation cost of light source calibration is reduced, and the light source calibration efficiency is effectively improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, specifically to a light source calibration method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Photometric Stereo (PS) is an important shape reconstruction technique used in medical research, archaeology, and online industrial measurement. By analyzing the relationship between image grayscale and image shape, it analyzes the surface brightness changes of an object under different light sources. Using only a small number of images, it can acquire three-dimensional surface information, enabling accurate 3D shape reconstruction of object details, features, and even smooth, textureless surfaces. Photometric Stereo relies on light source information; accurate calibration of the light source's position is crucial for precise 3D shape reconstruction. Similarly, the pupil-corneal reflex method in eye tracking also requires precise calibration of the light source's position. For Lambertian light sources that emit light uniformly in all directions, such as most LED light sources, traditional parallel light and point light source models are no longer applicable due to the inherent characteristics of the light source. To achieve accurate 3D reconstruction, precise calibration of the light source and illumination field is essential.

[0003] During the research and practice of existing technologies, it was found that existing light source calibration methods often directly assume that the light source is uniformly and symmetrically distributed around the center of the camera's entrance pupil, or use manual measurement methods to calibrate the position of the light source. Since the actual spatial position of the camera's entrance pupil center cannot be accurately given, the accuracy of light source calibration is poor. At the same time, the calibration cost of the light source calibration method based on stereo vision is high and the operation is cumbersome, which leads to poor light source calibration efficiency. Summary of the Invention

[0004] This application provides a light source calibration method, apparatus, electronic device, and computer-readable storage medium, which can improve the accuracy of target light source calibration while reducing the operational cost of light source calibration, thereby effectively improving the efficiency of light source calibration.

[0005] This application provides a light source calibration method, including:

[0006] Under multiple poses of the preset calibration board, a set of calibration images of the preset calibration board is acquired based on the target camera and the target light source;

[0007] Based on the calibration image set, the camera parameters of the target camera are determined, and the incident point coordinates of the target light source on the preset calibration plate are calculated under each of the aforementioned postures.

[0008] Based on the camera parameters, the coordinates of the incident point, and the surface normal vector of the preset calibration plate, the incident ray vector of the target light source is calculated under at least two different postures.

[0009] The position of the target light source is determined based on at least two different incident ray vectors.

[0010] Accordingly, embodiments of this application provide a light source calibration device, including:

[0011] An image calibration unit is used to acquire a set of calibration images of the preset calibration board based on a target camera and a target light source under multiple poses of the preset calibration board;

[0012] An incident point calculation unit is used to determine the camera parameters of the target camera based on the calibration image set, and to calculate the incident point coordinates of the target light source on the preset calibration plate under each of the aforementioned postures.

[0013] An incident ray determination unit is used to calculate the incident ray vector of the target light source under at least two different postures based on the camera parameters, the coordinates of the incident point, and the surface normal vector of the preset calibration plate.

[0014] A light source position determination unit is used to determine the position of the target light source based on at least two different incident ray vectors.

[0015] In one embodiment, the calibration image set includes a first calibration image set and a second calibration image set, and the image calibration unit is used for:

[0016] In each of the postures of the preset calibration board, the preset calibration board is set to a first working state, and the calibration pattern of the preset calibration board is captured by the target camera to obtain a first calibration image. The first calibration images corresponding to each posture constitute the first calibration image set.

[0017] In each of the aforementioned postures of the preset calibration board, the preset calibration board is set to a second working state, and the preset calibration board is illuminated by a target light source. The light from the target light source reflected by the mirror of the preset calibration board is captured by the target camera to obtain a second calibration image. The second calibration images corresponding to each of the aforementioned postures constitute the second calibration image set.

[0018] In one embodiment, the incident point calculation unit includes:

[0019] The camera parameter determination subunit is used to determine the camera parameters of the target camera based on the first calibration image set;

[0020] The incident point coordinate calculation subunit is used to calculate the incident point coordinates of the target light source on the preset calibration plate under each of the aforementioned attitudes, based on the camera parameters, the first calibration image set, and the second calibration image set.

[0021] In one embodiment, the incident point coordinate calculation subunit includes:

[0022] The mapping relationship acquisition module is used to acquire the planar mapping relationship between the image plane of the first calibration image set and the plane set of the preset calibration plate;

[0023] The image coordinate acquisition module is used to acquire the image coordinates of the target light source in the second calibration image set;

[0024] The initial incident point coordinate calculation module is used to calculate the initial incident point coordinates of the target light source incident on the preset calibration plate in the calibration plate coordinate system based on the plane mapping relationship and the light source image coordinates.

[0025] The incident point coordinate calculation module is used to calculate, based on the camera parameters and the initial incident point coordinates, the incident point coordinates of the target light source incident on the preset calibration plate in each of the aforementioned postures, in the camera coordinate system of the target camera.

[0026] In one embodiment, the mapping relationship acquisition module is used to:

[0027] For each first calibration image in the first calibration image set, at least four feature points in the first calibration image are determined, and the feature point image coordinates of the feature points on the first calibration image are obtained;

[0028] For each of the first calibration images in the first calibration image set, the feature point spatial coordinates on the preset calibration board are calculated based on the camera parameters and the feature point image coordinates.

[0029] For each first calibration image in the first calibration image set, based on the feature point image coordinates and the feature point spatial coordinates, a homography matrix is ​​calculated between the image plane of the first calibration image and the plane of the preset calibration plate. The homography matrix corresponding to each first calibration image constitutes a homography matrix set, which represents the planar mapping relationship between the image plane of the first calibration image set and the plane set of the preset calibration plate.

[0030] In one embodiment, the incident light determining unit is configured to:

[0031] Obtain the optical center coordinates of the target camera in the camera coordinate system, and for at least two different poses of the preset calibration plate, determine the surface normal vector of the preset calibration plate in the camera coordinate system based on the camera parameters respectively;

[0032] For at least two different orientations of the preset calibration plate, based on the principle of light reflection, the incident ray vector of the target light source in the camera coordinate system is determined according to the coordinates of the optical center, the surface normal vector, and the coordinates of the incident point.

[0033] In one embodiment, the light source position determination unit is used for:

[0034] Based on at least two different incident ray vectors, the coordinates of the intersection point between the incident rays corresponding to each incident ray vector are calculated, and the coordinates of the intersection point are the position of the target light source.

[0035] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program adapted for loading by a processor to execute the steps in any of the light source calibration methods provided in embodiments of this application.

[0036] Furthermore, this application also provides an electronic device, including a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to implement the light source calibration method provided in this application.

[0037] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. When the processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps in the light source calibration method provided in this application.

[0038] This application embodiment acquires a calibration image set of the preset calibration board under multiple postures based on the target camera and the target light source; determines the camera parameters of the target camera based on the calibration image set, and calculates the incident point coordinates of the target light source on the preset calibration board under each posture; calculates the incident ray vector of the target light source under at least two different postures based on the camera parameters, the incident point coordinates, and the surface normal vector of the preset calibration board; and determines the position of the target light source based on at least two different incident ray vectors. In this way, the accuracy of target light source calibration is improved while the operational cost of light source calibration is reduced, thereby effectively improving the efficiency of light source calibration. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram illustrating an implementation scenario of a light source calibration method provided in this application.

[0041] Figure 2 This is a schematic flowchart of a light source calibration method provided in an embodiment of this application;

[0042] Figure 3 This is a schematic diagram of a light source calibration scenario provided in an embodiment of this application for a light source calibration method;

[0043] Figure 4a This is a schematic diagram of a light source calibration method provided in an embodiment of this application;

[0044] Figure 4b This is another light source calibration schematic diagram of a light source calibration method provided in this application embodiment;

[0045] Figure 4c This is a schematic diagram illustrating the determination of incident light in a light source calibration method provided in this application embodiment;

[0046] Figure 4d This is another schematic diagram of a light source calibration method provided in the embodiments of this application;

[0047] Figure 5 This is a schematic diagram of the structure of the light source calibration device provided in the embodiments of this application;

[0048] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0049] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] Furthermore, in the description of the embodiments of this application, the use of terms such as "first" and "second" is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the embodiments of this application.

[0051] Photometric stereo vision is an important shape reconstruction technique in medical research, archaeology, and online industrial measurement. By analyzing the relationship between image grayscale and image shape, it analyzes the surface brightness changes of an object under different light sources. It can acquire three-dimensional surface information using only a small number of images, enabling good 3D shape reconstruction of object details, features, and even smooth surfaces without texture. Photometric stereo vision relies on light source information; accurate calibration of the light source's position is necessary for precise 3D shape reconstruction. Furthermore, the pupil-corneal reflex method in eye tracking also requires accurate calibration of the light source's position. However, for target light sources such as Lambertian sources that emit light uniformly in all directions, like most LED light sources, traditional parallel light and point light source models are no longer applicable due to the inherent characteristics of the light source. To achieve accurate 3D reconstruction, precise calibration of the light source and illumination field is essential. Existing Lambertian source position calibration techniques have several shortcomings:

[0052] (1) Traditional photometric stereo vision algorithms directly assume that the light source is uniformly and symmetrically distributed around the entrance pupil center of the camera, or use manual measurement methods to determine the position of the Lambertian light source. However, the actual spatial position of the entrance pupil center of the camera cannot be given accurately, so the position calibration result of the Lambertian light source will have a large error. At the same time, the calibration method based on stereo vision uses a lot of cameras and is complicated to operate, which makes the cost of light source calibration high.

[0053] (2) Although the traditional calibration method using a mirror reflection calibrator can effectively obtain the position of the light source, the camera optical axis cannot be strictly perpendicular to the image plane of the calibrator, and the calibration operation steps are cumbersome and complicated. The difficulty in extracting feature points during reflection imaging is relatively large, resulting in a large error in the calibration result of the light source position. The three-dimensional shape reconstruction result also produces a certain degree of blurring and deformation.

[0054] It is evident that existing light source calibration methods suffer from poor accuracy, high calibration costs, and cumbersome operations, resulting in low calibration efficiency. To address these technical problems, this application provides a light source calibration method. This method acquires a set of calibration images of a preset calibration board in multiple poses based on a target camera and a target light source. The camera parameters of the target camera are then determined from the calibration image set. The coordinates of the incident point of the target light source on the preset calibration board in each pose are calculated. Furthermore, the incident ray vectors of the target light source on the preset calibration board in at least two different poses are calculated based on the incident point coordinates. The position of the target light source can then be calculated using at least two different incident ray vectors. Thus, by using only a camera and a calibration board, the accuracy of target light source calibration is improved while reducing the operational cost of light source calibration, thereby effectively improving the efficiency of light source calibration.

[0055] This application provides a light source calibration method, apparatus, electronic device, and computer-readable storage medium. The light source calibration apparatus can be integrated into an electronic device, which may be a server or a terminal, etc.

[0056] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN) acceleration services, and big data and artificial intelligence platforms. The terminal can include, but is not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0057] Please see Figure 1 Taking the integration of a light source calibration device into electronic equipment as an example, Figure 1 This is a schematic diagram illustrating an implementation scenario of the light source calibration method provided in this application. The electronic device can be a computer device, such as a server or a terminal. The electronic device can acquire a calibration image set of the preset calibration board based on the target camera and the target light source under multiple postures of the preset calibration board; determine the camera parameters of the target camera based on the calibration image set; calculate the incident point coordinates of the target light source on the preset calibration board under each posture; calculate the incident ray vector of the target light source under at least two different postures based on the camera parameters, the incident point coordinates, and the surface normal vector of the preset calibration board; and determine the position of the target light source based on at least two different incident ray vectors.

[0058] It should be noted that, Figure 1 The schematic diagram illustrating the implementation environment of the light source calibration method is merely an example. The implementation environment of the light source calibration method described in this application is intended to more clearly illustrate the technical solutions of this application and does not constitute a limitation on the technical solutions provided in this application. Those skilled in the art will recognize that, with the evolution of data processing and the emergence of new business scenarios, the technical solutions provided in this application are equally applicable to similar technical problems.

[0059] The solutions provided in this application are specifically illustrated through the following embodiments. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0060] This embodiment will be described from the perspective of a light source calibration device, which can be integrated into an electronic device, such as a computer device, which can be a server or a terminal. This application does not impose any limitations on this.

[0061] Please see Figure 2 , Figure 2 This is a schematic flowchart of the light source calibration method provided in an embodiment of this application. The light source calibration method includes:

[0062] In step 101, under multiple poses of the preset calibration plate, a set of calibration images of the preset calibration plate is acquired based on the target camera and the target light source.

[0063] The preset calibration plate can be a pre-set calibration plate used to calibrate the positions of the target camera and the target light source. The preset calibration plate can be set to multiple postures, such as 7, 9, or 14 different postures, with each posture allowing for different placement positions or rotation angles. The target camera can be a camera used to calibrate the position of the target light source. The target light source can be a Lambertian light source, also known as a Lambertian luminescent body, which refers to a radiation source whose radiant brightness remains constant in all directions, and whose radiant intensity varies with the angle between the observation direction and the surface source normal according to a cosine law. A Lambertian body can refer to the phenomenon where incident energy is reflected uniformly in all directions, i.e., incident energy is reflected isotropically in all directions from the incident point within the entire hemispherical space. This is called diffuse reflection, also known as isotropic reflection. A perfect diffuse body is called a Lambertian body. A Lambertian light source emits light uniformly in all directions. The light emitted by the Lambertian light source can be projected onto the surface of an object, reflected by the object, and then incident on the camera, thus forming an image on the camera. The calibration image set can be a set of images based on the calibration images captured by the target camera and the target light source. These calibration images can be used to calibrate the positions of the target camera and the target light source.

[0064] Optionally, the target camera can be a camera in a structured light system, the target light source can be a light source in a structured light system, and the structured light system can be a system structure composed of a light source and a camera. The structured light system can project specific light information onto the surface of an object and its background through the light source, and then the camera can collect this light information, thereby using the changes in light signals caused by the object to calculate the position and depth of the object, and thus reconstruct the entire three-dimensional space.

[0065] In one specific embodiment, please refer to Figure 3 , Figure 3 This is a schematic diagram of a light source calibration scenario provided in an embodiment of this application. Taking a system composed of a virtual reality head-mounted display device (VR glasses or VR head-mounted display) as an example, the target camera can be a camera configured in the virtual reality head-mounted display device, and the target light source can be a Lambertian light source configured in the virtual reality head-mounted display device.

[0066] Optionally, the structured light system may include one or more target light sources and at least one target camera. The specific number of target cameras and target light sources can be set according to the actual situation, and this application embodiment does not limit it.

[0067] In this context, there are multiple ways to acquire a set of calibration images of the preset calibration board based on the target camera and the target light source under various poses of the preset calibration board. For example, the set of calibration images may include a first set of calibration images and a second set of calibration images. Thus, under each pose of the preset calibration board, the preset calibration board is set to a first working state, and the calibration pattern of the preset calibration board is captured by the target camera to obtain a first calibration image. The first calibration images corresponding to each pose constitute the first set of calibration images. Under each pose of the preset calibration board, the preset calibration board is set to a second working state, and the preset calibration board is illuminated by the target light source, and the light reflected from the mirror surface of the preset calibration board is captured by the target camera to obtain a second calibration image. The second calibration images corresponding to each pose constitute the second set of calibration images.

[0068] The first calibration image set can be a set of images captured by the target camera when the preset calibration board is in its first working state. The calibration pattern can be a pattern displayed on the preset calibration board. The calibration pattern can be a pattern with obvious corner features and known two-dimensional coordinates of each corner point. Corner points are important feature points; they have sharp angles and obvious brightness changes, and can be used as input to the calibration algorithm. For example, the calibration pattern can be a checkerboard pattern, circular corner points, or a QR code. Correspondingly, when the calibration pattern is a black and white checkerboard pattern, the corner points in the calibration pattern can refer to the points where black and white squares intersect. The first calibration image can be an image captured by the target camera when the preset calibration board is in its first working state under various poses of the preset calibration board, and can display the calibration pattern of the preset calibration board. The second calibration image set can be a set of images captured by the target camera when the preset calibration board is in its second working state under various poses of the preset calibration board. The first working state can be a working state for displaying calibration patterns, and the second working state can be a working state for specular reflection. In this way, a first calibration image and a second calibration image can be obtained for each posture of the preset calibration plate, thus obtaining a first calibration image set and a second calibration image set.

[0069] Optionally, the preset calibration plate may include a first working state and a second working state. In the first working state, the preset calibration plate may display a calibration pattern, which can provide the precise position of feature points such as surface corners. In the second working state, the preset calibration plate does not display a calibration pattern and can be used as a reflective surface of the target light source to reflect the position of the target light source for imaging.

[0070] In one embodiment, the preset calibration board is a display screen with a backlight. When the backlight is on, the preset calibration board is in a first working state. At this time, the preset calibration board can display a calibration pattern, so that the target camera can capture the calibration pattern on the preset calibration board and thus obtain a first calibration image. When the backlight is off, the preset calibration board is in a second working state. At this time, the preset calibration board does not display a checkerboard pattern. At the same time, a specular reflective film is attached to the surface of the display screen of the preset calibration board. The specular reflective film reflects the incident light from the target light source, so that the target camera can capture the reflected light from the target light source on the surface of the display screen and thus obtain a second calibration image.

[0071] In another optional embodiment, the preset calibration plate can be a composite calibration plate. It can be manufactured using laser printing or spraying techniques, where a calibration pattern is printed or affixed to the front of a transparent calibration plate (made of glass or plastic, etc.), and a surface light source is provided on the back of the transparent calibration plate to illuminate it, thus obtaining the preset calibration plate in this embodiment. When the surface light source on the back of the preset calibration plate is turned on, the calibration pattern on the front of the preset calibration plate is illuminated, allowing the target camera to capture the calibration pattern features, corresponding to the first working state of the preset calibration plate. When the surface light source on the back of the preset calibration plate is turned off, the calibration pattern on the front of the preset calibration plate is not illuminated. In this case, the preset calibration plate merely acts as a reflector for the target light source, specularly reflecting the light emitted by the target light source, corresponding to the second working state of the preset calibration plate.

[0072] In one embodiment, please refer to Figure 4a , Figure 4a This is a schematic diagram of a light source calibration method provided in an embodiment of this application. The method allows the preset calibration board (i.e., the pre-calibrated calibration board) to be set to a first working state in multiple orientations. In this first working state, the calibration pattern of the preset calibration board is captured by a target camera to obtain a first calibration image set. Please refer to... Figure 4b , Figure 4b This is another light source calibration diagram of a light source calibration method provided in this application embodiment. The preset calibration plate is set to a second working state. In the second working state, the preset calibration plate is illuminated by a Lambertian light source (i.e., a target light source), and the light of the target light source reflected by the specular surface of the preset calibration plate is captured by a target camera to obtain a second calibration image set.

[0073] In step 102, the camera parameters of the target camera are determined based on the calibration image set, and the incident point coordinates of the target light source on the preset calibration plate are calculated under each attitude.

[0074] The camera parameters can be parameters of the target camera, including internal parameters, external parameters, and distortion parameters. Internal parameters are those related to the target camera's own characteristics, such as focal length and pixel size, and can be used to transform coordinates from the camera coordinate system to the image (or pixel) coordinate system. External parameters are those of the target camera in the world coordinate system, such as its position and rotation direction, and can be used to transform coordinates from the world coordinate system to the camera coordinate system. Distortion parameters can include radial and tangential distortion parameters. Camera distortion is an image distortion phenomenon caused by imperfections in the optical system; distortion parameters can be used to describe and correct this distortion to improve the camera's image quality. The target camera calibration process involves determining the values ​​of these parameters to ensure that the information obtained from the image accurately reflects the size and position of objects in the real world. The incident point coordinates can be the coordinates of the point on the preset calibration plate where the target light source is incident.

[0075] There are several ways to determine the camera parameters of the target camera based on the calibration image set and to calculate the incident point coordinates of the target light source on the preset calibration plate under each attitude. For example, the camera parameters of the target camera can be determined based on the first calibration image set, and the incident point coordinates of the target light source on the preset calibration plate under each attitude can be calculated based on the camera parameters, the first calibration image set, and the second calibration image set.

[0076] There are several ways to determine the camera parameters of the target camera based on the first calibration image set. For example, a single-camera calibration method can be used to determine the camera parameters of the target camera based on the first calibration image set. This single-camera calibration method can include calibration methods such as Zhang Zhengyou calibration, Direct Linear Transform (DLT), and active vision methods. For example, Zhang Zhengyou calibration can be used to determine the camera parameters of the target camera. These camera parameters can include the target camera's intrinsic parameters A, distortion parameters D, and extrinsic parameters RT. The intrinsic parameters A can be a 3×3 matrix, the distortion parameters D can be a 1×5 matrix, and the extrinsic parameters RT can include a rotation matrix R and a translation vector T. The rotation matrix R can be a 3×3 matrix, and the translation vector T can be a 3×1 matrix.

[0077] After determining the camera parameters of the target camera based on the first calibration image set, the coordinates of the incident point of the target light source on the preset calibration plate under each attitude can be calculated according to the camera parameters, the first calibration image set, and the second calibration image set. There are several ways to calculate the coordinates of the incident point of the target light source on the preset calibration plate under each attitude based on the camera parameters, the first calibration image set, and the second calibration image set. For example, the planar mapping relationship between the image plane of the first calibration image set and the plane set of the preset calibration plate can be obtained, and the image coordinates of the light source where the target light source is located in the second calibration image set can be obtained. Based on the planar mapping relationship and the light source image coordinates, the initial incident point coordinates of the target light source incident on the preset calibration plate in the calibration plate coordinate system can be calculated. Finally, based on the camera parameters and the initial incident point coordinates, the incident point coordinates of the target light source incident on the preset calibration plate under each attitude in the camera coordinate system of the target camera can be calculated.

[0078] The image plane can be the imaging plane corresponding to the calibration image, i.e., the image plane on which the target camera captures the calibration image. The two-dimensional coordinate system corresponding to this image plane can be the pixel coordinate system. The set of planes of the preset calibration plate can be the set composed of the planes of the preset calibration plate under each pose. The plane mapping relationship can be the mapping relationship between the image plane of the calibration image and the plane of the preset calibration plate. According to the plane mapping relationship, the coordinates corresponding to a coordinate in the image plane in the plane of the preset calibration plate can be determined, and the coordinates corresponding to a coordinate in the plane of the preset calibration plate in the image plane can also be determined. The coordinate system corresponding to a coordinate in the image plane can be different from the coordinate system corresponding to a coordinate in the plane of the preset calibration plate. For example, the plane mapping relationship can be understood as the mapping relationship between the two-dimensional image coordinate system established based on the image plane and the three-dimensional spatial coordinate system constructed based on the plane of the preset calibration plate. The light source image coordinates can be the coordinates of the position of the reflected target light source in each second calibration image. The camera coordinate system can be a coordinate system built based on the target camera, or a three-dimensional Cartesian coordinate system established with the focus center of the target camera as the origin and the optical axis as the Z-axis. The calibration plate coordinate system can be a three-dimensional spatial coordinate system built based on a preset calibration plate, or a user-defined three-dimensional world coordinate system. Points on the preset calibration plate are represented using three-dimensional spatial coordinates, and their corresponding origins can be user-defined. This is introduced to describe the position of the preset calibration plate in the real world. In other words, this planar mapping relationship can be a mapping relationship between the pixel coordinate system based on the image plane and the calibration plate coordinate system based on the preset calibration plate. The initial incident point coordinates can be the coordinates of the incident point of the target light source on the preset calibration plate in the calibration plate coordinate system, or the coordinates of the incident point of the target light source on the preset calibration plate in the camera coordinate system.

[0079] There are several ways to obtain the planar mapping relationship between the image plane of the first calibration image set and the plane set of the preset calibration plate. For example, for each first calibration image in the first calibration image set, at least four feature points in the first calibration image can be determined, and the feature point image coordinates on the first calibration image can be obtained. For each first calibration image in the first calibration image set, the feature point spatial coordinates on the preset calibration plate are calculated based on the camera parameters and the feature point image coordinates. For each first calibration image in the first calibration image set, the homography matrix between the image plane of the first calibration image and the plane of the preset calibration plate is calculated based on the feature point image coordinates and the feature point spatial coordinates. The homography matrices corresponding to each first calibration image constitute a homography matrix set, which represents the planar mapping relationship between the image plane of the first calibration image set and the plane set of the preset calibration plate.

[0080] The feature point can be a corner point determined in the calibration pattern displayed in the first calibration image. The feature point image coordinates can be the coordinates of the feature point in the first calibration image, or the coordinates in a two-dimensional coordinate system of the image plane. The feature point spatial coordinates can be the spatial coordinates of the feature point in the calibration plate coordinate system of the preset calibration plate. The homography matrix can be information indicating the mapping relationship between the image plane of the calibration image and the plane of the preset calibration plate. The homography of a plane can be defined as a projection mapping from one plane to another, and can be used to describe the positional mapping relationship of an object between the world coordinate system and the pixel coordinate system.

[0081] There are several ways to determine at least four feature points in the first calibration image and obtain the feature point image coordinates of the feature points on the first calibration image. For example, at least four feature points can be randomly selected from the corner points of the calibration pattern displayed in the first calibration image. Then, the two-dimensional coordinates (u, v) of the feature points on the image plane can be obtained according to the corresponding image detection algorithm, thereby obtaining the feature point image coordinates of the feature points on the first calibration image.

[0082] There are several ways to calculate the spatial coordinates of a feature point on a preset calibration board based on camera parameters and feature point image coordinates. For example, the camera parameters of the target camera and the feature point image coordinates can be used to determine the point on the preset calibration board that corresponds to the feature point in the second calibration image. This allows us to determine the spatial coordinates of the point on the preset calibration board that corresponds to the feature point in the second calibration image in the calibration board coordinate system, thus obtaining the spatial coordinates (X, Y, Z) of the feature point on the preset calibration board.

[0083] There are several ways to calculate the homography matrix between the image plane of the first calibration image and the plane of the preset calibration plate based on the image coordinates and spatial coordinates of the feature points. For example, under each pose of the preset calibration plate, based on the homography mapping relationship between at least four feature points in the image plane and the preset calibration plate plane, a system of equations can be established according to the image coordinates and spatial coordinates of the feature points. This system of equations can be an overdetermined system of equations, which can be solved using the least linear squares method to obtain the homography matrix H between the image plane of the target camera and the plane of the preset calibration plate. In one embodiment, the plane mapping relationship between the image plane of the first calibration image and the plane of the preset calibration plate can be expressed as:

[0084]

[0085] Where H can be represented as the homography matrix between the image plane of the target camera and the plane of the preset calibration plate, u can be represented as the u component in the feature point image coordinates of a feature point, v can be represented as the v component in the feature point image coordinates, and X, Y, and Z can be represented as the X component, Y component, and Z component in the feature point spatial coordinates of a feature point.

[0086] There are several ways to calculate the incident point coordinates of the target light source on the preset calibration plate at each attitude in the camera coordinate system of the target camera, based on the camera parameters and the initial incident point coordinates. For example, a coordinate system transformation can be performed based on the camera parameters to convert the initial incident point coordinates Ps = (Xs, Ys, Zs) in the calibration plate coordinate system to coordinates in the camera coordinate system. This allows us to obtain the incident point coordinates Psc of the target light source on the preset calibration plate at each attitude in the camera coordinate system of the target camera. Specifically, the incident point coordinates Psc can be expressed as Psc = (Xs, Ys, Zs) in the camera coordinate system of the target camera. sc ,Y sc Z sc ) T = [RT]*Ps, where R and T represent the external parameters of the target camera, and X... sc Y sc Z sc These represent the X, Y, and Z components of the incident point coordinates Psc on the preset calibration plate in the camera coordinate system.

[0087] The initial incident point coordinates Ps can be determined based on the homography matrix H and the light source image coordinates (u). s ,v s ,1) Determine, that is, based on [u s ,v s ,1] T=H*Ps, H=A*[RT] / s, thus obtaining the initial incident point coordinates Ps=H -1 *[u s ,v s ,1] T ,in,[] T The matrix symbol [] represents the transpose of a matrix. -1 The matrix symbol represents the inverse of the matrix, A represents the intrinsic parameters of the target camera, and s represents the scale factor.

[0088] In step 103, based on camera parameters, incident point coordinates, and the surface normal vector of the preset calibration plate, the incident ray vector of the target light source is calculated under at least two different postures.

[0089] The surface normal vector can be the normal vector of the plane on which the preset calibration plate is located under various attitudes. The incident ray vector can be a vector indicating the direction of the incident ray from the target light source into the preset calibration plate.

[0090] There are several ways to calculate the incident ray vector of the target light source in at least two different postures, based on camera parameters, incident point coordinates, and the surface normal vector of the preset calibration plate. For example, the optical center coordinates of the target camera in the camera coordinate system can be obtained, and the surface normal vector of the preset calibration plate in the camera coordinate system can be determined based on the camera parameters for at least two different postures of the preset calibration plate. Alternatively, based on the principle of light reflection, the incident ray vector of the target light source in the camera coordinate system can be determined based on the optical center coordinates, the surface normal vector, and the incident point coordinates for at least two different postures of the preset calibration plate.

[0091] The optical center coordinates refer to the optical center of the lens, which is the center point when light passes through the lens. The position of the optical center of the camera is very important for the accuracy of camera imaging, as it determines the transformation relationship between pixels in the image and physical coordinates in the real world. The optical center coordinates of the target camera can be included in the internal parameters of the target camera. The principle of light reflection refers to the phenomenon that when light strikes the interface between two media, some of the light changes its direction of propagation and returns to the original medium to continue propagating. The reflection of light follows three basic laws, namely the laws of light reflection: (1) The reflected ray, the incident ray, and the normal are in the same plane; (2) The reflected ray and the incident ray are on opposite sides of the normal; (3) The angle of reflection is equal to the angle of incidence, that is, the angle between the reflected ray and the normal is equal to the angle between the incident ray and the normal.

[0092] There are several ways to determine the surface normal vector of the preset calibration plate in the camera coordinate system based on camera parameters for at least two different poses of the preset calibration plate. For example, since three points can define a plane, for at least two different poses of the preset calibration plate, the coordinates of at least three feature points in the preset calibration plate in the camera coordinate system can be determined according to the camera parameters for each pose. Then, based on the coordinates of the at least three feature points in the camera coordinate system, the plane of the preset calibration plate in the camera coordinate system can be determined, and the normal vector of the plane of the preset calibration plate in the camera coordinate system can be calculated, thus obtaining the surface normal vector of the preset calibration plate in each pose in the camera coordinate system. Alternatively, methods such as solving the laser plane equation or using a camera laser calibration algorithm based on plane constraints can be used to calculate the surface normal vector of the preset calibration plate in the camera coordinate system for each pose.

[0093] After determining the surface normal vector of the preset calibration plate in the camera coordinate system based on camera parameters for at least two different poses of the preset calibration plate, the incident ray vector of the target light source in the camera coordinate system can be determined for at least two different poses of the preset calibration plate, based on the principle of light reflection, using the coordinates of the optical center, the surface normal vector, and the coordinates of the incident point. There are multiple ways to determine the incident ray vector of the target light source in the camera coordinate system for at least two different poses of the preset calibration plate, based on the principle of light reflection, using the coordinates of the optical center, the surface normal vector, and the coordinates of the incident point. For example, please refer to... Figure 4c , Figure 4c This is a schematic diagram of the incident ray determination method of a light source calibration method provided in this application embodiment. For at least two different orientations of the preset calibration plate, based on the principle of light reflection, the reflected ray of the target light source to the optical center of the camera can be determined according to the optical center coordinates of the camera and the incident point coordinates. Thus, based on the principle of light reflection, it can be determined that the angle between the reflected ray and the surface normal vector is equal to the angle between the incident ray and the surface normal vector, thereby determining the incident ray vector of the target light source on the preset calibration plate.

[0094] For example, if the target light source is a Lambertian light source, please refer to [reference needed]. Figure 4d , Figure 4dThis is another schematic diagram of a light source calibration method provided in this application embodiment. In the first posture of the preset calibration plate, the incident light OA of the Lambertian light source is reflected by the mirror of the preset calibration plate and imaged on the target camera. Based on the principle of light reflection, the incident light ray Aa or OA of the Lambertian light source incident on the preset calibration plate in the current posture can be calculated. In the second posture of the preset calibration plate, the incident light OB of the Lambertian light source is reflected by the mirror of the preset calibration plate and imaged on the target camera. Based on the principle of light reflection, the incident light ray Bb or OB of the Lambertian light source incident on the preset calibration plate in the current posture can be solved. In the third posture of the preset calibration plate, the incident light ray OC of the Lambertian light source is imaged on the target camera after being reflected by the mirror of the preset calibration plate. Based on the principle of light reflection, the incident light ray Cc or OC of the Lambertian light source incident on the preset calibration plate in the current posture can be solved. In this way, the incident light ray vector of the target light source incident on the preset calibration plate in each posture can be determined.

[0095] In step 104, the position of the target light source is determined based on at least two different incident ray vectors.

[0096] The position of the target light source can be the position of the target light source relative to the target camera, or it can be based on the position of the target light source in the structured light system of the target camera and the target light source.

[0097] There are several ways to determine the position of the target light source based on at least two different incident ray vectors. For example, based on at least two different incident ray vectors, the coordinates of the intersection point between the incident rays corresponding to each incident ray vector can be calculated, and the coordinates of the intersection point are the position of the target light source.

[0098] The intersection point coordinates can be the coordinates of the intersection point between at least two incident rays corresponding to different postures, and the intersection point coordinates can be coordinates in the camera coordinate system.

[0099] Specifically, a set of constraint equations can be established based on the incident rays corresponding to at least two different orientations, and the constraint equations can be solved. For example, a set of constraint equations can be established based on the sum of the distances from the target light source to each incident ray, and then a nonlinear solution method can be used to solve the constraint equations to obtain the coordinates of the intersection point of the incident rays, i.e., the position of the target light source.

[0100] Optionally, the nonlinear solution method can be the LM (Levenberg-Marquardt) algorithm.

[0101] The number of equations in the constraint equation set or the number of incident light attitudes can be flexibly selected according to actual needs, and this application embodiment does not limit it.

[0102] As can be seen from the above, the embodiments of this application acquire a first calibration image set containing calibration patterns in the first working state of the preset calibration plate, and acquire a second calibration image set of specular reflection of the target light source in the second working state of the preset calibration plate. Based on the first calibration image set, the camera parameters of the target camera are calculated. According to the camera parameters, the first calibration image set, and the second calibration image set, the coordinates of the incident point of the target light source on the preset calibration plate in each posture are determined. Based on the principle of light reflection, the optical center coordinates of the target camera, the incident point coordinates, and the surface normal vector of the preset calibration plate, the incident ray vectors of the target light source on the preset calibration plate in at least two different postures are calculated. Based on the intersection point between at least two different incident ray vectors, the position of the target light source can be calculated. Thus, using only a camera and a calibration plate, the position of the target light source relative to the target camera can be easily and efficiently calibrated, improving the accuracy of light source calibration, reducing the cost of light source calibration, improving the ease of operation of light source calibration, and further improving the efficiency of light source calibration. This method can be widely applied in various fields such as visual sensing and measurement.

[0103] To better implement the above methods, embodiments of the present invention also provide a light source calibration device, which can be integrated into an electronic device, such as a server or a terminal.

[0104] For example, such as Figure 5 The diagram shown is a schematic representation of the light source calibration device provided in this embodiment of the application. The light source calibration device may include an image calibration unit 201, an incident point calculation unit 202, an incident ray determination unit 203, and a light source position determination unit 204, as follows:

[0105] The image calibration unit 201 is used to acquire a set of calibration images of the preset calibration board based on the target camera and the target light source under multiple poses of the preset calibration board;

[0106] The incident point calculation unit 202 is used to determine the camera parameters of the target camera based on the calibration image set, and to calculate the incident point coordinates of the target light source on the preset calibration plate under each attitude.

[0107] The incident ray determination unit 203 is used to calculate the incident ray vector of the target light source under at least two different postures based on camera parameters, incident point coordinates and the surface normal vector of the preset calibration plate.

[0108] The light source position determination unit 204 is used to determine the position of the target light source based on at least two different incident light vectors.

[0109] In one embodiment, the calibration image set includes a first calibration image set and a second calibration image set, and the image calibration unit 201 is used for:

[0110] In each pose of the preset calibration board, the preset calibration board is set to the first working state, and the calibration pattern of the preset calibration board is captured by the target camera to obtain the first calibration image. The first calibration images corresponding to each pose constitute the first calibration image set.

[0111] In each pose of the preset calibration board, the preset calibration board is set to the second working state, and the preset calibration board is illuminated by the target light source, and the light reflected by the target light source from the mirror surface of the preset calibration board is captured by the target camera to obtain the second calibration image. The second calibration images corresponding to each pose constitute the second calibration image set.

[0112] In one embodiment, the incident point calculation unit 202 includes:

[0113] The camera parameter determination subunit is used to determine the camera parameters of the target camera based on the first calibration image set;

[0114] The incident point coordinate calculation subunit is used to calculate the incident point coordinates of the target light source on the preset calibration plate under each attitude based on the camera parameters, the first calibration image set, and the second calibration image set.

[0115] In one embodiment, the incident point coordinate calculation subunit includes:

[0116] The mapping relationship acquisition module is used to acquire the planar mapping relationship between the image plane of the first calibration image set and the plane set of the preset calibration plate;

[0117] The image coordinate acquisition module is used to acquire the image coordinates of the target light source in the second calibration image set;

[0118] The initial incident point coordinate calculation module is used to calculate the initial incident point coordinates of the target light source incident on the preset calibration plate in the calibration plate coordinate system based on the plane mapping relationship and the light source image coordinates.

[0119] The incident point coordinate calculation module is used to calculate the incident point coordinates of the target light source on the preset calibration plate in each attitude, in the camera coordinate system of the target camera, based on the camera parameters and the initial incident point coordinates.

[0120] In one embodiment, the mapping relationship acquisition module is used for:

[0121] For each first calibration image in the first calibration image set, at least four feature points in the first calibration image are determined, and the feature point image coordinates of the feature points on the first calibration image are obtained;

[0122] For each first calibration image in the first calibration image set, the spatial coordinates of the feature points on the preset calibration board are calculated based on the camera parameters and the feature point image coordinates.

[0123] For each first calibration image in the first calibration image set, based on the image coordinates and spatial coordinates of the feature points, the homography matrix between the image plane of the first calibration image and the plane of the preset calibration plate is calculated. The homography matrix corresponding to each first calibration image constitutes a homography matrix set, which represents the planar mapping relationship between the image plane of the first calibration image set and the plane set of the preset calibration plate.

[0124] In one embodiment, the incident ray determining unit 203 is configured to:

[0125] Obtain the optical center coordinates of the target camera in the camera coordinate system, and determine the surface normal vector of the preset calibration plate in the camera coordinate system based on the camera parameters for at least two different poses of the preset calibration plate.

[0126] For at least two different orientations of the preset calibration plate, based on the principle of light reflection, the incident ray vector of the target light source in the camera coordinate system is determined according to the coordinates of the optical center, the surface normal vector, and the coordinates of the incident point.

[0127] In one embodiment, the light source position determination unit 204 is used for:

[0128] Based on at least two different incident ray vectors, calculate the coordinates of the intersection point between the incident rays corresponding to each incident ray vector. The coordinates of the intersection point are the position of the target light source.

[0129] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0130] As can be seen from the above, in this embodiment of the application, the image calibration unit 201 acquires a calibration image set of the preset calibration board based on the target camera and the target light source under multiple postures of the preset calibration board; the incident point calculation unit 202 determines the camera parameters of the target camera based on the calibration image set, and calculates the incident point coordinates of the target light source on the preset calibration board under each posture; the incident ray determination unit 203 calculates the incident ray vector of the target light source under at least two different postures based on the camera parameters, the incident point coordinates and the surface normal vector of the preset calibration board; and the light source position determination unit 204 determines the position of the target light source based on at least two different incident ray vectors. In this way, by acquiring a set of calibration images of a preset calibration board in multiple postures based on the target camera and the target light source, the camera parameters of the target camera are determined based on the calibration image set, and the incident point coordinates of the target light source on the preset calibration board in each posture are calculated. Based on the incident point coordinates, the incident ray vectors of the target light source on the preset calibration board in at least two different postures are calculated. Thus, the position of the target light source can be calculated based on at least two different incident ray vectors. In this way, the accuracy of target light source calibration can be improved while reducing the operation cost of light source calibration, thereby effectively improving the efficiency of light source calibration.

[0131] This application also provides an electronic device, such as... Figure 6 The diagram shows a schematic representation of the structure of an electronic device according to an embodiment of this application. This electronic device can be a server or a terminal. Specifically:

[0132] The electronic device may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more computer-readable storage media, a power supply 303, and an input unit 304. Those skilled in the art will understand that... Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0133] The processor 301 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in the memory 302, and calls data stored in the memory 302, to perform various functions and process data. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 301.

[0134] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and data processing by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0135] The electronic device also includes a power supply 303 that supplies power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 303 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0136] The electronic device may also include an input unit 304, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0137] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the electronic device loads the executable files corresponding to the processes of one or more application programs into the memory 302 according to the following computer program, and the processor 301 runs the application programs stored in the memory 302 to realize various functions, as follows:

[0138] Under multiple poses of the preset calibration plate, a calibration image set of the preset calibration plate is acquired based on the target camera and the target light source; the camera parameters of the target camera are determined based on the calibration image set, and the incident point coordinates of the target light source on the preset calibration plate under each pose are calculated; based on the camera parameters, the incident point coordinates and the surface normal vector of the preset calibration plate, the incident ray vector of the target light source is calculated under at least two different poses; the position of the target light source is determined based on at least two different incident ray vectors.

[0139] The specific implementation of each of the above operations can be found in the preceding embodiments, and will not be repeated here. It should be noted that the electronic device provided in this application embodiment and the light source calibration method in the above embodiments belong to the same concept, and its specific implementation process can be found in the above method embodiments, and will not be repeated here.

[0140] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0141] Therefore, embodiments of this application provide a computer-readable storage medium storing a computer program that can be loaded by a processor to execute the steps in any of the light source calibration methods provided in the embodiments of this application. For example, the computer program can execute the following steps:

[0142] Under multiple poses of the preset calibration plate, a calibration image set of the preset calibration plate is acquired based on the target camera and the target light source; the camera parameters of the target camera are determined based on the calibration image set, and the incident point coordinates of the target light source on the preset calibration plate under each pose are calculated; based on the camera parameters, the incident point coordinates and the surface normal vector of the preset calibration plate, the incident ray vector of the target light source is calculated under at least two different poses; the position of the target light source is determined based on at least two different incident ray vectors.

[0143] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0144] Since the computer program stored in the computer-readable storage medium can execute the steps in any of the light source calibration methods provided in the embodiments of this application, the beneficial effects that any of the light source calibration methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0145] According to one aspect of this application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps in the light source calibration method provided in the embodiments of this application.

[0146] The above provides a detailed description of a light source calibration method, apparatus, electronic device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for calibrating a light source, characterized in that, include: Under multiple poses of the preset calibration board, a set of calibration images of the preset calibration board is acquired based on the target camera and the target light source; Based on the calibration image set, the camera parameters of the target camera are determined, and the incident point coordinates of the target light source on the preset calibration plate are calculated under each of the aforementioned postures. Based on the camera parameters, the coordinates of the incident point, and the surface normal vector of the preset calibration plate, the incident ray vector of the target light source is calculated under at least two different postures. The position of the target light source is determined based on at least two different incident ray vectors.

2. The light source calibration method as described in claim 1, characterized in that, The calibration image set includes a first calibration image set and a second calibration image set. The step of acquiring the calibration image set of the preset calibration board under multiple poses of the preset calibration board, based on the target camera and the target light source, includes: In each of the postures of the preset calibration board, the preset calibration board is set to a first working state, and the calibration pattern of the preset calibration board is captured by the target camera to obtain a first calibration image. The first calibration images corresponding to each posture constitute the first calibration image set. In each of the aforementioned postures of the preset calibration board, the preset calibration board is set to a second working state, and the preset calibration board is illuminated by the target light source. The light from the target light source reflected by the target light source on the preset calibration board is captured by the target camera to obtain a second calibration image. The second calibration images corresponding to each of the aforementioned postures constitute the second calibration image set.

3. The light source calibration method as described in claim 2, characterized in that, The step of determining the camera parameters of the target camera based on the calibration image set, and calculating the incident point coordinates of the target light source on the preset calibration plate under each of the aforementioned postures, includes: Based on the first calibrated image set, the camera parameters of the target camera are determined; Based on the camera parameters, the first calibration image set, and the second calibration image set, calculate the incident point coordinates of the target light source on the preset calibration plate under each of the aforementioned postures.

4. The light source calibration method as described in claim 3, characterized in that, The step of calculating the incident point coordinates of the target light source on the preset calibration plate under each of the aforementioned postures, based on the camera parameters, the first calibration image set, and the second calibration image set, includes: Obtain the planar mapping relationship between the image plane of the first calibration image set and the plane set of the preset calibration plate; Obtain the coordinates of the target light source in the second calibrated image set; Based on the planar mapping relationship and the light source image coordinates, calculate the initial incident point coordinates of the target light source incident on the preset calibration plate in the calibration plate coordinate system; Based on the camera parameters and the initial incident point coordinates, calculate the incident point coordinates of the target light source on the preset calibration plate in each of the stated postures, in the camera coordinate system of the target camera.

5. The light source calibration method as described in claim 4, characterized in that, The step of obtaining the planar mapping relationship between the image planes of the first calibration image set and the planar set of the preset calibration board includes: For each first calibration image in the first calibration image set, at least four feature points in the first calibration image are determined, and the feature point image coordinates of the feature points on the first calibration image are obtained; For each of the first calibration images in the first calibration image set, the feature point spatial coordinates on the preset calibration board are calculated based on the camera parameters and the feature point image coordinates. For each first calibration image in the first calibration image set, based on the feature point image coordinates and the feature point spatial coordinates, a homography matrix is ​​calculated between the image plane of the first calibration image and the plane of the preset calibration plate. The homography matrix corresponding to each first calibration image constitutes a homography matrix set, which represents the planar mapping relationship between the image plane of the first calibration image set and the plane set of the preset calibration plate.

6. The light source calibration method according to any one of claims 1-5, characterized in that, The calculation of the incident ray vector of the target light source under at least two different postures, based on the camera parameters, the incident point coordinates, and the surface normal vector of the preset calibration plate, includes: Obtain the optical center coordinates of the target camera in the camera coordinate system, and for at least two different poses of the preset calibration plate, determine the surface normal vector of the preset calibration plate in the camera coordinate system based on the camera parameters respectively; For at least two different orientations of the preset calibration plate, based on the principle of light reflection, the incident ray vector of the target light source in the camera coordinate system is determined according to the coordinates of the optical center, the surface normal vector, and the coordinates of the incident point.

7. The light source calibration method according to any one of claims 1-5, characterized in that, Determining the position of the target light source based on at least two different incident ray vectors includes: Based on at least two different incident ray vectors, the coordinates of the intersection point between the incident rays corresponding to each incident ray vector are calculated, and the coordinates of the intersection point are the position of the target light source.

8. A light source calibration device, characterized in that, include: An image calibration unit is used to acquire a set of calibration images of the preset calibration board based on a target camera and a target light source under multiple poses of the preset calibration board; An incident point calculation unit is used to determine the camera parameters of the target camera based on the calibration image set, and to calculate the incident point coordinates of the target light source on the preset calibration plate under each of the aforementioned postures. An incident ray determination unit is used to calculate the incident ray vector of the target light source under at least two different postures based on the camera parameters, the coordinates of the incident point, and the surface normal vector of the preset calibration plate. A light source position determination unit is used to determine the position of the target light source based on at least two different incident ray vectors.

9. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of any one of the methods described in claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps of any of the methods described in claims 1 to 7.

Images