Light source calibration method and device, electronic equipment and storage medium

Abstract

The invention relates to a light source calibration method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a plurality of first posture images of a combined target through a first camera under a uniform illumination condition, and determining a target sphere center coordinate of a sphere target in a target coordinate system; acquiring a plurality of second attitude images of the combined target by using the target camera under a target illumination condition, and determining information of reflected light of each second attitude image under a target camera coordinate system; based on the target sphere center coordinates and the information of the reflected light rays, determining information of incident light rays corresponding to the reflected light rays in the target camera coordinate system, and determining light source coordinates of the to-be-calibrated light source in the target camera coordinate system according to the information of all the incident light rays; and determining radiation intensity distribution of the to-be-calibrated light source based on the plurality of second attitude images and the light source coordinates. According to the method and device, the electronic equipment and the storage medium disclosed by the invention, high-precision calibration of the light source spatial position and the radiation intensity distribution can be realized.

Description

Technical Field

[0001] The present disclosure relates to the technical field of light source calibration, and in particular to a light source calibration method and device, electronic equipment and storage medium. Background Art

[0002] Under near-field conditions, the light source needs to be accurately modeled and calibrated to achieve high-quality 3D reconstruction. Light source calibration generally includes the calibration of the light source's geometric spatial position and the calibration of the radiance intensity distribution (RID).

[0003] The current light source calibration method often uses a small reflective ball to calibrate the spatial position of the light source. However, when the sphere is illuminated by a point light source, it is difficult to separate the edge of the sphere from the black background, resulting in a deviation in the positioning of the sphere relative to the camera, which in turn leads to errors in the calibration of the spatial position of the light source. The current calibration model for radiation intensity distribution is limited to isotropic RID or radially symmetric RID, which is not flexible enough, and the current RID model requires prior information about the light source, which increases the complexity of the calibration. In addition, the current calibration process of the geometric spatial position of the light source and the calibration process of the radiation intensity distribution are independent of each other and need to be calibrated separately, resulting in a cumbersome calibration operation process. Summary of the invention

[0004] In view of this, the present disclosure proposes a light source calibration method and device, an electronic device and a storage medium, which can achieve high-precision calibration of the spatial position and radiation intensity distribution of the light source based on a set of images without requiring any prior information about the light source.

[0005] In one aspect according to the present disclosure, a method for calibrating a light source is provided, comprising: under preset uniform illumination conditions, using a first camera to acquire multiple first posture images of a combined target, wherein the combined target has a different first posture in each of the first posture images, and the combined target includes a plane target and a spherical target with fixed relative coordinates, and determining the target sphere center coordinates of the spherical target in a preset target coordinate system based on the multiple first posture images; under target illumination conditions formed by a light source to be calibrated, using a target camera to acquire multiple second posture images of the combined target, and determining information of reflected light of each of the second posture images in the target camera coordinate system, wherein the combined target has a different second posture in each of the second posture images; based on the target sphere center coordinates and the information of each reflected light, determining information of incident light corresponding to each reflected light in the target camera coordinate system, and determining the light source coordinates of the light source to be calibrated in the target camera coordinate system according to the information of all incident light; and determining the radiation intensity distribution of the light source to be calibrated based on the multiple second posture images and the light source coordinates.

[0006] In a possible implementation, determining the target sphere center coordinates of the spherical target in a preset target coordinate system based on the multiple first posture images includes: performing edge detection on the spherical target in each of the first posture images to obtain first edge point coordinates of each edge point of the spherical target in each of the first posture images in the first image coordinate system; performing coordinate transformation on each of the first edge point coordinates using the camera intrinsic parameters of the first camera to obtain corresponding second edge point coordinates in the first camera coordinate system, wherein the camera intrinsic parameters and camera extrinsic parameters of the first camera are obtained based on the first camera calibration performed using the plane targets in the multiple first posture images; performing coordinate transformation on each of the second edge point coordinates and obtaining corresponding second edge point coordinates in the first camera coordinate system based on the second edge point coordinates and the first camera coordinate system. The first spherical center coordinates corresponding to the spherical target in a posture image, the spherical radius of the spherical target, and the preset first optical center coordinates are used to determine the first tangent point coordinates of the tangent points corresponding to each of the edge points in the first camera coordinate system, wherein the first optical center coordinates represent the coordinates of the optical center of the first camera in the first camera coordinate system; the first optical center coordinates, the second edge point coordinates, and the first tangent point coordinates are respectively transformed by using the camera extrinsic parameters of the first camera to obtain the corresponding second optical center coordinates, the third edge point coordinates, and the second tangent point coordinates in the target coordinate system; based on the second optical center coordinates and the second tangent point coordinates, the target spherical center coordinates of the spherical target in the target coordinate system are determined.

[0007] In a possible implementation, the target center coordinates of the spherical target in the target coordinate system are determined based on the second optical center coordinates and the second tangent point coordinates, including: constructing a distance function that changes with the center coordinates of the spherical target in the target coordinate system based on the second optical center coordinates and the second tangent point coordinates, the distance function representing the distance between the center of the spherical target and the tangent point of the spherical target; constructing a target function that changes with the center coordinates of the spherical target in the target coordinate system according to the difference between the distance function and the spherical radius of the spherical target, and minimizing the target function to obtain the target center coordinates of the spherical target in the target coordinate system.

[0008] In a possible implementation manner, the first camera and the target camera are the same camera.

[0009] In a possible implementation, determining the information of reflected light of each second posture image in the target camera coordinate system includes: performing highlight area detection on each second posture image to obtain a highlight area in the corresponding second posture image, wherein the highlight area includes at least one pixel point; processing the pixel points of the highlight area in each second posture image to obtain a corresponding highlight point, and determining a first highlight point coordinate of each highlight point in the second image coordinate system, wherein the second image coordinate system corresponds to the imaging plane of the target camera; performing coordinate transformation on each first highlight point coordinate using the camera intrinsic parameter of the target camera to obtain the corresponding second highlight point coordinates in the target camera coordinate system, wherein the camera intrinsic parameter and camera extrinsic parameter of the target camera are obtained based on target camera calibration performed using the plane target in the multiple second posture images; determining the information of reflected light of each second posture image in the target camera coordinate system according to each second highlight point coordinate and a preset third optical center coordinate, wherein the third optical center coordinate represents the coordinate of the optical center of the target camera in the target camera coordinate system.

[0010] In a possible implementation, based on the target sphere center coordinates and the information of each reflected light, the information of the incident light corresponding to each reflected light in the target camera coordinate system is determined, including: using the camera extrinsic parameters of the target camera to perform coordinate transformation on the target sphere center coordinates to obtain the transformed sphere center coordinates in the target camera coordinate system; based on the sphere radius of the spherical target and the second sphere center coordinates corresponding to the spherical target in each second posture image in the target camera coordinate system, the intersection coordinates of the target intersection points on each reflected light in the target camera coordinate system are determined, and each target intersection point is the intersection point of the corresponding reflected light and the corresponding spherical target; according to the sphere radius, the transformed sphere center coordinates and the intersection coordinates, the information of the normal corresponding to each reflected light in the target camera coordinate system is determined; according to the information of each reflected light and the information of its corresponding normal, the information of the incident light corresponding to each reflected light is determined.

[0011] In a possible implementation, determining the radiation intensity distribution of the light source to be calibrated based on the multiple second posture images and the light source coordinates includes: sampling the plane targets in each of the second posture images according to a preset sampling rule to obtain the first sampling point coordinates of each sampling point in the second image coordinate system; performing coordinate transformation on each of the first sampling point coordinates using the camera intrinsic parameters of the target camera to obtain the corresponding second sampling point coordinates in the target camera coordinate system; obtaining the normal information corresponding to each of the second sampling point coordinates in the target camera coordinate system according to the light source coordinates, the second spherical center coordinates corresponding to the spherical target in each of the second posture images in the target camera coordinate system, and the second sampling point coordinates; determining a preset number of spherical harmonic basis functions in the spherical harmonic function and the spherical harmonic coefficients corresponding to each of the spherical harmonic basis functions based on the information of each of the second sampling point coordinates and the corresponding normals and the light source coordinates; and determining the radiation intensity distribution of the light source to be calibrated based on the light source coordinates, all spherical harmonic basis functions, and all spherical harmonic coefficients.

[0012] According to another aspect of the present disclosure, a light source calibration device is provided, including: a spherical center coordinate determination module, which is used to obtain multiple first posture images of a combined target using a first camera under preset uniform illumination conditions, wherein the combined target has a different first posture in each of the first posture images, and the combined target includes a plane target and a spherical target with fixed relative coordinates, and the target spherical center coordinates of the spherical target in a preset target coordinate system are determined based on the multiple first posture images; a light information determination module, which is used to obtain multiple second posture images of the combined target using a target camera under target illumination conditions formed by a light source to be calibrated. an image, and determines the information of the reflected light of each second posture image in the target camera coordinate system, wherein the combined target in each second posture image has a different second posture; a light source coordinate determination module, for determining the information of the incident light corresponding to each reflected light in the target camera coordinate system based on the target sphere center coordinates and the information of each reflected light, and determining the light source coordinates of the light source to be calibrated in the target camera coordinate system according to the information of all incident light; a light source intensity distribution determination module, for determining the radiation intensity distribution of the light source to be calibrated based on the multiple second posture images and the light source coordinates.

[0013] In a possible implementation, determining the target sphere center coordinates of the spherical target in a preset target coordinate system based on the multiple first posture images includes: performing edge detection on the spherical target in each of the first posture images to obtain first edge point coordinates of each edge point of the spherical target in each of the first posture images in the first image coordinate system; performing coordinate transformation on each of the first edge point coordinates using the camera intrinsic parameters of the first camera to obtain corresponding second edge point coordinates in the first camera coordinate system, wherein the camera intrinsic parameters and camera extrinsic parameters of the first camera are obtained based on the first camera calibration performed using the plane targets in the multiple first posture images; performing coordinate transformation on each of the second edge point coordinates and obtaining corresponding second edge point coordinates in the first camera coordinate system based on the second edge point coordinates and the first camera coordinate system. The first spherical center coordinates corresponding to the spherical target in a posture image, the spherical radius of the spherical target, and the preset first optical center coordinates are used to determine the first tangent point coordinates of the tangent points corresponding to each of the edge points in the first camera coordinate system, wherein the first optical center coordinates represent the coordinates of the optical center of the first camera in the first camera coordinate system; the first optical center coordinates, the second edge point coordinates, and the first tangent point coordinates are respectively transformed by using the camera extrinsic parameters of the first camera to obtain the corresponding second optical center coordinates, the third edge point coordinates, and the second tangent point coordinates in the target coordinate system; based on the second optical center coordinates and the second tangent point coordinates, the target spherical center coordinates of the spherical target in the target coordinate system are determined.

[0014] In a possible implementation, the target center coordinates of the spherical target in the target coordinate system are determined based on the second optical center coordinates and the second tangent point coordinates, including: constructing a distance function that changes with the center coordinates of the spherical target in the target coordinate system based on the second optical center coordinates and the second tangent point coordinates, the distance function representing the distance between the center of the spherical target and the tangent point of the spherical target; constructing a target function that changes with the center coordinates of the spherical target in the target coordinate system according to the difference between the distance function and the spherical radius of the spherical target, and minimizing the target function to obtain the target center coordinates of the spherical target in the target coordinate system.

[0015] In a possible implementation manner, the first camera and the target camera are the same camera.

[0016] In a possible implementation, determining the information of reflected light of each second posture image in the target camera coordinate system includes: performing highlight area detection on each second posture image to obtain a highlight area in the corresponding second posture image, wherein the highlight area includes at least one pixel point; processing the pixel points of the highlight area in each second posture image to obtain a corresponding highlight point, and determining a first highlight point coordinate of each highlight point in the second image coordinate system, wherein the second image coordinate system corresponds to the imaging plane of the target camera; performing coordinate transformation on each first highlight point coordinate using the camera intrinsic parameter of the target camera to obtain the corresponding second highlight point coordinates in the target camera coordinate system, wherein the camera intrinsic parameter and camera extrinsic parameter of the target camera are obtained based on target camera calibration performed using the plane target in the multiple second posture images; determining the information of reflected light of each second posture image in the target camera coordinate system according to each second highlight point coordinate and a preset third optical center coordinate, wherein the third optical center coordinate represents the coordinate of the optical center of the target camera in the target camera coordinate system.

[0017] In a possible implementation, based on the target sphere center coordinates and the information of each reflected light, the information of the incident light corresponding to each reflected light in the target camera coordinate system is determined, including: using the camera extrinsic parameters of the target camera to perform coordinate transformation on the target sphere center coordinates to obtain the transformed sphere center coordinates in the target camera coordinate system; based on the sphere radius of the spherical target and the second sphere center coordinates corresponding to the spherical target in each second posture image in the target camera coordinate system, the intersection coordinates of the target intersection points on each reflected light in the target camera coordinate system are determined, and each target intersection point is the intersection point of the corresponding reflected light and the corresponding spherical target; according to the sphere radius, the transformed sphere center coordinates and the intersection coordinates, the information of the normal corresponding to each reflected light in the target camera coordinate system is determined; according to the information of each reflected light and the information of its corresponding normal, the information of the incident light corresponding to each reflected light is determined.

[0018] In a possible implementation, determining the radiation intensity distribution of the light source to be calibrated based on the multiple second posture images and the light source coordinates includes: sampling the plane targets in each of the second posture images according to a preset sampling rule to obtain the first sampling point coordinates of each sampling point in the second image coordinate system; performing coordinate transformation on each of the first sampling point coordinates using the camera intrinsic parameters of the target camera to obtain the corresponding second sampling point coordinates in the target camera coordinate system; obtaining the normal information corresponding to each of the second sampling point coordinates in the target camera coordinate system according to the light source coordinates, the second spherical center coordinates corresponding to the spherical target in each of the second posture images in the target camera coordinate system, and the second sampling point coordinates; determining a preset number of spherical harmonic basis functions in the spherical harmonic function and the spherical harmonic coefficients corresponding to each of the spherical harmonic basis functions based on the information of each of the second sampling point coordinates and the corresponding normals and the light source coordinates; and determining the radiation intensity distribution of the light source to be calibrated based on the light source coordinates, all spherical harmonic basis functions, and all spherical harmonic coefficients.

[0019] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0020] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0021] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0022] Through the light source calibration method and device, electronic device and storage medium provided by the present disclosure, under preset uniform illumination conditions, a first camera is used to obtain multiple first posture images of a combined target, in each first posture image, the combined target has a different first posture, the combined target includes a plane target and a spherical target with fixed relative coordinates, and the target sphere center coordinates of the spherical target in a preset target coordinate system are determined based on the multiple first posture images; under target illumination conditions formed by a light source to be calibrated, a target camera is used to obtain multiple second posture images of the combined target, and information of reflected light of each second posture image in the target camera coordinate system is determined, and the combined target has a different second posture in each second posture image; based on the target sphere center coordinates and the information of each reflected light, the information of the incident light corresponding to each reflected light in the target camera coordinate system is determined, and the light source coordinates of the light source to be calibrated in the target camera coordinate system are determined according to the information of all incident light; based on the multiple second posture images and the light source coordinates, the radiation intensity distribution of the light source to be calibrated is determined, so that high-precision calibration of the spatial position and radiation intensity distribution of the light source can be achieved based on a group of images without any prior information about the light source.

[0023] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0025] Figure 1 A schematic diagram showing a combined target provided according to an embodiment of the present disclosure.

[0026] Figure 2 A schematic diagram of a photometric stereo system provided according to an embodiment of the present disclosure is shown.

[0027] Figure 3 A flow chart of a light source calibration method provided according to an embodiment of the present disclosure is shown.

[0028] Figure 4 A schematic diagram showing a plurality of first posture images provided by an embodiment of the present disclosure.

[0029] Figure 5 A schematic diagram showing a clear edge of a spherical target in a first posture image provided by an embodiment of the present disclosure.

[0030] Figure 6 A schematic diagram showing the imaging process of a spherical target in a first camera coordinate system provided by an embodiment of the present disclosure.

[0031] Figure 7A schematic diagram showing the imaging of a spherical target during the target calibration process provided by an embodiment of the present disclosure is shown.

[0032] Figure 8 A schematic diagram showing multiple second posture images provided by an embodiment of the present disclosure.

[0033] Fig. 9 A schematic diagram showing highlight points in a second posture image provided by an embodiment of the present disclosure.

[0034] Fig.10 A schematic diagram of light source position calibration provided by an embodiment of the present disclosure is shown.

[0035] Fig.11 A schematic diagram showing multiple second posture images provided by an embodiment of the present disclosure.

[0036] Fig.12 A schematic diagram of sampling points provided by an embodiment of the present disclosure is shown.

[0037] Fig.13 A schematic diagram showing the relative position relationship between each point light source and the combined target in the photometric stereo system provided by an embodiment of the present disclosure.

[0038] Fig.14 A schematic diagram showing the reconstruction result of the Lambert standard sphere provided by an embodiment of the present disclosure.

[0039] Fig.15 A schematic diagram showing the error distribution of the reconstruction result of the Lambert standard sphere provided by an embodiment of the present disclosure.

[0040] Fig.16 A schematic diagram showing the reconstruction result of the Lambertian plane provided by an embodiment of the present disclosure.

[0041] Fig.17 A schematic diagram showing the error distribution of the reconstruction result of the Lambertian plane provided by an embodiment of the present disclosure.

[0042] Fig.18 A schematic diagram showing the reconstruction result of plaster provided by an embodiment of the present disclosure.

[0043] Figure 19 to Figure 20 A block diagram of a light source calibration device provided according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0044] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0045] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0046] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0047] In order to facilitate those skilled in the art to understand the technical solution provided by the embodiments of the present disclosure, the technical environment in which the technical solution is implemented is described below.

[0048] Photometric stereo technology recovers the 3D shape of an object by capturing multiple images under different lighting conditions. A typical device includes a camera and multiple light sources. Under near-field conditions, the light source needs to be accurately modeled and calibrated to achieve high-quality 3D reconstruction. The calibration includes the calibration of the light source's geometric spatial position and the calibration of the radiance intensity distribution (RID).

[0049] In order to simplify mathematical analysis, the light source is often assumed to be parallel light in traditional photometric stereo methods. In the far field, the estimation error caused by this assumption is acceptable. However, in the near field, the parallel light condition is difficult to meet, and the application of smaller light-emitting diode (LED) light sources in photometric stereo systems is becoming more and more common. At this time, the traditional parallel light assumption will bring non-negligible errors to the reconstruction results. This is because when a single LED light source is lit, the light source radiates energy into space in the form of an approximate spherical wave, and the brightness of the object surface is affected by the surface normal, reflectivity and direction of the incident light. In order to achieve accurate shape recovery, the point light sources involved need to be accurately calibrated, including their spatial position and radiant intensity distribution (RID), that is, the radiation intensity of the light source under different emission directions.

[0050] There are still many problems in the current calibration methods. For example, the first point is that for the spatial position geometry calibration, the current calibration method often uses a small reflective ball to calibrate the spatial position of the light source. However, the process of calibrating the light source is carried out in a dark environment. When the sphere is illuminated by a point light source, the illumination on the surface of the sphere is uneven, and a shadow is generated at the edge of the sphere imaging. At this time, it is difficult to separate the edge of the sphere from the black background, resulting in edge extraction errors, which leads to deviations in the positioning of the sphere relative to the camera. Although increasing the camera exposure can enhance the edge clarity, it will also cause the quality of the highlight points on the image plane to deteriorate. Therefore, during operation, the camera exposure needs to strike a balance between achieving clear edges and maintaining smaller highlight points. Regarding the first point, in order to avoid the shadow problem, the related technology selects some points at the clear edge for fitting. Although this operation ultimately achieved good experimental results, the selection of edge points depends on manual operation, the fitting data is small, and the calibration results are not robust.

[0051] Second, due to the perspective imaging of the camera, the projection of the sphere on the image plane is an ellipse, and the center of the ellipse is not the image point of the center of the sphere, and there is an eccentricity error. These factors affect the spatial positioning of the calibration sphere, and further affect the ray tracing process, resulting in errors in the calibration of the spatial position of the light source. Regarding the second point, the related art uses the bundle adjustment method to determine the coordinates of the sphere in the target coordinate system, and the image of the sphere on the image plane is regarded as a circle for fitting. The coordinates of the center of the circle are not the image point of the center of the sphere. This is because the image of the sphere should be an ellipse due to the perspective imaging of the camera, so this method has an eccentricity error.

[0052] Third, with regard to the calibration of radiation intensity distribution, the current calibration model for radiation intensity distribution is limited to isotropic RID or radially symmetric RID. This model is not flexible enough to describe the diversity of RID in the real world. The related technology uses an axially symmetric RID luminescence model to describe the RID of the light source. This axially symmetric luminescence model cannot accurately describe the luminescence of the light source in the real scene, and the current RID model needs to obtain prior information about the half-decay angle of the light source brightness from the manufacturer, which increases the complexity of the calibration.

[0053] Fourthly, the calibration process of the light source geometric space position and the radiation intensity distribution is independent, and requires the acquisition of images and calibration separately, which makes the calibration operation process cumbersome.

[0054] In order to solve the above technical problems, with respect to the first point, the light source calibration method provided by the embodiment of the present disclosure takes advantage of the fact that the edge extraction of the plane circular mark point is less affected by uneven lighting, and establishes a light source calibration model based on a combined reference object, so as to achieve high-precision calibration of the spatial position of the light source under non-uniform lighting conditions. With respect to the second point, in the process of target calibration, the light source calibration method provided by the embodiment of the present disclosure uses the camera internal and external parameters provided by the plane target to convert all edge points and the optical center of the sphere to the target coordinate system, and uses the least squares method to solve the coordinates of the sphere center in the target coordinate system with the constraint that the distance from the sphere center to all tangents is equal to the sphere radius. In response to the third point above, the light source calibration method provided by the embodiment of the present disclosure adopts spherical harmonics to describe the RID of the light source. This model is suitable for the expression of anisotropic asymmetric radiation intensity fields. In order to achieve the fitting of spherical harmonic basis coefficients, the light source calibration method provided by the embodiment of the present disclosure designs the plane target part as a white background with black marking points. The white background material is a Lambert reflection material to achieve dense sampling of the light source luminous field. The image intensity information of the white area and the posture information provided by the black marking points are used to achieve the fitting of the spherical harmonic basis functions to achieve the fitting calibration of the light source RID. In response to the fourth point above, the combined target designed by the light source calibration method provided by the embodiment of the present disclosure can realize the calibration of the spatial position of the light source and the RID of the light source in one shot, avoiding the repetitive operation of two calibrations.

[0055] Figure 1 Schematic diagram of a combined target provided according to an embodiment of the present disclosure is shown. Figure 1 The combined target shown in FIG. 1 includes a plane target and a spherical target. The plane target and the spherical target can be fixed by a rigid body. The target coordinate system of the combined target is established at the center of the plane area in the plane target. The direction of the target coordinate system can be as follows: Figure 1 The plane target may include a white Lambert material plane and 144 black marking points around it. The distance between the black marking points is fixed and can be adjusted according to the size of the camera field of view. In some embodiments, the distance between the centers of two adjacent black marking points may be 3 mm. Figure 1 As shown, the black marking points include black small circles and black large circles. There are 5 black large circles, which are used to sort the marking points. The size of the sphere target can be adjusted according to the size of the camera field of view. In some embodiments, the radius of the sphere target is 19 mm. It should be noted that although 144, 5, 3 mm, 19 mm and Figure 1The combined target provided by the embodiment of the present disclosure is introduced as an example, but those skilled in the art will appreciate that the present disclosure should not be limited thereto. In fact, the user can flexibly set the size and position of the plane area in the plane target, the number and sorting method, size and position of the marker points in the plane target according to personal preferences and / or actual application scenarios. Similarly, the spherical radius and position of the spherical target can be flexibly set.

[0056] The light source calibration method provided by the embodiment of the present disclosure is calibrated using a combined target. The calibration process provided by the embodiment of the present disclosure may include two processes: target calibration and light source calibration. Target calibration is to determine the coordinates of the center of the spherical target in the target coordinate system based on the different posture images of the combined target obtained. Light source calibration includes light source position calibration and light source radiation intensity distribution RID calibration. Light source position calibration is to determine the spatial position of the light source to be calibrated based on the different posture images of the combined target obtained. RID calibration is to determine the light source radiation intensity distribution of the light source to be calibrated based on the different posture images of the combined target obtained, and the RID calibration needs to be performed in combination with the result of the light source position calibration, that is, the spatial position of the light source to be calibrated.

[0057] In some embodiments, the posture images used for light source position calibration and the posture images used for RID calibration can be the same set of images. In other words, the light source calibration provided by the embodiments of the present disclosure can achieve the two calibration processes of target calibration and light source calibration by acquiring different posture images of the combined target only once, without acquiring images separately, thus avoiding the repetitive operation of the two calibrations. Of course, in some embodiments, the posture images used for light source position calibration and the posture images used for RID calibration may not be the same set of images, which can be flexibly selected according to actual needs.

[0058] In some embodiments, different posture images of the combined target for target calibration, i.e., multiple first posture images, can be obtained by using the first camera under uniform lighting conditions, and different posture images of the combined target for light source calibration, i.e., multiple second posture images, can be obtained by using the target camera under target lighting conditions. The target camera and the first camera can be the same camera, so that only one camera can be calibrated to obtain the corresponding internal and external parameters during the entire calibration process, saving some time. Of course, in some embodiments, the target camera and the first camera can also be different cameras, so that during the entire calibration process, the two cameras need to be calibrated separately to obtain their own internal and external parameters to participate in the subsequent calculation process. Although using two cameras to obtain posture images separately may increase the length of the calibration process, this is conducive to avoiding the introduction of inaccurate calibration results of a single camera into the subsequent calibration process. In this embodiment, the uniform lighting condition is formed by utilizing multiple light sources, and the target lighting condition is formed based on a single light source to be calibrated, which makes the uniform lighting condition different from the target lighting condition. Even if the same camera is used to acquire posture images, the images acquired under these two lighting conditions are different. However, it should be noted that each first posture image and each second posture image are essentially posture images of the combined target in a certain posture, and the use of "first" and "second" is to distinguish that the two are obtained under different acquisition methods.

[0059] A photometric stereo system may generally include a camera and at least three point light sources that are relatively fixed in position. Figure 2 FIG. 1 is a schematic diagram of a photometric stereo system provided according to an embodiment of the present disclosure. Figure 2 The camera in the photometric stereo system shown is used as a target camera, a point light source is used as a light source to be calibrated, and the first camera is not a target camera to schematically illustrate the light source calibration method provided by the embodiment of the present disclosure.

[0060] Figure 3 FIG. 1 is a flow chart showing a method for calibrating a light source according to an embodiment of the present disclosure. Figure 3 As shown, the light source calibration method provided by the embodiment of the present disclosure may include the following steps S101 to S104.

[0061] Step S101: Under preset uniform illumination conditions, a first camera is used to obtain a plurality of first posture images of a combined target, and the target sphere center coordinates of the spherical target in a preset target coordinate system are determined based on the plurality of first posture images.

[0062] The combined target includes a plane target and a spherical target with fixed relative coordinates. The combined target in each first posture image has a different first posture. Figure 4 A schematic diagram showing a plurality of first posture images provided by an embodiment of the present disclosure. Figure 5A schematic diagram showing a clear edge of a spherical target in a first posture image provided by an embodiment of the present disclosure. In some embodiments, the combined target may be placed at different positions, so that the first camera can be used to photograph the combined target at each position to obtain Figure 4 The 21 first posture images P1L9 to P21L9 shown in FIG. 1 have different postures for the combined targets in these first posture images, and the edge of the spherical target in each first posture image is clear. The clarity can be seen in FIG. Figure 5 .

[0063] In fact, the uniform illumination condition, the number of first posture images, and the first posture corresponding to the first posture image can be flexibly set according to actual needs, and the embodiments of the present disclosure do not limit this. It is sufficient to ensure that the edge of the spherical target in each first posture image is clear. As for the judgment standard of edge clarity, it can be determined according to actual conditions. Figure 5 The sharp edges shown are for example only.

[0064] Figure 6 A schematic diagram showing the imaging process of a spherical target in a first camera coordinate system provided by an embodiment of the present disclosure. Figure 7 FIG. 2 is a schematic diagram showing the imaging of a spherical target during the target calibration process provided by an embodiment of the present disclosure. Figure 6 , Figure 7 , the step S101 of determining the target sphere center coordinates of the spherical target in the preset target coordinate system based on the multiple first posture images may include: first, performing edge detection on the spherical target in each first posture image, for example, executing the Canny edge detection algorithm on each first posture image, and obtaining the first edge point coordinates of each edge point of the spherical target in each first posture image in the first image coordinate system, wherein the first image coordinate system corresponds to the imaging plane of the first camera, and the first edge point coordinates are pixel coordinates; using the camera intrinsic parameters of the first camera to detect the first edge point coordinates; Coordinate transformation is performed to obtain the coordinates of each second edge point corresponding to the edge point in the first camera coordinate system. For example, the coordinate transformation can be performed by the following formula 1: The coordinates of the second edge point

[0065]

[0066] In formula 1, represents the first edge point coordinates of the edge point in the jth first posture (i.e., in the jth first posture image) in the first image coordinate system, represents the second edge point coordinates of the edge point in the first camera coordinate system, c represents the first camera coordinate system corresponding to the first camera, the first edge point coordinates and the second edge point coordinates essentially represent the position of the same edge point, except that the former is the coordinates in the first image coordinate system and the latter is the coordinates in the first camera coordinate system, f x 、f y Represents the focal length of the camera intrinsic parameters of the first camera, wherein the camera intrinsic parameters and camera extrinsic parameters of the first camera are obtained based on the first camera calibration using the plane targets in the multiple first posture images, and the camera extrinsic parameters will change with the first posture. The first camera calibration process is detailed in the example below. In addition, it should be noted that, similarly to the first edge point coordinates and the second edge point coordinates, the two coordinates corresponding to other coordinate transformations in this article are all for the position representation of the same object, which will not be repeated in the following text.

[0067] Then, if Figure 6 As shown, based on each edge point The second edge point coordinates, the first spherical center coordinates corresponding to the spherical target in each first posture image in the first camera coordinate system, the spherical radius of the spherical target, and the preset first optical center coordinates are used to determine each edge point The corresponding cut point The first tangent point coordinates in the first camera coordinate system, wherein the first optical center coordinates represent the coordinates of the optical center of the first camera in the first camera coordinate system. In this embodiment, the first optical center coordinates are the origin in the first camera coordinate system. Taking the edge point a1 in the third first posture image as an example, a straight line equation is determined based on the second edge point coordinates of the edge point a1, the first optical center coordinates, and the shortest distance between the optical center of the first camera and the imaging plane of the first camera (for the convenience of calculation, the normalized result is 1 in this example), and based on the spherical radius R of the spherical target and the first spherical center coordinates corresponding to the spherical target in the third first posture image in the first camera coordinate system (which can be calculated based on Figure 6 The point in the sphere representing the center (determine) determine a spherical equation, solve the linear equation and the spherical equation simultaneously, and use the solution with the smaller z coordinate as the tangent point b corresponding to the edge point a1; use the camera extrinsic parameters of the first camera to perform coordinate transformation on the first optical center coordinate, each second edge point coordinate, and each first tangent point coordinate, respectively, to obtain the corresponding second optical center coordinate, each third edge point coordinate, and each second tangent point coordinate in the target coordinate system. For example, the coordinate transformation can be performed by the following formula 2:

[0068]

[0069] In formula 2, represents the optical center of the first camera in the jth first posture (i.e., in the jth first posture image) in the target coordinate system w, R j and tj represents the camera extrinsic parameters of the first camera in the jth first posture, represents the optical center of the first camera in the jth first posture (i.e., in the jth first posture image) in the first camera coordinate system c, and the first optical center coordinates are used as the calculation Thus, through the formula 2 It can be calculated The corresponding second optical center coordinates, in fact, the optical center coordinates of the first camera in each first posture in the first camera coordinate system are all the origin of the first camera coordinate system; represents the edge point of the spherical target in the jth first posture (i.e., in the jth first posture image) in the target coordinate system w, represents the edge point of the spherical target in the jth first posture (i.e., in the jth first posture image) in the first camera coordinate system c, and the coordinates of the second edge point are used as Thus, through the formula 2 It can be calculated The corresponding third edge point coordinates; represents the tangent point of the spherical target in the jth first posture (i.e., in the jth first posture image) in the target coordinate system w, represents the tangent point of the spherical target in the jth first posture (i.e., in the jth first posture image) in the first camera coordinate system c, and the first tangent point coordinates are used as the calculation Thus, through the formula 2 It can be calculated Indicates the corresponding second tangent point coordinates.

[0070] Finally, the target sphere center coordinates of the spherical target in the target coordinate system are determined based on the second optical center coordinates and the second tangent point coordinates. This step may further include: constructing a distance function that changes with the sphere center coordinates of the spherical target in the target coordinate system based on the second optical center coordinates and the second tangent point coordinates. The distance function represents the distance between the sphere center of the spherical target and the tangent point of the spherical target. For example, the distance function d can be represented by the following formula 3:

[0071]

[0072] In formula 3, d represents the coordinates of the target sphere center (i.e. Figure 7 The point representing the center of the sphere Distance in target coordinate system The distance of this vector, The vector that indicates the optical center of the first camera corresponding to the j-th first posture in the target coordinate system w points to the center of the spherical target, Determined according to the target sphere center coordinates and each second optical center coordinates, Indicates the vectors of the tangent points corresponding to the jth first posture of the first camera in the target coordinate system w, Determined according to the coordinates of each second optical center and each second tangent point. Then, according to the difference between the distance function d and the spherical radius R of the spherical target, an objective function that changes with the coordinates of the sphere center of the spherical target in the target coordinate system is constructed, and the objective function is minimized to obtain the target sphere center coordinates of the spherical target in the target coordinate system. For example, the objective function E can be expressed by the following formula 4:

[0073]

[0074] In Formula 4, E represents the objective function, m represents the number of first postures, and n j represents the n edge points in the jth first posture, and the number of edge points in each first posture may be different, d represents the distance function, and R represents the spherical radius of the spherical target. The distance function d should be equal to the spherical radius R, so the most accurate target sphere center coordinates of the spherical target in the target coordinate system can be obtained by minimizing the objective function. In this embodiment, the least squares method can be used to solve the target sphere center coordinates. Since the target calibration process is carried out under uniform lighting conditions, the sphere edge with clear contrast is obtained, ensuring the vector and The collinearity of can be solved by minimizing the objective function shown in Equation 5:

[0075]

[0076] In formula 5, The vector indicating the edge point of the spherical target pointed by the optical center of the first camera corresponding to the jth first posture in the target coordinate system w, The coordinates of the target sphere center and the coordinates of each third edge point are determined. The remaining parameters are as mentioned above and will not be repeated here. In this way, since the plane target provides a relative posture relationship with respect to the first camera, this increases the vector The number of points is reduced, so the calculation result is more robust. After the target is calibrated, point light sources of any arrangement can be calibrated.

[0077] Camera calibration technology refers to the process of establishing a geometric model of camera imaging in order to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image during image measurement and machine vision applications. These geometric model parameters are the camera parameters. Camera parameters include camera intrinsic parameters and camera extrinsic parameters. In fact, the specific calibration process of the camera can be flexibly set according to actual conditions, and the embodiments of the present disclosure do not limit this. Now, take the first camera as an example to briefly explain: place the combined target in different postures in space, use the first camera to take a picture in each posture, and identify the black marker points in the plane target in the captured image and automatically sort them based on the distance relationship between the center of the small black circle and the center of the large black circle to obtain the pixel coordinates (u+△u,v+△v) of the center of each black marker point, and combine them with the spatial point [X] represented by the following formula 6. w ,Y w ,Z w ]The internal and external parameters of the first camera are solved by the geometric process of imaging from the first camera to the image plane:

[0078]

[0079] In formula 6, [X w ,Y w ,Z w ] represents the coordinates of the black mark point in the plane target (in this example, the coordinates of the center of the black mark point are known and fixed when the combined target is designed), f x 、f y represents the focal length of the first camera in the camera intrinsic parameter, u0 and v0 represent the camera principal point, u+△u and v+△v represent the pixel coordinates of the center of the black mark point, and Z c represents the scale factor, k1, k2, p1, p2 represent the lens distortion factor, R j and t j represents the camera external parameters of the first camera in the jth first posture, △u and △v represent the offset of the black mark point imaged on the image plane due to the distortion of the first camera lens. After solving, the intrinsic and extrinsic parameters of the first camera can be obtained. The intrinsic parameters include the camera focal length (f x 、f y ), principal point coordinates (u0, v0), distortion factors (k1, k2, p1, p2), the external parameters are [R j |t j ], where j represents the jth posture. In some embodiments, after obtaining the internal and external parameters of the first camera, the distortion factor can be used to perform distortion correction on all first posture images taken by the first camera, and then edge detection is performed on the corrected first posture images.

[0080] In addition, it should be noted that this paper has the same way of expressing the internal and external parameters and other related parameters of the target camera and the first camera. For example, the focal lengths of the target camera and the first camera are both expressed using f x 、f y The camera coordinate systems corresponding to the target camera and the first camera are all denoted by c. This is for the convenience of understanding, and the specific meaning should be determined in combination with the context.

[0081] Step S102: Under target illumination conditions formed by the light source to be calibrated, a target camera is used to acquire a plurality of second posture images of the combined target, and information of reflected light of each second posture image in the target camera coordinate system is determined.

[0082] The combined target in each second posture image has a different second posture. Figure 8 A schematic diagram showing multiple second posture images provided by an embodiment of the present disclosure. Fig. 9 A schematic diagram of highlight points in the second posture image provided by an embodiment of the present disclosure is shown. In some embodiments, during the light source calibration process, a single light source to be calibrated is lit, the target camera exposure is adjusted to minimize the highlight image point, and the combined target is placed in multiple postures in space and photographed to obtain the following: Figure 8 In fact, the number of the second posture images and the second postures corresponding to the second posture images can be flexibly set according to actual needs, and the embodiments of the present disclosure do not limit this.

[0083] In some embodiments, similarly to the first camera, the target camera may be calibrated to obtain the intrinsic parameters (f x ,f y ,u0,v0,k1,k2,p1,p2) and external parameters [R|t], use the distortion factors (k1,k2,p1,p2) in the internal parameters of the target camera to perform distortion correction on all the second posture images, and then determine the corresponding reflected light information based on the corrected second posture images.

[0084] Fig.10 The schematic diagram of the light source position calibration provided by the embodiment of the present disclosure is shown. The information of the reflected light of each second posture image in the target camera coordinate system in step S102 may include: performing highlight area detection on each second posture image to obtain the highlight area in the corresponding second posture image. The mirror ball of the spherical target will produce a highlight point on its surface under the condition of illumination by a single point light source, i.e., the light source to be calibrated. Generally, a highlight area is formed on the image plane. The highlight area includes at least one pixel point. In this embodiment, the highlight area is Fig. 9The 7-pixel rectangular area shown in FIG. 1 is a rectangular area of ​​7 pixels in size; the pixel points of the highlight area in each second posture image are processed to obtain the corresponding highlight point, and the first highlight point coordinates of each highlight point in the second image coordinate system are determined, wherein the second image coordinate system corresponds to the imaging plane of the target camera, and for example, an ellipse fitting is used as the processing method, so that the central pixel point obtained by fitting can be used as the highlight point, and the central pixel coordinates are used as the pixel coordinates of the highlight point (u h ,v h ); coordinate transformation is performed on the coordinates of each first highlight point using the camera intrinsic parameters of the target camera to obtain the corresponding coordinates of each second highlight point in the target camera coordinate system, wherein the camera intrinsic parameters and camera extrinsic parameters of the target camera are obtained based on the target camera calibration using the plane targets in the plurality of second posture images. For example, the coordinate transformation can be performed using the following formula 7 to obtain the coordinates of the second highlight point (u c ,v c ,1):

[0085]

[0086] In formula 7, f x 、f y represents the focal length of the camera intrinsic parameter of the target camera, u0 and v0 represent the camera principal point, and u h 、u h represents the pixel coordinates of the highlight point, and c represents the target camera coordinate system. Next, according to the coordinates of each second highlight point and the preset third optical center coordinates, the information of the reflected light of each second posture image in the target camera coordinate system is determined. In other words, by connecting the optical center and the highlight point in the target camera coordinate system, each second posture image, i.e., the reflected light l under each posture, can be obtained. re , wherein the third optical center coordinates represent the coordinates of the optical center of the target camera in the target camera coordinate system. In this embodiment, the optical center coordinates of the optical center of the first camera in each second posture (in each second posture image) in the target camera coordinate system are all the origin of the target camera coordinate system.

[0087] Step S103: Based on the target sphere center coordinates and the information of each reflected light ray, determine the information of the incident light ray corresponding to each reflected light ray in the target camera coordinate system, and determine the light source coordinates of the light source to be calibrated in the target camera coordinate system according to the information of all incident light rays.

[0088] In some embodiments, determining the information of the incident light corresponding to each reflected light in the target camera coordinate system based on the target sphere center coordinates and the information of each reflected light in step S103 may include: performing coordinate transformation on the target sphere center coordinates using the camera extrinsic parameters of the target camera to obtain the transformed sphere center coordinates in the target camera coordinate system. For example, the coordinate transformation may be performed using the following formula 8:

[0089]

[0090] In formula 8, R and t represent the camera external parameters of the target camera, and represents the center of the sphere in the target coordinate system. The corresponding target sphere center coordinates are obtained in step S101, and the sphere center is used in the calculation of equation 8. The corresponding target sphere center coordinates are used to calculate the sphere center in the target camera coordinate system The corresponding transformed sphere center coordinates.

[0091] Next, refer to Fig.10 Since the radius of the spherical target and the spatial position of the sphere center in the target camera coordinate system (i.e. the coordinates of the sphere center after transformation) are known, the reflected light l can be calculated. re The intersection point m with the spherical target in space, that is, based on the spherical radius of the spherical target and the second spherical center coordinates corresponding to the spherical target in each second posture image in the target camera coordinate system, determine the intersection coordinates of the target intersection points on each reflected light in the target camera coordinate system. Each target intersection point is the intersection point of each corresponding reflected light and the corresponding spherical target. The number of the intersection points is two, and the intersection point with a smaller z-axis is generally selected as the target intersection point. According to the spherical radius, the transformed spherical center coordinates and the intersection coordinates, determine the normal information corresponding to each reflected light in the target camera coordinate system. For example, the normal information can be determined by the following formula 9:

[0092]

[0093] In formula 9, n represents the normal line at the intersection point m, represents the sphere center in the target camera coordinate system and the transformed sphere center coordinates are used in the calculation of formula 9. The intersection point m is similarly The intersection coordinates of the target intersection point in the target camera coordinate system are used in the calculation of formula 9, and R represents the radius of the sphere. Finally, according to the information of each reflected light ray and its corresponding normal information, the information of the incident light ray corresponding to each reflected light ray is determined. For example, the information of the incident light ray can be determined by the following formula 10:

[0094] l in = l re -2(n T l re )n Formula 10 In formula 10, l in Represents the information of the incident light, l re Represents the information of reflected light, n represents the normal, and T represents the transpose.

[0095] like Fig.10As shown, when n′ second posture images are obtained, n′ incident light rays can be determined, and then the spatial position s of the light source to be calibrated can be obtained by calculating the intersection of the n′ incident light rays.

[0096] If the photometric stereo system includes multiple light sources to be calibrated, the above steps S102 to S103 may be performed for each light source to be calibrated, thereby completing the calibration of the spatial geometric positions of all point light sources of the photometric stereo system.

[0097] Step S104: determining the radiation intensity distribution of the light source to be calibrated based on the plurality of second posture images and the light source coordinates.

[0098] The embodiment of the present disclosure uses spherical harmonics to describe the radiant intensity distribution (RID) of the light source to be calibrated, i.e., Φ(x). The emission field of the light source to be calibrated, i.e., a point light source, can be conceptualized as a spherical wave radiating in all directions. Therefore, the radiant intensity distribution Φ(x) of the light source to be calibrated can be represented by a linear combination of the spherical harmonic basis functions of the following equation 11:

[0099]

[0100] In formula 11, c k represents the coefficient of the kth spherical harmonic function of the light source to be calibrated, y k (·) represents the basis function of the kth spherical harmonic function of the light source to be calibrated (spherical harmonic basis function for short), and s represents the spatial position of the light source to be calibrated. k (·) parameterized as a function of the spatial coordinates (x, y, z), the spherical harmonics are represented by y k (x, y, z), in the Cartesian coordinate system, the first 9 terms (h = 9) y1 to y9 of the spherical harmonic basis function can be expressed as the following equation 12:

[0101]

[0102] The parameters in Formula 12 are as mentioned above and will not be described again here.

[0103] Due to the orthogonality of spherical harmonics, any piecewise continuous function on a sphere can be expressed as a linear combination of an infinite series of spherical harmonics. Therefore, the embodiment of the present disclosure samples the planar area in the planar target in each second posture image to fit the spherical harmonics, thereby determining the RID of the light source to be calibrated.

[0104] Fig.11 A schematic diagram showing multiple second posture images provided by an embodiment of the present disclosure. Fig.12Schematic diagram of sampling points provided by the embodiment of the present disclosure is shown. In some embodiments, step S104 may include: sampling the plane targets in each second posture image according to a preset sampling rule to obtain the first sampling point coordinates of each sampling point in the second image coordinate system. The sampling rule can be flexibly set according to actual needs. In this embodiment, a brightness threshold can be set, and points in the plane area of ​​the plane targets in each second posture image that are equal to or greater than the brightness threshold are used as sampling points, and points below the brightness threshold are discarded, so that Fig.11 The 12 second posture images marked with multiple yellow sampling points are shown. Fig.12 It can be seen that not all points in the plane area of ​​the plane target are used as sampling points; the coordinates of each first sampling point are transformed by using the camera intrinsic parameters of the target camera to obtain the corresponding second sampling point coordinates in the target camera coordinate system. The coordinate transformation is similar to the coordinate transformation process of the first highlight point coordinates and the second highlight point coordinates in the previous text, which will not be repeated here; according to the light source coordinates, the second spherical center coordinates corresponding to the spherical target in each second posture image in the target camera coordinate system, and the coordinates of each second sampling point, the normal information corresponding to the coordinates of each second sampling point in the target camera coordinate system is obtained. The normal determination process can be similar to the process of determining the normal corresponding to the intersection point on the spherical target in the previous text, which will not be repeated here; based on the information of the coordinates of each second sampling point and its corresponding normal, and the light source coordinates, a preset number of spherical harmonic basis functions in the spherical harmonic function and the spherical harmonic coefficients corresponding to each spherical harmonic basis function are determined. In this embodiment, the preset number is set to 9, and the least squares form is constructed by combining the light intensity value at each sampling point to solve, wherein the light intensity value of each sampling point can be expressed by the following formula 13:

[0105]

[0106] In formula 13, I(p) represents the light intensity value at the sampling point p, the sampling point p is represented by the pixel coordinates in the second image coordinate system, γ represents the camera response coefficient, β represents the proportional coefficient of image clarity, which depends on many factors, such as lens aperture and magnification. represents the reflectivity of the white Lambert area (i.e., the plane area) in the plane target, x represents the spatial point corresponding to the sampling point p in the target camera coordinate system, and the coordinates of the spatial point are the coordinates of the second sampling point corresponding to the target camera coordinate system, n(x) represents the information of the normal corresponding to the coordinates of the second sampling point, and the other parameters are as mentioned above and will not be repeated here. Based on equation 13, equation 11 can be written as the following equation 14:

[0107]

[0108] In formula 14, In this embodiment, ψ is regarded as a constant, which is related to the system parameters and the reflectivity of the plane target surface. In this way, the unknown variables in the model shown in equation 14 are only the coefficients of the spherical harmonics. If p second postures of the combined target are obtained and each second posture has q sampling points on the white Lambert area, based on this, the least square form shown in equation 15 below can be constructed to solve the coefficients:

[0109]

[0110] In formula 15, y=[y1y2 y3 y4 y5 y6 y7 y8 y9],x ij represents the space point corresponding to the jth sampling point in the i-th second posture, and the coordinates of this space point are the coordinates of the corresponding second sampling point in the target camera coordinate system. The remaining parameters are as mentioned above and will not be repeated here. It is difficult to separate ψ and [c1 c2 c3 c4c5 c6 c7 c8 c9] in , so in this embodiment, the solution is As c k , and then based on the light source coordinates, all spherical harmonic basis functions, and all spherical harmonic coefficients, the radiation intensity distribution Φ(x) of the light source to be calibrated is determined. In addition, it should be noted that when using the calibrated photometric stereo system for image reconstruction, this composite term coefficient can be directly used In this way, high-precision ray inverse tracing is achieved through the designed combined target, and the radiation field of the light source to be calibrated is sampled using the white Lambert area on the plane target. The anisotropic LED radiation intensity distribution is calibrated by fitting the spherical harmonic function coefficients. The spherical harmonic function is used to describe the light source luminous field, which is suitable for the luminous conditions of real-world light sources and does not require any prior information about the light source.

[0111] If the photometric stereo system includes multiple light sources to be calibrated, the above step S104 may be performed for each light source to be calibrated, so as to complete the calibration of the radiation intensity distribution of all point light sources of the photometric stereo system.

[0112] Fig.13 A schematic diagram showing the relative positional relationship between each point light source and the combined target in the photometric stereo system provided by an embodiment of the present disclosure is shown. In one example, the photometric stereo system includes 8 point light sources and a camera. After one point light source is calibrated using the above light source calibration method, the other point light sources in the photometric stereo system are sequentially lit and the last calibrated point light source is turned off. The light source calibration process is repeated to obtain the coordinates of each point light source. Table 1 shows the coordinates of each point light source in the camera coordinate system corresponding to the camera. Fig.13 The relative position relationship between each point light source and the combined target in the camera coordinate system is shown.

[0113] Table 1 Point light source coordinates of each point light source in the photometric stereo system in the camera coordinate system

[0114]

[0115] Fig.14 A schematic diagram showing the reconstruction result of the Lambert standard sphere provided by an embodiment of the present disclosure. Fig.15 A schematic diagram showing the error distribution of the reconstruction result of the Lambert standard sphere provided by an embodiment of the present disclosure. Fig.16 A schematic diagram showing the reconstruction result of the Lambertian plane provided by an embodiment of the present disclosure. Fig.17 A schematic diagram showing the error distribution of the reconstruction result of the Lambertian plane provided by an embodiment of the present disclosure. Fig.18 A schematic diagram showing the reconstruction result of the plaster provided by the embodiment of the present disclosure. In some embodiments, in combination with the RID calibration result, a Lambert standard sphere with a diameter of 38.0995 mm, a Lambert plane, and plaster can be reconstructed. The reconstruction result of the Lambert standard sphere is as follows: Figure 14 to Figure 15 As shown, the root mean square error of the reconstruction result of the Lambert standard sphere is 0.2178mm, and the reconstruction result of the Lambert plane is as follows Figure 16 to Figure 17 As shown in Figure 2, the root mean square error of the reconstruction result of the Lambertian plane is 0.1349 mm, and the reconstruction result of the plaster is as follows: Fig.18 shown.

[0116] The present disclosure adds a plane target on the basis of the traditional single mirror ball calibration, and the two are fixed by a rigid body to realize the simultaneous high-precision calibration of the spatial position of the light source and the RID. The present disclosure takes advantage of the fact that the edge extraction of the plane circular mark point is less affected by uneven illumination, establishes a light source calibration model based on a combined reference object, and realizes the high-precision calibration of the spatial position of the light source under non-uniform illumination conditions. The light source calibration method provided by the present disclosure may include two parts. The first is target calibration, the purpose of which is to determine the relative posture relationship between the plane target and the sphere, that is, the target sphere center coordinates in the target coordinate system. In the process of calibrating the target, the camera internal and external parameters provided by the plane target are used to convert all edge points and the optical center of the spherical target to the target coordinate system, and the least squares method is used to solve the target sphere center coordinates in the target coordinate system with the distance from the sphere center to all tangents being equal to the sphere radius as a constraint. After completing the target calibration, the point light source arranged arbitrarily in space can be calibrated. The second is to use the calibrated target to realize the calibration of the point light source, including the calibration of the spatial position of the light source and the calibration of the radiation intensity distribution. In the process of calibrating the spatial position of the light source, high-precision sphere positioning is achieved, avoiding the sphere positioning problem caused by shadow and eccentricity errors, thereby realizing high-precision reverse ray tracing and accurately calibrating the spatial position of the light source. In addition, spherical harmonics are used to describe the light source's luminous field, avoiding the prior information of the light source parameters, and are suitable for the expression of anisotropic asymmetric radiation intensity fields. It can describe various anisotropic asymmetric luminous fields and achieve an accurate description of realistic point light sources. At the same time, the designed combined target unifies the light source spatial position calibration process and the RID calibration process. Only one calibration operation is required to achieve simultaneous calibration of the two, simplifying the calibration steps. The light source calibration method disclosed in the present invention can calibrate point light sources at any position in space after executing the target calibration. It is not only suitable for photometric stereo systems, but also can simultaneously estimate LED positions and anisotropic radiation intensity distributions in near-field lighting.

[0117] The disclosed embodiment also provides a light source calibration device. Fig.19 FIG. 2 is a block diagram of a light source calibration device provided according to an embodiment of the present disclosure. Fig.19 As shown, the light source calibration device 100 may include the following spherical center coordinate determination module 101, light information determination module 102, light source coordinate determination module 103, and light source intensity distribution determination module 104.

[0118] The spherical center coordinate determination module 101 is used to obtain multiple first posture images of the combined target using a first camera under preset uniform lighting conditions, wherein the combined target has a different first posture in each of the first posture images, and the combined target includes a plane target and a spherical target with fixed relative coordinates, and determine the target spherical center coordinates of the spherical target in a preset target coordinate system based on the multiple first posture images.

[0119] The light information determination module 102 is used to obtain multiple second posture images of the combined target using a target camera under target lighting conditions formed by the light source to be calibrated, and determine the information of reflected light of each second posture image in the target camera coordinate system, wherein the combined target has a different second posture in each second posture image.

[0120] The light source coordinate determination module 103 is used to determine the information of the incident light corresponding to each reflected light in the target camera coordinate system based on the target sphere center coordinates and the information of each reflected light, and determine the light source coordinates of the light source to be calibrated in the target camera coordinate system according to the information of all incident light rays.

[0121] The light source intensity distribution determining module 104 is used to determine the radiation intensity distribution of the light source to be calibrated based on the multiple second posture images and the light source coordinates.

[0122] In a possible implementation, determining the target sphere center coordinates of the spherical target in a preset target coordinate system based on the multiple first posture images includes: performing edge detection on the spherical target in each of the first posture images to obtain first edge point coordinates of each edge point of the spherical target in each of the first posture images in the first image coordinate system; performing coordinate transformation on each of the first edge point coordinates using the camera intrinsic parameters of the first camera to obtain corresponding second edge point coordinates in the first camera coordinate system, wherein the camera intrinsic parameters and camera extrinsic parameters of the first camera are obtained based on the first camera calibration performed using the plane targets in the multiple first posture images; performing coordinate transformation on each of the second edge point coordinates and obtaining corresponding second edge point coordinates in the first camera coordinate system based on the second edge point coordinates and the first camera coordinate system. The first spherical center coordinates corresponding to the spherical target in a posture image, the spherical radius of the spherical target, and the preset first optical center coordinates are used to determine the first tangent point coordinates of the tangent points corresponding to each of the edge points in the first camera coordinate system, wherein the first optical center coordinates represent the coordinates of the optical center of the first camera in the first camera coordinate system; the first optical center coordinates, the second edge point coordinates, and the first tangent point coordinates are respectively transformed by using the camera extrinsic parameters of the first camera to obtain the corresponding second optical center coordinates, the third edge point coordinates, and the second tangent point coordinates in the target coordinate system; based on the second optical center coordinates and the second tangent point coordinates, the target spherical center coordinates of the spherical target in the target coordinate system are determined.

[0123] In a possible implementation, the target center coordinates of the spherical target in the target coordinate system are determined based on the second optical center coordinates and the second tangent point coordinates, including: constructing a distance function that changes with the center coordinates of the spherical target in the target coordinate system based on the second optical center coordinates and the second tangent point coordinates, the distance function representing the distance between the center of the spherical target and the tangent point of the spherical target; constructing a target function that changes with the center coordinates of the spherical target in the target coordinate system according to the difference between the distance function and the spherical radius of the spherical target, and minimizing the target function to obtain the target center coordinates of the spherical target in the target coordinate system.

[0124] In a possible implementation manner, the first camera and the target camera are the same camera.

[0125] In a possible implementation, determining the information of reflected light of each second posture image in the target camera coordinate system includes: performing highlight area detection on each second posture image to obtain a highlight area in the corresponding second posture image, wherein the highlight area includes at least one pixel point; processing the pixel points of the highlight area in each second posture image to obtain a corresponding highlight point, and determining a first highlight point coordinate of each highlight point in the second image coordinate system, wherein the second image coordinate system corresponds to the imaging plane of the target camera; performing coordinate transformation on each first highlight point coordinate using the camera intrinsic parameter of the target camera to obtain the corresponding second highlight point coordinates in the target camera coordinate system, wherein the camera intrinsic parameter and camera extrinsic parameter of the target camera are obtained based on target camera calibration performed using the plane target in the multiple second posture images; determining the information of reflected light of each second posture image in the target camera coordinate system according to each second highlight point coordinate and a preset third optical center coordinate, wherein the third optical center coordinate represents the coordinate of the optical center of the target camera in the target camera coordinate system.

[0126] In a possible implementation, based on the target sphere center coordinates and the information of each reflected light, the information of the incident light corresponding to each reflected light in the target camera coordinate system is determined, including: using the camera extrinsic parameters of the target camera to perform coordinate transformation on the target sphere center coordinates to obtain the transformed sphere center coordinates in the target camera coordinate system; based on the sphere radius of the spherical target and the second sphere center coordinates corresponding to the spherical target in each second posture image in the target camera coordinate system, the intersection coordinates of the target intersection points on each reflected light in the target camera coordinate system are determined, and each target intersection point is the intersection point of the corresponding reflected light and the corresponding spherical target; according to the sphere radius, the transformed sphere center coordinates and the intersection coordinates, the information of the normal corresponding to each reflected light in the target camera coordinate system is determined; according to the information of each reflected light and the information of its corresponding normal, the information of the incident light corresponding to each reflected light is determined.

[0127] In a possible implementation, determining the radiation intensity distribution of the light source to be calibrated based on the multiple second posture images and the light source coordinates includes: sampling the plane targets in each of the second posture images according to a preset sampling rule to obtain the first sampling point coordinates of each sampling point in the second image coordinate system; performing coordinate transformation on each of the first sampling point coordinates using the camera intrinsic parameters of the target camera to obtain the corresponding second sampling point coordinates in the target camera coordinate system; obtaining the normal information corresponding to each of the second sampling point coordinates in the target camera coordinate system according to the light source coordinates, the second spherical center coordinates corresponding to the spherical target in each of the second posture images in the target camera coordinate system, and the second sampling point coordinates; determining a preset number of spherical harmonic basis functions in the spherical harmonic function and the spherical harmonic coefficients corresponding to each of the spherical harmonic basis functions based on the information of each of the second sampling point coordinates and the corresponding normals and the light source coordinates; and determining the radiation intensity distribution of the light source to be calibrated based on the light source coordinates, all spherical harmonic basis functions, and all spherical harmonic coefficients.

[0128] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0129] The present disclosure also provides an image reconstruction method, including performing image reconstruction using a photometric stereo system calibrated by the above light source calibration method. In some embodiments, the functions or modules included in the image reconstruction method provided by the present disclosure can be used to execute the method described in the above method embodiment, and its specific implementation can refer to the description of the above light source calibration method embodiment, which will not be repeated here for the sake of brevity.

[0130] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0131] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0132] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0133] Fig. 20 1900 is a block diagram of a light source calibration device according to an embodiment of the present disclosure. For example, the device 1900 may be provided as a server or a terminal device. Fig. 20 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0134] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0135] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.

[0136] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0137] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0138] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0139] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0140] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0141] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0142] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0143] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0144] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A light source calibration method, characterized in that: include: Under preset uniform illumination conditions, using a first camera to acquire a plurality of first posture images of a combined target, wherein the combined target in each of the first posture images has a different first posture, the combined target includes a plane target and a spherical target with fixed relative coordinates, and determining the target sphere center coordinates of the spherical target in a preset target coordinate system based on the plurality of first posture images; Under target illumination conditions formed by the light source to be calibrated, a plurality of second posture images of the combined target are acquired by using a target camera, and information of reflected light of each second posture image in a target camera coordinate system is determined, wherein the combined target has a different second posture in each second posture image; Based on the coordinates of the target sphere center and the information of each reflected light, determine the information of the incident light corresponding to each reflected light in the target camera coordinate system, and determine the light source coordinates of the light source to be calibrated in the target camera coordinate system according to the information of all incident light rays; The radiation intensity distribution of the light source to be calibrated is determined based on the multiple second posture images and the light source coordinates.

2. The method according to claim 1, characterized in that: Determining the target sphere center coordinates of the spherical target in a preset target coordinate system based on the multiple first posture images includes: Performing edge detection on the spherical target in each of the first posture images to obtain first edge point coordinates of each edge point of the spherical target in each of the first posture images in a first image coordinate system; Performing coordinate transformation on the coordinates of each of the first edge points using the camera intrinsic parameters of the first camera to obtain the coordinates of each of the second edge points corresponding to the first camera coordinate system, wherein the camera intrinsic parameters and the camera extrinsic parameters of the first camera are obtained based on first camera calibration performed using a plane target in the plurality of first posture images; Determine the first tangent point coordinates of the tangent point corresponding to each of the edge points in the first camera coordinate system based on the coordinates of each of the second edge points, the first spherical center coordinates corresponding to the spherical target in each of the first posture images in the first camera coordinate system, the spherical radius of the spherical target, and the preset first optical center coordinates, wherein the first optical center coordinates represent the coordinates of the optical center of the first camera in the first camera coordinate system; Using the camera extrinsic parameters of the first camera, coordinate transformation is performed on the first optical center coordinates, the second edge point coordinates, and the first tangent point coordinates to obtain the corresponding second optical center coordinates, the third edge point coordinates, and the second tangent point coordinates in the target coordinate system; The target spherical center coordinates of the spherical target in the target coordinate system are determined based on the second optical center coordinates and the second tangent point coordinates.

3. The method according to claim 2, characterized in that Determining the target sphere center coordinates of the spherical target in the target coordinate system based on the second optical center coordinates and each of the second tangent point coordinates includes: Based on the second optical center coordinates and the coordinates of each second tangent point, a distance function is constructed that changes with the coordinates of the center of the spherical target in the target coordinate system, wherein the distance function represents the distance between the center of the spherical target and the tangent point of the spherical target; According to the difference between the distance function and the spherical radius of the spherical target, an objective function that changes with the coordinates of the center of the spherical target in the target coordinate system is constructed, and the objective function is minimized to obtain the target center coordinates of the spherical target in the target coordinate system.

4. The method according to any one of claims 1 to 3, characterized in that The first camera and the target camera are the same camera.

5. The method according to claim 1, characterized in that Determining information of reflected light of each of the second posture images in the target camera coordinate system includes: Perform highlight area detection on each of the second posture images to obtain a highlight area in the corresponding second posture image, where the highlight area includes at least one pixel point; Processing the pixel points of the highlight area in each of the second posture images to obtain a corresponding highlight point, and determining the first highlight point coordinates of each of the highlight points in a second image coordinate system, wherein the second image coordinate system corresponds to the imaging plane of the target camera; Performing coordinate transformation on the coordinates of each of the first highlight points using the camera intrinsic parameters of the target camera to obtain the corresponding coordinates of each of the second highlight points in the target camera coordinate system, wherein the camera intrinsic parameters and camera extrinsic parameters of the target camera are obtained based on target camera calibration performed using a plane target in the plurality of second posture images; According to each of the second highlight point coordinates and the preset third optical center coordinates, the information of the reflected light of each of the second posture images in the target camera coordinate system is determined, wherein the third optical center coordinates represent the coordinates of the optical center of the target camera in the target camera coordinate system.

6. The method according to claim 1 or 5, characterized in that: Based on the target sphere center coordinates and the information of each reflected light ray, determining the information of the incident light ray corresponding to each reflected light ray in the target camera coordinate system, including: Performing coordinate transformation on the target sphere center coordinates using the camera extrinsic parameters of the target camera to obtain transformed sphere center coordinates in the target camera coordinate system; Based on the spherical radius of the spherical target and the second spherical center coordinates corresponding to the spherical target in each of the second posture images in the target camera coordinate system, determining the intersection coordinates of the target intersection points on each of the reflected light rays in the target camera coordinate system, each of the target intersection points being the intersection points of the corresponding reflected light rays and the corresponding spherical target; Determining information of a normal corresponding to each of the reflected light rays in the target camera coordinate system according to the sphere radius, the transformed sphere center coordinates and the intersection coordinates; According to the information of each reflected light ray and the information of its corresponding normal line, the information of the incident light ray corresponding to each reflected light ray is determined.

7. The method according to claim 1 or 5, characterized in that: Determining the radiation intensity distribution of the light source to be calibrated based on the plurality of second posture images and the light source coordinates includes: According to a preset sampling rule, sampling is performed on the plane targets in each of the second posture images to obtain the first sampling point coordinates of each sampling point in the second image coordinate system; Performing coordinate transformation on the coordinates of each of the first sampling points using the camera intrinsic parameters of the target camera to obtain corresponding coordinates of the second sampling points in the target camera coordinate system; Obtaining information on a normal corresponding to each of the second sampling point coordinates in the target camera coordinate system according to the light source coordinates, the second sphere center coordinates corresponding to the spherical target in each of the second posture images in the target camera coordinate system, and the second sampling point coordinates; Based on the information of the coordinates of each of the second sampling points and the normal lines corresponding thereto, and the coordinates of the light source, determining a preset number of spherical harmonic basis functions in the spherical harmonic function and spherical harmonic coefficients corresponding to each of the spherical harmonic basis functions; Based on the light source coordinates, all spherical harmonic basis functions, and all spherical harmonic coefficients, the radiation intensity distribution of the light source to be calibrated is determined.

8. A light source calibration device, characterized in that: include: A spherical center coordinate determination module is used to obtain a plurality of first posture images of a combined target using a first camera under preset uniform illumination conditions, wherein the combined target has a different first posture in each of the first posture images, and the combined target includes a plane target and a spherical target with fixed relative coordinates, and determine the target spherical center coordinates of the spherical target in a preset target coordinate system based on the plurality of first posture images; a light information determination module, configured to obtain, under target illumination conditions formed by a light source to be calibrated, a plurality of second posture images of the combined target using a target camera, and determine information of reflected light of each of the second posture images in a target camera coordinate system, wherein the combined target has a different second posture in each of the second posture images; A light source coordinate determination module, used to determine the information of the incident light corresponding to each reflected light in the target camera coordinate system based on the target sphere center coordinates and the information of each reflected light, and determine the light source coordinates of the light source to be calibrated in the target camera coordinate system according to the information of all incident light rays; A light source intensity distribution determination module is used to determine the radiation intensity distribution of the light source to be calibrated based on the multiple second posture images and the light source coordinates.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 7 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images