Light source calibration method, related equipment and storage medium in light field acquisition system

Abstract

The present disclosure relates to a method for calibrating a light source in a light field acquisition system, related equipment, and storage media. The method includes: acquiring multiple first images synchronously captured of a reflective object based on different viewing angles under a first ambient brightness; obtaining the first ambient brightness based on incident light from a target light source in the light field acquisition system, where the target light source is any light source in the light field acquisition system; determining the image position information of the target light source in each first image based on the position of the reflected light spot in each first image; determining the first reflection position information and first incident light direction information corresponding to each first image based on the spatial position information of the reflective object and the above-mentioned image position information of the target light source; and determining the spatial position information of the target light source based on the first reflection position information and first incident light direction information corresponding to each first image in the multiple first images. The present disclosure improves the accuracy of light source calibration and reduces the precision requirements for light source installation by individually calibrating the light source.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a light source calibration method, related equipment, and storage medium in a light field acquisition system. Background Art

[0002] The Light Stage is a spherical, high-precision, three-dimensional information acquisition device based on high-frequency, controllable lighting and exposure. It includes a controllable light system and a synchronized camera array. The controllable light system includes multiple independently controllable light sources evenly mounted on a spherical bracket. The synchronized camera array is tended to be distributed on the equator of the bracket or on the entire sphere and aimed at the center of the sphere.

[0003] In practical applications, the object being photographed is placed at the center of a sphere. A controllable light system emits specific lighting patterns, and multiple cameras in a synchronized camera array simultaneously capture images of the object under these lighting patterns from their respective perspectives. This allows the object's 3D geometry and material information, such as reflectivity, to be calculated. This geometry and material information is often used for virtual human modeling or to provide high-precision training data for other basic algorithms, such as face and object modeling and editing. Therefore, the accuracy of the light emitted by the light source in the controllable light system (such as brightness and polarization state) is crucial to the accuracy of light field acquisition information, necessitating calibration of the light source in the light field acquisition system.

[0004] In related technologies, based on the assumption that the relative positions between light sources are the designed positions, the light source positions in the light field acquisition system are calibrated as a whole, resulting in poor accuracy in light source calibration and high requirements for light source installation precision, which increases installation difficulty and cost. Summary of the Invention

[0005] The present disclosure provides a method, apparatus, device, and storage medium for calibrating a light source in a light field acquisition system to at least address the problems of poor accuracy, difficulty in installation, and high cost in related art light source calibration. The technical solutions of the present disclosure are as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, a method for calibrating a light source in a light field acquisition system is provided, comprising:

[0007] Acquire a plurality of first images synchronously captured of a reflective object based on different viewing angles under a first ambient brightness; the first ambient brightness is obtained based on incident light from a target light source in a light field acquisition system, where the target light source is any light source in the light field acquisition system;

[0008] Determining image position information of the target light source in each first image based on the position of the reflected light spot in each first image;

[0009] determining first reflection position information and first incident light direction information corresponding to each first image based on the spatial position information of the reflecting object and the image position information of the target light source in each first image; the first reflection position information indicates the spatial position of a corresponding reflection point of the target light source on the reflecting object;

[0010] Based on the first reflection position information and the first incident light direction information corresponding to each first image in the plurality of first images, light source calibration information of the target light source is determined; the light source calibration information includes spatial position information of the target light source.

[0011] In an exemplary embodiment, determining the first reflection position information and the first incident light direction information corresponding to each first image based on the spatial position information of the reflecting object and the image position information of the target light source in each first image includes:

[0012] Determining a first ray corresponding to each first image based on image acquisition parameters of the viewing angle corresponding to each first image and information about the center position of the contour graphic corresponding to the reflective object in the first image; the first ray being a ray starting from a viewpoint corresponding to the viewing angle of the corresponding first image to the center position of the contour graphic corresponding to the reflective object in the first image;

[0013] Determining a second ray corresponding to each first image based on image acquisition parameters of the viewing angle corresponding to each first image and image position information of the target light source in the first image; the second ray is a ray starting from a viewpoint corresponding to the viewing angle of the corresponding first image to the position of the target light source in the first image;

[0014] Based on the spatial position information of the reflection object, the size information of the reflection object, and the first ray and the second ray corresponding to each first image, first reflection position information and first incident light direction information corresponding to each first image are determined.

[0015] In an exemplary embodiment, determining first reflection position information and first incident light direction information corresponding to each first image based on the spatial position information of the reflection object, the size information of the reflection object, and the first ray and the second ray corresponding to each first image includes:

[0016] determining first reflection position information corresponding to each first image based on the spatial position information of the reflection object, the size information of the reflection object, and the first ray and the second ray corresponding to each first image;

[0017] determining normal information of the reflection point corresponding to each first image based on the spatial position information of the reflection object and the first reflection position information corresponding to each first image;

[0018] Based on the normal information of the reflection point corresponding to each first image, the first reflection position information corresponding to the first image, and the second ray corresponding to the first image, the first incident light direction information corresponding to each first image is determined.

[0019] In an exemplary embodiment, determining the light source calibration information of the target light source based on the first reflection position information and the first incident light direction information corresponding to each first image in the plurality of first images includes:

[0020] determining a plurality of incident light rays corresponding to the target light source based on first reflection position information and first incident light direction information corresponding to each first image in the plurality of first images;

[0021] Determine a target space point closest to a plurality of incident light rays corresponding to the target light source;

[0022] The spatial position information of the target spatial point is used as the spatial position information of the target light source.

[0023] In an exemplary embodiment, the target light source includes a plurality of asymmetric light-emitting elements, and determining the image position information of the target light source in each first image based on the position of the reflected light spot in each first image includes:

[0024] For each first image, determining position information of a reflected light spot corresponding to each light-emitting element in the first image to obtain a plurality of light spot positions;

[0025] An average value of position information corresponding to each pixel in the multiple light spot positions is determined to obtain image position information of the target light source in the first image.

[0026] In an exemplary embodiment, the method further comprises:

[0027] For each of the light-emitting elements, based on the position information of the reflection spot corresponding to the light-emitting element in each of the first images, determining second reflection position information and second incident light direction information corresponding to each of the first images; the second reflection position information indicates the spatial position of the corresponding reflection point of the corresponding light-emitting element on the reflective object;

[0028] Determining position offset information of each of the light-emitting elements relative to the center of the target light source;

[0029] Determining the posture information of the target light source based on the spatial position information of the target light source, the position offset information corresponding to each of the light-emitting elements, and the second reflection position information and second incident light direction information corresponding to each first image determined for each light-emitting element; the light source calibration information includes the posture information;

[0030] In which, under the posture indicated by the posture information, the sum of the distances from each of the light-emitting elements to multiple target incident light rays is minimum, and the multiple target incident light rays are obtained based on the second reflection position information and second incident light direction information corresponding to each first image determined for the corresponding light-emitting element.

[0031] In an exemplary embodiment, the method further comprises:

[0032] Based on the spatial position information of each light source in the light field acquisition system, segmenting the spherical surface of the spherical acquisition support of the light field acquisition system to obtain segmentation polygons corresponding to each light source;

[0033] Determining a relative brightness coefficient corresponding to each light source based on the area of ​​the segmentation polygon corresponding to each light source; wherein the relative brightness coefficient is positively correlated with the area of ​​the segmentation polygon;

[0034] Based on the product of the relative brightness coefficient corresponding to each light source and the target brightness parameter, the light source brightness information of each light source in the target lighting mode is obtained; wherein the light source calibration information also includes the light source brightness information, and the target lighting mode is the lighting mode corresponding to the target brightness parameter.

[0035] In an exemplary embodiment, before determining the first reflection position information and the first incident light direction information corresponding to each first image based on the spatial position information of the reflecting object and the image position information of the target light source in each first image, the method further includes:

[0036] Acquire multiple second images of the reflective object synchronously captured based on the different viewing angles under a second ambient brightness; the second ambient brightness is obtained based on incident light from multiple light sources in the light field acquisition system;

[0037] Determining subspace position information of the reflective object corresponding to each second image based on size information of the reflective object and center position information of a contour graphic corresponding to the reflective object in each second image;

[0038] Subspace position information that meets a preset condition is selected from the subspace position information of the reflection object corresponding to the multiple second images as the spatial position information of the reflection object.

[0039] In an exemplary embodiment, determining the subspace position information of the reflective object corresponding to each second image based on the size information of the reflective object and the center position information of the contour graphic corresponding to the reflective object in each second image includes:

[0040] Determining a third ray corresponding to each second image based on image acquisition parameters of the viewing angle corresponding to each second image and information about the center position of the contour graphic corresponding to the reflective object in the second image; the third ray being a ray starting from a viewpoint corresponding to the viewing angle of the corresponding second image to the center position of the contour graphic corresponding to the reflective object in the second image;

[0041] determining, based on image acquisition parameters of the viewing angle corresponding to each second image and a target contour point on the contour graph corresponding to the reflective object in the second image, a fourth ray corresponding to each second image; the fourth ray being a ray starting from a viewpoint corresponding to the viewing angle of the corresponding second image to the target contour point, the target contour point being any point on the corresponding contour graph;

[0042] Based on the size information of the reflective object and the third ray and the fourth ray corresponding to each second image, subspace position information of the reflective object corresponding to each second image is determined.

[0043] In an exemplary embodiment, selecting subspace position information that meets a preset condition from the subspace position information of the reflective object corresponding to the plurality of second images as the spatial position information of the reflective object includes:

[0044] For each second image, project the subspace position information of the reflection object corresponding to the subspace position information into the image coordinate system of the corresponding viewing angle of each other second image to obtain the image position corresponding to the subspace position information in each other second image;

[0045] determining, based on the distance between an image position corresponding to the subspace position information on each other second image and a center position of a contour graphic corresponding to the reflective object in the corresponding other second image, the number of other second images to which the subspace position information matches; wherein the distance corresponding to the other second images to which the subspace position information matches is less than a preset distance threshold;

[0046] The subspace position information with the largest number is used as the spatial position information of the reflection object.

[0047] In an exemplary embodiment, the method further comprises:

[0048] performing position correction processing based on the spatial position information of the reflecting object and the spatial position information corresponding to each light source in the light field acquisition system to obtain corrected spatial position information corresponding to the reflecting object and corrected spatial position information corresponding to each light source;

[0049] Among them, the cumulative distance obtained based on the corrected spatial position information corresponding to the reflecting object and the corrected spatial position information corresponding to each light source is the smallest, the cumulative distance is the sum of the target distances corresponding to each light source, and the target distance is the sum of the distances from the light source position indicated by the corrected spatial position information corresponding to the corresponding light source to the multiple corrected incident light rays corresponding to the light source, and the multiple corrected incident light rays are obtained by correcting the multiple incident light rays of the corresponding light source based on the corrected spatial position information corresponding to the reflecting object.

[0050] According to a second aspect of an embodiment of the present disclosure, a light source calibration device in a light field acquisition system is provided, comprising:

[0051] A first image acquisition unit is configured to acquire a plurality of first images of the reflective object synchronously captured at different viewing angles under a first ambient brightness, wherein the first ambient brightness is obtained based on incident light from a target light source in a light field acquisition system, wherein the target light source is any light source in the light field acquisition system;

[0052] a light source image position determining unit, configured to determine image position information of the target light source in each first image based on the position of the reflected light spot in each first image;

[0053] a first information determining unit configured to determine first reflection position information and first incident light direction information corresponding to each first image based on the spatial position information of the reflecting object and the image position information of the target light source in each first image; the first reflection position information indicates the spatial position of a corresponding reflection point of the target light source on the reflecting object;

[0054] The first calibration information determination unit is configured to determine the light source calibration information of the target light source based on the first reflection position information and the first incident light direction information corresponding to each first image in the multiple first images; the light source calibration information includes the spatial position information of the target light source.

[0055] In an exemplary embodiment, the first information determining unit includes:

[0056] A first ray determining unit is configured to determine a first ray corresponding to each first image based on image acquisition parameters of a viewing angle corresponding to each first image and information about a center position of a contour graphic corresponding to the reflective object in the first image; the first ray being a ray starting from a viewpoint corresponding to the viewing angle of the corresponding first image to a center position of the contour graphic corresponding to the reflective object in the first image;

[0057] a second ray determining unit configured to determine a second ray corresponding to each first image based on image acquisition parameters of a viewing angle corresponding to each first image and image position information of the target light source in the first image; the second ray being a ray starting from a viewpoint corresponding to the viewing angle of the corresponding first image to the position of the target light source in the first image;

[0058] The second information determination unit is configured to determine the first reflection position information and the first incident light direction information corresponding to each first image based on the spatial position information of the reflection object, the size information of the reflection object, and the first ray and the second ray corresponding to each first image.

[0059] In an exemplary embodiment, the second information determining unit includes:

[0060] a first reflection position information determining unit configured to determine first reflection position information corresponding to each first image based on the spatial position information of the reflection object, the size information of the reflection object, and the first ray and the second ray corresponding to each first image;

[0061] a normal information determining unit configured to determine normal information of the reflection point corresponding to each first image based on the spatial position information of the reflection object and the first reflection position information corresponding to each first image;

[0062] The first incident light direction information determination unit is configured to determine the first incident light direction information corresponding to each first image based on the normal information of the reflection point corresponding to each first image, the first reflection position information corresponding to the first image, and the second ray corresponding to the first image.

[0063] In an exemplary embodiment, the first calibration information determining unit includes:

[0064] an incident light ray determining unit, configured to determine a plurality of incident light rays corresponding to the target light source based on first reflection position information and first incident light direction information corresponding to each of the plurality of first images;

[0065] The target space point determination unit is configured to determine the target space point closest to the multiple incident light rays corresponding to the target light source; and use the spatial position information of the target space point as the spatial position information of the target light source.

[0066] In an exemplary embodiment, the target light source includes a plurality of asymmetric light-emitting elements, and the light source image position determination unit includes:

[0067] a light spot position determining unit configured to determine, for each first image, position information of a reflected light spot corresponding to each light emitting element in the first image, to obtain a plurality of light spot positions;

[0068] The image position determination subunit is configured to determine an average value of position information corresponding to each pixel in the multiple light spot positions to obtain image position information of the target light source in the first image.

[0069] In an exemplary embodiment, the apparatus further comprises:

[0070] a third information determining unit configured to determine, for each of the light-emitting elements, second reflection position information and second incident light direction information corresponding to each first image based on position information of a reflection spot corresponding to the light-emitting element in each first image; wherein the second reflection position information indicates a spatial position of a corresponding reflection point of the corresponding light-emitting element on the reflective object;

[0071] a position offset information determining unit, configured to determine position offset information of each of the light-emitting elements relative to the center of the target light source;

[0072] a posture information determining unit configured to determine the posture information of the target light source based on the spatial position information of the target light source, the position offset information corresponding to each of the light-emitting elements, and the second reflection position information and the second incident light direction information corresponding to each first image determined for each light-emitting element; the light source calibration information includes the posture information;

[0073] In which, under the posture indicated by the posture information, the sum of the distances from each of the light-emitting elements to multiple target incident light rays is minimum, and the multiple target incident light rays are obtained based on the second reflection position information and second incident light direction information corresponding to each first image determined for the corresponding light-emitting element.

[0074] In an exemplary embodiment, the apparatus further comprises:

[0075] a spherical surface segmentation unit configured to segment the spherical surface of the spherical acquisition support of the light field acquisition system based on the spatial position information of each light source in the light field acquisition system, and obtain a segmentation polygon corresponding to each light source;

[0076] a relative brightness coefficient determining unit configured to determine a relative brightness coefficient corresponding to each light source based on the area of ​​the segmentation polygon corresponding to each light source; wherein the relative brightness coefficient is positively correlated with the area of ​​the segmentation polygon;

[0077] The light source brightness information determination unit is configured to perform a multiplication of the relative brightness coefficient corresponding to each light source and the target brightness parameter to obtain the light source brightness information of each light source under the target lighting mode; wherein the light source calibration information also includes the light source brightness information, and the target lighting mode is the lighting mode corresponding to the target brightness parameter.

[0078] In an exemplary embodiment, the apparatus further comprises:

[0079] A second image acquisition unit is configured to acquire a plurality of second images of the reflective object synchronously captured based on the different viewing angles under a second ambient brightness, wherein the second ambient brightness is obtained based on incident light from a plurality of light sources in the light field acquisition system;

[0080] a first reflecting object position determining unit configured to determine subspace position information of the reflecting object corresponding to each second image based on size information of the reflecting object and center position information of a contour graphic corresponding to the reflecting object in each second image;

[0081] The second reflecting object position determining unit is configured to select subspace position information that meets a preset condition from the subspace position information of the reflecting object corresponding to the multiple second images as the spatial position information of the reflecting object.

[0082] In an exemplary embodiment, the first reflecting object position determining unit includes:

[0083] a third ray determining unit configured to determine a third ray corresponding to each second image based on image acquisition parameters corresponding to the viewing angle of each second image and information about a center position of a contour graphic corresponding to the reflective object in the second image; the third ray being a ray starting from a viewpoint corresponding to the viewing angle of the corresponding second image to a center position of the contour graphic corresponding to the reflective object in the second image;

[0084] a fourth ray determining unit configured to determine a fourth ray corresponding to each second image based on image acquisition parameters of the viewing angle corresponding to each second image and a target contour point on the contour graph corresponding to the reflective object in the second image; the fourth ray being a ray from the viewpoint corresponding to the viewing angle of the corresponding second image to the target contour point, the target contour point being any point on the corresponding contour graph;

[0085] The reflection object subspace position information determining unit is configured to determine the subspace position information of the reflection object corresponding to each second image based on the size information of the reflection object and the third ray and the fourth ray corresponding to each second image.

[0086] In an exemplary embodiment, the second reflecting object position determining unit includes:

[0087] a spatial position projection unit configured to project the subspace position information of the reflective object corresponding to each second image into an image coordinate system of a corresponding perspective of each other second image, thereby obtaining an image position corresponding to the subspace position information in each other second image;

[0088] a matching image number determining unit configured to determine the number of other second images to which the subspace position information matches based on the distance between an image position corresponding to the subspace position information on each other second image and a center position of a contour figure corresponding to the reflective object in the corresponding other second image; wherein the distance corresponding to the other second images to which the subspace position information matches is less than a preset distance threshold;

[0089] The reflecting object position determining subunit is configured to use the sub-space position information with the largest number as the spatial position information of the reflecting object.

[0090] In an exemplary embodiment, the apparatus further comprises:

[0091] a position correction unit configured to perform position correction processing based on the spatial position information of the reflecting object and the spatial position information corresponding to each light source in the light field acquisition system, to obtain corrected spatial position information corresponding to the reflecting object and corrected spatial position information corresponding to each light source;

[0092] Among them, the cumulative distance obtained based on the corrected spatial position information corresponding to the reflecting object and the corrected spatial position information corresponding to each light source is the smallest, the cumulative distance is the sum of the target distances corresponding to each light source, and the target distance is the sum of the distances from the light source position indicated by the corrected spatial position information corresponding to the corresponding light source to the multiple corrected incident light rays corresponding to the light source, and the multiple corrected incident light rays are obtained by correcting the multiple incident light rays of the corresponding light source based on the corrected spatial position information corresponding to the reflecting object.

[0093] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0094] processor;

[0095] a memory for storing instructions executable by the processor;

[0096] The processor is configured to execute the instructions to implement the light source calibration method in the light field acquisition system of the first aspect.

[0097] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the calibration method of the light source in the light field acquisition system of the first aspect described above.

[0098] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the calibration method of the light source in the light field acquisition system of the first aspect.

[0099] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0100] By synchronously capturing multiple first images of a reflective object based on different viewing angles under a first ambient brightness formed by a single light source in a light field acquisition system, and then determining, for each first image, the image position information of the single light source in the first image based on the position of the reflected light spot in the first image, and determining the first reflection position information and the first incident light direction information corresponding to the first image based on the spatial position information of the reflective object and the image position information of the single light source in the first image, thereby determining the spatial position information of the single light source based on the above information corresponding to each first image in the multiple first images, individual calibration of each light source in the light field acquisition system is achieved, the accuracy and flexibility of light source calibration are improved, and the precision requirements, installation difficulty and cost of light source installation are significantly reduced.

[0101] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0103] Figure 1 The present invention is a flowchart showing a method for calibrating a light source in a light field acquisition system according to an exemplary embodiment.

[0104] Figure 2 The figure is a flow chart of another method for calibrating a light source in a light field acquisition system according to an exemplary embodiment.

[0105] Figure 3 The present invention is a flowchart showing a method for determining the posture information of a light source in a light field acquisition system according to an exemplary embodiment.

[0106] Figure 4 The present invention is a flowchart showing a method for determining light source brightness information of a light source in a light field acquisition system according to an exemplary embodiment.

[0107] Figure 5 The present invention is a block diagram showing a device for calibrating a light source in a light field acquisition system according to an exemplary embodiment.

[0108] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0109] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0110] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0111] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0112] Light field acquisition systems are widely used in film shooting, video special effects production, 3D face scanning and other fields. The accuracy of the light emitted by the light source in the light field acquisition system (such as brightness, polarization state, etc.) is very important to the accuracy of the light field acquisition information (such as 3D geometric information and material information of the photographed object). Therefore, it is necessary to calibrate the light source parameters of the light source in the light field acquisition system. The light source parameters can include the installation position of the light source, the posture of the light source (such as direction), and the brightness of the light source.

[0113] In related technologies, when calibrating the light source in a light field acquisition system, the overall position of the light source is directly calibrated based on the assumption that the light source meets the designed position and brightness. However, due to issues such as light source tolerance and installation accuracy, there are inevitable errors between the actual light source parameters (such as position and posture) and the designed parameters. This leads to poor calibration accuracy in related technologies, which in turn reduces the acquisition accuracy of the light field acquisition system. Furthermore, the requirements for the light source and installation accuracy of the equipment are very high, resulting in high installation difficulty and high cost. Furthermore, related technologies also have high requirements for light source position and camera parameters, resulting in poor universality and lack of stability.

[0114] In view of the above problems, an embodiment of the present disclosure provides a method for calibrating light sources in a light field acquisition system. This method independently calibrates the spatial position of each light source in the light field acquisition system after installation. While improving the accuracy and flexibility of light source calibration, it can significantly reduce the installation precision requirements of the light source, thereby reducing the installation difficulty and cost, and greatly improving the stability and universality of the calibration.

[0115] The method for calibrating a light source in a light field acquisition system in an embodiment of the present disclosure can be performed by a calibration device for the light source in the light field acquisition system. The calibration device for the light source in the light field acquisition system can be configured in an electronic device, which can be a terminal or a server. The terminal can be, but is not limited to, a smartphone, a tablet computer, a laptop computer, a desktop computer, etc. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0116] First, the hardware structure of the light field acquisition system is introduced, including a controllable light system and a synchronized camera array. The controllable light system includes multiple independently controllable light sources, which are evenly mounted on a spherical bracket. Each light source includes multiple asymmetrically distributed light-emitting elements, such as white light LEDs. The synchronized camera array is tended to be distributed on the equator of the spherical bracket or on the entire sphere and is aligned with the center of the spherical bracket. The object to be photographed is placed near the center of the spherical bracket. In the light source calibration scenario, the object to be photographed is a regularly shaped object made of reflective material, which is referred to as a reflective object in the disclosed embodiment, such as a chrome-plated metal reflective ball. In the actual light field information acquisition scenario, the object to be photographed is an object corresponding to the geometric information, material information, and other information that need to be collected.

[0117] Figure 1 FIG. 1 is a flow chart showing a method for calibrating a light source in a light field acquisition system according to an exemplary embodiment. Figure 1 As shown, taking the execution subject as an electronic device as an example, the following steps are included.

[0118] In step S101 , a plurality of first images of a reflective object captured synchronously based on different viewing angles under a first ambient brightness are acquired.

[0119] The first ambient brightness is obtained based on incident light from a target light source in the light field acquisition system, and the target light source is any light source in the light field acquisition system.

[0120] A reflective object is a regularly shaped object made of reflective material, such as a chrome-plated metal reflective ball. In the light source calibration scenario, the reflective object is placed close to the center of the spherical support of the light field acquisition system.

[0121] The different perspectives correspond to the individual cameras in the synchronized camera array of the light field acquisition system. That is, each camera synchronously captures the reflective object located at the center of the spherical support sphere from its own perspective, thereby obtaining a first image corresponding to each perspective (i.e., each camera). The first images corresponding to all perspectives (i.e., each camera in the camera array) constitute the multiple first images corresponding to the target light source that will be processed later in the embodiments of the present disclosure.

[0122] It should be noted that a preset number of simultaneous captures can be performed at each viewing angle. This preset number can be set based on actual experience, for example, once per viewing angle, or multiple times per viewing angle. Taking simultaneous capture at each viewing angle as an example, assuming the camera array has 24 cameras, 24 first images can be obtained by synchronously capturing the reflective object at different viewing angles under a first ambient brightness.

[0123] In a specific implementation, one light source in the light field acquisition system is turned on each time, and each camera in the camera array is triggered to synchronously capture the reflective ball located near the center of the spherical acquisition frame, so that under the ambient brightness formed by each light source, multiple first images corresponding to the light source can be obtained by the camera array.

[0124] It should be noted that the image acquisition parameters corresponding to different viewpoints (including the camera's intrinsic parameters, extrinsic parameters, and distortion parameters) are determined by pre-calibrating the camera array. That is, before the above step S101, the method may further include calibrating the cameras in the light field acquisition system to determine the intrinsic parameters, extrinsic parameters, and distortion parameters of each camera in the camera array, and thereby obtain the image acquisition parameters corresponding to each viewpoint in the light field acquisition system.

[0125] The cameras in the light field acquisition system can be calibrated using any camera parameter calibration method in the related art. For example, calibrating the intrinsic parameters, extrinsic parameters, and distortion parameters of each camera in the camera array can include the following steps:

[0126] 1) Initial intrinsic parameter calibration: Each camera takes several checkerboard images and uses the Zhang Zhengyou calibration method to obtain the initial intrinsic parameters of each camera;

[0127] 2) Extraction and matching of calibration sphere corner points: The cameras in the camera array synchronously shoot the calibration sphere several times, and adjust the position and rotation angle of the calibration sphere for each shot. Then, edge detection is performed on the captured image, and the circular outline of the calibration sphere boundary in the image is detected and determined by a random sampling consensus algorithm. That is, 3 edge points are randomly selected, the circle position is calculated, and the position with the most matches is selected as the circular outline of the calibration sphere boundary. After that, the corner points in the image are extracted, and invalid corner points are removed based on the range of the calibration sphere and the features of the calibration image. The corner points in the images taken by different cameras are matched by a random sampling consensus algorithm. That is, for any pair of two cameras, two corner points are randomly selected, and the initial internal parameters obtained by calibration in step 1) are used to solve the PNP problem to obtain the camera external parameters, and other corner points are used for verification. The camera external parameters with the most matches are selected as the result;

[0128] 3) Use the bundle adjustment method to solve high-precision camera intrinsic parameters, extrinsic parameters, and distortion parameters: input the filtered corner point positions and jointly optimize all camera intrinsic and extrinsic parameters and distortion parameters.

[0129] It is understandable that the above is only an example of calibration of camera intrinsic parameters, extrinsic parameters and distortion parameters. Other camera parameter calibration methods can also be used in actual applications. The embodiment of the present disclosure does not specifically limit the camera parameter calibration method.

[0130] In step S103 , based on the position of the reflected light spot in each first image, the image position information of the target light source in each first image is determined.

[0131] Specifically, for each of the plurality of first images corresponding to the target light source, image position information of the target light source in the first image is determined based on the position of the reflected light spot in the first image. The image position information of the target light source in each first image indicates the position of the target light source in the first image.

[0132] In practical applications, each light source in the light field acquisition system includes multiple asymmetric light-emitting elements, such as multiple asymmetric LED lamp beads. Each asymmetric light-emitting element corresponds to a reflection spot on the reflective object (such as a reflective ball), so that each asymmetric light-emitting element corresponds to the position of a corresponding reflection spot in the first image. Then, the image position information of the target light source in each first image can be determined based on the positions of the reflection spots corresponding to the multiple asymmetric light-emitting elements it contains in each first image.

[0133] Based on this, in an exemplary embodiment, the above step 103 may include:

[0134] For each first image corresponding to the target light source, determine the position information of the reflected light spot corresponding to each light-emitting element contained therein in the first image to obtain multiple light spot positions;

[0135] An average value of position information corresponding to each pixel in the plurality of light spot positions is determined to obtain image position information of the target light source in the first image.

[0136] In a specific implementation, for each first image corresponding to the target light source, after the first image is binarized, each light-emitting element is an independent light spot in the first image, and then a plurality of light spot positions can be obtained. The position information of the pixels contained in each light spot position in the multiple light spot positions is then summed up to calculate the average value, and the average value is used as the image position information of the target light source in the first image.

[0137] For example, the target light source includes four asymmetric LED lamp beads. For the first image A corresponding to the target light source, after binarizing the first image A, four light spot positions can be obtained. The average position of the pixels contained in these four light spot positions is calculated, so that the image position information of the target light source in the first image A can be obtained.

[0138] In the above embodiment, considering that the target light source includes multiple asymmetric light-emitting elements, the image position information of the target light source in each first image is determined by the position of the reflected light spot of each light-emitting element in each first image. This allows the position of the target light source in each first image to be accurately obtained, thereby facilitating improved accuracy in calibrating the position of the target light source. Furthermore, the first image corresponding to the target light source only contains the position of the reflected light spot of the light-emitting element on the target light source, and does not contain the reflected light spots of the light-emitting elements on other light sources. This ensures the efficiency and accuracy of determining the position of the reflected light spot of the light-emitting element in the target light source from the first image, thereby improving the universality and robustness of the disclosed embodiment.

[0139] In practical applications, in order to improve the accuracy of calibration, before the above step S103, the first image captured at the corresponding viewing angle can be dedistorted based on the distortion parameters in the image acquisition parameters corresponding to each viewing angle, so that the subsequent calibration steps can be implemented on the dedistorted image.

[0140] In step S105 , first reflection position information and first incident light direction information corresponding to each first image are determined based on the spatial position information of the reflection object and the image position information of the target light source in each first image.

[0141] The first reflection position information indicates the spatial position of the corresponding reflection point of the target light source on the reflection object, and the first incident light direction information indicates the direction of the incident light of the target light source.

[0142] Among them, the spatial position information of the reflecting object indicates the spatial position of the reflecting object in the light field acquisition system. The spatial position information of the reflecting object can be determined by multiple second images of the reflecting object synchronously captured based on different perspectives under a second ambient brightness, wherein the second ambient brightness is obtained based on the incident light of multiple light sources in the light field acquisition system. The multiple light sources can be all or part of the light sources in the light field acquisition system, that is, when determining the spatial position information of the reflecting object, all or part of the light sources in the light field acquisition system can be turned on, and then the camera array can be triggered to shoot synchronously, thereby obtaining multiple second images of the reflecting object synchronously captured based on different perspectives.

[0143] That is, before the above step S105, the method may further include determining the spatial position information of the reflecting object. The determination of the spatial position information of the reflecting object is described in detail below. Figure 2 As shown, determining the spatial position information of the reflecting object may include:

[0144] In step S201 , a plurality of second images of a reflective object captured synchronously based on different viewing angles under a second ambient brightness are acquired.

[0145] The second ambient brightness is obtained based on incident light from multiple light sources in the light field acquisition system.

[0146] In practical applications, in order to improve the accuracy of calibration, after acquiring multiple second images, the second images captured at the corresponding viewing angles can be dedistorted based on the distortion parameters in the image acquisition parameters corresponding to each viewing angle, so that the subsequent calibration steps can be implemented on the dedistorted second images.

[0147] In step S203 , based on the size information of the reflective object and the center position information of the contour graphic corresponding to the reflective object in each second image, the subspace position information of the reflective object corresponding to each second image is determined.

[0148] The size information of the reflective object may be set based on the shape of the reflective object. For example, for a reflective ball, the size information of the reflective object is the radius of the reflective ball.

[0149] The center position information of the contour graphic corresponding to the reflective object in each second image indicates the center position of the contour graphic corresponding to the reflective object in the corresponding second image. In practical applications, the contour graphic corresponding to the reflective object in each second image needs to be determined by detecting the second image. For example, if the reflective object is a reflective ball and the detection method is the Hough transform in edge detection, a circle in each second image can be detected through the Hough transform. The circle can be used as the contour graphic corresponding to the reflective ball in the corresponding second image, and the center position of the contour graphic corresponding to the reflective ball in the second image is the center position of the circle in the second image.

[0150] The subspace position information of the reflection object corresponding to each second image indicates the spatial position of the reflection object determined based on each second image. For example, when the reflection object is a reflection ball, the spatial coordinates of the center of the reflection ball can be determined based on each second image.

[0151] In an exemplary embodiment, the above step S203 may include:

[0152] determining a third ray corresponding to each second image based on image acquisition parameters of a viewing angle corresponding to each second image and central position information of a contour graphic corresponding to the reflective object in the second image;

[0153] determining a fourth ray corresponding to each second image based on image acquisition parameters of a corresponding viewing angle of each second image and a target contour point on a contour graphic corresponding to a reflective object in the second image;

[0154] Based on the size information of the reflecting object and the third ray and the fourth ray corresponding to each second image, subspace position information of the reflecting object corresponding to each second image is determined.

[0155] Among them, the third ray refers to the ray starting from the viewpoint of the corresponding perspective of the corresponding second image to the center position of the contour figure corresponding to the reflected object in the second image; the fourth ray refers to the ray starting from the viewpoint of the corresponding perspective of the corresponding second image to the target contour point, which is any point on the corresponding contour figure.

[0156] In a specific implementation, it can be assumed that the camera in the light field acquisition system is a perspective projection, and the image acquisition parameters corresponding to the viewing angle of each second image include the camera intrinsic parameter matrix K, the camera extrinsic parameter rotation matrix R, and the camera extrinsic parameter translation matrix t. According to the projection relationship of the camera imaging, it can be known that the ray starting from the camera (i.e., the viewpoint) to the center position of the contour figure is It can be expressed as the following formula (1):

[0157]

[0158] Among them, p0 represents the center position information of the contour figure; ||·|| represents the modulus of the vector.

[0159] Similarly, the ray from the camera (i.e., viewpoint) to the target contour point on the contour graph It can be expressed as the following formula (2):

[0160]

[0161] Where p1 represents the position information of the target contour point on the contour graph; ||·|| represents the modulus of the vector.

[0162] For each second image, the above-mentioned and Then, based on the size information of the reflecting object and the above information corresponding to each second image and The subspace position information of the reflection object corresponding to each second image can be determined.

[0163] Taking the reflection object as a reflection ball as an example, the radius of the reflection ball is r0. According to the geometric relationship, the spatial coordinate p of the center of the reflection ball can be calculated based on the following formula (3) for each second image: c (i.e., subspace position information) is:

[0164]

[0165] Among them, Dot() represents the dot product calculation.

[0166] In practical applications, in order to improve the robustness of the calculation results, different target contour points can be selected multiple times for each second image. After each target contour point is selected, the corresponding target contour point can be determined based on the above formula (2). Based on the above formula (3), we can get the p corresponding to the target contour point c , and then calculate the target contour points corresponding to p selected multiple times c Average value The average value As the subspace position information of the reflection object corresponding to the second image.

[0167] In the above embodiment, two rays starting from the viewpoint of the corresponding perspective to the center position and contour point of the contour graphic corresponding to the reflected object in the second image are determined for each second image, and then the subspace position information of the reflected object corresponding to the second image is quickly and accurately obtained by combining these two rays and the size information of the reflected object.

[0168] In step S205 , subspace position information that meets a preset condition is selected from the subspace position information of the reflection object corresponding to the plurality of second images as the spatial position information of the reflection object.

[0169] It can be understood that, through the above step S203, the subspace position information of the reflection object corresponding to each second image can be obtained, that is, multiple spatial positions p of the reflection object can be obtained. c In order to obtain a more accurate spatial position of the reflecting object, the embodiment of the present disclosure determines a spatial position that meets a preset condition from multiple spatial positions of the reflecting object as the spatial position representation of the reflecting object, wherein the preset condition can be that the degree of matching between the subspace position information and the multiple second images meets a preset matching degree.

[0170] In an exemplary embodiment, the above step 205 may include:

[0171] For each second image, project the subspace position information of the reflection object corresponding to the subspace position information into the image coordinate system of the corresponding viewing angle of each other second image to obtain the image position corresponding to the subspace position information in each other second image;

[0172] determining the number of other second images to which the subspace position information matches based on the distance between the image position corresponding to the subspace position information on each other second image and the center position of the contour graphic corresponding to the reflective object in the corresponding other second image;

[0173] The subspace position information with the largest number is used as the spatial position information of the reflection object.

[0174] The other second images are second images other than the second image corresponding to the currently projected subspace position information. For example, if there are three second images {A, B, C}, if the current projection is based on the subspace position information corresponding to A, then the other second images include B and C. If the current projection is based on the subspace position information corresponding to B, then the other second images include A and C. If the current projection is based on the subspace position information corresponding to C, then the other second images include A and B.

[0175] The distance corresponding to the other matched second images is smaller than a preset distance threshold, and the preset distance threshold can be set according to actual needs.

[0176] Assume that the subspace position information of the reflection object is p c , when projecting the subspace position information into the image coordinate system of the corresponding perspective of each other second image, it can be achieved based on the following formula (4):

[0177] o c =K(Rp c +t) (4)

[0178] Among them, K, R, and t represent the image acquisition parameters of the corresponding viewing angle of the other second image projected, namely, the camera intrinsic parameter matrix K, the camera extrinsic parameter rotation matrix R, and the camera extrinsic parameter translation matrix t; c Indicates p c The corresponding image position in the other projected second image.

[0179] If p0 indicates the center position of the corresponding contour figure of the reflection object in the other second image projected, then o can be calculated. c The distance between p0 and p1 is compared with the preset distance threshold. If it is less than the preset distance threshold, it can be determined that p c Match to the other second image, and so on to determine the p c The number of other second images matched, and finally count the number of each p c The number of other second images that match each other is the p with the largest number c As the spatial position information of the reflection object.

[0180] To facilitate understanding of the above selection process, a specific example is given below. Assume that there are three second images with different perspectives, {A, B, C}, where the subspace position information of the reflection ball corresponding to A is p c A , the subspace position information of the reflection ball corresponding to B is p c B , the subspace position information of the reflection ball corresponding to C is p cC The image acquisition parameter corresponding to the viewing angle A is K A 、R A and t A , the image acquisition parameter corresponding to the viewing angle of B is K B 、R B and t B , the image acquisition parameter corresponding to the viewing angle C is K C 、R C and t C The center position information of the corresponding outline of the reflection ball in A is p0 A , the center position information of the contour figure corresponding to the reflection ball in B is p0 B , the center position information of the corresponding contour figure of the reflection ball in C is p0 C .So:

[0181] For p c A , p c A Projected into the image coordinate systems corresponding to B and C respectively to obtain o c A→B and o c A→C , where o c A→B =K B (R B p c A +t B ), o c A→C =K C (R C p c A +t c ); Then, calculate o respectively c A→B With p0 B The distance between c A→C With p0 C The distance between them is compared with the preset distance threshold respectively. Assume that o c A→B With p0 B The distance between them is less than the preset distance threshold, o c A→C With p0 c The distance between them is greater than the preset distance threshold, then it is determined that p c A Matched to B, that is, the number of other second images matched to it is 1.

[0182] Similarly, for p cB , p c B Projected into the image coordinate systems corresponding to A and C respectively to obtain o c B→A and o c B→C , where o c B→A =K A (R A p c B +t A ), o c B→C =K C (R C p c B +t C ); Then, calculate o respectively c B→A With p0 A The distance between c B→C With p0 C The distance between them is compared with the preset distance threshold respectively. Assume that o c B→A With p0 A The distance between them is less than the preset distance threshold, o c B→C With p0 C The distance between them is less than the preset distance threshold, then it is determined that p c B A and C are matched, that is, the number of other second images matched is 2.

[0183] Similarly, for p c C , p c C Projected into the image coordinate systems corresponding to A and B respectively to obtain o c C→A and o c C→B , where o c C→A =K A (R A p c C +t A ), o c C→B =K B (R B p c C +t B ); Then, calculate o respectively cC→A With p0 A The distance between c C→B With p0 B The distance between them is compared with the preset distance threshold respectively. Assuming that oc C→A With p0 A The distance between them is greater than the preset distance threshold, o c C→B With p0 B The distance between them is greater than the preset distance threshold, then it is determined that p c C No other second images are matched, that is, the number of other second images matched is 0.

[0184] From the above we can know that p c B The number of other second images that are matched is the largest, and then p c B As the spatial position information of the reflecting ball.

[0185] In the above embodiment, the subspace position information of the reflection object corresponding to each second image is projected into the image coordinate system of each other second image, and the number of other second images matched by each subspace position information is determined in combination with the center position of the contour figure corresponding to the reflection object in the other second images. Then, the subspace position information of the other second images with the largest number of matches is used as the spatial position information of the reflection object, which greatly improves the accuracy of determining the spatial position of the reflection object, and is conducive to improving the accuracy of the light source position calibrated based on the spatial position of the reflection object.

[0186] The embodiment of the present disclosure obtains multiple second images of the reflective object that are synchronously captured based on different viewing angles under a second ambient brightness, and determines the subspace position information of the reflective object corresponding to each second image based on the size information of the reflective object and the center position information of the contour graphic corresponding to the reflective object in each second image, and then selects the subspace position information that meets preset conditions from the multiple subspace position information as the spatial position information of the reflective object, thereby greatly improving the accuracy and efficiency of determining the spatial position of the reflective object.

[0187] In an exemplary embodiment, the above step S105, based on the spatial position information of the reflecting object and the image position information of the target light source in each first image, determining the first reflection position information and the first incident light direction information corresponding to each first image may include:

[0188] determining a first ray corresponding to each first image based on image acquisition parameters of a viewing angle corresponding to each first image and central position information of a contour graphic corresponding to a reflective object in the first image;

[0189] determining a second ray corresponding to each first image based on image acquisition parameters of a viewing angle corresponding to each first image and image position information of a target light source in the first image;

[0190] Based on the spatial position information of the reflecting object, the size information of the reflecting object, and the first ray and the second ray corresponding to each first image, first reflection position information and first incident light direction information corresponding to each first image are determined.

[0191] Among them, the first ray refers to the ray starting from the viewpoint of the corresponding perspective of the corresponding first image to the center position of the contour figure corresponding to the reflecting object in the first image; the second ray refers to the ray starting from the viewpoint of the corresponding perspective of the corresponding first image to the position of the target light source in the first image.

[0192] In a specific implementation, the image acquisition parameters of each first image corresponding to the viewing angle include the camera intrinsic parameter matrix K, the camera extrinsic parameter rotation matrix R, and the camera extrinsic parameter translation matrix t. According to the projection relationship of the camera imaging, it can be known that the ray from the camera (i.e., the viewpoint) to the center position of the contour figure is It can be obtained by the above formula (1).

[0193] Similarly, the ray from the camera (i.e., the viewpoint) to the position of the target light source in the first image It can be expressed by the following formula (5):

[0194]

[0195] Wherein, l0 represents the image position information of the target light source in the first image.

[0196] Then, based on each first image object and By combining the spatial position information of the reflecting object and the size information of the reflecting object, the first reflection position information and the first incident light direction information corresponding to each first image can be accurately determined.

[0197] In an exemplary embodiment, determining the first reflection position information and the first incident light direction information corresponding to each first image based on the spatial position information of the reflecting object, the size information of the reflecting object, and the first ray and the second ray corresponding to each first image may include:

[0198] determining first reflection position information corresponding to each first image based on spatial position information of the reflection object, size information of the reflection object, and the first ray and the second ray corresponding to each first image;

[0199] determining normal information of a reflection point corresponding to each first image based on the spatial position information of the reflection object and the first reflection position information corresponding to each first image;

[0200] Based on the normal information of the reflection point corresponding to each first image, the first reflection position information corresponding to the first image, and the second ray corresponding to the first image, the first incident light direction information corresponding to each first image is determined.

[0201] In a specific implementation, taking the reflection object as a reflection ball as an example, the radius of the reflection ball is r0. According to the relationship between geometry and light reflection, the coordinates p of the corresponding reflection point on the reflection ball can be calculated based on the following formula (6) for each first image: ref (i.e. first reflection position information):

[0202]

[0203] in, represents the second ray corresponding to the first image; p c Represents the spatial position information of the reflection object; Indicates the first ray corresponding to the first image.

[0204] Furthermore, the normal information n of the reflection point corresponding to each first image can be expressed as n=p ref -p c .

[0205] Then, according to the relationship between geometry and light reflection, the incident light direction v can be calculated for each first image based on the following formula (7): in (i.e. first incident light direction information):

[0206]

[0207] In the above embodiment, for each first image, after determining the first reflection position information corresponding to the first image, the normal information of the reflection point corresponding to the first image is determined based on the first reflection position information and the spatial position information of the reflecting object, and then based on the normal information and the first reflection position information, the first incident light direction information corresponding to the first image can be accurately determined.

[0208] In step S107 , light source calibration information of the target light source is determined based on the first reflection position information and the first incident light direction information corresponding to each of the plurality of first images of the target light source.

[0209] The light source calibration information includes spatial position information of the target light source, and the spatial position information of the target light source indicates the actual installation spatial position of the target light source, such as three-dimensional spatial coordinates.

[0210] In the embodiment of the present disclosure, for multiple first images of the target light source, the first reflection position information and first incident light direction information corresponding to each first image can be obtained based on steps S103 to S105, and then multiple incident light rays corresponding to the target light source can be obtained. From the geometric relationship, it can be seen that the spatial position information of the point closest to the light source position q on the incident light ray i can be expressed as The spatial position of the light source finally obtained should be the point closest to all incident light rays corresponding to the light source.

[0211] Based on this, in an exemplary embodiment, the above step S107 may include:

[0212] determining a plurality of incident light rays corresponding to a target light source based on first reflection position information and first incident light direction information corresponding to each first image in the plurality of first images;

[0213] Determine the target space point closest to the multiple incident light rays corresponding to the target light source;

[0214] The spatial position information of the target spatial point is used as the spatial position information of the target light source.

[0215] Among them, based on the first reflection position information and the first incident light direction information corresponding to each first image, an incident light ray with the first incident light direction information as the direction and passing through the reflection position indicated by the first reflection position information can be determined, and then multiple incident light rays corresponding one by one to the first image can be obtained.

[0216] The sum of the distances from the target space point to each incident light ray is less than a preset distance value, and the preset distance value can be set according to actual needs.

[0217] In a specific implementation, the target space point can be determined by the optimization problem, namely the following formula (8):

[0218]

[0219] Wherein, i is the first image serial number. In a scenario where each camera in the camera array synchronously captures a first image, i may be the camera serial number. represents the first reflection position information corresponding to the first image i; represents the first incident light direction information corresponding to the first image i; q represents the position information of the target space point.

[0220] In the above embodiment, the spatial position information of the target light source can be obtained quickly and accurately by determining the target spatial point closest to the multiple incident light rays corresponding to the target light source.

[0221] It can be seen from the above technical solutions of the embodiments of the present disclosure that the embodiments of the present disclosure can individually calibrate each light source in the light field acquisition system to obtain the spatial position information of each light source, thereby improving the accuracy, flexibility, stability and universality of the light source calibration, significantly reducing the installation accuracy requirements for the light source, and thereby reducing the difficulty and cost of light source installation.

[0222] In an exemplary embodiment, the light source calibration information of the target light source may further include attitude information. Then, after obtaining the spatial position information of the target light source, the attitude information of the target light source may be further determined, wherein the attitude information indicates the direction of the target light source. Figure 3 As shown, after obtaining the spatial position information of the target light source, for each of the light-emitting elements included in the target light source, the method may further include:

[0223] In step S301, based on the position information of the reflection light spot corresponding to the light emitting element in each first image, the second reflection position information and the second incident light direction information corresponding to each first image are determined.

[0224] The second reflection position information indicates the spatial position of the corresponding reflection point of the corresponding light emitting element on the reflection object, and the second incident light direction information indicates the direction of the incident light of the corresponding light emitting element.

[0225] Wherein, step S301 may be implemented with reference to the aforementioned step S105 of the embodiment of the present disclosure. Taking a light-emitting element k included in the target light source as an example, the second reflection position information corresponding to the first image i may be obtained based on the following formula (9):

[0226]

[0227] Among them, p ref k，i represents the second reflection position information corresponding to the first image i for the light-emitting element k; represents the position information of the reflected light spot corresponding to the light emitting element k in the first image i; r0 is the size information of the reflecting object (such as the radius of the reflecting ball); p c Represents the spatial position information of the reflection object; represents the first ray corresponding to the first image i determined based on the image acquisition parameters of the viewing angle corresponding to the first image i and the center position information of the contour graphic corresponding to the reflection object in the first image i. For details, please refer to the above formula (1).

[0228] Then, the second incident light direction information corresponding to the first image i can be obtained based on the following formula (10):

[0229]

[0230] Among them, n k，i =p ref k，i -p c ;v in k，i represents the second incident light direction information corresponding to the first image i for the light-emitting element k.

[0231] In step S303, the position offset information of each light-emitting element relative to the center of the target light source is determined.

[0232] Specifically, the position offset information indicates the distance between the corresponding light-emitting element and the center of the target light source. In actual application, the position offset information can be obtained from the factory configuration information of the light source.

[0233] In step S305 , the posture information of the target light source is determined based on the spatial position information of the target light source, the position offset information corresponding to each light emitting element, and the second reflection position information and second incident light direction information corresponding to each first image determined for each light emitting element.

[0234] In which, under the posture indicated by the posture information, the sum of the distances from each light-emitting element of the target light source to multiple target incident light rays is minimized, and the multiple target incident light rays are obtained based on the second reflection position information and second incident light direction information corresponding to each first image determined for the corresponding light-emitting element.

[0235] In a specific implementation, the posture information of the target light source can be solved by designing an optimization problem, where the optimization goal is to minimize the sum of the distances from each light-emitting element on the target light source to its corresponding incident light ray, which can be expressed as formula (11):

[0236]

[0237] in, is a rotation matrix used to represent the posture information of the target light source j; d k represents the position offset information of the light emitting element k on the target light source; q j represents the spatial position information of the target light source; i represents the sequence number of the first image. In the scenario where each camera in the camera array synchronously captures a first image, i can be the camera sequence number.

[0238] In the above embodiment, after obtaining the spatial position information of the target light source, the posture information of the target light source can be accurately determined by further analyzing the position of the reflected light spot corresponding to each light-emitting element on the target light source in the first image, and then based on the posture information, it can be determined whether the installation direction of the target light source is incorrect.

[0239] In an exemplary embodiment, the light source calibration information of the target light source may also include light source brightness information. Then, after obtaining the spatial position information of each light source in the light field acquisition system, the brightness information of each light source may also be obtained based on the spatial position information of each light source, so as to calibrate the brightness of each light source, improve the luminous accuracy of the controllable light system, and make the actual luminous effect of the light source acquisition system closer to the target lighting mode. Specifically, Figure 4 As shown, the light source brightness information of each light source in the light field acquisition system can be determined by the following steps:

[0240] In step S401, based on the spatial position information of each light source in the light field acquisition system, the spherical surface of the spherical acquisition support of the light field acquisition system is segmented to obtain a segmentation polygon corresponding to each light source;

[0241] The spatial position information of each light source in this step can be obtained by using the aforementioned embodiment of the present disclosure. Figure 1 The method embodiment shown is obtained by individually calibrating each light source in the light field acquisition system.

[0242] In step S403, based on the area of ​​the segmentation polygon corresponding to each light source, a relative brightness coefficient corresponding to each light source is determined; the relative brightness coefficient is positively correlated with the area of ​​the segmentation polygon;

[0243] In step S405 , light source brightness information of each light source in a target illumination mode is obtained based on the product of the relevant brightness coefficient corresponding to each light source and the target brightness parameter, where the target illumination mode is the illumination mode corresponding to the target brightness parameter.

[0244] In a specific implementation, based on the spatial position information of each light source, the longitude and latitude coordinates of each light source on the spherical collection support (or spherical collection frame) can be calculated. Taking light source j as an example, its corresponding spatial position information is q j , then the longitude and latitude coordinates of light source j on the spherical collection support are Where r represents the radius of the spherical collection support, Represents spatial position information q i The component in the y direction of the three-dimensional coordinate system, Represents spatial position information q j The component in the z direction of the three-dimensional coordinate system.

[0245] Then, based on the longitude and latitude coordinates of all light sources in the light field acquisition system, the spherical surface of the spherical acquisition support can be segmented according to the longitude and latitude coordinates of the light sources, obtaining segmentation polygons for each light source in the longitude and latitude coordinate system. Specifically, the segmentation can be performed using a Voronoi diagram, a spatial segmentation algorithm developed by Russian mathematicians that converts points into a graph of regions. In this graph, any point in each segmented region is closer to the region center than to the center points of other regions.

[0246] Furthermore, based on the assumption that the relative brightness coefficient is positively correlated with the area of ​​the segmentation polygon, the relative brightness coefficient of each light source in the light field acquisition system is determined based on the segmentation polygon corresponding to the light source. In a specific implementation, the ratio of the areas of the segmentation polygons can be calculated and then normalized to a value between 0 and 1. The normalized value corresponding to each segmentation polygon can then be used as the relative brightness coefficient of the light source corresponding to the segmentation polygon.

[0247] Next, the target brightness parameter of the target illumination mode is obtained. The product of the relative brightness coefficient of each light source and the target brightness parameter is used as the light source brightness information of the corresponding light source under the target illumination mode. The target illumination mode can be set according to the illumination mode required for actual light field information collection, for example, the target illumination mode can be a gradient illumination mode.

[0248] In the above embodiment, the spatial position information of each light source in the light field acquisition system obtained through calibration can be used to further calibrate the brightness of each light source to obtain the brightness information of each light source. The brightness information can be used to correct the error caused by the uneven distribution of the light source position, thereby improving the luminous accuracy of the controllable light system, so that the actual luminous effect of the light field acquisition system is closer to the target lighting mode.

[0249] In actual applications, when the reflective object is a reflective ball, a standard circle detected based on the Hough transform is usually used as the outline of the reflective object in the image captured by the camera. However, considering that the actual outline of the reflective ball in the image in the camera perspective projection is not a standard circle, and the Hough transform detection method also has large errors, after obtaining the spatial position information of the reflective object and the spatial position information of each light source in the light field acquisition system based on the aforementioned method of the embodiment of the present disclosure, the spatial position information of the reflective object and the spatial position information of each light source can also be corrected to obtain more accurate spatial position information of the reflective object and the spatial position information of each light source after correction. Based on this, in an exemplary embodiment, the method can also include:

[0250] Performing position correction processing based on the spatial position information of the reflecting object and the spatial position information corresponding to each light source in the light field acquisition system to obtain corrected spatial position information corresponding to the reflecting object and corrected spatial position information corresponding to each light source;

[0251] The spatial position information of each light source can be obtained by using the aforementioned embodiment of the present disclosure. Figure 1 The method embodiment shown is obtained by individually calibrating each light source in the light field acquisition system.

[0252] Among them, the cumulative distance obtained based on the corrected spatial position information corresponding to the reflecting object and the corrected spatial position information corresponding to each light source is the smallest. The cumulative distance is the sum of the target distances corresponding to each light source. The target distance is the sum of the distances from the light source position indicated by the corrected spatial position information corresponding to the corresponding light source to the multiple corrected incident light rays corresponding to the light source. The multiple corrected incident light rays are obtained by correcting the multiple incident light rays of the corresponding light source based on the corrected spatial position information corresponding to the reflecting object.

[0253] In a specific implementation, the following formula (12) can be used to simultaneously optimize the spatial position information p of the reflection object: c And the spatial position information q of each light source j , to obtain the corrected and accurate spatial position information p of the reflecting object c ′ and the spatial position information q of each light source j ′:

[0254]

[0255] Wherein, i represents the serial number of the first image. In a scenario where each camera in the camera array simultaneously captures a first image, i may be the serial number of the camera; j represents the serial number of the light source in the light field acquisition system; and Please refer to the description in the aforementioned formula (8) of the embodiment of the present disclosure.

[0256] Specifically, the spatial position information p of the reflection object c And the spatial position information q of each light source j As the initial value of the corresponding parameter in the above formula (12), the current function value is calculated. If the current function value is greater than the preset function threshold, the update gradient is determined based on the current function value, and then the spatial position information p of the reflection object is directly updated based on the update gradient. c And the spatial position information q of each light source j, and based on the updated information, the current function value is updated and the iteration is continued until the current function value is less than or equal to the preset function threshold value, thereby ending the iteration, thereby using the spatial position information of the reflection object at the end of the iteration as the corrected spatial position information of the reflection object, and using the spatial position information of each light source at the end of the iteration as the corrected spatial position information of each light source.

[0257] The above implementation further improves the accuracy of light source calibration by correcting the spatial position information of the reflecting object and the spatial position information of each light source, thereby improving the luminous accuracy of the controllable light system and improving the acquisition effect of the entire light field acquisition system. In the experiment, after the light source calibration and correction, the object normal information collected by the system has clearer details.

[0258] It is understandable that the posture information and light source brightness information of each light source in the light field acquisition system can be obtained based on the corrected spatial position information of each light source to improve the accuracy of the posture information and light source brightness information.

[0259] In order to facilitate understanding of the technical solution of the embodiments of the present disclosure, a specific example is given below for illustration.

[0260] Scenario Description: The light field acquisition system's spherical collection support consists of a 3.6-meter-diameter spherical collection frame. 357 light panels are mounted as evenly as possible on the frame as the light source. A camera array consisting of 24 cameras is fixed around the equator of the spherical collection frame and aligned with the center of the sphere. Each light panel has red, green, blue, and two different color temperature white LEDs evenly but asymmetrically distributed. The room where the light field acquisition system is located has no other light sources, and the surrounding walls are completely covered with black material.

[0261] Calibration process description:

[0262] Step 1: Perform a checkerboard calibration on each of the 24 cameras to obtain rough intrinsic parameters. A 250mm diameter calibration sphere covered with a circular checkerboard calibration pattern is placed near the center of the spherical acquisition frame. The calibration sphere is rotated through three different positions, each rotated five times, and the cameras are synchronized to capture 15 images. Using these images, the intrinsic and extrinsic parameters, as well as the distortion parameters, are calculated for each camera. Calibration uses a tangential distortion model, and the final calibration error is typically less than 0.5 pixels.

[0263] Step 2: Turn on all white LEDs in the light source and place a 250mm diameter chrome-plated metal reflector near the center of the spherical collection frame. Place a green screen behind the reflector and move the screen. All cameras simultaneously capture four images, obtaining a total of 96 images. Use the distortion parameters obtained in Step 1 to correct these images. Use the corrected images to calculate the rough coordinates of the reflector's center.

[0264] Step 3: Maintaining the reflector position from Step 2, turn on one of the 357 light sources at a time, focusing on only one group of four asymmetrically distributed white LEDs, and trigger the camera to capture the image simultaneously. Use the camera distortion parameters obtained in Step 1 to correct the image distortion. For each light source, 24 images are taken using 24 cameras. Four individual reflected light spots from each LED can be extracted from each image. The arithmetic average of these is taken as the light spot position of that light source in the image, and the initial rough position of the light source is calculated. After obtaining the initial rough position of each light source, this position and the rough center position of the reflector are used as the initial values. The corrected light source position and reflector center coordinates are solved by joint optimization using the light spot position constraints in the images used. The light panel pose is then determined using the four asymmetric light spots. Light sources that differ significantly from the designed pose can be detected, their positions manually corrected, and calibration is repeated in Step 3.

[0265] Step 4: In the specific implementation, a gradient light illumination mode is required to collect facial reflectance and detail normals. Therefore, the light source positions are calibrated using step 3, and the relative brightness coefficient corresponding to each light source is calculated. The relative brightness coefficient is then used to correct the brightness parameters of the gradient light mode, making the overall illumination of the controllable light system closer to the ideal gradient light mode.

[0266] The disclosed embodiment significantly reduces the precision requirements, installation difficulty, and cost of light source installation by independently calibrating the position and posture of each light source in the light field acquisition system after installation. It also improves the luminous accuracy of the controllable light system, which is beneficial to improving the acquisition effect of the overall light field acquisition system. In the experiment, after the light source calibration and correction, the object normal information collected by the system has clearer details.

[0267] Figure 5 FIG1 is a block diagram of a light source calibration device in a light field acquisition system according to an exemplary embodiment. Figure 5 , the device:

[0268] A first image acquisition unit 510 is configured to acquire a plurality of first images of a reflective object synchronously captured at different viewing angles under a first ambient brightness, wherein the first ambient brightness is obtained based on incident light from a target light source in a light field acquisition system, where the target light source is any light source in the light field acquisition system;

[0269] The light source image position determination unit 520 is configured to determine the image position information of the target light source in each first image based on the position of the reflected light spot in each first image;

[0270] A first information determining unit 530 is configured to determine first reflection position information and first incident light direction information corresponding to each first image based on the spatial position information of the reflecting object and the image position information of the target light source in each first image; the first reflection position information indicates the spatial position of the corresponding reflection point of the target light source on the reflecting object;

[0271] The first calibration information determination unit 540 is configured to determine the light source calibration information of the target light source based on the first reflection position information and the first incident light direction information corresponding to each first image in the multiple first images; the light source calibration information includes the spatial position information of the target light source.

[0272] In an exemplary embodiment, the first information determining unit 530 includes:

[0273] A first ray determining unit is configured to determine a first ray corresponding to each first image based on image acquisition parameters of a viewing angle corresponding to each first image and information about a center position of a contour graphic corresponding to the reflective object in the first image; the first ray being a ray starting from a viewpoint corresponding to the viewing angle of the corresponding first image to a center position of the contour graphic corresponding to the reflective object in the first image;

[0274] a second ray determining unit configured to determine a second ray corresponding to each first image based on image acquisition parameters of a viewing angle corresponding to each first image and image position information of the target light source in the first image; the second ray being a ray starting from a viewpoint corresponding to the viewing angle of the corresponding first image to the position of the target light source in the first image;

[0275] The second information determination unit is configured to determine the first reflection position information and the first incident light direction information corresponding to each first image based on the spatial position information of the reflection object, the size information of the reflection object, and the first ray and the second ray corresponding to each first image.

[0276] In an exemplary embodiment, the second information determining unit includes:

[0277] a first reflection position information determining unit configured to determine first reflection position information corresponding to each first image based on the spatial position information of the reflection object, the size information of the reflection object, and the first ray and the second ray corresponding to each first image;

[0278] a normal information determining unit configured to determine normal information of the reflection point corresponding to each first image based on the spatial position information of the reflection object and the first reflection position information corresponding to each first image;

[0279] The first incident light direction information determination unit is configured to determine the first incident light direction information corresponding to each first image based on the normal information of the reflection point corresponding to each first image, the first reflection position information corresponding to the first image, and the second ray corresponding to the first image.

[0280] In an exemplary embodiment, the first calibration information determining unit 540 includes:

[0281] an incident light ray determining unit, configured to determine a plurality of incident light rays corresponding to the target light source based on first reflection position information and first incident light direction information corresponding to each of the plurality of first images;

[0282] The target space point determination unit is configured to determine the target space point closest to the multiple incident light rays corresponding to the target light source; and use the spatial position information of the target space point as the spatial position information of the target light source.

[0283] In an exemplary embodiment, the target light source includes a plurality of asymmetric light-emitting elements, and the light source image position determination unit 520 includes:

[0284] a light spot position determining unit configured to determine, for each first image, position information of a reflected light spot corresponding to each light emitting element in the first image, to obtain a plurality of light spot positions;

[0285] The image position determination subunit is configured to determine an average value of position information corresponding to each pixel in the multiple light spot positions to obtain image position information of the target light source in the first image.

[0286] In an exemplary embodiment, the apparatus further comprises:

[0287] a third information determining unit configured to determine, for each of the light-emitting elements, second reflection position information and second incident light direction information corresponding to each first image based on position information of a reflection spot corresponding to the light-emitting element in each first image; wherein the second reflection position information indicates a spatial position of a corresponding reflection point of the corresponding light-emitting element on the reflective object;

[0288] a position offset information determining unit, configured to determine position offset information of each of the light-emitting elements relative to the center of the target light source;

[0289] a posture information determining unit configured to determine the posture information of the target light source based on the spatial position information of the target light source, the position offset information corresponding to each of the light-emitting elements, and the second reflection position information and the second incident light direction information corresponding to each first image determined for each light-emitting element; the light source calibration information includes the posture information;

[0290] In which, under the posture indicated by the posture information, the sum of the distances from each of the light-emitting elements to multiple target incident light rays is minimum, and the multiple target incident light rays are obtained based on the second reflection position information and second incident light direction information corresponding to each first image determined for the corresponding light-emitting element.

[0291] In an exemplary embodiment, the apparatus further comprises:

[0292] a spherical surface segmentation unit configured to segment the spherical surface of the spherical acquisition support of the light field acquisition system based on the spatial position information of each light source in the light field acquisition system, and obtain a segmentation polygon corresponding to each light source;

[0293] a relative brightness coefficient determining unit configured to determine a relative brightness coefficient corresponding to each light source based on the area of ​​the segmentation polygon corresponding to each light source; wherein the relative brightness coefficient is positively correlated with the area of ​​the segmentation polygon;

[0294] The light source brightness information determination unit is configured to perform a multiplication of the relative brightness coefficient corresponding to each light source and the target brightness parameter to obtain the light source brightness information of each light source under the target lighting mode; wherein the light source calibration information also includes the light source brightness information, and the target lighting mode is the lighting mode corresponding to the target brightness parameter.

[0295] In an exemplary embodiment, the apparatus further comprises:

[0296] A second image acquisition unit is configured to acquire a plurality of second images of the reflective object synchronously captured based on the different viewing angles under a second ambient brightness, wherein the second ambient brightness is obtained based on incident light from a plurality of light sources in the light field acquisition system;

[0297] a first reflecting object position determining unit configured to determine subspace position information of the reflecting object corresponding to each second image based on size information of the reflecting object and center position information of a contour graphic corresponding to the reflecting object in each second image;

[0298] The second reflecting object position determining unit is configured to select subspace position information that meets a preset condition from the subspace position information of the reflecting object corresponding to the multiple second images as the spatial position information of the reflecting object.

[0299] In an exemplary embodiment, the first reflecting object position determining unit includes:

[0300] a third ray determining unit configured to determine a third ray corresponding to each second image based on image acquisition parameters corresponding to the viewing angle of each second image and information about a center position of a contour graphic corresponding to the reflective object in the second image; the third ray being a ray starting from a viewpoint corresponding to the viewing angle of the corresponding second image to a center position of the contour graphic corresponding to the reflective object in the second image;

[0301] a fourth ray determining unit configured to determine a fourth ray corresponding to each second image based on image acquisition parameters of the viewing angle corresponding to each second image and a target contour point on the contour graph corresponding to the reflective object in the second image; the fourth ray being a ray from the viewpoint corresponding to the viewing angle of the corresponding second image to the target contour point, the target contour point being any point on the corresponding contour graph;

[0302] The reflection object subspace position information determining unit is configured to determine the subspace position information of the reflection object corresponding to each second image based on the size information of the reflection object and the third ray and the fourth ray corresponding to each second image.

[0303] In an exemplary embodiment, the second reflecting object position determining unit includes:

[0304] a spatial position projection unit configured to project the subspace position information of the reflective object corresponding to each second image into an image coordinate system of a corresponding perspective of each other second image, thereby obtaining an image position corresponding to the subspace position information in each other second image;

[0305] a matching image number determining unit configured to determine the number of other second images to which the subspace position information matches based on the distance between an image position corresponding to the subspace position information on each other second image and a center position of a contour figure corresponding to the reflective object in the corresponding other second image; wherein the distance corresponding to the other second images to which the subspace position information matches is less than a preset distance threshold;

[0306] The reflecting object position determining subunit is configured to use the sub-space position information with the largest number as the spatial position information of the reflecting object.

[0307] In an exemplary embodiment, the apparatus further comprises:

[0308] a position correction unit configured to perform position correction processing based on the spatial position information of the reflecting object and the spatial position information corresponding to each light source in the light field acquisition system, to obtain corrected spatial position information corresponding to the reflecting object and corrected spatial position information corresponding to each light source;

[0309] Among them, the cumulative distance obtained based on the corrected spatial position information corresponding to the reflecting object and the corrected spatial position information corresponding to each light source is the smallest, the cumulative distance is the sum of the target distances corresponding to each light source, and the target distance is the sum of the distances from the light source position indicated by the corrected spatial position information corresponding to the corresponding light source to the multiple corrected incident light rays corresponding to the light source, and the multiple corrected incident light rays are obtained by correcting the multiple incident light rays of the corresponding light source based on the corrected spatial position information corresponding to the reflecting object.

[0310] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0311] In an exemplary embodiment, an electronic device is also provided, including a processor; a memory for storing instructions executable by the processor; wherein, when the processor is configured to execute the instructions stored in the memory, it implements the calibration method of the light source in the light field acquisition system provided in any of the above embodiments.

[0312] The electronic device may be a terminal, a server or a similar computing device. Figure 6 is a block diagram of an electronic device for calibrating a light source in a light field acquisition system according to an exemplary embodiment. Figure 6 As shown, the electronic device 60 may include one or more (illustrated by 602a, 602b, ..., 602n in the figure) processors 602 (the processor 602 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 604 for storing data, and a transmission device 606 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 6 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 6 More or fewer components than shown, or with Figure 6 Different configurations shown.

[0313] It should be noted that the one or more processors 602 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the electronic device 60 (or mobile device). As described in the embodiments of the present disclosure, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0314] The memory 604 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the methods described in the embodiments of the present disclosure. The processor 602 executes the software programs and modules stored in the memory 604 to perform various functional applications and data processing, thereby implementing the aforementioned method for calibrating a light source in a light field acquisition system. The memory 604 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 604 may further include memory remotely located relative to the processor 602, and these remote memories may be connected to the electronic device 60 via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0315] The transmission device 606 is used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by the communication provider of the electronic device 60. In one embodiment, the transmission device 606 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In one embodiment, the transmission device 606 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0316] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the electronic device 60 (or mobile device).

[0317] In an exemplary embodiment, a computer-readable storage medium including instructions is further provided, such as a memory 604 including instructions, which can be executed by the processor 602 of the apparatus 600 to perform the above method. Alternatively, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0318] In an exemplary embodiment, a computer program product is further provided, including a computer program / instruction, which, when executed by a processor, implements the method for calibrating a light source in a light field acquisition system provided in any of the above embodiments.

[0319] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0320] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for calibrating a light source in a light field acquisition system, characterized in that: include: Acquire a plurality of first images synchronously captured from different viewing angles of the reflective object under a first ambient brightness; The first ambient brightness is obtained based on incident light from a target light source in the light field acquisition system, where the target light source is any light source in the light field acquisition system; Determining image position information of the target light source in each first image based on the position of the reflected light spot in each first image; determining first reflection position information and first incident light direction information corresponding to each first image based on the spatial position information of the reflective object and the image position information of the target light source in each first image; the first reflection position information indicating the spatial position of a corresponding reflection point of the target light source on the reflective object, and the first incident light direction information indicating the direction of incident light of the target light source; determining light source calibration information of the target light source based on first reflection position information and first incident light direction information corresponding to each first image in the plurality of first images; The light source calibration information includes spatial position information of the target light source.

2. The method according to claim 1, characterized in that The determining, based on the spatial position information of the reflecting object and the image position information of the target light source in each first image, first reflection position information and first incident light direction information corresponding to each first image includes: Determining a first ray corresponding to each first image based on image acquisition parameters of the viewing angle corresponding to each first image and information about the center position of the contour graphic corresponding to the reflective object in the first image; the first ray being a ray starting from a viewpoint corresponding to the viewing angle of the corresponding first image to the center position of the contour graphic corresponding to the reflective object in the first image; Determining a second ray corresponding to each first image based on image acquisition parameters of the viewing angle corresponding to each first image and image position information of the target light source in the first image; the second ray is a ray starting from a viewpoint corresponding to the viewing angle of the corresponding first image to the position of the target light source in the first image; Based on the spatial position information of the reflection object, the size information of the reflection object, and the first ray and the second ray corresponding to each first image, first reflection position information and first incident light direction information corresponding to each first image are determined.

3. The method according to claim 2, characterized in that The determining, based on the spatial position information of the reflecting object, the size information of the reflecting object, and the first ray and the second ray corresponding to each first image, first reflection position information and first incident light direction information corresponding to each first image includes: determining first reflection position information corresponding to each first image based on the spatial position information of the reflection object, the size information of the reflection object, and the first ray and the second ray corresponding to each first image; determining normal information of the reflection point corresponding to each first image based on the spatial position information of the reflection object and the first reflection position information corresponding to each first image; Based on the normal information of the reflection point corresponding to each first image, the first reflection position information corresponding to the first image, and the second ray corresponding to the first image, the first incident light direction information corresponding to each first image is determined.

4. The method according to claim 1, wherein The determining the light source calibration information of the target light source based on the first reflection position information and the first incident light direction information corresponding to each first image in the plurality of first images includes: determining a plurality of incident light rays corresponding to the target light source based on first reflection position information and first incident light direction information corresponding to each first image in the plurality of first images; Determine a target space point closest to a plurality of incident light rays corresponding to the target light source; The spatial position information of the target spatial point is used as the spatial position information of the target light source.

5. The method according to claim 1, wherein The target light source includes a plurality of asymmetric light-emitting elements, and determining the image position information of the target light source in each first image based on the position of the reflected light spot in each first image includes: For each first image, determining position information of a reflected light spot corresponding to each light-emitting element in the first image to obtain a plurality of light spot positions; An average value of position information corresponding to each pixel in the multiple light spot positions is determined to obtain image position information of the target light source in the first image.

6. The method according to claim 5, characterized in that The light source calibration information also includes posture information of the target light source; and the method further includes: For each of the light-emitting elements, based on the position information of the reflection spot corresponding to the light-emitting element in each first image, determining second reflection position information and second incident light direction information corresponding to each first image; the second reflection position information indicates the spatial position of the corresponding reflection point of the corresponding light-emitting element on the reflective object, and the second incident light direction information indicates the direction of the incident light of the corresponding light-emitting element; determining positional offset information of each of the light-emitting elements relative to the center of the target light source; Determining the posture information of the target light source based on the spatial position information of the target light source, the position offset information corresponding to each of the light-emitting elements, and the second reflection position information and second incident light direction information corresponding to each first image determined for each light-emitting element; In which, under the posture indicated by the posture information, the sum of the distances from each of the light-emitting elements to multiple target incident light rays is minimum, and the multiple target incident light rays are obtained based on the second reflection position information and second incident light direction information corresponding to each first image determined for the corresponding light-emitting element.

7. The method according to claim 1, characterized in that The method further comprises: Based on the spatial position information of each light source in the light field acquisition system, a spherical surface of a spherical acquisition support of the light field acquisition system is segmented to obtain segmentation polygons corresponding to each light source; wherein the light source calibration information of each light source in the light field acquisition system is determined; the light source calibration information of each light source includes the spatial position information of each light source; Determining a relative brightness coefficient corresponding to each light source based on the area of ​​the segmentation polygon corresponding to each light source; wherein the relative brightness coefficient is positively correlated with the area of ​​the segmentation polygon; Based on the product of the relative brightness coefficient corresponding to each light source and the target brightness parameter, the light source brightness information of each light source in the target lighting mode is obtained; wherein the light source calibration information also includes the light source brightness information, and the target lighting mode is the lighting mode corresponding to the target brightness parameter.

8. The method according to any one of claims 1 to 7, characterized in that Before determining first reflection position information and first incident light direction information corresponding to each first image based on the spatial position information of the reflection object and the image position information of the target light source in each first image, the method further includes: Acquire multiple second images of the reflective object synchronously captured based on the different viewing angles under a second ambient brightness; the second ambient brightness is obtained based on incident light from multiple light sources in the light field acquisition system; Determining subspace position information of the reflective object corresponding to each second image based on size information of the reflective object and center position information of a contour graphic corresponding to the reflective object in each second image; Subspace position information that meets a preset condition is selected from the subspace position information of the reflection object corresponding to the multiple second images as the spatial position information of the reflection object.

9. The method according to claim 8, characterized in that The determining, based on the size information of the reflective object and the center position information of the contour graphic corresponding to the reflective object in each second image, the subspace position information of the reflective object corresponding to each second image includes: Determining a third ray corresponding to each second image based on image acquisition parameters of the viewing angle corresponding to each second image and information about the center position of the contour graphic corresponding to the reflective object in the second image; the third ray being a ray starting from a viewpoint corresponding to the viewing angle of the corresponding second image to the center position of the contour graphic corresponding to the reflective object in the second image; determining, based on image acquisition parameters of the viewing angle corresponding to each second image and a target contour point on the contour graph corresponding to the reflective object in the second image, a fourth ray corresponding to each second image; the fourth ray being a ray starting from a viewpoint corresponding to the viewing angle of the corresponding second image to the target contour point, the target contour point being any point on the corresponding contour graph; Based on the size information of the reflective object and the third ray and the fourth ray corresponding to each second image, subspace position information of the reflective object corresponding to each second image is determined.

10. The method according to claim 8, characterized in that The selecting, from the subspace position information of the reflection object corresponding to the plurality of second images, subspace position information that meets a preset condition as the spatial position information of the reflection object includes: For each second image, project the subspace position information of the reflection object corresponding to the subspace position information into the image coordinate system of the corresponding viewing angle of each other second image to obtain the image position corresponding to the subspace position information in each other second image; determining, based on the distance between an image position corresponding to the subspace position information on each other second image and a center position of a contour graphic corresponding to the reflective object in the corresponding other second image, the number of other second images to which the subspace position information matches; wherein the distance corresponding to the other second images to which the subspace position information matches is less than a preset distance threshold; The subspace position information with the largest number is used as the spatial position information of the reflection object.

11. The method according to claim 8, characterized in that The method further includes: Performing position correction processing based on the spatial position information of the reflecting object and the spatial position information corresponding to each light source in the light field acquisition system to obtain corrected spatial position information corresponding to the reflecting object and corrected spatial position information corresponding to each light source; wherein the light source calibration information of each light source in the light field acquisition system is determined; the light source calibration information of each light source includes the spatial position information of each light source; Among them, the cumulative distance obtained based on the corrected spatial position information corresponding to the reflecting object and the corrected spatial position information corresponding to each light source is the smallest, the cumulative distance is the sum of the target distances corresponding to each light source, and the target distance is the sum of the distances from the light source position indicated by the corrected spatial position information corresponding to the corresponding light source to the multiple corrected incident light rays corresponding to the light source, and the multiple corrected incident light rays are obtained by correcting the multiple incident light rays of the corresponding light source based on the corrected spatial position information corresponding to the reflecting object.

12. A light source calibration device in a light field acquisition system, characterized in that: include: A first image acquisition unit is configured to acquire a plurality of first images synchronously captured from different viewing angles of the reflective object under a first ambient brightness; The first ambient brightness is obtained based on incident light from a target light source in the light field acquisition system, where the target light source is any light source in the light field acquisition system; a light source image position determining unit, configured to determine image position information of the target light source in each first image based on the position of the reflected light spot in each first image; a first information determining unit configured to determine first reflection position information and first incident light direction information corresponding to each first image based on the spatial position information of the reflecting object and the image position information of the target light source in each first image; the first reflection position information indicates the spatial position of a corresponding reflection point of the target light source on the reflecting object; a first calibration information determining unit configured to determine light source calibration information of the target light source based on first reflection position information and first incident light direction information corresponding to each first image in the plurality of first images; The light source calibration information includes spatial position information of the target light source.

13. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method for calibrating a light source in a light field acquisition system according to any one of claims 1 to 11.

14. A computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to execute the method for calibrating a light source in a light field acquisition system according to any one of claims 1 to 11.

Images