Near-field radiation intensity reconstruction method and device and storage medium

Abstract

The invention relates to the technical field of radiation metrics, and discloses a near-field radiation intensity reconstruction method and device and a storage medium. The method comprises the following steps: storing a multi-scale position coding feature vector of a three-dimensional space through a multi-scale hash table, and decoding the feature vector through a decoding model to obtain radiation field information of a space point location; combining the radiation field information of the spatial points and then predicting a first pixel predicted value of a pixel point in a two-dimensional brightness image collected at a first pose by using a ray tracing and volume rendering technology, and calculating a difference between the first pixel predicted value and a pixel actual value of the pixel point; the decoding model and the multi-scale hash table are optimized through the gradient descent method, and the overall three-dimensional space light field radiation intensity information can be coded, so that three-dimensional modeling of the light field radiation intensity is realized, deduction and reconstruction of the light field radiation intensity information at any angle and distance (namely, the second pose) are realized, and the accuracy of the light field radiation intensity information is improved. And more accurate and reliable support can be provided for measurement and analysis of the radiation intensity of the light field.

Description

Technical Field

[0001] The present disclosure relates to the technical field of radiometrics, and in particular to a near-field radiometric reconstruction method, device and storage medium. Background Art

[0002] Radiometry is a science that studies electromagnetic radiation. The instrument that measures radiometric parameters is called a radiometer. In the field of optics, the research object of radiometry is mainly electromagnetic radiation in visible light and its nearby bands. In practical applications, the study of the visible light band needs to consider not only the objective measurement of electromagnetic radiation, but also the physiological and psychological response of human vision to electromagnetic radiation. Radiometry that includes human vision factors is also called photometry. The physical parameters involved in photometry include luminous intensity, luminous flux, illumination and brightness. The instrument that measures the above photometric parameters is called a near-field photometer (NFG).

[0003] In a typical NFG structure, the NFG includes two mutually perpendicular rotation axes, a scanning frame connected to the rotation axes, and an imaging luminance meter mounted on the scanning frame. When the scanning frame rotates, the motion trajectory of the imaging luminance meter is located on a scanning sphere with a radius of R, and the luminous body to be measured is placed inside the scanning sphere. The maximum size of the luminous body to be measured is less than the diameter 2R of the scanning sphere. The core device of the imaging luminance meter is a two-dimensional imaging sensor such as a charge-coupled device (CCD) and a complementary metal-oxide-semiconductor (CMOS), which can collect brightness images of all accessible positions during the spherical scanning process. Since the brightness value in a specified direction is independent of the propagation distance of the light, even if the motion trajectory of the imaging luminance meter is always located on the scanning sphere, the brightness image collected at a specific position can describe the light radiation characteristics at any distance in that direction, and can solve parameters such as luminous intensity and illumination at different distances based on the conversion relationship between photometric parameters. Among them, the process of solving parameters such as luminous intensity, luminous flux, and illumination based on the brightness image measured by the imaging brightness meter and drawing the full-space luminosity distribution is the process of near-field radiance reconstruction.

[0004] In order to ensure the accuracy of near-field radiometry reconstruction, traditional near-field radiometry reconstruction methods usually require an imaging luminance meter to scan a large number of angles and record the two-dimensional light field brightness image corresponding to each angle, and then reconstruct the near-field radiometry based on the collected brightness image.

[0005] However, to collect brightness images at multiple scanning angles, it is necessary to control the imaging brightness meter to move at multiple positions, and to calculate and reconstruct the near-field radiance for a large number of brightness images (for example, generally 2664 brightness images at different angles are required). The calculation cost is high and the calculation time is long, all of which lead to the low efficiency of traditional near-field radiance reconstruction. Summary of the invention

[0006] In view of this, the present disclosure proposes a near-field radiometry reconstruction method, device and storage medium. Based on the principle of implicit representation of radiation field, a multi-scale hash table is used to store the feature vectors of voxels in three-dimensional space, and a decoding model is used as a feature vector decoder. Under the supervision of multiple brightness images taken at the first pose and the volume rendering formula, the multi-scale hash table and the decoding model are optimized at the same time, thereby realizing the reconstruction of the three-dimensional radiation field and the inversion of the radiation field at any position in space (second pose). At this time, it is not necessary to collect brightness images at the first pose and the second pose, and near-field radiometry reconstruction can be realized using a small number of brightness images, thereby improving the efficiency of near-field radiometry reconstruction.

[0007] According to one aspect of the present disclosure, a near-field radiometry reconstruction method is provided, which is used in an imaging near-field photometer, wherein the imaging near-field photometer comprises: a rotating scanning mechanism and an imaging luminance meter mounted on the rotating scanning mechanism; the rotating scanning mechanism is used to drive the imaging luminance meter to move on the surface of a scanning sphere; the imaging luminance meter is used to collect a luminance image of a light source to be measured located inside the scanning sphere; the method comprises:

[0008] Acquire a brightness image collected by the imaging brightness meter on the scanning sphere in a first position; wherein different brightness images have different first positions;

[0009] Dividing the three-dimensional space corresponding to the scanning sphere into voxels according to different resolutions, and constructing a multi-scale hash table based on the voxels corresponding to the different resolutions, so as to store the feature vector of each voxel through the multi-scale hash table;

[0010] For each light sampling point on the light corresponding to at least one pixel point in each brightness image, determining a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtaining a position code corresponding to each light sampling point;

[0011] Performing angle coding on the light sampling points based on an angle coding function to obtain an angle coding corresponding to each light sampling point;

[0012] Decoding the position code and the angle code based on a pre-created decoding model to obtain a density prediction value and an optical parameter prediction value of each light sampling point;

[0013] Determine a first pixel prediction value of the pixel point based on a density prediction value and an optical parameter prediction value of each light sampling point on the light corresponding to the pixel point, and a preset volume rendering model;

[0014] Iteratively optimize the feature vectors in the multiscale hash table and the model parameters of the decoding model based on the first pixel prediction value and the actual pixel value of the pixel point on the brightness image to obtain a trained multiscale hash table and a trained decoding model;

[0015] Based on the trained multi-scale hash table and the trained decoding model, a second pixel prediction value corresponding to a second posture on the scanning sphere is determined to reconstruct the near-field radiance of the light source to be measured based on the second pixel prediction value; wherein the second posture is different from the first posture.

[0016] In a possible implementation, the multi-scale hash table is used to store feature vectors corresponding to each vertex of each voxel; accordingly,

[0017] The step of determining, for each light sampling point on a light corresponding to at least one pixel point in each brightness image, a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtaining a position code corresponding to each light sampling point includes:

[0018] For each voxel of each resolution corresponding to the light sampling point, if the light sampling point is located inside the voxel, determining the vertex index of the vertex of the voxel based on the position index of the voxel;

[0019] Determine, in the multi-scale hash table, a feature vector corresponding to each vertex index;

[0020] Perform feature fusion on the feature vectors corresponding to each vertex index to obtain the position encoding of the position index corresponding to each resolution;

[0021] The position codes of the position indexes corresponding to various resolutions are merged to obtain the position codes corresponding to the light sampling points.

[0022] In a possible implementation, the feature fusion of the feature vectors corresponding to the vertex indices to obtain the position encoding of the position index corresponding to each resolution includes:

[0023] Based on the distance between the three-dimensional coordinates of the light sampling point and the three-dimensional coordinates of each vertex, the feature vectors corresponding to each vertex index are fused to obtain the position encoding of the position index corresponding to each resolution.

[0024] In a possible implementation, for each light sampling point on the light corresponding to at least one pixel in each brightness image, determining a feature vector corresponding to the light sampling point based on the multi-scale hash table to obtain a position code corresponding to each light sampling point includes:

[0025] For each pixel in the brightness image, connect the position of the pixel and the position of the optical center in the pinhole camera model corresponding to the imaging brightness meter to determine the light information corresponding to the pixel, wherein the light information includes the direction and position of the light;

[0026] Based on the light information, discrete point sampling is performed on at least one light ray corresponding to each pixel point to obtain a plurality of light ray sampling points on each light ray;

[0027] The feature vector corresponding to each light sampling point is determined based on the multi-scale hash table to obtain the position code corresponding to each light sampling point.

[0028] In a possible implementation, the decoding model includes an optical density network and an optical parameter network connected to the optical density network; accordingly,

[0029] The decoding of the position code and the angle code based on the pre-created decoding model to obtain a density prediction value and an optical parameter prediction value of each light sampling point includes:

[0030] Inputting the position code into the optical density network to decode the position code to obtain a multi-channel density prediction result, outputting the density prediction result of the target channel in the multi-channel density prediction result, and obtaining the density prediction value;

[0031] The density prediction results of other channels except the target channel in the multi-channel density prediction results and the angle code are input into the optical parameter network to obtain the optical parameter prediction value.

[0032] In a possible implementation, the optical density network and / or the optical parameter network is established based on a fully connected neural network.

[0033] In a possible implementation, determining the first pixel prediction value of the pixel point based on the density prediction value and the optical parameter prediction value of each light sampling point on the light corresponding to the pixel point and a preset volume rendering model includes:

[0034] Determining a starting point and an end point of each ray, wherein the starting point and the end point are located within the scanning sphere;

[0035] Based on the volume rendering model, the density prediction value and the optical parameter prediction value of each light sampling point from the starting point to the end point are integrated to obtain a first pixel prediction value of the pixel point.

[0036] In a possible implementation, the volume rendering model is expressed by the following formula:

[0037] G = ∫T(t)σ(t)c(t,d)dt;

[0038]

[0039] Among them, G represents the first pixel prediction value of the pixel point; σ(t) represents the density prediction value of the light sampling point on the light corresponding to the pixel point at the position x(t); c(t, d) is the optical parameter prediction value of the light sampling point at x(t) that changes with the direction; T(t) is the transmittance from the starting point to the position t, which represents the probability that the light is not absorbed before the position t.

[0040] According to another aspect of the present disclosure, a near-field radiometry reconstruction device is provided, comprising: a processor; and a memory for storing processor executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0041] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0042] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0043] The method obtains a brightness image collected by an imaging brightness meter at the first position of a scanning sphere; divides the three-dimensional space corresponding to the scanning sphere into voxels according to different resolutions, and constructs a multi-scale hash table based on the voxels corresponding to different resolutions, so as to store the feature vector of each voxel through the multi-scale hash table; for each light sampling point on a light ray corresponding to at least one pixel point in each brightness image, determines the feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtains the position code corresponding to each light sampling point; performs angle coding on the light sampling point based on the angle coding function, and obtains the angle code corresponding to each light sampling point; decodes the position code and the angle code based on a pre-created decoding model, and obtains the density prediction value and the optical parameter prediction value of each light sampling point; based on the density prediction value and the optical parameter prediction value of each light sampling point on the light ray corresponding to the pixel point, The method uses a plurality of pixel prediction values ​​and a preset volume rendering model to determine a first pixel prediction value of a pixel point; based on the first pixel prediction value and the actual pixel value of the pixel point on the brightness image, iteratively optimizes the feature vector in the multi-scale hash table and the model parameters of the decoding model to obtain a trained multi-scale hash table and a trained decoding model; based on the trained multi-scale hash table and the trained decoding model, determines a second pixel prediction value corresponding to a second posture on the scanning sphere, so as to reconstruct the near-field radiance of the luminous body to be measured based on the second pixel prediction value; the method can solve the problem that a large number of brightness images need to be collected in the traditional near-field radiance reconstruction method, resulting in low reconstruction efficiency; since it is no longer necessary to collect brightness images at both the first posture and the second posture, near-field radiance reconstruction can be achieved using a small number of brightness images collected at the first posture, thereby improving the near-field radiance reconstruction efficiency.

[0044] In addition, the near-field radiometry reconstruction method of this embodiment can also estimate the three-dimensional radiation field information well in the case of a small number of brightness images, and has good data utilization.

[0045] In addition, the near-field radiometry reconstruction method of this embodiment can measure luminous bodies of various shapes and properties and has good versatility.

[0046] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0048] Figure 1 A structural diagram of an imaging near-field photometer according to an embodiment of the present disclosure is shown;

[0049] Figure 2A flowchart showing a near-field radiometry reconstruction method according to an embodiment of the present disclosure is shown;

[0050] Figure 3 A schematic diagram showing a brightness image acquisition process according to an embodiment of the present disclosure is shown;

[0051] Figure 4 A schematic diagram showing a world coordinate system according to an embodiment of the present disclosure;

[0052] Figure 5 A schematic diagram showing a process of acquiring light sampling points according to an embodiment of the present disclosure;

[0053] Figure 6 A schematic diagram showing a process of obtaining a position code according to an embodiment of the present disclosure;

[0054] Figure 7 A schematic diagram showing a decoding model according to an embodiment of the present disclosure;

[0055] Figure 8 A schematic diagram showing a near-field radiometric image reconstructed according to an embodiment of the present disclosure;

[0056] Fig. 9 A flowchart showing a near-field radiometry reconstruction method according to another embodiment of the present disclosure is shown;

[0057] Fig.10 A block diagram of a near-field radiometry reconstruction device according to an embodiment of the present disclosure is shown;

[0058] Fig.11 A block diagram of a near-field radiometry reconstruction device according to another embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0059] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0060] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0061] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0062] First, several terms involved in this application are introduced.

[0063] Voxel: short for Volume Pixel, it combines the concepts of pixel, volume and element, and represents the smallest unit in three-dimensional space. In computer graphics, voxels are represented by small cubes in three-dimensional models, and each voxel can be given different attributes, such as color, material, etc., to form a complex three-dimensional scene.

[0064] The pinhole camera model is a simplified optical imaging model. The basic principle is central perspective projection, which projects objects in three-dimensional space onto a two-dimensional imaging plane through a pinhole. In order to describe the pinhole camera model in mathematical language, four coordinate systems are generally required: world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system. The imaging process of three-dimensional objects on two-dimensional images is expressed through coordinate system transformation.

[0065] The world coordinate system is an absolute coordinate system that describes the objective world. Since the camera can be located at any position in the objective world, it is necessary to establish a reference coordinate system to describe the position of the camera, and use it to describe the position of any spatial point in the environment. Generally, it is expressed by (X w ,Y w ,Z w ) represents the world coordinate value.

[0066] Camera coordinate system: It is a three-dimensional coordinate system established with the camera optical center as the reference point. It is used to describe the position of any spatial point in the camera space. The establishment method is: the camera optical center is the coordinate origin, the camera main optical axis is the Z axis, the horizontal direction parallel to the image plane is the X axis, and the vertical direction parallel to the image plane is the Y axis. It is generally expressed by (X c ,Y c ,Z c ) represents the camera coordinate value.

[0067] Image coordinate system: It is a two-dimensional plane coordinate system defined on the imaging plane, expressed in actual physical units (mm). It is established as follows: the intersection of the camera's main optical axis and the image plane is taken as the origin. The intersection is called the principal point. The X-axis is parallel to the horizontal direction of the image to the right, and the Y-axis is parallel to the vertical direction of the image downward. The image coordinate value is generally represented by (x, y).

[0068] Pixel coordinate system: In computers, digital images are stored in the form of pixel matrices after being discretized. Therefore, a pixel coordinate system needs to be established to describe camera images. Pixel coordinates can be understood as the position index of pixels in the image plane. The establishment method is: take the upper left corner of the image plane as the origin, the u axis is parallel to the X axis of the image coordinate system, horizontally to the right, and the v axis is parallel to the Y axis of the image coordinate system, vertically downward. The pixel coordinate value is generally represented by (u, v).

[0069] Camera imaging can be understood as the process of projecting an object in three-dimensional space onto a two-dimensional image plane. The projection process ultimately establishes a mapping relationship between three-dimensional space points and image plane points. Therefore, the projection process can be expressed by the coordinate transformation between three-dimensional space points and two-dimensional image plane points. Correspondingly, the pixel points on the two-dimensional image can be converted inversely to obtain the position of the object in three-dimensional space.

[0070] Figure 1 FIG. 2 shows a structural diagram of an imaging near-field photometer according to an embodiment of the present disclosure. Figure 1 As shown, the imaging near-field photometer includes: a rotating scanning mechanism 110 and an imaging luminance meter 120 installed on the rotating scanning mechanism.

[0071] The imaging luminance meter 120 is used to collect the brightness image of the luminous body to be measured located inside the scanning sphere. Optionally, the imaging luminance meter 120 can be a camera or a video camera with the ability to collect brightness images. The imaging sensor in the imaging luminance meter 120 can be a CCD or a CMOS, etc. This embodiment does not limit the implementation method of the imaging luminance meter 120.

[0072] The rotating scanning mechanism 110 is used to drive the imaging luminance meter 120 to move on the surface of the scanning sphere. Schematically, the rotating scanning mechanism 110 includes: two rotating shafts 111 perpendicular to each other, and a scanning frame 112 connected to the rotating shafts.

[0073] The scanning frame 112 is used to install the imaging luminance meter 120. In this way, when the rotating shaft 111 rotates, the scanning frame 112 can be driven to rotate around the rotating shaft 111, and then the scanning frame drives the imaging luminance meter 120 to move. At this time, the imaging luminance meter 120 can move on the surface of the scanning ball with the luminous body to be measured as the sphere center with its center of mass as the moving point, and the optical axis of the imaging luminance meter 120 always points to the sphere center. The luminous body to be measured is located inside the scanning ball, and specifically can be located at the sphere center of the scanning ball.

[0074] In other embodiments, the rotating scanning mechanism 110 may be implemented in other ways. For example, the number of rotating axes in the rotating scanning mechanism 110 may be less or more, or the rotating scanning mechanism 110 may also include a supporting structure connected to the rotating axis 111, etc. This embodiment does not limit the implementation method of the rotating scanning mechanism 110.

[0075] Optionally, the rotating scanning mechanism 110 is also connected to a control device, which sets the moving path of the imaging luminance meter 120 according to the characteristics of the luminous body to be measured and the measurement requirements. Accordingly, the control device is connected to the motor in the rotating shaft 111 to control the motor to drive the rotating shaft 111 to rotate, thereby driving the imaging luminance meter 120 to move along the moving path through the scanning frame, forming a moving trajectory corresponding to the moving path on the surface of the scanning ball, so as to shoot at different angles of the luminous body to be measured. Each time the imaging luminance meter 120 reaches the designated shooting position to shoot the luminous body to be measured, the acquisition position of the imaging luminance meter 120 is recorded to obtain a series of two-dimensional brightness images and the acquisition posture corresponding to each brightness image, and the acquisition posture includes the acquisition position and orientation data of the imaging luminance meter 120.

[0076] In the traditional near-field radiometry reconstruction method, an imaging luminance meter is required to collect luminance images at a large number of collection positions to stitch the luminance images of the full-space light field and realize near-field radiometry reconstruction. At this time, the imaging luminance meter needs to move and collect multiple times within the scanning sphere, and then calculate a large number of collected luminance images (generally more than 1,000 luminance images), which makes the near-field radiometry reconstruction less efficient.

[0077] In order to improve the efficiency of near-field radiometry reconstruction, in some implementations, the efficiency of near-field radiometry reconstruction is improved by increasing the control and positioning speed of the scanning frame, and / or increasing the splicing calculation speed of the brightness image. However, in these implementations, since the near-field radiometry reconstruction still requires the acquisition of a large number of brightness images, the degree of efficiency improvement is limited. Based on this, in the present application, a near-field radiometry reconstruction method is provided, which can collect a small amount of brightness images to train a multi-scale hash table and a decoding model, and realize pixel reconstruction at other positions in the light field based on the trained multi-scale hash table and decoding model, thereby realizing near-field radiometry reconstruction. In this process, since the number of brightness images used is reduced, the reconstruction efficiency of the near-field radiometry can be greatly improved.

[0078] The following is a detailed introduction to the near-field radiometry reconstruction method provided by the present application. In the present application, the near-field radiometry reconstruction method can be used in an imaging near-field photometer, or in an electronic device with computing capabilities that is communicatively connected to an imaging near-field photometer, the electronic device including but not limited to a user terminal or a server, the user terminal can be a computer, a tablet computer, a mobile phone, etc., and this embodiment does not limit the implementation method of the electronic device.

[0079] Figure 2 FIG. 5 is a flowchart of a near-field radiometric reconstruction method according to an embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0080] Step 201, obtaining a brightness image captured by an imaging brightness meter on a scanning sphere in a first posture.

[0081] In this embodiment, the imaging luminance meter collects luminance images at multiple first position when scanning the surface of the sphere, different luminance images have different first position, and the number of luminance images is less than the number of images required for image stitching in the traditional near-field radiometry reconstruction method.

[0082] refer to Figure 3 The schematic diagram of the acquisition process of the brightness image shown in the figure, the imaging brightness meter acquires brightness images at multiple first positions on the surface of the upper hemisphere of the scanning sphere. In other embodiments, the first position can also be distributed on the surface of the lower hemisphere, and this embodiment does not limit the selection method of the first position.

[0083] The number of first poses (i.e., the number of brightness images) can ensure the training accuracy of the following multi-scale hash table and decoding model. Schematically, the number can be 50 to 100 brightness images (much smaller than the at least 1000 brightness images required for the traditional near-field radiometry reconstruction method). This embodiment does not limit the number of first poses.

[0084] In this embodiment, the first posture is represented by the position coordinate value and direction vector of the imaging luminance meter in the world coordinate system, wherein the position coordinate value is used to indicate the position of the imaging luminance meter on the surface of the scanning sphere, and the direction vector is used to indicate the shooting angle of the imaging luminance meter when shooting the luminance image.

[0085] Schematically, refer to Figure 4 , the world coordinate system is established by taking the center of the scanning ball as the origin, determining the x-axis and y-axis respectively on the plane parallel to the horizontal plane, and determining the positive direction of the z-axis upwards on the vertical horizontal plane, establishing a three-dimensional rectangular coordinate system to obtain the world coordinate system. At this time, the coordinates of the centroid of the imaging brightness meter can be obtained by the two azimuth angles θ and Determine, where θ is the angle between the center of mass of the imaging brightness meter and the center of the sphere and the z-axis, is the angle between the projection point of the imaging brightness meter on the xoy plane and the line connecting the sphere center and the positive half axis of the x-axis. c ,y c ,z c ) can be determined by the following formula:

[0086]

[0087] Where r represents the radius of the scanning sphere;

[0088] Direction Vector It can be determined by the following formula:

[0089]

[0090] In other embodiments, the world coordinate system may be established in other ways, and accordingly, the formula for determining the first pose may also be adaptively changed. This embodiment does not limit the way to establish the world coordinate system.

[0091] Optionally, each time the imaging luminance meter captures a luminance image, the imaging near-field photometer may record the first position corresponding to the luminance image, thereby obtaining a series of two-dimensional luminance images and the first position corresponding to each luminance image.

[0092] Step 202 , divide the three-dimensional space corresponding to the scanning sphere into voxels according to different resolutions, and construct a multi-scale hash table based on the voxels corresponding to the different resolutions, so as to store the feature vector of each voxel through the multi-scale hash table.

[0093] Assuming that the resolution includes M types, each resolution corresponds to a voxel division result of a three-dimensional space, and the H corresponding to the resolution is obtained m ×W m ×L m A cube block of a certain size is a voxel corresponding to the resolution. Each voxel corresponds to a voxel index, which is used to indicate the position of the voxel in the three-dimensional space. Schematically, the voxel index can be represented by the numbering of the voxels in the length, width and height directions, such as: the voxel index is (a, b, c), which represents the ath voxel in the length direction, the bth voxel in the width direction, and the cth voxel in the height direction. In other embodiments, the voxel index can also be implemented by other representations, such as: represented by the coordinate position of at least one point (such as a vertex) in the voxel. This embodiment does not limit the implementation method of the voxel index. Wherein, m = 1, 2, ..., M. Wherein, M is a positive integer, for example: M = 8. In other embodiments, M can also be other values. This embodiment does not limit the value of M.

[0094] In the multi-scale hash table, each voxel is composed of K 1-row V-column column vectors as the feature vector of the cube block. Schematically, each feature vector of length V (i.e., the column vector of 1 row V columns in the above text) is used to indicate the features of a certain position in the voxel, for example: K feature vectors of length V include feature vectors of 8 vertices in the voxel, and the feature vector of each vertex corresponds to a feature vector of length V, which is used to represent the features of the vertex. In this case, K is greater than or equal to 8; in other embodiments, K feature vectors of length V may also include feature vectors of other positions in the voxel, such as the feature vector of the center point, or each feature vector of length V is used to indicate the features after fusion of multiple positions, etc. This embodiment does not limit the content of K feature vectors of length V. Each position in the voxel corresponds to M voxel division methods, so the K×V feature vector corresponding to each voxel has K×V×M numerical representations. The multi-scale hash table includes two-dimensional matrices corresponding to M resolutions, and the two-dimensional matrices corresponding to each resolution are combined to obtain a three-dimensional matrix. For the voxels divided according to the mth resolution, the voxel index of the voxel is hashed by the hash function corresponding to the multi-scale hash table to obtain the storage location corresponding to the voxel in the two-dimensional matrix corresponding to the mth resolution, and the K×N matrix corresponding to the mth resolution is stored in the storage location. The voxels divided at various resolutions are processed as described above to obtain the M×K×N matrix corresponding to the voxel.

[0095] Where K represents the number of feature vectors, V represents the length of the feature vector, K is a positive integer, and V is an integer greater than 1, for example: K is 2 19 , V is 16. In other embodiments, K and V may also be other values. This embodiment does not limit the values ​​of K and V. In this embodiment, by dividing the three-dimensional space into different resolution levels, each level has a different voxel size, a higher level has a larger voxel, and a lower level has a smaller voxel. This hierarchical method can maintain the overall structural information at a large scale and retain the detail information at a small scale. Afterwards, near-field radiosity reconstruction is performed based on the eigenvectors of different scales corresponding to each voxel, which can improve the reconstruction accuracy.

[0096] Optionally, the value of the eigenvector stored in the initialized multi-scale hash table is a normally distributed random number. Since the eigenvector will be optimized and trained later, even if the initial value has a large error, it can be corrected through the subsequent training optimization process and will not affect the accuracy of near-field radiosity reconstruction.

[0097] Optionally, the three-dimensional space corresponding to the scanning sphere is divided into voxels according to different resolutions, including: normalizing the three-dimensional space corresponding to the scanning sphere to obtain a normalized space within the range of [-1,1]; and dividing the normalized space into voxels. In this way, the accuracy of subsequent neural network model learning can be improved.

[0098] Step 203 : for each light sampling point on the light corresponding to at least one pixel in each brightness image, determine a feature vector corresponding to the light sampling point based on a multi-scale hash table, and obtain a position code corresponding to each light sampling point.

[0099] In one example, the light corresponding to at least one pixel in each brightness image is determined based on ray tracing technology. Schematically, for each pixel in the brightness image, the position of the pixel and the position of the optical center in the pinhole camera model corresponding to the imaging brightness meter are connected to determine the light information corresponding to the pixel, and the light information includes the direction and position of the light; based on the light information, at least one of the rays corresponding to each pixel is sampled at discrete points to obtain multiple light sampling points on each ray. Afterwards, the feature vector corresponding to each light sampling point is determined based on the multi-scale hash table to obtain the position code corresponding to each light sampling point.

[0100] Among them, the position of the pixel point and the position of the optical center are both positions in the world coordinate system. The position of the pixel point can be converted from the pixel coordinate system to the image coordinate system based on the intrinsic parameters of the imaging luminance meter and the conversion relationship between the pixel coordinate system and the image coordinate system; then based on the focal length of the imaging luminance meter and the conversion relationship between the image coordinate system and the camera coordinate system, the position of the pixel point in the image coordinate system is converted to the camera coordinate system; finally, based on the extrinsic parameters of the imaging luminance meter and the conversion relationship between the camera coordinate system and the world coordinate system, the position of the pixel point in the camera coordinate system is converted to the world coordinate system.

[0101] Accordingly, the position of the optical center can be based on the external parameters of the imaging brightness meter and the conversion relationship between the camera coordinate system and the world coordinate system, and the origin of the camera coordinate system (i.e., the position of the optical center in the camera coordinate system) can be converted to the world coordinate system. Among them, at least one ray can be randomly determined from the rays corresponding to each pixel point, or it can also be the ray of a specified pixel point among each pixel point. This embodiment does not limit the determination method of at least one ray.

[0102] The discrete point sampling of each light ray in the at least one light ray may be performed at equal intervals within the scanning sphere, or may be performed randomly within the scanning sphere. This embodiment does not limit the sampling method.

[0103] refer to Figure 5In the light sampling point acquisition process shown in the figure, the optical center position 501 of the pinhole camera model corresponding to the imaging brightness meter and the line connecting the pixel point 502 in the brightness image form the light corresponding to the pixel point 502. By sampling the light at discrete points, the light sampling point 503 on the light can be obtained.

[0104] Assume that the size of the brightness image collected by the imaging brightness meter at the current position is A×B, according to Figure 5 It can be seen that a ray of light can be obtained by connecting the pixel and the optical center of the pinhole camera model. By traversing each pixel, a total of A×B rays of light are obtained. Among them, the light path and propagation direction are determined by the pixel position of the pixel in the pixel coordinate system and the external parameters of the imaging brightness meter. Randomly sample N rays from the A×B rays, and then sample R discrete points at equal intervals from far to near for each ray. Each discrete point has its own position coordinate x and corresponding light direction d. Among them, N and R are positive integers; the external parameters of the imaging brightness meter are based on the posture information θ and Calculated to determine.

[0105] In other embodiments, the line between the optical center and the specified pixel point can be directly determined to obtain the light corresponding to the specified pixel point, and discrete point sampling can be performed on the light, thereby eliminating the step of selecting at least one light from the light corresponding to each pixel point. This embodiment does not limit the process of obtaining the light sampling points.

[0106] In one example, a multi-scale hash table is used to store feature vectors corresponding to each vertex of each voxel. Accordingly, for each light sampling point on a light corresponding to at least one pixel in each brightness image, a feature vector corresponding to the light sampling point is determined based on the multi-scale hash table, and a position code corresponding to each light sampling point is obtained, including steps 31-34:

[0107] Step 31 : For each voxel of each resolution corresponding to the light sampling point, if the light sampling point is located inside the voxel, the vertex index of the vertex of the voxel is determined based on the position index of the voxel.

[0108] Assume that the mth resolution level divides the normalized 3D scan sphere space into H m ×W m ×L m The length, width and height of each voxel are 1 / L respectively. m ,1 / W m ,1 / H m If the normalized three-dimensional scanning sphere, the three-dimensional coordinates of the light sampling point in the world coordinate system are (x R ,y R ,z R ), then the position index of the light sampling point in the length, width and height directions can be expressed by the following formula:

[0109]

[0110] Among them, (id L ,id W ,id H ) are the indices of the light sampling points in the length, width and height directions of the voxel, Indicates rounding down.

[0111] Optionally, after the three-dimensional scanning sphere space is divided into voxels according to different resolutions, a correspondence is established between the position index of each voxel and the vertex index of the vertex of the voxel. According to the correspondence, the vertex index of 8 vertices corresponding to the voxel can be determined based on the position index of the voxel of each resolution corresponding to the light sampling point, and the vertex index is used to indicate the position coordinates of the corresponding vertex.

[0112] Step 32: Determine the feature vector corresponding to each vertex index in the multi-scale hash table.

[0113] The vertex index is calculated based on the hash function corresponding to the multi-scale hash table to obtain the storage location of the feature vector corresponding to the vertex index in the multi-scale hash table, and the data stored in the storage location is read to obtain the feature vector corresponding to the vertex index. For example: each voxel corresponds to K feature vectors of length V. In this embodiment, the feature vectors corresponding to 8 vertices are determined from the K feature vectors of length V.

[0114] Step 33, feature fusion is performed on the feature vectors corresponding to each vertex index to obtain the position code of the position index corresponding to each resolution.

[0115] In one example, feature fusion is performed on the feature vectors corresponding to each vertex index to obtain a position encoding of the position index corresponding to each resolution, including: based on the distance between the three-dimensional coordinates of the light sampling point and the three-dimensional coordinates of each vertex, the feature vectors corresponding to each vertex index are fused to obtain a position encoding of the position index corresponding to each resolution.

[0116] Schematically, the feature vectors corresponding to each vertex index are interpolated and fused according to the distance based on the trilinear interpolation method. Alternatively, different weights are set for the feature vectors corresponding to different vertex indices, and the weights are negatively correlated with the distance, and the feature vectors corresponding to each vertex index are weightedly fused.

[0117] For example, the storage locations of the vertex index mappings of 8 vertices in the multi-scale hash table are determined by a hash function, and 8 feature vectors of length V are obtained. The distance relationship between the light sampling point and the 8 vertices in the voxel is used to interpolate and merge these 8 feature vectors of length V to obtain a position code of length V. The same hash operation is performed on the voxels corresponding to each resolution to obtain the M position codes of length V corresponding to the light sampling point. These position codes are merged into a feature vector of length M×V in length to obtain the position code corresponding to the light sampling point.

[0118] In other embodiments, for different resolutions, the feature vectors corresponding to the vertex indices may also have different feature fusion methods. For example, for a resolution whose voxel size is greater than a preset threshold, the feature vectors corresponding to the vertex indices of the resolution are fused based on the distance between the position index and each vertex index; for a resolution whose voxel size is less than a preset threshold, the feature vectors corresponding to the vertex indices of the resolution are fused based on the average value of the feature vectors corresponding to each vertex index. This embodiment does not limit the feature fusion method of the feature vectors corresponding to the vertex indices.

[0119] In other embodiments, the multi-scale hash table may also store more or fewer feature vectors for each voxel. In this case, the position index of the voxel may be calculated based on the hash function to determine all feature vectors corresponding to the position index, and all feature vectors may be fused bit by bit to obtain the position code. The position codes corresponding to the various resolutions are merged in the length direction to obtain the position code corresponding to the light sampling point. This embodiment does not limit the specific implementation method of the position code.

[0120] Step 34, merging the position codes of the position indexes corresponding to the various resolutions to obtain the position codes corresponding to the light sampling points.

[0121] In order to more clearly understand the process of obtaining the position code corresponding to the light sampling point in this embodiment, refer to Figure 6 The position coding acquisition process shown in Figure 6 It can be seen that after the light sampling point x in the normalized three-dimensional scanning sphere space is divided into voxels according to different resolutions, the voxels corresponding to the light sampling point x at different resolutions can be obtained. For example: Figure 6In , the light sampling point x belongs to the voxel corresponding to the blue cube and the voxel corresponding to the red cube respectively. The voxel corresponding to the blue cube and the voxel corresponding to the red cube are voxels obtained after voxel division according to different resolutions, wherein the size of the voxel corresponding to the red cube is larger than the size of the voxel corresponding to the blue cube. Based on the position index of each voxel to which the light sampling point x belongs, the position code of length V corresponding to each voxel is determined in the multi-scale hash table; the position codes of length V corresponding to M resolutions are merged to obtain the position code of length V×M corresponding to the light sampling point.

[0122] Step 204 , angle coding the light sampling points based on the angle coding function to obtain the angle coding corresponding to each light sampling point.

[0123] Optionally, the angle encoding function may be a spherical harmonic function, or a sine function, etc. This embodiment does not limit the implementation method of the angle encoding function.

[0124] Among them, spherical harmonics is a function defined on a sphere, which uses a set of orthogonal basis functions to expand any function defined on a unit sphere. These basis functions are called spherical harmonic basis functions, which have a specific mathematical form and can represent the intensity and direction of light at different angles. By applying these basis functions to light signals, spherical harmonics can decompose light signals into a combination of a series of basis functions. Light signals at different angles can be uniquely represented by linear combinations of orthogonal basis functions. Each basis function corresponds to a specific angle, and the selection of these basis functions ensures the orthogonality of spherical harmonics. This decomposition process converts light signals from the spatial domain to the frequency domain, so that the different angular characteristics of light signals can be represented and stored by the coefficients of the basis functions. For the light direction in three-dimensional space, after spherical harmonic encoding, a feature vector with a length of 16 can be obtained, that is, the angle encoding corresponding to the light sampling point.

[0125] The sine function encodes the angle of the light from 0 to 2π through periodic changes in amplitude.

[0126] Optionally, the angle codes corresponding to different light sampling points on the same light ray may be calculated only once, that is, different light sampling points on the same light ray share the same angle code; or, different light sampling points on the same light ray may also be calculated separately based on the angle code function.

[0127] Step 205 : decoding the position code and the angle code based on the pre-created decoding model to obtain the density prediction value and the optical parameter prediction value of each light sampling point.

[0128] Among them, the decoding model is established based on the neural network model.

[0129] In one example, a decoding model includes an optical density network and an optical parameter network connected to the optical density network; accordingly, the position code and the angle code are decoded based on a pre-created decoding model to obtain a density prediction value and an optical parameter prediction value for each light sampling point, including: inputting the position code into the optical density network to decode the position code to obtain a multi-channel density prediction result, outputting the density prediction result of the target channel in the multi-channel density prediction result, and obtaining a density prediction value; inputting the density prediction results of other channels except the target channel in the multi-channel density prediction result and the angle code into the optical parameter network to obtain an optical parameter prediction value.

[0130] Optionally, the optical density network and / or the optical parameter network are established based on a fully connected neural network. Figure 7 The decoding model shown, Figure 7 The optical density network 710 and the optical parameter network 720 are both established based on a fully connected neural network as an example for explanation. Figure 7 It can be seen that the optical density network 710 includes three fully connected intermediate layers 711 with activation functions, and the activation function is a linear rectified unit (Rectified Linear Unit, Relu), and the number of channels of each fully connected intermediate layer 711 is 64. After the position code is processed by the fully connected intermediate layer 711, the density prediction results of 16 channels are obtained. The density prediction result of the first channel of the 16 channels is output as the density prediction result of the target channel to obtain the density prediction value; the density prediction results of the last 15 channels are combined with the angle code of length 16 on the channel to obtain the feature data of 31 channels input into the optical parameter network.

[0131] The optical parameter network 720 includes four fully connected intermediate layers 721 with activation functions. After the feature data of 31 channels are processed by the four fully connected intermediate layers 721, the optical parameter prediction values ​​of C channels are obtained. The value of C is set based on the optical parameter requirement. For example: if the optical parameter requirement indicates that a color image needs to be constructed, the value of C can be 3, and the optical parameter prediction values ​​of each channel represent the prediction values ​​of the three colors of red, green and blue. For another example: if the optical parameter requirement indicates that a multispectral image needs to be constructed, the value of C can be 16, and the optical parameter prediction values ​​of each channel represent the parameter prediction values ​​of 16 bands.

[0132] In other embodiments, the number of layers and channels of the fully connected intermediate layers 711 and 721 may also be greater or lesser values, and this embodiment does not limit the configuration of the fully connected intermediate layers 711 and 721. The activation function in the fully connected intermediate layer 721 may be a Relu or a Sigmoid function, and the activation functions in the optical density network 710 and the optical parameter network 720 may be the same or different, and this embodiment does not limit the configuration of the activation function.

[0133] In other embodiments, the optical density network and / or the optical parameter network may also be established based on other types of neural networks, such as: based on a convolutional neural network. This embodiment does not limit the implementation method of the optical density network and / or the optical parameter network.

[0134] Step 206 : determining a first pixel prediction value of the pixel point based on the density prediction value and the optical parameter prediction value of each light sampling point on the light corresponding to the pixel point, and a preset volume rendering model.

[0135] In one example, the starting point and the end point of each ray are determined, and the starting point and the end point are located within a scanning sphere; based on a volume rendering model, the density prediction value and the optical parameter prediction value of each ray sampling point from the starting point to the end point are integrated to obtain a first pixel prediction value of the pixel point.

[0136] Optionally, the starting point and the end point are respectively located on both sides of the light source to be measured on the light, the starting point and the end point are selected within a preset range close to the light source to be measured, and the light sampling point is located between the starting point and the end point.

[0137] Schematically, the volume rendering model is represented by the following formula:

[0138] G = ∫T(t)σ(t)c(t,d)dt;

[0139]

[0140] Among them, G represents the first pixel prediction value of the pixel; σ(t) represents the density prediction value of the light sampling point on the light corresponding to the pixel at position x(t); c(t, d) is the predicted value of the optical parameter of the light sampling point at x(t) that changes with direction; T(t) is the transmittance from the starting point to position t, which represents the probability that the light is not absorbed before position t.

[0141] Optionally, in actual calculation, the above volume rendering model can be approximated by discretization. Discretize the integral range and write the integral of the volume rendering formula as a summation. In this way, the volume rendering model uses the density prediction value and optical parameter prediction value of each light sampling point to perform volume rendering on the light to obtain the first pixel prediction value of the pixel point. After traversing the pixel points corresponding to all the light rays, the first pixel prediction value corresponding to each pixel point obtained by the current imaging near-field photometer posture can be obtained.

[0142] Step 207, based on the first pixel prediction value and the actual pixel value of the pixel point on the brightness image, iteratively optimize the feature vector in the multi-scale hash table and the model parameters of the decoding model to obtain a trained multi-scale hash table and a trained decoding model.

[0143] For each pixel, the first predicted pixel value I of the current pixel can be obtained by using light sampling, position encoding, neural network prediction, and volume rendering. pre In addition, in step 201, the actual pixel value I of the pixel point collected by the imaging brightness meter is also obtained. obs The loss function can be used to calculate the difference between the first pixel prediction value and the actual pixel value, and based on the difference, the gradient descent method is used to update the network parameters of the decoding model and the feature vector stored in the multi-scale hash table.

[0144] Optionally, the loss function may be an L2 norm or an L1 norm, etc. This embodiment does not limit the implementation method of the loss function.

[0145] Among them, the L1 norm L1-loss is expressed by the following formula:

[0146] L1-loss=||I pre -I obs ||1;

[0147] The L2-norm L2-loss is expressed by the following formula:

[0148]

[0149] Based on this difference, the gradient descent method is used to update the network parameters of the decoding model and the feature vectors stored in the multi-scale hash table, including: using automatic derivation to calculate the gradient of the loss function value to the network parameters and feature vectors; using the Adam optimizer to update the network parameters and feature vectors. The gradient is back-propagated through the decoding model and linear interpolation, and then accumulated in the found feature vector.

[0150] Schematically, let the parameter of the i-th iteration be ω i , Adam update parameter formula is expressed as follows:

[0151]

[0152] Among them, g is the current loss function parameter ω i The gradient of β1 and β2 are adjustable parameters, generally β1 = 0.9, β2 = 0.999. In other embodiments, β1 and β2 may also be other values. This embodiment does not limit the values ​​of β1 and β2. μ and ν represent the first-order momentum and second-order momentum of the gradient respectively; η is a preset coefficient to increase the stability of the denominator. For multi-scale hash tables, a smaller η, such as 10, needs to be selected. -15 , for the decoding model, we can set η = 10 -8 .

[0153] Step 208, based on the trained multi-scale hash table and the trained decoding model, determine the second pixel prediction value corresponding to the second posture on the scanning sphere, so as to reconstruct the near-field radiance of the light source to be measured based on the second pixel prediction value; wherein the second posture is different from the first posture.

[0154] After training the multi-scale hash table and decoding model, the hash table and the fully connected network are used to predict the two-dimensional brightness image (not the real image, but a predicted image composed of the predicted values ​​of each second pixel) corresponding to the imaging brightness meter under a given new imaging brightness meter posture (i.e., the second posture).

[0155] The second pixel prediction value corresponding to the second posture includes: the second pixel prediction value corresponding to each pixel point on the image when the image is taken in the second posture. By combining the brightness images corresponding to each first posture with each predicted second pixel prediction value, a near-field radiance image of the luminous body to be measured can be reconstructed. Assume that the reconstructed near-field radiance image is as follows: Figure 8 As shown, Figure 8 In this example, C=1, which indicates grayscale, is used to illustrate. Figure 8 It can be seen that a relatively realistic near-field radiance image of the luminous object to be tested can be reconstructed using a small amount of brightness images.

[0156] In summary, the near-field radiometry reconstruction method provided in the present embodiment obtains a brightness image collected by an imaging brightness meter at the first position of a scanning sphere; performs voxel division on the three-dimensional space corresponding to the scanning sphere according to different resolutions, and constructs a multi-scale hash table based on the voxels corresponding to the different resolutions, so as to store the feature vector of each voxel through the multi-scale hash table; for each light sampling point on the light corresponding to at least one pixel point in each brightness image, determines the feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtains the position code corresponding to each light sampling point; performs angle coding on the light sampling point based on the angle coding function, and obtains the angle code corresponding to each light sampling point; decodes the position code and the angle code based on a pre-created decoding model, and obtains the density prediction value and the optical parameter prediction value of each light sampling point; and performs a multi-scale hash table based on each light sampling point on the light corresponding to the pixel point. The method uses a density prediction value and an optical parameter prediction value of a point, and a preset volume rendering model to determine a first pixel prediction value of the pixel point; based on the first pixel prediction value and the actual pixel value of the pixel point on the brightness image, iteratively optimizes the feature vector in the multi-scale hash table and the model parameters of the decoding model to obtain a trained multi-scale hash table and a trained decoding model; based on the trained multi-scale hash table and the trained decoding model, determines a second pixel prediction value corresponding to a second posture on the scanning sphere, so as to reconstruct the near-field radiance of the luminous body to be measured based on the second pixel prediction value; this method can solve the problem that the traditional near-field radiance reconstruction method needs to collect a large number of brightness images, resulting in low reconstruction efficiency; since it is no longer necessary to collect brightness images at both the first posture and the second posture, near-field radiance reconstruction can be achieved using a small number of brightness images collected at the first posture, thereby improving the near-field radiance reconstruction efficiency.

[0157] In addition, the near-field radiometry reconstruction method of this embodiment can also estimate the three-dimensional radiation field information well in the case of a small number of brightness images, and has good data utilization.

[0158] In addition, the near-field radiometry reconstruction method of this embodiment can measure luminous bodies of various shapes and properties and has good versatility.

[0159] In order to more clearly understand the near-field radiometric reconstruction method provided by the present application, an example is given below to illustrate the method. Fig. 9 , the method includes the following steps:

[0160] Step 91, obtaining a brightness image collected by an imaging brightness meter on a scanning sphere in a first posture;

[0161] Step 92, establishing a multi-scale hash table and a decoding model;

[0162] Step 93, determining whether each brightness image has been traversed; if not, executing step 94; if yes, executing step 99;

[0163] Step 94, for each untraversed brightness image, determine whether each pixel in the current brightness image has been traversed; if so, execute step 93; if not, execute step 95;

[0164] Step 95, for each pixel point that has not been traversed, determine the position code of the light sampling point on the light corresponding to the pixel point based on the multi-scale hash table; determine the angle code of the light sampling point based on the angle coding function;

[0165] Step 96, inputting the position code and the angle code into a decoding model for decoding to obtain a first pixel prediction value corresponding to the current pixel point;

[0166] Step 97, calculating the difference between the first pixel prediction value and the actual pixel value corresponding to the current pixel point based on the loss function to obtain a loss function value;

[0167] Step 98, updating the network parameters of the decoding model and the feature vector stored in the multi-scale hash table based on the loss function value; executing step 94;

[0168] Step 99, based on the trained multi-scale hash table and the trained decoding model, determine the second pixel prediction value corresponding to the second posture on the scanning sphere, so as to reconstruct the near-field radiance of the luminous object to be measured based on the second pixel prediction value.

[0169] Fig.10 FIG. 4 is a flowchart of a near-field radiometric reconstruction apparatus according to an embodiment of the present disclosure. Fig.10 As shown, the device includes: an image acquisition module 1010, a voxel division module 1020, a position encoding module 1030, an angle encoding module 1040, a parameter prediction module 1050, a pixel prediction module 1060, a model training module 1070 and a radiometry reconstruction module 1080.

[0170] An image acquisition module 1010 is used to acquire a brightness image collected by the imaging brightness meter on the scanning sphere in a first position; wherein different brightness images have different first positions;

[0171] A voxel partitioning module 1020 is used to perform voxel partitioning on the three-dimensional space corresponding to the scanning sphere according to different resolutions, and to construct a multi-scale hash table based on the voxels corresponding to the different resolutions, so as to store the feature vector of each voxel through the multi-scale hash table;

[0172] A position encoding module 1030 is used to determine, for each light sampling point on the light corresponding to at least one pixel point in each brightness image, a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtain a position encoding corresponding to each light sampling point;

[0173] An angle coding module 1040, used to perform angle coding on the light sampling points based on an angle coding function to obtain an angle coding corresponding to each light sampling point;

[0174] A parameter prediction module 1050 is used to decode the position code and the angle code based on a pre-created decoding model to obtain a density prediction value and an optical parameter prediction value of each light sampling point;

[0175] A pixel prediction module 1060, configured to determine a first pixel prediction value of the pixel point based on a density prediction value and an optical parameter prediction value of each light sampling point on the light corresponding to the pixel point, and a preset volume rendering model;

[0176] A model training module 1070, configured to iteratively optimize the feature vectors in the multiscale hash table and the model parameters of the decoding model based on the first pixel prediction value and the actual pixel value of the pixel point on the brightness image, so as to obtain a trained multiscale hash table and a trained decoding model;

[0177] The radiometry reconstruction module 1080 is used to determine a second pixel prediction value corresponding to a second posture on the scanning sphere based on the trained multi-scale hash table and the trained decoding model, so as to reconstruct the near-field radiometry of the light source to be measured based on the second pixel prediction value; wherein the second posture is different from the first posture.

[0178] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0179] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0180] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0181] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0182] Fig.111 is a block diagram of an apparatus 1900 for near-field radiometric reconstruction according to an exemplary embodiment. For example, the apparatus 1900 may be provided as a server or a terminal device. Fig.11 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0183] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0184] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.

[0185] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A near-field radiosity reconstruction method, characterized in that: Used in an imaging near-field photometer, the imaging near-field photometer comprises: a rotating scanning mechanism and an imaging luminance meter mounted on the rotating scanning mechanism; the rotating scanning mechanism is used to drive the imaging luminance meter to move on the surface of a scanning sphere; the imaging luminance meter is used to collect a luminance image of a light source to be measured located inside the scanning sphere; the method comprises: Acquire a brightness image collected by the imaging brightness meter on the scanning sphere in a first position; wherein different brightness images have different first positions; Dividing the three-dimensional space corresponding to the scanning sphere into voxels according to different resolutions, and constructing a multi-scale hash table based on the voxels corresponding to the different resolutions, so as to store the feature vector of each voxel through the multi-scale hash table; For each light sampling point on the light corresponding to at least one pixel point in each brightness image, determining a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtaining a position code corresponding to each light sampling point; Performing angle coding on the light sampling points based on an angle coding function to obtain an angle coding corresponding to each light sampling point; Decoding the position code and the angle code based on a pre-created decoding model to obtain a density prediction value and an optical parameter prediction value of each light sampling point; Determine a first pixel prediction value of the pixel point based on a density prediction value and an optical parameter prediction value of each light sampling point on the light corresponding to the pixel point, and a preset volume rendering model; Iteratively optimize the feature vectors in the multiscale hash table and the model parameters of the decoding model based on the first pixel prediction value and the actual pixel value of the pixel point on the brightness image to obtain a trained multiscale hash table and a trained decoding model; Based on the trained multi-scale hash table and the trained decoding model, a second pixel prediction value corresponding to a second posture on the scanning sphere is determined to reconstruct the near-field radiance of the light source to be measured based on the second pixel prediction value; wherein the second posture is different from the first posture.

2. The method according to claim 1, characterized in that: The multi-scale hash table is used to store the feature vectors corresponding to each vertex of each voxel; accordingly, The step of determining, for each light sampling point on a light corresponding to at least one pixel point in each brightness image, a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtaining a position code corresponding to each light sampling point includes: For each voxel of each resolution corresponding to the light sampling point, if the light sampling point is located inside the voxel, determining the vertex index of the vertex of the voxel based on the position index of the voxel; Determine, in the multi-scale hash table, a feature vector corresponding to each vertex index; Perform feature fusion on the feature vectors corresponding to each vertex index to obtain the position encoding of the position index corresponding to each resolution; The position codes of the position indexes corresponding to various resolutions are merged to obtain the position codes corresponding to the light sampling points.

3. The method according to claim 2, characterized in that The feature fusion of the feature vectors corresponding to each vertex index to obtain the position encoding of the position index corresponding to each resolution includes: Based on the distance between the three-dimensional coordinates of the light sampling point and the three-dimensional coordinates of each vertex, the feature vectors corresponding to each vertex index are fused to obtain the position encoding of the position index corresponding to each resolution.

4. The method according to claim 1, characterized in that The step of determining, for each light sampling point on a light corresponding to at least one pixel point in each brightness image, a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtaining a position code corresponding to each light sampling point includes: For each pixel in the brightness image, connect the position of the pixel and the position of the optical center in the pinhole camera model corresponding to the imaging brightness meter to determine the light information corresponding to the pixel, wherein the light information includes the direction and position of the light; Based on the light information, discrete point sampling is performed on at least one light ray corresponding to each pixel point to obtain a plurality of light ray sampling points on each light ray; The feature vector corresponding to each light sampling point is determined based on the multi-scale hash table to obtain the position code corresponding to each light sampling point.

5. The method according to claim 1, characterized in that The decoding model includes an optical density network and an optical parameter network connected to the optical density network; accordingly, The decoding of the position code and the angle code based on the pre-created decoding model to obtain a density prediction value and an optical parameter prediction value of each light sampling point includes: Inputting the position code into the optical density network to decode the position code to obtain a multi-channel density prediction result, outputting the density prediction result of the target channel in the multi-channel density prediction result, and obtaining the density prediction value; The density prediction results of other channels except the target channel in the multi-channel density prediction results and the angle code are input into the optical parameter network to obtain the optical parameter prediction value.

6. The method according to claim 5, characterized in that The optical density network and / or the optical parameter network are established based on a fully connected neural network.

7. The method according to claim 1, characterized in that The determining the first pixel prediction value of the pixel point based on the density prediction value and the optical parameter prediction value of each light sampling point on the light corresponding to the pixel point and a preset volume rendering model includes: Determining a starting point and an end point of each ray, wherein the starting point and the end point are located within the scanning sphere; Based on the volume rendering model, the density prediction value and the optical parameter prediction value of each light sampling point from the starting point to the end point are integrated to obtain a first pixel prediction value of the pixel point.

8. The method according to claim 7, characterized in that The volume rendering model is expressed by the following formula: G = ∫T(t)σ(t)c(t,d)dt; Among them, G represents the first pixel prediction value of the pixel point; σ(t) represents the density prediction value of the light sampling point on the light corresponding to the pixel point at the position x(t); c(t, d) is the optical parameter prediction value of the light sampling point at x(t) that changes with the direction; T(t) is the transmittance from the starting point to the position t, which represents the probability that the light is not absorbed before the position t.

9. A near-field radiometric reconstruction device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 8 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images