Near-field radiometry reconstruction method, device, and storage medium

Abstract

The present disclosure relates to the field of radiometric technology, including near-field radiometry reconstruction methods, devices and storage media. Multi-scale position coding feature vectors of three-dimensional space are stored through a multi-scale hash table, and the radiation field information of the spatial point is obtained by decoding the feature vectors through a decoding model. The radiation field information of these spatial points is combined and then the first pixel prediction value of the pixel point in the two-dimensional brightness image collected at the first pose is predicted using ray tracing and volume rendering technology. The difference between the first pixel prediction value and the actual pixel value of the pixel point is calculated, and the decoding model and the multi-scale hash table are optimized by the gradient descent method. It is possible to encode the overall three-dimensional space light field radiometry information, thereby realizing three-dimensional modeling of light field radiometry, and realizing the deduction and reconstruction of light field radiometry information at any angle and distance (i.e., the second pose), which can provide more accurate and reliable support for light field radiometry measurement and analysis.

Description

Technical Field

[0001] The present disclosure relates to the technical field of radiometry, and in particular to a near-field radiometry reconstruction method, device, and storage medium. Background Art

[0002] Radiometry is the science that studies electromagnetic radiation. The instrument that measures radiometric parameters is called a radiometer. In the field of optics, radiometry primarily studies electromagnetic radiation in and near visible light bands. In practical applications, the study of visible light bands requires not only objective measurement of electromagnetic radiation but also the physiological and psychological responses of the human eye to electromagnetic radiation. Radiometry, which incorporates human vision, is also known as photometry. The physical parameters involved in photometry include luminous intensity, luminous flux, illuminance, and brightness. The instrument that measures these photometric parameters is called a near-field photometer (NFG).

[0003] A typical NFG structure consists of two perpendicular rotating axes, a scanning frame connected to the axes, and an imaging luminance meter mounted on the scanning frame. As the scanning frame rotates, the imaging luminance meter's trajectory lies on a scanning sphere with a radius of R. The luminous object to be measured is placed within the scanning sphere. The maximum size of the luminous object to be measured is less than the diameter of the scanning sphere, 2R. The core components of the imaging luminance meter are two-dimensional imaging sensors, such as charge-coupled devices (CCDs) and complementary metal-oxide-semiconductors (CMOSs). These sensors can capture luminance images at all accessible locations during the spherical scanning process. Because the luminance value in a given direction is independent of the light propagation distance, even if the imaging luminance meter's trajectory remains on the scanning sphere, the luminance image captured at a specific location can describe the light radiation characteristics at any distance in that direction. Parameters such as luminous intensity and illuminance at different distances can be calculated based on conversion relationships between photometric parameters. Among them, the process of solving parameters such as luminous intensity, luminous flux, and illuminance based on the brightness image measured by the imaging luminance meter and drawing the full-space luminosity distribution is the process of near-field radiometry reconstruction.

[0004] To ensure the accuracy of near-field radiometry reconstruction, traditional near-field radiometry reconstruction methods usually require an imaging luminance meter to scan a large number of angles and record the two-dimensional light field brightness image corresponding to each angle. Then, the near-field radiometry is reconstructed based on the collected brightness images.

[0005] However, capturing brightness images at multiple scanning angles requires controlling the movement of the imaging luminance meter at multiple positions, and reconstructing the near-field radiance by calculating a large number of brightness images (for example, generally 2,664 brightness images at different angles are required). This results in high computational cost and long calculation time, all of which lead to the low efficiency of traditional near-field radiance reconstruction. Summary of the Invention

[0006] In view of this, the present disclosure proposes a near-field radiometry reconstruction method, device, and storage medium. Based on the principle of implicit representation of the radiation field, a multi-scale hash table is used to store the feature vectors of voxels in three-dimensional space. A decoding model is used as a feature vector decoder. Under the supervision of multiple brightness images taken at the first pose and the volume rendering formula, the multi-scale hash table and the decoding model are optimized simultaneously, thereby realizing the reconstruction of the three-dimensional radiation field and the inversion of the radiation field at any position in space (the second pose). At this time, there is no need to collect brightness images at the first pose and the second pose. Near-field radiometry reconstruction can be achieved using a small number of brightness images, thereby improving the efficiency of near-field radiometry reconstruction.

[0007] According to one aspect of the present disclosure, a near-field radiometry reconstruction method is provided for use in an imaging near-field photometer. The imaging near-field photometer comprises: a rotating scanning mechanism and an imaging luminance meter mounted on the rotating scanning mechanism; the rotating scanning mechanism is configured to drive the imaging luminance meter to move on the surface of a scanning sphere; the imaging luminance meter is configured to capture a luminance image of a luminous object to be measured located within the scanning sphere; the method comprises:

[0008] Acquire a brightness image captured by the imaging luminance meter on the scanning sphere in a first position; wherein different brightness images have different first positions;

[0009] Dividing the three-dimensional space corresponding to the scanning sphere into voxels according to different resolutions, and constructing a multi-scale hash table based on the voxels corresponding to the different resolutions, so as to store a feature vector of each voxel through the multi-scale hash table;

[0010] For each light sampling point on the light corresponding to at least one pixel in each brightness image, determining a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtaining a position code corresponding to each light sampling point;

[0011] Performing angle coding on the light sampling points based on an angle coding function to obtain an angle coding corresponding to each light sampling point;

[0012] Decoding the position code and the angle code based on a pre-created decoding model to obtain a density prediction value and an optical parameter prediction value of each light sampling point;

[0013] Determining a first pixel prediction value of the pixel point based on a density prediction value and an optical parameter prediction value of each light sampling point on the light corresponding to the pixel point, and a preset volume rendering model;

[0014] Iteratively optimizing the feature vectors in the multi-scale hash table and the model parameters of the decoding model based on the first pixel prediction value and the actual pixel value of the pixel point on the luminance image to obtain a trained multi-scale hash table and a trained decoding model;

[0015] Based on the trained multi-scale hash table and the trained decoding model, a second pixel prediction value corresponding to a second posture on the scanning sphere is determined to reconstruct the near-field radiance of the luminous object to be measured based on the second pixel prediction value; wherein, the second posture is different from the first posture.

[0016] In a possible implementation, the multi-scale hash table is used to store the feature vectors corresponding to each vertex of each voxel; accordingly,

[0017] The step of determining, for each light sampling point on a light ray corresponding to at least one pixel in each brightness image, a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtaining a position code corresponding to each light sampling point includes:

[0018] For each voxel of each resolution corresponding to the light sampling point, if the light sampling point is located inside the voxel, determining a vertex index of a vertex of the voxel based on a position index of the voxel;

[0019] Determining the feature vector corresponding to each vertex index in the multi-scale hash table;

[0020] Perform feature fusion on the feature vectors corresponding to each vertex index to obtain the position encoding of the position index corresponding to each resolution;

[0021] The position codes of the position indexes corresponding to the various resolutions are merged to obtain the position codes corresponding to the light sampling points.

[0022] In a possible implementation, the feature fusion of the feature vectors corresponding to the vertex indices to obtain the position encoding of the position index corresponding to each resolution includes:

[0023] Based on the distance between the three-dimensional coordinates of the light sampling point and the three-dimensional coordinates of each vertex, the feature vectors corresponding to each vertex index are fused to obtain the position code of the position index corresponding to each resolution.

[0024] In one possible implementation, for each light sampling point on a light corresponding to at least one pixel in each brightness image, determining a feature vector corresponding to the light sampling point based on the multi-scale hash table to obtain a position code corresponding to each light sampling point includes:

[0025] For each pixel in the brightness image, connecting the position of the pixel with the position of the optical center in the pinhole camera model corresponding to the imaging brightness meter to determine light information corresponding to the pixel, the light information including the direction and position of the light;

[0026] Based on the light information, discrete point sampling is performed on at least one light ray corresponding to each pixel point to obtain a plurality of light sampling points on each light ray;

[0027] The feature vector corresponding to each light sampling point is determined based on the multi-scale hash table to obtain the position code corresponding to each light sampling point.

[0028] In a possible implementation, the decoding model includes an optical density network and an optical parameter network connected to the optical density network; accordingly,

[0029] The decoding of the position code and the angle code based on the pre-created decoding model to obtain a density prediction value and an optical parameter prediction value of each light sampling point includes:

[0030] Inputting the position code into the optical density network to decode the position code to obtain a multi-channel density prediction result, outputting the density prediction result of the target channel in the multi-channel density prediction result to obtain the density prediction value;

[0031] The density prediction results of other channels except the target channel in the multi-channel density prediction results and the angle code are input into the optical parameter network to obtain the optical parameter prediction value.

[0032] In a possible implementation, the optical density network and / or the optical parameter network are established based on a fully connected neural network.

[0033] In one possible implementation, determining the first pixel prediction value of the pixel point based on the density prediction value and the optical parameter prediction value of each ray sampling point on the ray corresponding to the pixel point and a preset volume rendering model includes:

[0034] Determining a start point and an end point of each ray, wherein the start point and the end point are located within the scanning sphere;

[0035] Based on the volume rendering model, density prediction values ​​and optical parameter prediction values ​​of each light sampling point from the starting point to the end point are integrated to obtain a first pixel prediction value of the pixel point.

[0036] In one possible implementation, the volume rendering model is expressed by the following formula:

[0037] G=∫T(t)σ(t)c(t,d)dt;

[0038]

[0039] Where G represents the first pixel prediction value of the pixel; σ(t) represents the density prediction value of the light sampling point on the light corresponding to the pixel at position x(t); c(t, d) is the predicted value of the optical parameter of the light sampling point at x(t) as it changes with direction; T(t) is the transmittance from the starting point to position t, which indicates the probability that the light is not absorbed before position t.

[0040] According to another aspect of the present disclosure, a near-field radiometry reconstruction device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0041] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0042] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0043] By acquiring the brightness image captured by the imaging luminance meter at the first posture of the scanning sphere; dividing the three-dimensional space corresponding to the scanning sphere into voxels according to different resolutions, and constructing a multi-scale hash table based on the voxels corresponding to different resolutions, so as to store the feature vector of each voxel through the multi-scale hash table; for each light sampling point on the light corresponding to at least one pixel point in each brightness image, determining the feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtaining the position code corresponding to each light sampling point; performing angle coding on the light sampling point based on the angle coding function, and obtaining the angle code corresponding to each light sampling point; decoding the position code and angle code based on the pre-created decoding model, and obtaining the density prediction value and optical parameter prediction value of each light sampling point; based on the density prediction value and optical parameter prediction value of each light sampling point on the light corresponding to the pixel point, The method uses a plurality of pixel prediction values ​​and a preset volume rendering model to determine a first pixel prediction value of the pixel point; based on the first pixel prediction value and the actual pixel value of the pixel point on the luminance image, iteratively optimizes the feature vector in the multi-scale hash table and the model parameters of the decoding model to obtain a trained multi-scale hash table and a trained decoding model; based on the trained multi-scale hash table and the trained decoding model, determines a second pixel prediction value corresponding to a second posture on the scanning sphere, so as to reconstruct the near-field radiance of the luminous object to be measured based on the second pixel prediction value; this method can solve the problem that the traditional near-field radiance reconstruction method needs to collect a large number of luminance images, resulting in low reconstruction efficiency; since it is no longer necessary to collect luminance images at both the first and second postures, near-field radiance reconstruction can be achieved using a small number of luminance images collected at the first posture, thereby improving the near-field radiance reconstruction efficiency.

[0044] In addition, the near-field radiometry reconstruction method of this embodiment can also well estimate the three-dimensional radiation field information in the case of a small number of brightness images, and has good data utilization.

[0045] In addition, the near-field radiometry reconstruction method of this embodiment can measure luminous objects of various shapes and properties, and has good versatility.

[0046] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0048] Figure 1 A structural diagram of an imaging near-field photometer according to an embodiment of the present disclosure is shown;

[0049] Figure 2A flowchart showing a near-field radiometry reconstruction method according to an embodiment of the present disclosure is shown;

[0050] Figure 3 A schematic diagram illustrating a brightness image acquisition process according to an embodiment of the present disclosure is shown;

[0051] Figure 4 A schematic diagram illustrating a world coordinate system according to an embodiment of the present disclosure is shown;

[0052] Figure 5 A schematic diagram illustrating a process of acquiring light sampling points according to an embodiment of the present disclosure is shown;

[0053] Figure 6 A schematic diagram illustrating a process of obtaining a position code according to an embodiment of the present disclosure is shown;

[0054] Figure 7 A schematic diagram illustrating a decoding model according to an embodiment of the present disclosure is shown;

[0055] Figure 8 A schematic diagram showing a near-field radiometry image reconstructed according to an embodiment of the present disclosure;

[0056] Figure 9 A flowchart showing a near-field radiometry reconstruction method according to another embodiment of the present disclosure is shown;

[0057] Figure 10 A block diagram of a near-field radiometry reconstruction apparatus according to an embodiment of the present disclosure is shown;

[0058] Figure 11 A block diagram of a near-field radiometry reconstruction apparatus according to another embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0059] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

[0060] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0061] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0062] First, several terms involved in this application are introduced.

[0063] Voxel: Short for Volume Pixel, it combines the concepts of pixel, volume, and element, representing the smallest unit in three-dimensional space. In computer graphics, voxels are represented by small cubes in a 3D model. Each voxel can be assigned different attributes, such as color and material, to create complex 3D scenes.

[0064] The pinhole camera model is a simplified optical imaging model based on central perspective projection, projecting an object in three-dimensional space onto a two-dimensional imaging plane through a pinhole. To describe the pinhole camera model mathematically, four coordinate systems are generally required: the world coordinate system, the camera coordinate system, the image coordinate system, and the pixel coordinate system. The imaging process of a three-dimensional object on a two-dimensional image is expressed through coordinate system transformations.

[0065] Among them, the world coordinate system is an absolute coordinate system that describes the objective world. Since the camera can be located at any position in the objective world, it is necessary to establish a reference coordinate system to describe the position of the camera, and use it to describe the position of any spatial point in the environment. Generally, it is expressed by (X w ,Y w ,Z w ) represents the world coordinate value.

[0066] Camera coordinate system: It is a three-dimensional coordinate system established with the optical center of the camera as the reference point. It is used to describe the position of any spatial point in the camera space. The establishment method is: the optical center of the camera is the coordinate origin, the main optical axis of the camera is the Z axis, the horizontal direction parallel to the image plane is the X axis, and the vertical direction parallel to the image plane is the Y axis. It is generally expressed by (X c ,Y c ,Z c ) represents the camera coordinate value.

[0067] Image coordinate system: It is a two-dimensional plane coordinate system defined on the imaging plane, expressed in actual physical units (mm). It is established as follows: the intersection of the camera's main optical axis and the image plane is used as the origin. The intersection is called the principal point. The X-axis is parallel to the image horizontally to the right, and the Y-axis is parallel to the image vertically downward. The image coordinate value is generally represented by (x, y).

[0068] Pixel Coordinate System: In computers, digital images are discretized and stored as a pixel matrix. Therefore, a pixel coordinate system is necessary to describe camera images. Pixel coordinates can be understood as the position index of a pixel in the image plane. This system is established by taking the upper-left corner of the image plane as the origin, with the u-axis parallel to the image coordinate system's X-axis, pointing horizontally to the right, and the v-axis parallel to the image coordinate system's Y-axis, pointing vertically downward. Pixel coordinate values ​​are generally represented by (u, v).

[0069] Camera imaging can be understood as the process of projecting an object in three-dimensional space onto a two-dimensional image plane. This projection process ultimately establishes a mapping relationship between points in three-dimensional space and points on the image plane. Therefore, the projection process can be expressed through coordinate transformations between points in three-dimensional space and points on the two-dimensional image plane. Similarly, the position of the object in three-dimensional space can be determined by inversely transforming the coordinates of the pixels on the two-dimensional image.

[0070] Figure 1 FIG. 1 shows a structural diagram of an imaging near-field photometer according to an embodiment of the present disclosure. Figure 1 As shown, the imaging near-field photometer includes a rotating scanning mechanism 110 and an imaging luminance meter 120 mounted on the rotating scanning mechanism.

[0071] Imaging luminance meter 120 is used to capture a luminance image of the luminous object to be measured within the scanning sphere. Alternatively, imaging luminance meter 120 may be a camera or camcorder capable of capturing luminance images. The imaging sensor in imaging luminance meter 120 may be a CCD or CMOS sensor. This embodiment does not limit the implementation of imaging luminance meter 120.

[0072] The rotating scanning mechanism 110 is used to drive the imaging luminance meter 120 to move on the surface of the scanning sphere. Schematically, the rotating scanning mechanism 110 includes: two rotating shafts 111 perpendicular to each other, and a scanning frame 112 connected to the rotating shafts.

[0073] The scanning frame 112 is used to mount the imaging luminance meter 120. Rotation of the rotation axis 111 drives the scanning frame 112 to rotate around the rotation axis 111, which in turn drives the imaging luminance meter 120 to move. The imaging luminance meter 120 can then move on the surface of a scanning sphere centered on the luminous object to be measured, with its center of mass as the moving point. The optical axis of the imaging luminance meter 120 always points toward the center of the sphere. The luminous object to be measured is located within the scanning sphere, specifically at its center.

[0074] In other embodiments, the rotating scanning mechanism 110 may be implemented in other ways, for example, the number of rotating axes in the rotating scanning mechanism 110 may be less or more, or the rotating scanning mechanism 110 may also include a supporting structure connected to the rotating axis 111, etc. This embodiment does not limit the implementation method of the rotating scanning mechanism 110.

[0075] Optionally, the rotating scanning mechanism 110 is also connected to a control device, which sets the moving path of the imaging luminance meter 120 according to the characteristics of the luminous body to be measured and the measurement requirements. Accordingly, the control device is connected to the motor in the rotating shaft 111 to control the motor to drive the rotating shaft 111 to rotate, thereby driving the imaging luminance meter 120 to move along the moving path through the scanning frame, forming a moving trajectory corresponding to the moving path on the surface of the scanning sphere, so as to shoot at different angles of the luminous body to be measured. Each time the imaging luminance meter 120 reaches the designated shooting position to shoot the luminous body to be measured, the acquisition position of the imaging luminance meter 120 is recorded to obtain a series of two-dimensional brightness images and the acquisition posture corresponding to each brightness image, which includes the acquisition position and orientation data of the imaging luminance meter 120.

[0076] Traditional near-field radiometry reconstruction methods require an imaging luminance meter to capture luminance images at numerous locations, stitching together the luminance images of the entire spatial light field to achieve near-field radiometry reconstruction. This requires multiple movements and acquisitions within the scanning sphere, followed by calculations on the numerous luminance images (typically over 1,000). This results in low near-field radiometry reconstruction efficiency.

[0077] In order to improve the efficiency of near-field radiometry reconstruction, in some embodiments, the efficiency of near-field radiometry reconstruction is improved by increasing the control and positioning speed of the scanning frame, and / or increasing the splicing calculation speed of the brightness image. However, in these embodiments, since the near-field radiometry reconstruction still requires the acquisition of a large number of brightness images, the degree of efficiency improvement is limited. Based on this, the present application provides a near-field radiometry reconstruction method, which can collect a small amount of brightness images to train a multi-scale hash table and a decoding model, and realize pixel reconstruction of other positions in the light field based on the trained multi-scale hash table and decoding model, thereby realizing near-field radiometry reconstruction. In this process, since the number of brightness images used is reduced, the reconstruction efficiency of the near-field radiometry can be greatly improved.

[0078] The following is a detailed description of the near-field radiometry reconstruction method provided in this application. In this application, the near-field radiometry reconstruction method can be used in an imaging near-field photometer, or in an electronic device with computing capabilities that is communicatively connected to the imaging near-field photometer, including but not limited to a user terminal or server. The user terminal can be a computer, tablet computer, mobile phone, etc. This embodiment does not limit the implementation method of the electronic device.

[0079] Figure 2 FIG. 5 is a flowchart of a near-field radiometric reconstruction method according to an embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0080] Step 201: Acquire a brightness image captured by an imaging luminance meter on a scanning sphere at a first posture.

[0081] In this embodiment, the imaging luminance meter collects luminance images at multiple first poses when scanning the surface of the sphere. Different luminance images have different first poses, and the number of luminance images is less than the number of images required for image stitching in the traditional near-field radiometry reconstruction method.

[0082] refer to Figure 3 The image capture process diagram shows the imaging luminance meter capturing luminance images at multiple first poses on the upper hemisphere of the scanning sphere. In other embodiments, the first poses can also be distributed on the lower hemisphere. This embodiment does not limit the selection of the first pose.

[0083] The number of first poses (i.e., the number of luminance images) can ensure the training accuracy of the multi-scale hash table and decoding model described below. Schematically, the number can be 50 to 100 luminance images (much less than the at least 1,000 luminance images required by the traditional near-field radiometry reconstruction method). This embodiment does not limit the number of first poses.

[0084] In this embodiment, the first pose is represented by the position coordinate value and direction vector of the imaging luminance meter in the world coordinate system. The position coordinate value indicates the position of the imaging luminance meter on the surface of the scanning sphere, and the direction vector indicates the shooting angle of the imaging luminance meter when capturing the luminance image.

[0085] Schematically, refer to Figure 4 The world coordinate system is established by taking the center of the scanning sphere as the origin, determining the x-axis and y-axis respectively on the plane parallel to the horizontal plane, and determining the positive direction of the z-axis upward perpendicular to the horizontal plane to establish a three-dimensional rectangular coordinate system to obtain the world coordinate system. At this time, the coordinates of the center of mass of the imaging luminance meter can be obtained by the two azimuth angles θ and Determine, where θ is the angle between the line connecting the center of mass of the imaging luminance meter and the center of the sphere and the z-axis, is the angle between the projection point of the imaging luminance meter on the xoy plane and the line connecting the sphere center and the positive half axis of the x-axis. c ,y c ,z c ) can be determined by the following formula:

[0086]

[0087] Where r represents the radius of the scanning sphere;

[0088] Direction vector It can be determined by the following formula:

[0089]

[0090] In other embodiments, the world coordinate system may be established in other ways. Accordingly, the formula for determining the first pose may also be adaptively changed. This embodiment does not limit the way to establish the world coordinate system.

[0091] Optionally, each time the imaging luminance meter captures a luminance image, the imaging near-field photometer may record the first position corresponding to the luminance image, thereby obtaining a series of two-dimensional luminance images and the first position corresponding to each luminance image.

[0092] Step 202 : Divide the three-dimensional space corresponding to the scanning sphere into voxels according to different resolutions, and construct a multi-scale hash table based on the voxels corresponding to the different resolutions, so as to store the feature vector of each voxel through the multi-scale hash table.

[0093] Assuming that the resolution includes M types, each resolution corresponds to a voxel division result of the three-dimensional space, and the H corresponding to the resolution is obtained. m ×W m ×L m A cubic block of a certain size, which is the voxel corresponding to the resolution, and each voxel corresponds to a voxel index, which is used to indicate the position of the voxel in the three-dimensional space. Schematically, the voxel index can be represented by the numbering of the voxels in the length, width and height directions, such as: the voxel index is (a, b, c), which represents the ath voxel in the length direction, the bth voxel in the width direction, and the cth voxel in the height direction. In other embodiments, the voxel index can also be implemented by other representations, such as: by the coordinate position of at least one point (such as a vertex) in the voxel. This embodiment does not limit the implementation method of the voxel index. Wherein, m = 1, 2, ..., M. Wherein, M is a positive integer, for example: M = 8. In other embodiments, M can also be other values. This embodiment does not limit the value of M.

[0094] In a multi-scale hash table, each voxel is represented by K 1-row, V-column column vectors as the feature vectors of the cube block. Schematically, each feature vector of length V (i.e., the 1-row, V-column column vector mentioned above) is used to indicate the characteristics of a certain position in the voxel. For example, the K feature vectors of length V include the feature vectors of the eight vertices in the voxel, and each vertex's feature vector corresponds to a feature vector of length V, which is used to represent the characteristics of that vertex. In this case, K is greater than or equal to 8. In other embodiments, the K feature vectors of length V may also include feature vectors of other positions in the voxel, such as the feature vector of the center point, or each feature vector of length V may be used to indicate the characteristics of multiple positions after fusion. This embodiment does not limit the content of the K feature vectors of length V. Each position in the voxel corresponds to M voxel partitioning methods. Therefore, the K×V feature vectors corresponding to each voxel have K×V×M numerical representations. The multi-scale hash table includes two-dimensional matrices corresponding to M resolutions. The two-dimensional matrices corresponding to each resolution are combined to obtain a three-dimensional matrix. For a voxel partitioned at the mth resolution, the voxel index is hashed using the hash function corresponding to the multi-scale hash table to obtain the corresponding storage location in the two-dimensional matrix corresponding to the mth resolution. The K×N matrix corresponding to the mth resolution is stored in this storage location. The above processing is repeated for voxels partitioned at various resolutions to obtain the corresponding M×K×N matrix.

[0095] Among them, K represents the number of eigenvectors, V represents the length of the eigenvector, K is a positive integer, and V is an integer greater than 1, for example: K is 2 19 , V is 16. In other embodiments, K and V may also be other values. This embodiment does not limit the values ​​of K and V. In this embodiment, by dividing the three-dimensional space into different resolution levels, each level has a different voxel size, the higher level has larger voxels, and the lower level has smaller voxels. This hierarchical method can maintain the overall structural information at a large scale and retain detail information at a small scale. Afterwards, near-field radiosity reconstruction is performed based on the eigenvectors of different scales corresponding to each voxel, which can improve the reconstruction accuracy.

[0096] Optionally, the value of the eigenvector stored in the initialized multi-scale hash table is a normally distributed random number. Since the eigenvector will be optimized and trained later, even if the initial value has a large error, it can be corrected through the subsequent training optimization process and will not affect the accuracy of near-field radiometry reconstruction.

[0097] Optionally, the three-dimensional space corresponding to the scanned sphere is voxel-divided according to different resolutions, including: normalizing the three-dimensional space corresponding to the scanned sphere to obtain a normalized space within the range of [-1, 1]; and voxel-dividing the normalized space. This can improve the accuracy of subsequent neural network model learning.

[0098] Step 203 : for each light sampling point on the light corresponding to at least one pixel in each brightness image, determine a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtain a position code corresponding to each light sampling point.

[0099] In one example, the ray corresponding to at least one pixel in each luminance image is determined using ray tracing technology. Specifically, for each pixel in the luminance image, the pixel's position is connected to the position of the optical center in a pinhole camera model corresponding to the imaging luminance meter to determine the corresponding ray information, including the ray's direction and position. Based on this ray information, discrete point sampling is performed on at least one ray corresponding to each pixel, obtaining multiple ray sampling points on each ray. Subsequently, a feature vector corresponding to each ray sampling point is determined using a multi-scale hash table to obtain a position code corresponding to each ray sampling point.

[0100] Among them, the position of the pixel point and the position of the optical center are both positions in the world coordinate system. The position of the pixel point can be converted from the pixel coordinate system to the image coordinate system based on the intrinsic parameters of the imaging luminance meter and the conversion relationship between the pixel coordinate system and the image coordinate system; then, based on the focal length of the imaging luminance meter and the conversion relationship between the image coordinate system and the camera coordinate system, the position of the pixel point in the image coordinate system is converted to the camera coordinate system; finally, based on the extrinsic parameters of the imaging luminance meter and the conversion relationship between the camera coordinate system and the world coordinate system, the position of the pixel point in the camera coordinate system is converted to the world coordinate system.

[0101] Accordingly, the position of the optical center can be converted from the origin of the camera coordinate system (i.e., the position of the optical center in the camera coordinate system) to the world coordinate system based on the extrinsic parameters of the imaging luminance meter and the conversion relationship between the camera coordinate system and the world coordinate system. The at least one ray can be randomly determined from the rays corresponding to each pixel, or can be the ray for a specified pixel among the pixels. This embodiment does not limit the method for determining the at least one ray.

[0102] The discrete point sampling of each ray in the at least one ray may be performed at equal intervals within the scanning sphere, or may be performed randomly within the scanning sphere. This embodiment does not limit the sampling method.

[0103] refer to Figure 5In the light sampling point acquisition process shown in FIG, the optical center position 501 of the pinhole camera model corresponding to the imaging luminance meter and the line connecting the pixel point 502 in the luminance image form the light corresponding to the pixel point 502. By performing discrete point sampling on this light, the light sampling point 503 on the light can be obtained.

[0104] Assume that the size of the brightness image collected by the imaging luminance meter at the current posture is A×B, according to Figure 5 It can be seen that a ray can be obtained by connecting the pixel and the optical center of the pinhole camera model. By traversing each pixel, a total of A×B rays are obtained. Among them, the ray path and propagation direction are determined by the pixel position of the pixel in the pixel coordinate system and the external parameters of the imaging luminance meter. Randomly sample N rays from the A×B rays, and then sample R discrete points at equal intervals from far to near for each ray. Each discrete point has its own position coordinate x and corresponding light direction d. Among them, N and R are positive integers; the external parameters of the imaging luminance meter are based on the posture information θ and Calculated and confirmed.

[0105] In other embodiments, the line between the optical center and the specified pixel point can be directly determined to obtain the light corresponding to the specified pixel point, and discrete point sampling can be performed on the light, thereby eliminating the step of selecting at least one light from the light corresponding to each pixel point. This embodiment does not limit the process of obtaining the light sampling point.

[0106] In one example, a multi-scale hash table is used to store the feature vectors corresponding to each vertex of each voxel. Accordingly, for each ray sampling point on a ray corresponding to at least one pixel in each luminance image, the feature vector corresponding to the ray sampling point is determined based on the multi-scale hash table to obtain the position code corresponding to each ray sampling point, including steps 31-34:

[0107] Step 31 : For each voxel of each resolution corresponding to the ray sampling point, if the ray sampling point is located inside the voxel, the vertex index of the voxel vertex is determined based on the position index of the voxel.

[0108] Assume that the mth resolution level divides the normalized 3D scan sphere space into H m ×W m ×L m The length, width and height of each voxel are 1 / L respectively. m ,1 / W m ,1 / H m If the normalized three-dimensional scanning sphere, the three-dimensional coordinates of the light sampling point in the world coordinate system are (x R ,y R ,z R ), the position index of the light sampling point in the length, width and height directions can be expressed by the following formula:

[0109]

[0110] Among them, (id L ,id W ,id H ) are the indices of the light sampling points in the length, width and height directions of the voxel, Indicates rounding down.

[0111] Optionally, after the three-dimensional scanning sphere space is divided into voxels according to different resolutions, a correspondence is established between the position index of each voxel and the vertex index of the vertex of the voxel. According to the correspondence, the vertex indexes of the eight vertices corresponding to the voxel can be determined based on the position index of the voxel of each resolution corresponding to the light sampling point. The vertex index is used to indicate the position coordinates of the corresponding vertex.

[0112] Step 32: Determine the feature vector corresponding to each vertex index in the multi-scale hash table.

[0113] The vertex index is calculated based on the hash function corresponding to the multi-scale hash table to obtain the storage location of the feature vector corresponding to the vertex index in the multi-scale hash table. The data stored in the storage location is read to obtain the feature vector corresponding to the vertex index. For example, each voxel corresponds to K feature vectors of length V. In this embodiment, the feature vectors corresponding to eight vertices are determined from these K feature vectors of length V.

[0114] Step 33: Perform feature fusion on the feature vectors corresponding to the vertex indices to obtain the position code of the position index corresponding to each resolution.

[0115] In one example, feature fusion is performed on the feature vectors corresponding to each vertex index to obtain a position code of the position index corresponding to each resolution, including: based on the distance between the three-dimensional coordinates of the light sampling point and the three-dimensional coordinates of each vertex, the feature vectors corresponding to each vertex index are fused to obtain a position code of the position index corresponding to each resolution.

[0116] Schematically, the feature vectors corresponding to each vertex index are interpolated and fused based on the distance using trilinear interpolation. Alternatively, different weights are set for the feature vectors corresponding to different vertex indices, and the weights are negatively correlated with the distance, and the feature vectors corresponding to each vertex index are weighted and fused.

[0117] For example, a hash function is used to determine the storage locations of the vertex indices of eight vertices in a multi-scale hash table, resulting in eight feature vectors of length V. Using the distance relationship between the ray sampling point and the eight vertices within the voxel, these eight feature vectors of length V are interpolated and fused to obtain a position code of length V. The same hash operation is performed on the voxels corresponding to each resolution, resulting in M ​​position codes of length V corresponding to the ray sampling point. These position codes are then merged into a feature vector of length M × V, yielding the position code corresponding to the ray sampling point.

[0118] In other embodiments, for different resolutions, the feature vectors corresponding to the vertex indices may also have different feature fusion methods. For example, for a resolution whose voxel size is greater than a preset threshold, the feature vectors corresponding to the vertex indices of the resolution are fused based on the distance between the position index and each vertex index; for a resolution whose voxel size is less than a preset threshold, the feature vectors corresponding to the vertex indices of the resolution are fused based on the average value of the feature vectors corresponding to each vertex index. This embodiment does not limit the feature fusion method of the feature vectors corresponding to the vertex indices.

[0119] In other embodiments, the multi-scale hash table can also store more or fewer feature vectors for each voxel. In this case, a hash function can be used to calculate the voxel's position index to determine all feature vectors corresponding to that position index. All feature vectors are then bit-wise merged to obtain a position code. The position codes corresponding to the various resolutions are merged along the length direction to obtain the position code corresponding to the ray sampling point. This embodiment does not limit the specific implementation of the position code.

[0120] Step 34 : Merge the position codes of the position indexes corresponding to the various resolutions to obtain the position codes corresponding to the light sampling points.

[0121] To more clearly understand the process of obtaining the position code corresponding to the light sampling point in this embodiment, refer to Figure 6 The position coding acquisition process shown in Figure 6 It can be seen that after the light sampling point x in the normalized 3D scanning sphere space is divided into voxels according to different resolutions, the voxels corresponding to the light sampling point x at different resolutions will be obtained. For example: Figure 6In the example, the ray sampling point x belongs to the voxel corresponding to the blue cube and the voxel corresponding to the red cube. The voxels corresponding to the blue cube and the red cube are obtained by voxel partitioning at different resolutions. The size of the voxel corresponding to the red cube is larger than that of the voxel corresponding to the blue cube. Based on the position index of each voxel to which the ray sampling point x belongs, the position code of length V corresponding to each voxel is determined in the multi-scale hash table. The position codes of length V corresponding to M resolutions are merged to obtain a position code of length V × M corresponding to the ray sampling point.

[0122] Step 204 : Angle-encode the light sampling points based on the angle encoding function to obtain an angle code corresponding to each light sampling point.

[0123] Optionally, the angle encoding function may be a spherical harmonic function, a sine function, or the like. This embodiment does not limit the implementation method of the angle encoding function.

[0124] Among them, spherical harmonics is a function defined on a sphere, which uses a set of orthogonal basis functions to expand any function defined on the unit sphere. These basis functions are called spherical harmonic basis functions and have a specific mathematical form. They can represent the intensity and direction of light at different angles. By applying these basis functions to light signals, spherical harmonics can decompose light signals into a combination of a series of basis functions. Light signals at different angles can be uniquely represented by a linear combination of orthogonal basis functions. Each basis function corresponds to a specific angle, and the selection of these basis functions ensures the orthogonality of spherical harmonics. This decomposition process converts the light signal from the spatial domain to the frequency domain, so that the different angular characteristics of the light signal can be represented and stored by the coefficients of the basis functions. For the light direction in three-dimensional space, after spherical harmonic encoding, a feature vector of length 16 can be obtained, which is the angle code corresponding to the light sampling point.

[0125] The sine function encodes the angle of the light from 0 to 2π through periodic amplitude changes.

[0126] Optionally, the angle codes corresponding to different light sampling points on the same light ray may be calculated only once, that is, different light sampling points on the same light ray share the same angle code; or, different light sampling points on the same light ray may be calculated separately based on the angle code function.

[0127] Step 205 : Decode the position code and the angle code based on a pre-created decoding model to obtain a density prediction value and an optical parameter prediction value of each light sampling point.

[0128] Among them, the decoding model is established based on the neural network model.

[0129] In one example, the decoding model includes an optical density network and an optical parameter network connected to the optical density network; accordingly, the position code and the angle code are decoded based on a pre-created decoding model to obtain a density prediction value and an optical parameter prediction value for each light sampling point, including: inputting the position code into the optical density network to decode the position code to obtain a multi-channel density prediction result, outputting the density prediction result of the target channel in the multi-channel density prediction result, and obtaining a density prediction value; inputting the density prediction results of other channels except the target channel in the multi-channel density prediction result and the angle code into the optical parameter network to obtain an optical parameter prediction value.

[0130] Optionally, the optical density network and / or the optical parameter network are established based on a fully connected neural network. Figure 7 The decoding model shown, Figure 7 The optical density network 710 and the optical parameter network 720 are both established based on a fully connected neural network as an example for explanation. Figure 7 It can be seen that the optical density network 710 includes three fully connected intermediate layers 711 with activation functions, and the activation function is a linear rectifier unit (Rectified Linear Unit, Relu). The number of channels in each fully connected intermediate layer 711 is 64. After the position code is processed by the fully connected intermediate layer 711, the density prediction results of 16 channels are obtained. The density prediction result of the first channel of the 16-channel density prediction results is output as the density prediction result of the target channel to obtain the density prediction value; the density prediction results of the last 15 channels are combined with the angle code of length 16 on the channel to obtain 31 channels of feature data for input into the optical parameter network.

[0131] The optical parameter network 720 includes four fully connected intermediate layers 721 with activation functions. After the feature data of 31 channels are processed by the four fully connected intermediate layers 721, the optical parameter prediction values ​​of C channels are obtained. The value of C is set based on the optical parameter requirements. For example, if the optical parameter requirements indicate that a color image needs to be constructed, the value of C can be 3, and the optical parameter prediction values ​​of each channel represent the prediction values ​​of the three colors red, green, and blue. For another example, if the optical parameter requirements indicate that a multispectral image needs to be constructed, the value of C can be 16, and the optical parameter prediction values ​​of each channel represent the parameter prediction values ​​of 16 bands.

[0132] In other embodiments, the number of layers and channels in the fully connected intermediate layers 711 and 721 may be greater or lesser. This embodiment does not limit the configuration of the fully connected intermediate layers 711 and 721. The activation function in the fully connected intermediate layer 721 may be a ReLU function or a Sigmoid function. The activation functions in the optical density network 710 and the optical parameter network 720 may be the same or different. This embodiment does not limit the configuration of the activation function.

[0133] In other embodiments, the optical density network and / or the optical parameter network may also be established based on other types of neural networks, such as: based on a convolutional neural network. This embodiment does not limit the implementation method of the optical density network and / or the optical parameter network.

[0134] Step 206 : Determine a first pixel prediction value of the pixel point based on the density prediction value and the optical parameter prediction value of each light sampling point on the light ray corresponding to the pixel point, and a preset volume rendering model.

[0135] In one example, the starting point and end point of each ray are determined, and the starting point and end point are located within a scanning sphere; based on a volume rendering model, the density prediction value and the optical parameter prediction value of each ray sampling point from the starting point to the end point are integrated to obtain a first pixel prediction value of the pixel point.

[0136] Optionally, the starting point and the end point are respectively located on both sides of the light source to be measured, the starting point and the end point are selected within a preset range close to the light source to be measured, and the light sampling point is located between the starting point and the end point.

[0137] Schematically, the volume rendering model is represented by the following formula:

[0138] G=∫T(t)σ(t)c(t,d)dt;

[0139]

[0140] Where G represents the first pixel prediction value of the pixel; σ(t) represents the density prediction value of the light sampling point on the light corresponding to the pixel at position x(t); c(t, d) is the predicted value of the optical parameter of the light sampling point at x(t) as it changes with direction; T(t) is the transmittance from the starting point to position t, which indicates the probability that the light is not absorbed before position t.

[0141] Optionally, during actual calculations, the volume rendering model can be approximated through discretization. The integral range is discretized, and the integral of the volume rendering formula is written as a summation. In this way, the volume rendering model uses the density predictions and optical parameter predictions for each ray sampling point to perform volume rendering on the ray, obtaining the first pixel prediction value for each pixel. After traversing the pixels corresponding to all rays, the first pixel prediction value corresponding to each pixel obtained by the current imaging near-field photometer posture is obtained.

[0142] Step 207 : Iteratively optimize the feature vectors in the multi-scale hash table and the model parameters of the decoding model based on the first pixel prediction value and the actual pixel value of the pixel point on the luminance image to obtain a trained multi-scale hash table and a trained decoding model.

[0143] For each pixel, the first predicted pixel value I of the current pixel can be obtained by using light sampling, position encoding, neural network prediction, and volume rendering. pre In addition, in step 201, the actual pixel value I of the pixel point collected by the imaging luminance meter is also obtained. obs The loss function can be used to calculate the difference between the first pixel prediction value and the actual pixel value, and based on the difference, the gradient descent method is used to update the network parameters of the decoding model and the feature vector stored in the multi-scale hash table.

[0144] Optionally, the loss function may be an L2 norm or an L1 norm, etc. This embodiment does not limit the implementation method of the loss function.

[0145] Among them, the L1 norm L1-loss is expressed by the following formula:

[0146] L1-loss=||I pre -I obs ||1;

[0147] The L2-norm L2-loss is expressed as follows:

[0148]

[0149] Based on this difference, the network parameters of the decoding model and the feature vectors stored in the multi-scale hash table are updated using gradient descent. This involves calculating the gradients of the loss function with respect to the network parameters and feature vectors using automatic differentiation and updating them using the Adam optimizer. The gradients are backpropagated through the decoding model and linear interpolation, and then accumulated in the found feature vector.

[0150] Schematically, let the parameter of the i-th iteration be ω i , Adam update parameter formula is expressed as follows:

[0151]

[0152] Among them, g is the current loss function parameter ω i The gradient of β1 and β2 are adjustable parameters, generally β1 = 0.9 and β2 = 0.999. In other embodiments, β1 and β2 can also be other values. This embodiment does not limit the values ​​of β1 and β2. μ and ν represent the first-order momentum and second-order momentum of the gradient respectively; η is a preset coefficient to increase the stability of the denominator. For multi-scale hash tables, a smaller η should be selected, such as 10 -15 , for the decoding model, we can set η=10 -8 .

[0153] Step 208: Determine a second pixel prediction value corresponding to a second posture on the scanning sphere based on the trained multi-scale hash table and the trained decoding model, so as to reconstruct the near-field radiance of the luminous object to be measured based on the second pixel prediction value; wherein the second posture is different from the first posture.

[0154] After training the multi-scale hash table and decoding model, the hash table and the fully connected network are used to predict the two-dimensional brightness image (not the actual captured image, but a predicted image composed of the predicted values ​​of each second pixel) corresponding to the imaging brightness meter under a given new imaging brightness meter posture (i.e., the second posture).

[0155] The second pixel prediction value corresponding to the second posture includes: the second pixel prediction value corresponding to each pixel point on the image when the image is taken at the second posture. By merging the brightness image corresponding to each first posture with the predicted second pixel prediction value, the near-field radiance image of the luminous object to be measured can be reconstructed. Assume that the reconstructed near-field radiance image is as follows Figure 8 As shown, Figure 8 In this paper, we take C=1, which represents grayscale, as an example. Figure 8 It can be seen that a relatively realistic near-field radiometry image of the luminous object to be measured can be reconstructed using a small number of brightness images.

[0156] In summary, the near-field radiometry reconstruction method provided by this embodiment obtains a brightness image captured by an imaging luminance meter in the first position of a scanning sphere; divides the three-dimensional space corresponding to the scanning sphere into voxels according to different resolutions, and constructs a multi-scale hash table based on the voxels corresponding to different resolutions, so as to store the feature vector of each voxel through the multi-scale hash table; for each light sampling point on the light corresponding to at least one pixel point in each brightness image, determines the feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtains the position code corresponding to each light sampling point; angle codes the light sampling points based on the angle coding function, and obtains the angle code corresponding to each light sampling point; decodes the position code and the angle code based on a pre-created decoding model, and obtains the density prediction value and the optical parameter prediction value of each light sampling point; based on each light sampling point on the light corresponding to the pixel point, the feature vector corresponding to the light sampling point is determined based on the multi-scale hash table, and the position code corresponding to each light sampling point is obtained. The density prediction value and optical parameter prediction value of the point, as well as the preset volume rendering model, are used to determine the first pixel prediction value of the pixel point; based on the first pixel prediction value and the actual pixel value of the pixel point on the brightness image, the feature vector in the multi-scale hash table and the model parameters of the decoding model are iteratively optimized to obtain the trained multi-scale hash table and the trained decoding model; based on the trained multi-scale hash table and the trained decoding model, the second pixel prediction value corresponding to the second posture on the scanning sphere is determined to reconstruct the near-field radiance of the luminous object to be measured based on the second pixel prediction value; this can solve the problem that the traditional near-field radiance reconstruction method needs to collect a large number of luminance images, resulting in low reconstruction efficiency; since it is no longer necessary to collect luminance images at both the first and second postures, near-field radiance reconstruction can be achieved using a small number of luminance images collected at the first posture, thereby improving the near-field radiance reconstruction efficiency.

[0157] In addition, the near-field radiometry reconstruction method of this embodiment can also well estimate the three-dimensional radiation field information in the case of a small number of brightness images, and has good data utilization.

[0158] In addition, the near-field radiometry reconstruction method of this embodiment can measure luminous objects of various shapes and properties, and has good versatility.

[0159] In order to more clearly understand the near-field radiometric reconstruction method provided by the present application, an example is given below to illustrate the method. Figure 9 , the method includes the following steps:

[0160] Step 91, obtaining a brightness image captured by an imaging luminance meter on a scanning sphere at a first posture;

[0161] Step 92: Establish a multi-scale hash table and decoding model;

[0162] Step 93, determining whether each brightness image has been traversed; if not, executing step 94; if so, executing step 99;

[0163] Step 94: for each untraversed brightness image, determine whether each pixel in the current brightness image has been traversed; if so, execute step 93; if not, execute step 95;

[0164] Step 95: for each untraversed pixel, determine the position code of the light sampling point on the light corresponding to the pixel based on the multi-scale hash table; and determine the angle code of the light sampling point based on the angle coding function;

[0165] Step 96: Input the position code and the angle code into the decoding model for decoding to obtain a first pixel prediction value corresponding to the current pixel point;

[0166] Step 97, calculating the difference between the first pixel prediction value and the actual pixel value corresponding to the current pixel point based on the loss function to obtain a loss function value;

[0167] Step 98: Update the network parameters of the decoding model and the feature vector stored in the multi-scale hash table based on the loss function value; and execute step 94.

[0168] Step 99 : determining a second pixel prediction value corresponding to a second posture on the scanning sphere based on the trained multi-scale hash table and the trained decoding model, so as to reconstruct the near-field radiance of the luminous object to be measured based on the second pixel prediction value.

[0169] Figure 10 FIG. 5 is a flow chart showing a near-field radiometry reconstruction apparatus according to an embodiment of the present disclosure. Figure 10 As shown, the device includes: an image acquisition module 1010, a voxel division module 1020, a position encoding module 1030, an angle encoding module 1040, a parameter prediction module 1050, a pixel prediction module 1060, a model training module 1070 and a radiometry reconstruction module 1080.

[0170] An image acquisition module 1010 is configured to acquire a brightness image captured by the imaging luminance meter on the scanning sphere in a first pose; wherein different brightness images have different first poses;

[0171] a voxel partitioning module 1020 for performing voxel partitioning on the three-dimensional space corresponding to the scanning sphere according to different resolutions, and constructing a multi-scale hash table based on the voxels corresponding to the different resolutions, so as to store a feature vector of each voxel in the multi-scale hash table;

[0172] a position encoding module 1030 configured to determine, for each light sampling point on a light corresponding to at least one pixel in each luminance image, a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtain a position encoding corresponding to each light sampling point;

[0173] An angle coding module 1040, configured to perform angle coding on the light sampling points based on an angle coding function to obtain an angle coding corresponding to each light sampling point;

[0174] A parameter prediction module 1050 is configured to decode the position code and the angle code based on a pre-created decoding model to obtain a density prediction value and an optical parameter prediction value for each light sampling point;

[0175] a pixel prediction module 1060 configured to determine a first pixel prediction value of the pixel point based on a density prediction value and an optical parameter prediction value of each ray sampling point on the ray corresponding to the pixel point, and a preset volume rendering model;

[0176] a model training module 1070 for iteratively optimizing the feature vectors in the multi-scale hash table and the model parameters of the decoding model based on the first pixel prediction value and the actual pixel value of the pixel point on the luminance image, to obtain a trained multi-scale hash table and a trained decoding model;

[0177] The radiometry reconstruction module 1080 is configured to determine a second pixel prediction value corresponding to a second posture on the scanning sphere based on the trained multi-scale hash table and the trained decoding model, so as to reconstruct the near-field radiometry of the luminous object to be measured based on the second pixel prediction value; wherein the second posture is different from the first posture.

[0178] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0179] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0180] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0181] An embodiment of the present disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0182] Figure 111 is a block diagram of an apparatus 1900 for near-field radiometry reconstruction according to an exemplary embodiment. For example, the apparatus 1900 may be provided as a server or a terminal device. Figure 11 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described method.

[0183] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2003. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0184] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by the processing component 1922 of the apparatus 1900 to perform the above-described method.

[0185] While various embodiments of the present disclosure have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A near-field radiometry reconstruction method, characterized in that: Used in an imaging near-field photometer, the imaging near-field photometer includes: a rotating scanning mechanism and an imaging luminance meter mounted on the rotating scanning mechanism; the rotating scanning mechanism is used to drive the imaging luminance meter to move on the surface of a scanning sphere; the imaging luminance meter is used to collect a luminance image of a luminous object to be measured located inside the scanning sphere; the method includes: Acquire a brightness image captured by the imaging luminance meter on the scanning sphere in a first position; wherein different brightness images have different first positions; Dividing the three-dimensional space corresponding to the scanning sphere into voxels according to different resolutions, and constructing a multi-scale hash table based on the voxels corresponding to the different resolutions, so as to store a feature vector of each voxel through the multi-scale hash table; For each light sampling point on the light corresponding to at least one pixel in each brightness image, determining a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtaining a position code corresponding to each light sampling point; Performing angle coding on the light sampling points based on an angle coding function to obtain an angle coding corresponding to each light sampling point; Decoding the position code and the angle code based on a pre-created decoding model to obtain a density prediction value and an optical parameter prediction value of each light sampling point; Determining a first pixel prediction value of the pixel point based on a density prediction value and an optical parameter prediction value of each light sampling point on the light corresponding to the pixel point, and a preset volume rendering model; Iteratively optimizing the feature vectors in the multi-scale hash table and the model parameters of the decoding model based on the first pixel prediction value and the actual pixel value of the pixel point on the luminance image to obtain a trained multi-scale hash table and a trained decoding model; Based on the trained multi-scale hash table and the trained decoding model, a second pixel prediction value corresponding to a second posture on the scanning sphere is determined to reconstruct the near-field radiance of the luminous object to be measured based on the second pixel prediction value; wherein, the second posture is different from the first posture.

2. The method according to claim 1, characterized in that The multi-scale hash table is used to store the feature vectors corresponding to each vertex of each voxel; accordingly, The step of determining, for each light sampling point on a light ray corresponding to at least one pixel in each brightness image, a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtaining a position code corresponding to each light sampling point includes: For each voxel of each resolution corresponding to the light sampling point, if the light sampling point is located inside the voxel, determining a vertex index of a vertex of the voxel based on a position index of the voxel; Determining the feature vector corresponding to each vertex index in the multi-scale hash table; Perform feature fusion on the feature vectors corresponding to each vertex index to obtain the position encoding of the position index corresponding to each resolution; The position codes of the position indexes corresponding to the various resolutions are merged to obtain the position codes corresponding to the light sampling points.

3. The method according to claim 2, characterized in that The feature fusion of the feature vectors corresponding to the vertex indices to obtain the position encoding of the position index corresponding to each resolution includes: Based on the distance between the three-dimensional coordinates of the light sampling point and the three-dimensional coordinates of each vertex, the feature vectors corresponding to each vertex index are fused to obtain the position code of the position index corresponding to each resolution.

4. The method according to claim 1, wherein The step of determining, for each light sampling point on a light ray corresponding to at least one pixel in each brightness image, a feature vector corresponding to the light sampling point based on the multi-scale hash table, and obtaining a position code corresponding to each light sampling point includes: For each pixel in the brightness image, connecting the position of the pixel with the position of the optical center in the pinhole camera model corresponding to the imaging brightness meter to determine light information corresponding to the pixel, the light information including the direction and position of the light; Based on the light information, discrete point sampling is performed on at least one light ray corresponding to each pixel point to obtain a plurality of light sampling points on each light ray; The feature vector corresponding to each light sampling point is determined based on the multi-scale hash table to obtain the position code corresponding to each light sampling point.

5. The method according to claim 1, wherein The decoding model includes an optical density network and an optical parameter network connected to the optical density network; accordingly, The decoding of the position code and the angle code based on the pre-created decoding model to obtain a density prediction value and an optical parameter prediction value of each light sampling point includes: Inputting the position code into the optical density network to decode the position code to obtain a multi-channel density prediction result, outputting the density prediction result of the target channel in the multi-channel density prediction result to obtain the density prediction value; The density prediction results of other channels except the target channel in the multi-channel density prediction results and the angle code are input into the optical parameter network to obtain the optical parameter prediction value.

6. The method according to claim 5, characterized in that The optical density network and / or the optical parameter network are established based on a fully connected neural network.

7. The method according to claim 1, characterized in that The determining the first pixel prediction value of the pixel point based on the density prediction value and the optical parameter prediction value of each light sampling point on the light corresponding to the pixel point and a preset volume rendering model includes: Determining a start point and an end point of each ray, wherein the start point and the end point are located within the scanning sphere; Based on the volume rendering model, density prediction values ​​and optical parameter prediction values ​​of each light sampling point from the starting point to the end point are integrated to obtain a first pixel prediction value of the pixel point.

8. The method according to claim 7, characterized in that The volume rendering model is expressed as follows: G=∫T(t)σ(t)c(t,d)dt; Where G represents the first pixel prediction value of the pixel; σ(t) represents the density prediction value of the light sampling point on the light corresponding to the pixel at position x(t); c(t, d) is the predicted value of the optical parameter of the light sampling point at x(t) as it changes with direction; T(t) is the transmittance from the starting point to position t, which indicates the probability that the light is not absorbed before position t.

9. A near-field radiometry reconstruction device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 8 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images