Event camera based joint reconstruction of surface normals and multispectral reflectance

Abstract

The event camera-based surface normal and multispectral reflectance joint reconstruction method comprises the following steps: a spectral-angle multiplexing illumination coding device is built; illumination calibration parameters are established; single event acquisition is performed; a joint imaging and event constraint model is established; the surface normal and multispectral reflectance are decoupled by using the same waveband multidirectional repetition; the availability under the conditions of attached shadow and highlight is improved through the redundant repetition mechanism; the absolute reflectance is recovered through the constant light component; and the event-driven spectral-angle decoder is used for joint optimization. The present application realizes the joint recovery under the conditions of single acquisition and single system configuration, significantly reduces the data bandwidth and acquisition time delay, has better robustness under the conditions of shadow and highlight, can recover the absolute multispectral reflectance instead of only the relative reflectance, and has strong expansibility.

Description

Technical Field

[0001] This invention belongs to the field of computational imaging and computer vision technology, and specifically relates to a method for joint reconstruction of surface normal and multispectral reflectance based on an event camera. Background Technology

[0002] Surface normal and multispectral reflectance are two key underlying quantities in physical scene understanding. Surface normal reflects geometry, while multispectral reflectance reflects the inherent response of a material in different wavelength bands. The joint reconstruction of these two parameters is of great value for applications such as industrial inspection, cultural relic analysis, material identification, and robot perception.

[0003] Existing multispectral photometric stereo methods based on frame cameras typically rely on multiple sequential images, narrowband light source switching, multispectral cameras, or high data throughput. These methods suffer from several drawbacks: slow acquisition process, high bandwidth, and complex systems; furthermore, most models default to approximating Lambertian reflection, which can lead to estimation errors when encountering specular highlights and attached shadows.

[0004] Event cameras output logarithmic changes in brightness asynchronously, offering advantages such as high temporal resolution, high dynamic range, and low data bandwidth. Existing event photometric stereo methods can estimate surface normals using event streams. For example, a multispectral / color photometric stereo technique based on frame cameras is described in references: "G. Fyffe. Single-shot photometric stereoby spectral multiplexing. ACM SIGGRAPH Asia, 2010." and "M. Kitahara, T. Okabe, C. Fuchs, HPA Lensch. Simultaneous estimation of spectral reflectance and normal from a small number of images. VISAPP, 2015." The typical process involves acquiring multiple frame images under various illumination conditions, recovering the surface normal from the intensity values ​​based on the Lambertian or extended reflectance model, and simultaneously recovering the reflectance of each band by combining multispectral imaging or spectral multiplexing information. This technical approach typically requires dense frame sampling, multiple exposures, or multi-stage spectral acquisition. It relies on complete frame data and high data throughput, which is not conducive to high-speed, low-bandwidth acquisition. The acquisition speed is slow, the bandwidth is high, and it is sensitive to highlights and attached shadows, limiting its applicability in complex materials and fast acquisition scenarios.

[0005] Existing event hyperspectral / multispectral imaging methods can recover spectral information from event streams. For example, a two-stage combined scheme based on an event camera, as referenced in "B. Yu, J. Ren, J. Han, F. Wang, J. Liang, B. Shi. EventPS: Real-time photometric stereo using an event camera. CVPR, 2024.", "J. Liang, B. Yu, S. Yang, H. Zhuang, J. Ren, P. Duan, B. Shi. EventUPS: Uncalibrated photometric stereo using an event camera. ICCV, 2025.", and "B. Yu, J. Liang, Z. Wang, B. Fan, A. Subpa-asa, B. Shi, I. Sato. Active hyperspectral imaging using an event camera. CVPR, 2025.", typically involves: first, recovering the surface normal using event photometric stereo under known or estimated illumination directions; second, recovering multispectral reflectance using another event spectral imaging coding scheme. This technical approach typically requires heterogeneous lighting protocols, different system configurations, or sequential processing. While it reduces bandwidth to some extent, it is essentially still a serial solution, unable to be directly and jointly recovered in a single system and a single acquisition stage, with errors accumulating in both stages. System integration is complex, and it is difficult to directly guarantee the accuracy of the multispectral reflectance scale, making it difficult to obtain absolute values. In summary, existing technologies have significant problems in the following aspects:

[0006] (1) Lack of single-stage joint recovery capability. Existing technologies can usually only recover one of the surface normal or multispectral reflectance, or must rely on two-stage cascade, making it difficult to obtain both the surface normal and absolute multispectral reflectance in a single acquisition.

[0007] (2) Lack of illumination coding design suitable for event domain decoupling. If each band is only illuminated once, the surface normal and multispectral reflectance will be multiplicatively coupled in event observation, and it will be impossible to form enough multispectral reflectance to offset the constraint.

[0008] (3) It is difficult to resist invalid observations caused by attached shadows and specular highlights. Existing technologies usually rely on additional elimination strategies or complex optical devices to compensate, which places a heavy burden on the system.

[0009] (4) The scale ambiguity of event measurement cannot be resolved naturally. When existing technology relies solely on the observation of event intensity ratios, different multispectral reflectance scales may produce the same event response, thus failing to recover the absolute multispectral reflectance.

[0010] In addition, simply splicing geometric restoration and multispectral reflectance restoration directly will face at least three core difficulties: First, in event observation, surface normal and multispectral reflectance are multiplicatively coupled, and cannot be stably decoupled without sufficient constraints; Second, attached shadows and specular highlights will cause some observations to fail, reducing the number of available constraints; Third, event cameras essentially only observe relative logarithmic brightness changes, which can easily lead to multispectral reflectance being restored only to an unknown scale, rather than directly obtaining absolute multispectral reflectance.

[0011] Therefore, a hardware-algorithm collaborative solution oriented towards event streams is needed to encode illumination direction information, band information, and absolute scale information into the event stream under the premise of single acquisition and low bandwidth, and to provide a matching decoding method to jointly recover the surface normal and absolute multispectral reflectance. Summary of the Invention

[0012] To address the shortcomings of existing technologies, such as lack of single-stage joint reconstruction capability, lack of illumination coding design suitable for event domain decoupling, difficulty in resisting invalid observations caused by attached shadows and specular highlights, and inability to naturally resolve scale ambiguities in event measurements, this invention provides a joint reconstruction method based on surface normal and multispectral reflectance from an event camera.

[0013] The technical solution adopted by this invention to solve the technical problem is as follows:

[0014] This invention provides a method for jointly reconstructing surface normal and multispectral reflectance based on an event camera, which mainly includes the following steps:

[0015] Step S1: Construct a spectral-angle multiplexed illumination encoding device;

[0016] Step S2: Establish lighting calibration parameters;

[0017] Step S3: Perform single event acquisition;

[0018] Step S4: Establish a joint imaging and event constraint model;

[0019] Step S5: Decouple the surface normal and multispectral reflectance by repeating the same wavelength in multiple directions;

[0020] Step S6: Improve usability under attached shadow and highlight conditions through a redundant repetition mechanism;

[0021] Step S7: Recover absolute reflectivity using a constant light component;

[0022] Step S8: Perform joint optimization using an event-driven spectrum-angle decoder.

[0023] Furthermore, in step S1, the spectral-angle multiplexing illumination encoding device includes an event camera, a beam splitter, a constant light source, a halo illumination module, and a controller; the object under test, the halo illumination module, the constant light source, the beam splitter, and the event camera are arranged in sequence, with the beam splitter positioned in front of the event camera so that the field of view of the event camera is coaxial with the constant light path, the constant light source is used to generate constant light components, and the halo illumination module is used to generate directional light.

[0024] Furthermore, the halo lighting module includes a ring-shaped bracket and multiple LEDs evenly arranged on the ring-shaped bracket along the azimuth angle, and is divided into... Each band channel is arranged repeatedly, and each band channel corresponds to multiple different lighting directions.

[0025] Furthermore, in step S2, the illumination directions at multiple different angles corresponding to each band channel, the spectral power distribution of each band channel, the spectral response of the event camera at each band channel, and the known radiance of the constant light component are pre-calibrated or recorded.

[0026] Furthermore, in step S3, the object under test, during one or more rounds of preset timing sequence lighting condition switching, outputs an event stream from the event camera. , For pixel position, For the polarity of the event, This is the event timestamp.

[0027] Furthermore, in step S4, the joint imaging model is represented as: , Indicates pixel position The normal vector to the surface of the object being measured. Indicates time The corresponding incident light direction; Indicates the number of band channels involved in the encoding; Indicates the first Parameters of the event camera for each band channel; Indicates pixel position The first The reflectivity of each band channel represents the reflectivity of that pixel location under the corresponding spectral or illumination conditions; Indicates the first Each band channel at time The timing function is used to represent the first... Each band channel at time The intensity.

[0028] Furthermore, in step S4, the event constraint model is represented as follows: , Indicates time The imaging intensity, Indicates time The imaging intensity, This is the threshold for triggering the event.

[0029] Furthermore, in step S5, geometric constraints for reflectivity cancellation are constructed for the intensity ratios of two sets of events in the same band; at least two sets of independent event intensity ratio constraints are needed to determine the surface normal direction; after the surface normal is constrained, the remaining multi-band intensity information is used to solve for the reflectivity of each band.

[0030] Furthermore, in step S7, a known radiance is additionally superimposed on the directional light. The constant light component makes the event intensity satisfy ,in, Indicates the imaging intensity at different times in step S4. The absolute value, Indicates the imaging intensity in step S4 Normalized values; for two known times , The event intensity ratio is obtained. Seeking .

[0031] Furthermore, in step S8, the unknowns are parameterized as pixel-level normal map n and low-rank multispectral reflectance model. , Represents pixel coefficient, Indicates the band base, The dimension of the low-rank matrix is ​​represented by . The final result is obtained by minimizing the consistency error of adjacent event pairs and combining non-negativity constraints, normal unit length constraints, forward visibility constraints and spatial consistency regularization terms.

[0032] The beneficial effects of this invention are:

[0033] (1) This invention achieves joint recovery under a single acquisition and single system configuration. Compared with the existing schemes that require multi-stage serial connection, this invention can simultaneously recover the surface normal and absolute multispectral reflectance in a single set of hardware and a continuous event stream, significantly reducing system complexity.

[0034] (2) This invention significantly reduces data bandwidth and acquisition latency. The event camera only outputs data when the brightness changes, which has lower bandwidth and higher speed compared to traditional multi-frame multispectral acquisition, making it suitable for dynamic or high-throughput application scenarios.

[0035] (3) The present invention has better robustness under shadow and highlight conditions. Through multi-angle repetition in the same band and corresponding decoding weighting mechanism, the system can still retain effective constraints when attached shadows and specular highlights are present, thus improving recovery stability.

[0036] (4) This invention can recover absolute multispectral reflectance rather than just relative reflectance. By introducing a constant light component, the scale invariance of event intensity ratio observation can be broken, and absolute scale results can be obtained without an additional frame intensity sensor.

[0037] (5) The present invention has strong scalability. The present invention can be extended to more bands, different ring layouts, different event cameras and different decoding models, which facilitates engineering deployment.

[0038] (6) This invention can be used for both offline high-precision reconstruction and online rapid detection; it can be used for single static objects and can also be extended to pipeline detection scenarios through synchronous control and timing scheduling. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the spectral-angle multiplexing illumination encoding device constructed according to the present invention.

[0040] Figure 2 This is a schematic diagram of the halo lighting module.

[0041] Figure 3 The diagram illustrates the principle of resolving scale ambiguity with constant light assistance. (a) Event triggering without a constant light source (same event at different scales); (b) Event triggering with a constant light source (different events at different scales).

[0042] In the diagram, there are: event camera 1, beam splitter 2, constant light source 3, halo illumination module 4, controller 5, and the object under test 6. Detailed Implementation

[0043] The present invention will be further described in detail below with reference to the accompanying drawings.

[0044] This invention provides a method for joint reconstruction of surface normal and multispectral reflectance based on an event camera.

[0045] This method consists of two parts: hardware encoding acquisition and event-driven joint decoding. The hardware encoding acquisition part is mainly responsible for encoding geometric information, band information, and absolute scale information into the event stream; the event-driven joint decoding part outputs the normal map and absolute multispectral reflectance of the target surface based on the known illumination sequence, event constraint relationships, and physical priors.

[0046] This invention provides a method for joint reconstruction of surface normal and multispectral reflectance based on an event camera. It employs an event-driven joint reconstruction scheme under a single acquisition and single-system configuration, enabling the surface normal and absolute multispectral reflectance to be simultaneously recovered in the same event stream. The specific implementation process is as follows:

[0047] Step S1: Construct a spectral-angle multiplexed illumination encoding device;

[0048] The spectral-angle multiplexing illumination encoding device constructed in this invention mainly includes an event camera 1, a beam splitter 2, a constant light source 3, a halo illumination module 4, and a controller 5. The object under test 6, the halo illumination module 4, the constant light source 3, the beam splitter 2, and the event camera 1 are arranged in sequence. The beam splitter 2 is positioned in front of the event camera 1 so that the field of view of the event camera 1 is coaxial with the constant light path. The constant light source 3 is used to generate constant light components, and the halo illumination module 4 is used to generate directional light.

[0049] Specifically, the constant light source 3 can be a mobile phone with a uniform white screen, a calibrated white LED board, an integrating sphere outlet, a diffused white light panel, or other calibrated constant radiance devices, etc. Its optical path can be incorporated into the system through a beam splitter, a coaxial illumination device, or other equivalent optical combinations. The directional light and constant light act simultaneously on the object under test 6, and the object under test 6 can be scanned by the event camera 1 without changing the viewing angle during single or multiple scans, thus obtaining a complete event stream.

[0050] Specifically, the event camera 1 can be a Prophesee (Prophesee EVK4), DAVIS, CeleX, or other devices with asynchronous brightness change output capability.

[0051] Specifically, beam splitter 2 can be a 50 / 50 beam splitter prism.

[0052] Specifically, such as Figure 2 As shown, the halo lighting module 4 includes a ring bracket and 24 LEDs evenly arranged on the ring bracket along the azimuth angle, and is divided into... Each band channel is arranged three times, and each band channel corresponds to three different lighting directions. These three different lighting directions are preferably 120° apart in azimuth.

[0053] It should be noted that the arrangement of light sources on the halo illumination module 4 is not limited to a ring. As long as the same wavelength can be repeated under multiple different illumination directions and sufficient geometric decoupling constraints are ensured, it can be implemented in the form of polygonal halo, arc array, rectangular array or mechanically scanned light source.

[0054] In addition, the number of bands and the number of repetitions can be adjusted. It is preferred to use 8 band channels with 3 repetitions per band channel. Alternatively, more or fewer bands can be used depending on application requirements, or each band channel can be repeated at least 3 times, provided that decoupling constraints and effective observation requirements are met.

[0055] Specifically, controller 5 can be an Arduino or a functionally equivalent timing control module (such as an FPGA, microcontroller, or industrial control board) to precisely drive the LEDs to switch according to a preset timing sequence. Controller 5 activates only one LED at any given time according to the preset timing sequence, so that information from different band channels and different lighting directions is injected into the event stream in chronological order.

[0056] This invention proposes an illumination coding rule that allows the same band to repeat under multiple different illumination directions. In particular, it can form at least two sets of repeating arrangements that can offset geometric constraints with reflectivity and improve usability under attached shadows / highlights. This allows the same band to repeat under different illumination directions, thereby constructing sufficient geometric constraints in the event domain to decouple the surface normal from the multispectral reflectivity.

[0057] Step S2: Establish lighting calibration parameters;

[0058] The illumination directions at three different angles corresponding to each band channel are pre-calibrated or recorded, as well as the spectral power distribution of each band channel, the spectral response of event camera 1 at each band channel, and the known radiance of the constant light component. To balance highlight separation and shadow availability, the aura polar angle that meets geometric requirements can be selected.

[0059] Specifically, a halo polar angle greater than 27.5° ensures that the high light pollution areas corresponding to different repetitions of the same wavelength under different illumination directions do not overlap.

[0060] This invention proposes a constant light enhancement mechanism that superimposes a constant light component with known radiance while switching the direction of illumination, in order to resolve the scale ambiguity caused by the observation of the event intensity ratio and restore the absolute multispectral reflectance.

[0061] Step S3: Perform single event acquisition;

[0062] Under one or more rounds of preset timing lighting condition switching, the event camera 1 outputs an event stream for the object under test 6. .in, For pixel position, For the polarity of the event, This is the event timestamp. Since event camera 1 records the logarithmic change in brightness, low-bandwidth acquisition can be achieved without outputting a full frame image.

[0063] Step S4: Establish a joint imaging and event constraint model;

[0064] For pixel position At any moment The imaging intensity, i.e., the joint imaging model, can be expressed as: .in, Indicates pixel position The normal vector of surface 6 of the object being measured. Indicates time The corresponding incident light direction; Indicates the number of band channels involved in the encoding; Indicates the first Parameters of event camera 1 for each band channel; Indicates pixel position The first The reflectivity of each band channel can represent the reflectivity of that pixel location under the corresponding spectral or illumination conditions; Indicates the first Each band channel at time The timing function is used to represent the first... Each band channel at time The intensity.

[0065] Specifically, reflectance can be represented by rank-based models, low-rank basis expansion, sparse dictionary representation, direct solution per band, piecewise smoothing models, or neural implicit representations, etc.

[0066] Event triggering satisfies that the logarithmic intensity change between adjacent time points is approximately equal to the event polarity multiplied by the event triggering threshold. Thus, adjacent event pairs can be transformed into solvable constraints on surface normal and multispectral reflectivity.

[0067] Specifically, the event constraint model can be represented as: ,in, Indicates time The imaging intensity, Indicates time The imaging intensity, This is the threshold for triggering the event.

[0068] Step S5: Decouple the surface normal and multispectral reflectance by repeating the same wavelength in multiple directions;

[0069] Since the same wavelength band can repeat under different illumination directions, a geometric constraint for reflectivity cancellation can be constructed for the intensity ratios of two sets of events within the same wavelength band. At least two sets of independent event intensity ratio constraints are needed to determine the surface normal direction; after the surface normal is constrained, the remaining multi-band intensity information can be further used to solve for the reflectivity of each band.

[0070] Step S6: Improve usability under attached shadow and highlight conditions through a redundant repetition mechanism;

[0071] For any given fixed illumination, the attached shadow covers at most a hemispherical area; however, after repeating the design using cubic azimuth separation in the same band, even if one or two observations are affected by attached shadows or highlights, at least one valid observation is usually still retained. During decoding, angle-gated weights based on the current normal estimate and the illumination direction can be used to reduce the weighting of highlight-dominated anomaly constraints.

[0072] In this invention, the robust mechanism for handling the effects of attached shadows or highlights can employ angle-gated weights, hard threshold removal, confidence graph prediction, RANSAC-style anomaly constraint removal, or other robust estimation methods.

[0073] This invention proposes a redundant observation mechanism and decoding constraint design for attached shadows and specular highlights, ensuring that each band retains at least one valid observation under non-ideal conditions.

[0074] Step S7: Recover the absolute multispectral reflectance using constant light components;

[0075] When observing only the event intensity ratio, different multispectral reflectance scales may produce the same event response. This invention additionally superimposes known radiance onto the directional light. The constant light component makes the event intensity satisfy ,in, Indicates the imaging intensity at different times in step S4. The absolute value, Indicates the imaging intensity in step S4 Normalized values; for two known times , The event intensity ratio is obtained. Seeking Therefore, no additional frame camera or additional intensity sensor is required to recover the absolute multispectral reflectance scale in the event domain.

[0076] Specifically, without superimposed constant light components, if the overall image intensity is multiplied by the scale factor... Then at two moments and The logarithmic intensity changes between them satisfy the following relationship:

[0077] ;

[0078] Among them, the scaling factor They are canceled out, therefore different absolute scales may produce the same event response. After superimposing a constant light component, the logarithmic intensity change is:

[0079] ;

[0080] This expression is related to the scaling factor. This is relevant, thus enabling different absolute scales to be distinguished through event responses.

[0081] like Figure 3 As shown in (a), without superimposed constant light components, the two sets of directional light imaging intensities, although having different absolute scales, maintain a consistent relative proportion over time. Since event camera 1 triggers events based on the logarithmic intensity changes between adjacent moments, the logarithmic intensity difference remains unchanged when the overall imaging intensity is multiplied by the same positive scale coefficient. Therefore, the two sets of imaging intensities with different absolute scales can produce the same or approximately the same event triggering time, event polarity, and event number. Thus, it is evident that relying solely on the relative brightness change information in the event stream is insufficient to distinguish the overall scale of the imaging intensity, and consequently, the absolute scale of the multispectral reflectance cannot be directly determined.

[0082] like Figure 3 As shown in (b), a constant light component with known radiance is superimposed on the directional light imaging intensity. Afterwards, the imaging intensities of directional light at different absolute scales became respectively Since the constant light component participates in imaging additively, when the scale coefficient is unknown... When changes occur, the ratio of total imaging intensity at adjacent time points and the corresponding logarithmic intensity change no longer remain constant. Therefore, different scales typically exhibit differences in event trigger time, number of events, or event polarity sequence. Based on this difference in event response and known constant light components... and normalized relative intensity The absolute strength scale factor can then be solved. This allows for the restoration of the absolute multispectral reflectance of each band.

[0083] Step S8: Perform joint optimization using an event-driven spectrum-angle decoder;

[0084] The unknowns are parameterized as pixel-level normal map n and low-rank multispectral reflectance model. ,in, Represents pixel coefficient, Indicates the band base, The dimension of the low-rank matrix is ​​represented. The final result is obtained by minimizing the consistency error of adjacent event pairs and combining non-negativity constraints, normal unit length constraints, forward visibility constraints, and spatial consistency regularization terms.

[0085] In this invention, the event-driven spectrum-angle decoder can be implemented by combining analytical solution with iterative optimization, or by using an end-to-end neural network, neural field, differentiable optimizer or a hybrid scheme thereof, as long as it essentially utilizes the spectrum-angle joint constraint of the event domain and the idea of ​​constant light to eliminate scale ambiguity.

[0086] In this invention, the decoding side can specifically employ an event-driven spectral-angle decoder implemented in PyTorch to perform joint optimization on a single scene. Specifically, the rank K of the low-rank reflectivity model is set to 8, the AdamW optimizer is used, beta can be taken as (0.5, 0.95), and the learning rate of the normal parameter can be set to... The learning rate for the reflectivity parameter can be taken as follows: Each scene is iterated approximately 1600 times. To suppress abnormal highlights and high-frequency noise, angle-gated weights and Gaussian consistency regularization terms can be introduced.

[0087] This invention proposes an event-driven spectral-angle decoding method that matches the spectral-angle multiplexing illumination coding scheme, enabling the system to stably output normal maps and absolute multispectral reflectance under feasible hardware conditions.

[0088] Step S9: Output the results and serve downstream tasks;

[0089] The system output includes at least a dense normal map of the target surface and the absolute multispectral reflectance vector of each pixel. It can be used for material identification, industrial defect detection, cultural relic analysis or other application tasks that require joint geometric-spectral information, such as joint geometric-spectral detection of 3D printed parts, industrial parts or standard objects.

[0090] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for joint reconstruction of surface normal and multispectral reflectance based on an event camera, characterized in that, Includes the following steps: Step S1: Construct a spectral-angle multiplexed illumination encoding device; Step S2: Establish lighting calibration parameters; Step S3: Perform single event collection; Step S4: Establish a joint imaging and event constraint model; Step S5: Decouple the surface normal and multispectral reflectance by repeating the same wavelength in multiple directions; Step S6: Improve usability under attached shadow and highlight conditions through a redundant repetition mechanism; Step S7: Recover absolute reflectivity using a constant light component; Step S8: Perform joint optimization using an event-driven spectrum-angle decoder.

2. The method for joint reconstruction of surface normal and multispectral reflectance based on an event camera according to claim 1, characterized in that, In step S1, the spectrum-angle multiplexing illumination encoding device includes an event camera, a beam splitter, a constant light source, a halo illumination module, and a controller; the object under test, the halo illumination module, the constant light source, the beam splitter, and the event camera are arranged in sequence, with the beam splitter positioned in front of the event camera so that the field of view of the event camera is coaxial with the constant light path, the constant light source is used to generate constant light components, and the halo illumination module is used to generate directional light.

3. The method for joint reconstruction of surface normal and multispectral reflectance based on an event camera according to claim 2, characterized in that, The halo lighting module includes a ring-shaped bracket and multiple LEDs evenly arranged on the ring-shaped bracket along the azimuth angle, and is divided into... Each band channel is arranged repeatedly, and each band channel corresponds to multiple different lighting directions.

4. The method for joint reconstruction of surface normal and multispectral reflectance based on an event camera according to claim 1, characterized in that, In step S2, the illumination directions at multiple different angles corresponding to each band channel, the spectral power distribution of each band channel, the spectral response of the event camera at each band channel, and the known radiance of the constant light component are pre-calibrated or recorded.

5. The method for joint reconstruction of surface normal and multispectral reflectance based on an event camera according to claim 1, characterized in that, In step S3, the object under test, under one or more rounds of preset timing sequence lighting condition switching, outputs an event stream from the event camera. , For pixel position, For the polarity of the event, This is the event timestamp.

6. The method for joint reconstruction of surface normal and multispectral reflectance based on an event camera according to claim 1, characterized in that, In step S4, the joint imaging model is represented as follows: , Indicates pixel position The normal vector to the surface of the object being measured. Indicates time The corresponding incident light direction; Indicates the number of band channels involved in the encoding; Indicates the first Parameters of the event camera for each band channel; Indicates pixel position The first The reflectivity of each band channel represents the reflectivity of that pixel location under the corresponding spectral or illumination conditions; Indicates the first Each band channel at time The timing tuning function is used to represent the first... Each band channel at time The intensity.

7. The method for joint reconstruction of surface normal and multispectral reflectance based on an event camera according to claim 1, characterized in that, In step S4, the event constraint model is represented as follows: , Indicates time The imaging intensity, Indicates time The imaging intensity, This is the threshold for triggering the event.

8. The method for joint reconstruction of surface normal and multispectral reflectance based on an event camera according to claim 1, characterized in that, In step S5, geometric constraints for reflectivity cancellation are constructed for the intensity ratios of two sets of events in the same band; at least two sets of independent event intensity ratio constraints are needed to determine the surface normal direction; after the surface normal is constrained, the remaining multi-band intensity information is used to solve for the reflectivity of each band.

9. The method for joint reconstruction of surface normal and multispectral reflectance based on an event camera according to claim 1, characterized in that, In step S7, a known radiance is superimposed on top of the directional light. The constant light component makes the event intensity satisfy ,in, Indicates the imaging intensity at different times in step S4. The absolute value, Indicates the imaging intensity in step S4 Normalized values; for two known times , The event intensity ratio is obtained. Seeking .

10. The method for joint reconstruction of surface normal and multispectral reflectance based on an event camera according to claim 1, characterized in that, In step S8, the unknowns are parameterized as pixel-level normal map n and low-rank multispectral reflectance model. , Represents pixel coefficient, Indicates the band base, The dimension of the low-rank matrix is ​​represented by . The final result is obtained by minimizing the consistency error of adjacent event pairs and combining non-negativity constraints, normal unit length constraints, forward visibility constraints and spatial consistency regularization terms.

Images