A spectral imaging based object surface measurement system and method

Abstract

The application discloses a kind of object surface measurement system and method based on spectral imaging, collect the color spectrum data of object surface under the light of multiple angles alone;Collect the image data of object surface under the light of multiple angles, different from the angle of spectral data acquisition;Collect the image data of object surface under the light of single diffusion and the light of multiple angles, different from the angle of spectral data acquisition;The weight of the reflectivity of the image feature under different light is assigned, and then the spatial flicker point feature and texture feature are extracted, the semantic feature of color and flicker coupling is extracted from spectral feature, the three features are used for the prediction of the flicker degree and texture degree of object surface respectively, the loss between the prediction result and the measurement result of standard instrument is constructed, to train neural network to carry out flicker degree and texture degree prediction.The application is suitable for high-precision color and surface effect detection scene of automobile painting, plastic building material, printing and packaging industry.

Description

Technical Field

[0001] This invention belongs to the field of color measurement technology, specifically relating to an object surface measurement system and method based on spectral imaging. Background Technology

[0002] In industrial production fields such as automotive painting, plastic building materials, and printing and packaging, the surface of objects is often treated with special components such as metallic powder and pearlescent materials to achieve three core visual effects:

[0003] 1. Color Travel: The sample exhibits different color abrupt changes under different observation or lighting angles;

[0004] 2. Sparkle effect: A bright flash produced by a highly reflective pigment sheet at a specific viewing angle under directional strong illumination;

[0005] 3. Graininess: The non-uniform macroscopic graininess that appears on a surface under diffused lighting.

[0006] The quantification of the above three types of visual effects corresponds to three core industrial measurement parameters: multi-angle color, glitter value, and texture value.

[0007] There are very few existing devices capable of simultaneously measuring three core parameters. For example, the measuring device with publication number CN103808666A establishes its hardware architecture based on the following three points:

[0008] 1. Coaxial common optical path design: The spectral sensor and the image sensor receive the reflected light from the sample from the same observation direction;

[0009] 2. Beam splitter hardware: A semi-transparent and semi-reflective beam splitter is used to split the common beam into two, which are then directed to the spectrometer and the color camera, respectively.

[0010] 3. Polarization equalization control: The beam splitter rotates around the common optical path and forms an angle of approximately 45° with the system plane to equalize the polarization components.

[0011] While the "same-angle acquisition + beam splitter" mode solves the synchronous sampling problem, this fixed hardware design means that the measurement of its flash value and texture value depends entirely on the sequential scanning of a single directional light source, which has serious limitations at the physical mechanism level.

[0012] 1. Beam splitters cause optical signal loss, resulting in a low system signal-to-noise ratio;

[0013] This patent employs a semi-transparent, semi-reflective beam splitter (typically with a 50% beam splitting ratio), which evenly distributes reflected light to the spectral sensor and the image sensor. When only color measurement is performed, 50% of the effective light signal is diverted to the image sensor, rendering this portion unusable for color measurement and resulting in a direct 50% loss of light intensity. Similarly, when only surface effect measurement is performed, the spectral sensor also wastes 50% of the light signal. This light signal loss directly leads to a significant reduction in the system's signal-to-noise ratio, and since the signal-to-noise ratio of color measurement directly determines the instrument's measurement repeatability, this deficiency severely impacts measurement accuracy and stability.

[0014] 2. The addition of a beam splitter leads to complex structural processing and high production costs;

[0015] The installation of beam splitters requires high-precision, multi-dimensional structural coordination. Conventional 3-axis CNC (Computer Numerical Control) machining centers cannot meet the processing requirements, and multi-axis high-precision CNC machining centers must be used. The equipment requirements are stringent and the processing technology is complex, which directly leads to long processing cycles, low yield rates, and high overall manufacturing costs for structural components.

[0016] 3. Fixed optical path design, resulting in poor data compatibility;

[0017] The fixed observation angle and the complete binding of the beam splitter structure make it impossible to adapt to the measurement angle standards of different manufacturers, making it difficult to match the measurement data with the patented device, resulting in poor industry versatility.

[0018] On the other hand, in traditional surface effect measurement techniques, sparkle assessment typically relies on the sequential unidirectional illumination of multiple directional LED light sources, with reflection images acquired independently from specific single angles using image sensors. However, this measurement mode based on discrete unidirectional illumination suffers from two extremely significant technical drawbacks:

[0019] 1. A severe disconnect between the measured light field and the actual visual perception of the human eye;

[0020] When humans observe the flickering and texture of an object's surface in a real environment (such as natural sunlight or exhibition hall lighting), the visual system is always in an extremely complex multidimensional light field, which is by no means an ideal single-direction directional illumination.

[0021] Existing devices for assessing flicker typically rely on multiple directional LED light sources being sequentially illuminated in one direction to capture discrete images. However, when humans observe objects in real-world environments (such as natural light or in an exhibition hall), their visual system is situated within an extremely complex multidimensional light field. This forced separation of unidirectional illumination patterns leads to a significant discrepancy between the data measured by the device and the actual macroscopic visual perception of the human eye, potentially creating a cognitive gap.

[0022] 2. Completely ignore the nonlinear optical coupling effect of special effect pigments;

[0023] The special effects coating contains a large number of microscopically arranged mica flakes, aluminum powder, and other sheet-like reflective media. Under simultaneous illumination from complex multi-source light sources, these media undergo extremely complex refraction, multiple scattering, internal occlusion, and spatial light and shadow interactions. For example, simply adding two images obtained by illuminating the sample surface "separately" from 45° and 75° directional light sources at the pixel level will not result in a physical equivalent to the nonlinear optical coupling effect induced by the two beams of light illuminating the sample surface "simultaneously." The traditional "sequential illumination" measurement method completely severs this interaction of light, severely limiting the measurement fidelity of complex paint surface effects. Summary of the Invention

[0024] To address the shortcomings of existing technologies and achieve simultaneous measurement of color, glitter value, and texture value, thereby improving the effective light signal, reducing processing costs, and enhancing data compatibility, this invention adopts the following technical solution:

[0025] A method for measuring the surface of an object based on spectral imaging includes the following steps:

[0026] Collect color spectral data of object surfaces under multiple angles of individual illumination to extract spectral features;

[0027] Collect object surface image data under multi-angle illumination, different from the spectral data acquisition angle, to extract the first image features;

[0028] Collect object surface image data under individual diffuse lighting and under diffuse lighting combined with the multi-angle lighting, at angles different from the spectral data acquisition angle, to extract second image features; wherein, collecting object surface image data under individual diffuse lighting at angles different from the spectral data acquisition angle, i.e., obtaining the object surface background color and micro-uniformity features, thereby establishing a physical benchmark for "texture (Graininess)" evaluation.

[0029] A multimodal heterogeneous fusion neural network is constructed. Through a spatial weighted attention mechanism, cursor scalars of each image feature are extracted, enabling the neural network to macroscopically perceive the overall energy state of each image feature. This allows for rapid localization of image features with abnormal energy. Weights are then assigned to the reflectivity of image features under different lighting conditions, and these weights are combined with the image features to obtain new image features. This automatically enhances lighting patterns with high physical saliency (such as step interferometry) while effectively suppressing overexposure of the mirror surface or low signal-to-noise ratio signals caused by sensor misalignment. Spatial scintillation features are extracted from the new image features to generate image spatial features. Semantic features coupling color and scintillation are extracted from the spectral features, and texture features are extracted from the new image features. Based on the joint features obtained through spatial cascading of image spatial features, semantic features, and texture features, the scintillation and texture of the object surface are predicted. The prediction results are compared with the measurement results of a standard instrument to construct a joint loss training neural network, ensuring that the predicted scintillation and texture of the object surface are consistent with the measurement results of the standard instrument.

[0030] The trained neural network is used to predict the flicker and texture of the surface of the object being tested.

[0031] Furthermore, the multi-angle combined lighting is multi-angle individual lighting and / or multi-angle composite lighting. The multi-angle composite lighting is based on a sequence of light sources at different angles. A sliding window is constructed by two light sources at different angles, and the light is slidably lit in the light source sequence with one or more light sources as the step size.

[0032] Furthermore, the multi-angle composite illumination includes two or more adjacent light sources, which are illuminated in a sliding window sequence with the interval between adjacent light sources as the step size, to obtain a set of adjacent angle sliding window illumination sequences. During the dark field image acquisition process, the adjacent angle sliding window illumination sequence enables the extracted first image features to capture the continuity of surface reflection of the pigment sheet when it is deflected at a small angle, thereby simulating and quantifying the "halo effect" under directional illumination. During the environmental field image acquisition process, the adjacent angle sliding window illumination sequence enables the extracted second image features to be used to evaluate the interference and enhancement effect of the macroscopic ambient light background on the physical fusion effect of adjacent scintillation points.

[0033] Furthermore, the multi-angle composite illumination includes two non-adjacent light sources spanning one or more light sources, which slide and illuminate in the light source sequence with one light source as the step size, to obtain a set of cross-step interference illumination sequences. The cross-step interference illumination sequences in the dark field image acquisition process enable the extracted first image features to obtain the nonlinear optical features of the object surface, such as complex occlusion, shadows, and multiple scattering inside the paint surface, through long-distance multi-directional cross illumination. The cross-step interference illumination sequences in the environmental field image acquisition process enable the extracted second image features to be used to simulate the depth visual response under the combination of real diffuse illumination and the multi-angle illumination, providing core visual criteria for the neural network.

[0034] Furthermore, the multi-angle combined illumination includes multi-angle individual illumination and multi-angle total illumination. In the dark field image acquisition process, multi-angle individual illumination enables the extracted first image features to obtain the image representation sequence under basic unidirectional illumination, while multi-angle total illumination enables the extracted first image features to increase the total luminous flux for dark surfaces, suppress sensor noise from the physical source, and improve the signal-to-noise ratio of feature extraction under extremely low brightness. In the environmental field image acquisition process, multi-angle individual illumination based on diffuse illumination enables the extracted second image features to obtain the visual saliency of flickering points at each angle under background ambient light interference, i.e., the "background color-flickering" contrast feature. While multi-angle total illumination based on diffuse illumination enables the extracted second image features to obtain the limit response boundary of the object surface under full ambient illumination, which is used to correct system deviations in the different angle acquisition mode.

[0035] Furthermore, a homography matrix is ​​constructed from the physical plane acquired by the original image sensor to the virtual orthogonal plane. Based on the homography matrix, the acquired image data is mapped to the pixel coordinates on the virtual orthogonal plane, realizing the "forward" restoration of the field of view and obtaining new image data. The new image data is the distortion-free image data mapped to the virtual orthogonal plane after perspective transformation. For the perspective (trapezoidal) distortion of the field of view caused by the irregular installation of the image sensor, the calibrated homography mapping principle is used to perform geometric alignment and reconstruction of the irregular angle image, realizing pixel-level accurate registration of multi-source image sequences in spatial dimension, thereby constructing a unified geometric reference coordinate system for subsequent multi-channel feature fusion.

[0036] Furthermore, using the number of new image data as the number of channels, a second-order statistical model is performed on multiple channels in the new image data using the gray-level co-occurrence matrix. The multi-channel energy, contrast, correlation, and entropy of each element in the gray-level co-occurrence matrix are integrated and vectorized and concatenated, transforming the pixel spatial distribution into a high-dimensional feature descriptor with clear physical meaning. This provides explicit physical attribute constraints for the neural network, improving the robustness of texture index regression. The element represents the probability density of the adjacent occurrence of the first gray level i and the second gray level j under a specific spatial direction and step size. Energy is used to measure the uniformity of the image gray-level distribution and the coarseness of the texture. When the paint texture distribution is more uniform (e.g., solid color paint), the co-occurrence matrix... The higher the concentration of elements, the greater the energy value. When there is a complex and rough distribution of metal particles on the paint surface, the energy value decreases. Contrast is used to reflect the drastic degree of local gray-level changes and the clarity of texture in an image. Large particles of aluminum powder or high-contrast pearlescent flakes on the coating surface will cause drastic jumps in gray-level of adjacent pixels, which will increase the contrast value. Correlation is used to measure the linear similarity of elements in the spatial gray-level co-occurrence matrix in the row or column direction to quantify the extensibility of the coating texture in a specific direction and the regularity of the local structure. Entropy is used to quantify the complexity and randomness of the paint surface texture information. When the surface contains a large number of irregularly arranged special effect pigments (such as mica flakes that reflect at multiple angles), the spatial distribution of pixels is highly random, and the entropy value reaches its peak at this time.

[0037] Furthermore, the mean and standard deviation of the spectral data are obtained as physical components characterizing the energy intensity reflected from the object's surface. The difference between the spectral data and its mean is then divided by the standard deviation to obtain the semantic components of the spectral data, which characterize the intrinsic color-changing properties as the angle changes. The physical components and the semantic components are used as spectral feature tensors, thereby realizing the collaborative representation of high-dimensional spectral semantics and physical energy intensity at the model input, providing a physically meaningful and dimensionally rigorous basic tensor for subsequent mapping of the neural network.

[0038] A surface measurement system based on spectral imaging includes a spectral acquisition device, an image acquisition device, a diffuse illumination source, and a set of multi-angle illumination sources arranged relative to the surface of the object. The system also includes a neural network module, which acquires spectral features and image features according to the spectral imaging-based surface measurement method to train the neural network module, and then uses the trained neural network module to measure the surface of the object.

[0039] Furthermore, the multi-angle spectrophotometer includes a fan-shaped body with a measurement hole at the bottom, and two spectral acquisition devices surrounding the measurement hole, an image acquisition device at an angle opposite to the spectral acquisition devices, and a set of multi-angle illumination sources are arranged on the arc surface of the fan-shaped body. A diffuse illumination source is also provided above the measurement hole.

[0040] The advantages and beneficial effects of this invention are as follows:

[0041] This invention simultaneously measures color, luminance, and texture values. By employing a design where image and spectral acquisition are at opposite angles, it eliminates the need for a semi-transparent, semi-reflective beam splitter to avoid beam splitting loss, preserving 100% of the effective light signal intensity from both the spectral and image acquisition. This improves the signal-to-noise ratio of both acquisitions and significantly reduces color measurement repeatability errors. Furthermore, it eliminates the need for a 45° beam splitter, simplifying the optical path. Multi-axis machining equipment is unnecessary; conventional 3-axis CNC machining is sufficient for aluminum alloy structural parts. This reduces equipment requirements by over 40%, shortens the machining cycle by 30%, increases the yield rate of structural parts to over 90%, and significantly reduces overall production costs. The invention addresses the issue of inconsistent measurement data with mainstream manufacturers caused by acquisition from different angles. Its proprietary correction algorithm, through a combination of multi-source images and neural network correction, ensures that the measured flash and texture values ​​have a deviation of ≤±1% from existing devices that simultaneously measure three core parameters. This allows for direct integration with existing industry testing systems, ensuring data interoperability and compatibility. Furthermore, without altering the optical path hardware structure, the invention can adapt to measurement results from similar instruments with different angle designs simply through algorithm iteration, maintaining consistency with third-party manufacturers' data, achieving multi-manufacturer data compatibility, expanding product adaptation scenarios, and extending product lifecycle. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the system structure in an embodiment of the present invention.

[0043] Figure 2 This is a schematic diagram of the structure of the multi-angle spectrophotometer in an embodiment of the present invention.

[0044] Figure 3 This is a flowchart of the method in an embodiment of the present invention.

[0045] Figure 4 This is a correlation coefficient diagram between the predicted texture parameters under the multi-angle composite light source of the present invention and the texture parameters measured by the BYK colorimeter.

[0046] Figure 5 This is a correlation coefficient graph between the predicted texture parameters under traditional multi-angle lighting and the texture parameters measured by a BYK colorimeter.

[0047] Figure 6 This is a correlation coefficient diagram between the predicted values ​​of scintillation parameters under the multi-angle composite light source of the present invention and the scintillation parameters measured by the BYK colorimeter.

[0048] Figure 7 This is a correlation coefficient graph between the predicted values ​​of scintillation parameters under traditional multi-angle illumination and the scintillation parameters measured by a BYK colorimeter. Detailed Implementation

[0049] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0050] like Figure 1 As shown, this invention proposes a surface measurement system for objects based on spectral imaging, including a spectral acquisition device, an image acquisition device, a diffuse illumination source, a set of multi-angle illumination sources, and a neural network module. The system also includes a multi-angle spectrophotometer, such as... Figure 2 As shown, the multi-angle spectrophotometer specifically includes a fan-shaped body and a measuring hole 12 disposed at the bottom of the body. An illumination system and a detection system are disposed around the measuring hole 12 on the arc surface of the fan-shaped body.

[0051] Measurement hole 12: Sample detection reference window, serving as a common platform for incident illumination light and outgoing reflected light;

[0052] The lighting system includes multiple sets of directional lighting sources and one set of diffuse lighting sources, which are uniformly arranged around the circumference of the measuring hole 12. They are used for multi-angle spectral measurement, flash measurement, and texture measurement, respectively. The diffuse lighting source is located above the measuring hole 12. In this embodiment of the invention, the multiple sets of directional lighting sources consist of 7 lighting sources.

[0053] The detection system includes a spectral sensor A8, a spectral sensor B9, and an image sensor 10. The three are arranged at different observation angles from the measurement aperture 12, without sharing an optical path or a beam splitter, and each independently collects reflected signals.

[0054] The multi-angle spectrophotometer eliminates the beam splitter design and adopts an independent layout for spectral and image acquisition at different angles. The acquisition process is executed in three standardized steps through the organic combination of spectral measurement and multi-dimensional image encoding:

[0055] Step 1: Multi-angle spectral measurement (spectral data acquisition);

[0056] 1.1. The first sequence of spectral data was acquired using a spectral sensor A8;

[0057] Execution method: Directional lighting source 1 to directional lighting source 7 are lit up individually in sequence, and spectral sensor A8 synchronously collects the reflected spectral signal;

[0058] Data Acquisition: 7 sets of multi-angle reflectance spectral data;

[0059] Physical significance: To obtain the basic reflectance spectral response of the sample under the first observation angle layout.

[0060] 1.2. The second sequence of spectral data was acquired using a B9 spectral sensor.

[0061] Execution method: Directional lighting source 1 to directional lighting source 7 are lit up individually in sequence, and the spectral sensor B9 synchronously collects the reflected spectral signal;

[0062] Data Acquisition: 7 sets of multi-angle reflectance spectral data;

[0063] Physical significance: By utilizing the angular difference between the two sensors (A and B), spectral information of the supplementary angle is obtained, resulting in a total of 14 sets of spectral data to improve the accuracy of color space reconstruction.

[0064] Step 2: Dark field image encoding and acquisition (image data, Ldiff diffuse light source 11 off)

[0065] 2.1 Basic unidirectional representation sequence;

[0066] Execution method: With the diffuse light source 11 off, the directional illumination sources 1 to 7 are lit up one by one in sequence, so that the image sensor 10 can acquire images synchronously.

[0067] Data Acquisition: 7 raw unidirectional illumination images.

[0068] Physical significance: To establish a linear reflection characteristic benchmark under each independent orientation angle for the extraction of basic scintillation parameters.

[0069] 2.2, Adjacent Corner Sliding Window Coupling Sequence;

[0070] Execution method: Keep diffuse light source 11 off, and light up the following combinations in sequence:

[0071] (1) Adjacent double lamp groups: (directional lighting source 1 + directional lighting source 2), (directional lighting source 2 + directional lighting source 3), (directional lighting source 3 + directional lighting source 4), (directional lighting source 4 + directional lighting source 5), (directional lighting source 5 + directional lighting source 6), (directional lighting source 6 + directional lighting source 7), 6 images were collected.

[0072] (2) Adjacent three light groups: (directional lighting source 1 + directional lighting source 2 + directional lighting source 3), (directional lighting source 2 + directional lighting source 3 + directional lighting source 4), (directional lighting source 3 + directional lighting source 4 + directional lighting source 5), (directional lighting source 4 + directional lighting source 5 + directional lighting source 6), (directional lighting source 5 + directional lighting source 6 + directional lighting source 7), collect 5 images.

[0073] Data Acquisition: A total of 11 coupled images were acquired in this project.

[0074] Physical significance: To capture the continuity of reflection when pigment flakes are deflected at a small angle, and to simulate and quantify the "halo effect" under directional lighting.

[0075] 2.3. Step-interference coding sequence;

[0076] Execution method: Keep diffuse light source 11 off, and light up the following combinations sequentially according to the light number interval step k:

[0077] (1) Spanning one corner (k=1): (Directional lighting source 1 + directional lighting source 3), (directional lighting source 2 + directional lighting source 4), (directional lighting source 3 + directional lighting source 5), (directional lighting source 4 + directional lighting source 6), (directional lighting source 5 + directional lighting source 7), collect 5 images.

[0078] (2) Crossing two angles (k=2): (directional lighting source 1 + directional lighting source 4), (directional lighting source 2 + directional lighting source 5), (directional lighting source 3 + directional lighting source 6), (directional lighting source 4 + directional lighting source 7), collect 4 images.

[0079] (3) Cross-triangle (k=3): (directional lighting source 1 + directional lighting source 5), (directional lighting source 2 + directional lighting source 6), (directional lighting source 3 + directional lighting source 7), collect 3 images.

[0080] (4) Spanning four corners (k=4): (directional lighting source 1 + directional lighting source 6), (directional lighting source 2 + directional lighting source 7), collect 2 images.

[0081] Data Acquisition: This item contains a total of 14 images.

[0082] Physical significance: By using long-distance multi-directional cross illumination, complex nonlinear optical features such as occlusion, shadows, and multiple scattering inside the paint surface can be extracted.

[0083] 2.4, All-round Enhancement Mode Sequence;

[0084] Execution method: Keep diffuse light source 11 off, and simultaneously turn on all directional lighting sources 1 to directional lighting sources 7.

[0085] Data acquired: 1 high-intensity radiation image.

[0086] Physical significance: To increase the total luminous flux for dark-colored paint surfaces, suppress sensor noise from the physical source, and improve the signal-to-noise ratio of feature extraction at extremely low brightness.

[0087] Step 3: Ambient lighting sequence (image data, diffuse light source 11 remains constantly lit)

[0088] 3.1 Environmental field benchmark testing;

[0089] Execution method: Only turn on diffuse light source 11, and turn off all 7 groups of directional lighting sources.

[0090] Data acquisition: 1 image with pure diffuse lighting.

[0091] Physical significance: To obtain the background color and microscopic uniformity characteristics of the sample and establish a physical benchmark for "graininess" evaluation.

[0092] 3.2 Unidirectional coupling sequence of the environmental field;

[0093] Execution method: Keep diffuse light source 11 constantly lit, and sequentially add the following directional lighting sources: (diffuse light source 11 + directional lighting source 1), (diffuse light source 11 + directional lighting source 2), (diffuse light source 11 + directional lighting source 3), (diffuse light source 11 + directional lighting source 4), (diffuse light source 11 + directional lighting source 5), (diffuse light source 11 + directional lighting source 6), (diffuse light source 11 + directional lighting source 7).

[0094] Data Acquisition: 7 environmental field comparison images.

[0095] Physical meaning: The measurement of the visual salience of the flickering points at various angles under background ambient light interference is the "background color-flickering" contrast characteristic.

[0096] 3.3, Coupling sequence of adjacent windows in the environmental field;

[0097] Execution method: Keep diffuse light source 11 constantly lit, and reproduce the adjacent corner sliding window combination:

[0098] (1) Adjacent dual-lamp coupling: (diffuse light source 11 + directional light source 1 + directional light source 2), (diffuse light source 11 + directional light source 2 + directional light source 3), (diffuse light source 11 + directional light source 3 + directional light source 4), (diffuse light source 11 + directional light source 4 + directional light source 5), (diffuse light source 11 + directional light source 5 + directional light source 6), (diffuse light source 11 + directional light source 6 + directional light source 7), 6 images were collected.

[0099] (2) Adjacent three-lamp coupling: (diffuse light source 11 + directional light source 1 + directional light source 2 + directional light source 3), (diffuse light source 11 + directional light source 2 + directional light source 3 + directional light source 4), (diffuse light source 11 + directional light source 3 + directional light source 4 + directional light source 5), (diffuse light source 11 + directional light source 4 + directional light source 5 + directional light source 6), (diffuse light source 11 + directional light source 5 + directional light source 6 + directional light source 7), 5 images were collected.

[0100] Data Acquisition: This item contains a total of 11 environmental field coupling images.

[0101] Physical significance: To evaluate the interference and enhancement effects of macroscopic ambient light background on the physical fusion effect of adjacent scintillation points.

[0102] 3.4 Environmental Field Step Coding Sequence

[0103] Execution method: Keep diffuse light source 11 constantly lit and reproduce the step interference sequence combination:

[0104] (1) Spanning one corner (k=1): (diffuse light source 11 + directional light source 1 + directional light source 3), (diffuse light source 11 + directional light source 2 + directional light source 4), (diffuse light source 11 + directional light source 3 + directional light source 5), (diffuse light source 11 + directional light source 4 + directional light source 6), (diffuse light source 11 + directional light source 5 + directional light source 7), collect 5 images.

[0105] (2) Crossing two angles (k=2): (diffuse light source 11 + directional light source 1 + directional light source 4), (diffuse light source 11 + directional light source 2 + directional light source 5), (diffuse light source 11 + directional light source 3 + directional light source 6), (diffuse light source 11 + directional light source 4 + directional light source 7), collect 4 images.

[0106] (3) Cross-triangle (k=3): (diffuse light source 11 + directional light source 1 + directional light source 5), (diffuse light source 11 + directional light source 2 + directional light source 6), (diffuse light source 11 + directional light source 3 + directional light source 7), collect 3 images.

[0107] (4) Spanning four corners (k=4): (diffuse light source 11 + directional light source 1 + directional light source 6), (diffuse light source 11 + directional light source 2 + directional light source 7), collect 2 images.

[0108] Data Acquisition: This item contains a total of 14 environmental field images.

[0109] Physical significance: It simulates the depth visual response in real complex scenes (ambient diffuse + multi-directional point light source) and provides core visual criteria for neural networks.

[0110] 3.5. Environmental Field Omnipotent Limit Mode Sequence

[0111] Execution method: Keep diffuse light source 11 constantly lit, and simultaneously turn on directional lighting sources 1 to directional lighting sources 7.

[0112] Data Acquisition: 1 image of the environmental field's limit saturation.

[0113] Physical significance: To obtain the limiting response boundary of the sample under all environmental loads, which is used to correct system biases in different angle acquisition modes.

[0114] Through the above three steps, a single measurement by a multi-angle spectrophotometer can acquire 14 sets of spectral data and a total of 67 multimodal images. This execution process achieves a high degree of fit between the measurement indicators and human visual perception through strict symmetrical coding of the "dark field" and the "ambient field," and can automatically compensate for hardware mechanical assembly tolerances, ensuring extremely high inter-station consistency.

[0115] The image sensor 10 and the spectral sensor adopt different observation angles relative to the surface of the sample being measured (measuring hole 12), which is completely different from the design of "spectral and image pickups being acquired at the same angle" in the existing technology. This eliminates the need for a semi-transparent and semi-reflective beam splitter. The spectral acquisition optical path and the image acquisition optical path are completely independent, with no shared optical path and no beam splitting loss. This solves the problem of light intensity loss caused by the beam splitter, allowing 100% of the effectively reflected light to enter the corresponding sensor, and greatly improving the system signal-to-noise ratio.

[0116] Furthermore, based on the aforementioned system, this invention also proposes a surface measurement method for objects based on spectral imaging. Based on the end-to-end mapping of the multimodal heterogeneous residual fusion network (MH-ResNet), 14 sets of spectra and 67 coded image sequences acquired under irregular angular layouts are mapped into industry standard device sparkle and graininess indices.

[0117] Specifically, in this embodiment of the invention, 1000 sets of actual samples with glitter and texture effects are prepared. First, standard glitter and texture values ​​are measured using an existing multi-angle colorimeter. Then, the device of this invention is used to measure 14 multi-angle spectral images and 67 multimodal images. Using the 14 sets of spectra and 67 multimodal images as input, and the standard glitter and texture values ​​as output, a dedicated neural network correction model is constructed and trained. The surface effect is corrected through the correction model, achieving accurate data matching while remaining compatible with measurement standards from other manufacturers, such as… Figure 3 As shown, the measurement method specifically includes the following steps:

[0118] Step 1: Perform tensor quantization preprocessing on multi-source heterogeneous data, including full-information feature structure in the spectral domain, geometric alignment and reconstruction of images at different angles, and extraction of texture feature-specific operators. Specifically, this includes the following steps:

[0119] Step 1.1: Decoupling of full information features in the spectral domain;

[0120] 14 sets of original spectral features Perform a standard normal variable transformation (SNV) to construct a two-component feature space:

[0121] Semantic component A1: Represents the intrinsic color-changing properties that vary with angle.

[0122]

[0123] in, Represents the mean of spectral characteristics , This represents the standard deviation of spectral characteristics. Since each set of curves covers the visible light band of 400~700nm, with a sampling interval of 10nm, and each set contains 31 spectral values, the output dimension is 14×31=434 semantic feature values. This processing eliminates the absolute difference in intensity and focuses on characterizing the intrinsic color change characteristics of the sample as the angle changes.

[0124] Physical component A2: Explicitly retains energy intensity. Extract the [[ from each set of curves]] , The output dimension is 14×2=28 intensity feature values. For special materials such as metallic paint, this feature records the core physical quantity for judging the scintillation intensity, ensuring that the algorithm can still capture the surface reflection energy of the sample under different angle observation.

[0125] Synthesis input: spectral feature tensor V spec =[A1, A2]∈R 462 This generates a two-component spectral joint feature with a total dimension of 462 (434+28).

[0126] This step achieves a collaborative representation of high-dimensional spectral semantics and physical energy intensity at the data input end, providing a physically meaningful and dimensionally rigorous foundation tensor for subsequent mapping of the neural network.

[0127] Step 1.2: Geometric alignment and reconstruction of the 67-channel coded light field at different angles;

[0128] To address the perspective (trapezoidal) distortion caused by the 10-angle mounting of the image sensor, a calibrated homography mapping principle is used to achieve pixel-level precise registration of multi-source image sequences in spatial dimensions. This establishes a unified geometric reference coordinate system for subsequent multi-channel feature fusion. The specific process is as follows:

[0129] Beforehand, using a standard calibration template, obtain the homography matrix from the original sensor physical plane to the virtual orthogonal projection plane. For each frame of the coded image acquired, pixel reconstruction is performed through homogeneous coordinate transformation:

[0130]

[0131]

[0132] in, This represents the two-dimensional pixel coordinates in the original acquired image (which contains perspective distortion). The original sampled image is the image with trapezoidal distortion actually acquired by the image sensor 10 at an angular position. This represents the pixel coordinates mapped onto the virtual orthogonal plane after reconstruction, achieving "positive" restoration of the field of view. The resulting target image is the distortion-free image mapped onto the virtual orthogonal plane after perspective transformation. The scale factor in homogeneous coordinates characterizes the nonlinear projection depth during perspective transformation. The actual Cartesian coordinates are normalized by dividing the homogeneous result by 1 / 2. ) to obtain, that is homography matrix The nine core mapping parameters comprehensively encapsulate the weights of rotation, translation, scaling, and projection transformations, describing the projective transformation relationship between two planes.

[0133] Through the above transformations, the system synchronously projects 33 dark field encoded images, 33 ambient field encoded images, and 1 diffuse reference image onto a unified geometric plane. Subsequently, a bilinear interpolation algorithm is used to resample the pixel values, ultimately constructing a high-dimensional image tensor T. img ∈R H×W×67 H represents the height of the image, W represents the width of the image, and 67 (33+33+1) represents the dimension of the image.

[0134] This tensor ensures pixel-level alignment of images under various illumination modes while fully preserving the microscopic reflection distribution characteristics of the sample surface under 67 composite light fields.

[0135] Step 1.3: Extract texture feature-specific operators;

[0136] To guide the neural network to quickly and accurately locate the graininess distribution range of macroscopic surface vision, this invention utilizes the Gray-Level Co-occurrence Matrix (GLCM) on the reconstructed high-dimensional image tensor T. img Second-order statistical modeling was performed on 67 channels. Through prior manual feature engineering, the complex pixel spatial distribution was transformed into high-dimensional features with clear physical meaning, providing explicit physical property constraints for the neural network and improving the robustness of texture index regression. Feature descriptor formula:

[0137]

[0138] in, This indicates that the extracted results from 67 channels are vectorized and concatenated, where c represents the channel index, ultimately forming a low-level texture feature vector F with feature dimensions N = 4 × 67 = 268. text .

[0139] For an image with G gray levels, the element P(i,j) in the normalized gray-level co-occurrence matrix represents the probability density of gray levels i and j being adjacent under a specific spatial direction and step size. The core operator calculation method for any of the 67 channels is as follows:

[0140] (1) Energy (ASM, Angular Second Moment):

[0141]

[0142] ASM is used to measure the uniformity of grayscale distribution and texture coarseness of an image. When the paint texture is more uniform (such as solid color paint) and the elements of the co-occurrence matrix are more concentrated, the ASM value is larger. When there is a complex and rough distribution of metal particles on the paint surface, the ASM value decreases.

[0143] (2) Contrast (CON):

[0144]

[0145] CON is used to reflect the severity of local grayscale changes and the clarity of texture in an image. Large particles of aluminum powder or high-contrast pearlescent film on the coating surface can cause drastic jumps in grayscale between adjacent pixels, resulting in a significant increase in the CON value.

[0146] (3) Correlation (COR):

[0147]

[0148] in, Represents the marginal mean of matrix elements. This represents the marginal standard deviation of matrix elements.

[0149] COR is used to measure the linear similarity of elements in a spatial gray-level co-occurrence matrix in the row or column direction, in order to quantify the extensibility of coating texture in a specific direction and the regularity of local structure.

[0150] (4) Entropy (ENT, Entropy):

[0151]

[0152] ENT is used to quantify the complexity and randomness of paint texture information. When the surface contains a large number of irregularly arranged special effect pigments (such as mica flakes that reflect at multiple angles), the pixel spatial distribution is highly random, and the ENT value reaches its peak at this time.

[0153] Step 2: Construct a multimodal heterogeneous fusion neural network architecture (MH-ResNet), including building a channel-space weighted attention module, three-branch feature dimensionality reduction and deep coupling, specifically including the following steps:

[0154] Step 2.1: Construct a 67-channel Spatial Weight Attention Module.

[0155] Based on the irregular layout and 67 sets of coded lighting modes (including 33 dark field images, 33 ambient field images and 1 diffuse reference image), the effective contribution of each channel signal to the surface effect evaluation varies significantly. This invention introduces an adaptive spatial weight attention mechanism, which dynamically weights the feature channels to achieve adaptive extraction of effective physical signals, thereby using the algorithm to combat sensor installation misalignment, overexposure of the mirror surface and interference from environmental noise.

[0156] Specifically, in existing multi-angle surface effect measurement systems, to achieve "co-angular acquisition" of spectra and images, it is necessary to forcibly use a semi-transparent, semi-reflective beam splitter or extremely complex multi-axis optical hardware. Although this co-axial design ensures the consistency of the observation angle, it suffers from severe optical signal loss and extremely high mechanical assembly costs. This invention eliminates the beam splitter and adopts a "different-angle independent layout" for image acquisition and spectral acquisition. However, different-angle observation inevitably leads to physical signal degradation under non-standard optical angles, specifically manifested as follows:

[0157] (1) Local glare: Under certain directional illumination angles, the incident light beams that are totally reflected on the surface being measured will directly enter the image sensor 10 set at different angles, causing local pixel saturation of the image and completely masking the real sparkle of the metallic paint.

[0158] (2) Effective signal attenuation and noise dominance: At other lighting angles, due to deviation from the optimal reflection angle, the effective flicker signal received by the sensor is extremely weak, causing the feature map of this channel to be completely submerged by the ambient noise.

[0159] To address the hardware-level photometric distortion caused by "uneven installation angles," this invention proposes a design approach of "combating hardware offset with algorithmic redundancy." By acquiring 67 sets of high-dimensional composite light fields (single-point in dark fields, combined illumination, step interference, etc.) in the early stages, the system creates significant "illumination redundancy." Among these 67 lighting modes, there are suitable angles that allow for clear observation of surface effects, but there are also unsuitable angles that result in overexposure or pure noise.

[0160] The 67-channel spatial weighted attention module includes the following components:

[0161] Global Energy Sensing (Squeeze) Unit: Extracts the photometric scalar z of each channel through global average pooling (GAP). c This allows the network to perceive the overall energy state of each of the 67 images from a macroscopic perspective, thereby quickly locating the channels that are "abnormally high energy (overexposure)" and "abnormally low energy (noise)".

[0162] Physical saliency scoring (Excitation) unit: Utilizing a two-layer fully connected network, based on physical priors trained on 1000 samples, it adaptively scores the effectiveness of 67 lighting modes under the current angle. For high-quality combined light channels that clearly represent the reflection of metallic particles, a high weight coefficient s approaching 1 is assigned. c For overexposed or purely noisy channels, assign extremely low weighting coefficients close to 0.

[0163] Feature Reconstruction and Optical Reweighting Unit: Multiplies the generated weight coefficients back into the original feature map, thereby "dynamically silencing" the degraded image. Physically equivalent, this is equivalent to reconstructing an "ideal observation angle" without overexposure or blind spots in virtual space through an algorithm.

[0164] By introducing a 67-channel spatial weighted attention module, the data deviation problem caused by non-angular acquisition is fundamentally solved. This allows the system to automatically extract flicker features with extremely high physical fidelity through deep learning algorithms, even under harsh hardware conditions that completely abandon expensive beam splitters and allow for certain mechanical assembly tolerances. This ensures that the final output flicker and graininess prediction values ​​can be accurately matched with industry standard instruments.

[0165] In this embodiment of the invention, a compression operation is performed by a global energy sensing (Squeeze) unit, and global average pooling (GAP) is used to compress the spatial information of each channel into a global photometric scalar z. c To obtain the macroscopic energy response under each lighting mode, the specific formula is as follows:

[0166]

[0167] in, This represents the global photometric scalar (i.e., channel descriptor) generated by the c-th feature channel. This represents the feature compression operation function. This represents the feature map of the c-th feature channel input to the global energy sensing unit, where H and W represent the height and width of the feature map, respectively, and i and j represent the spatial indexes of the pixels in the feature map at their height and width. This represents the activation value (pixel intensity) of the feature map at position (i,j).

[0168] In this embodiment of the invention, the physical saliency scoring unit performs an excitation operation, learning the complex nonlinear dependencies between illumination channels through a two-layer fully connected structure, and generating a weight vector s for feature rescaling:

[0169]

[0170] Where s represents the output 67-dimensional weighted feature vector, and its components s c ∈[0, 1], z represents the global description vector z = [z1, z2, ..., z], which is composed of the concatenation of scalars of each channel. c ,…, z 67 W1 and W2 represent the weight matrices of the first and second fully connected layers, respectively. Represents the ReLU non-linear activation function, used to enhance the expressive power of the network. This represents the Sigmoid activation function, which aims to map the output to the interval [0, 1] as a gating weight.

[0171] In this embodiment of the invention, the learned weights s are weighted by feature recombination and optical correction unit weighted output (Reweight). c Applied to the corresponding original feature channel u c superior.

[0172] The 67-channel spatial weighted attention mechanism can automatically enhance lighting patterns with high physical saliency (such as step interferometry patterns) while effectively suppressing overexposure of the mirror or low signal-to-noise ratio signals caused by sensor misalignment.

[0173] Step 2.2: Three-branch feature dimensionality reduction and deep coupling;

[0174] The standardized multimodal data is input into a three-branch parallel network. The latent features of each dimension are extracted through a specific decoupling strategy, and dimension alignment is performed for subsequent fusion.

[0175] (1) The image stream uses a CNN flicker feature extraction branch (CNN-Branch);

[0176] The input image tensor, processed by the spatial weighting module, is used to extract spatial scintillation features using the ResNet-18 backbone network. Finally, a 32-dimensional image spatial feature vector F is output through linear mapping. img .

[0177] (2) The spectral flow uses the MLP semantic feature extraction branch (MLP-Branch).

[0178] Input is a 462-dimensional spectral feature tensor V generated by the Standard Normal Variable Transform (SNV). spec A three-layer fully connected perceptron (128→64→32) is used to extract the 32-dimensional semantic feature vector F of color-flicker coupling. spec .

[0179] (3) Texture flow uses a texture feature mapping branch (Static-Branch);

[0180] The input is a 272-dimensional texture feature F extracted by the gray-level co-occurrence matrix (GLCM) operator. text A single-layer linear mapping is used to the 32-dimensional latent feature space F. text_map .

[0181] The features of the three branches mentioned above are spatially concatenated to construct a 96-dimensional joint feature tensor X. cat :

[0182] X cat = [ F img ⊕ F spec ⊕ F text_map ]

[0183] Among them, X cat⊕ represents the 96-dimensional multimodal joint feature vector ultimately used for multi-task regression prediction, and ⊕ represents the feature concatenation operator, which concatenates three independent 32-dimensional vectors along the channel dimension.

[0184] Step 3: Construct a multi-task regression head and a self-balancing loss function to train the neural network, including template preparation and data alignment, to establish an accurate correspondence between the signals acquired by the multi-angle spectrophotometer of this invention and the industry standard measured values; construct a parallel bi-branch regression topology to predict flicker and texture; based on the predicted and true values ​​of flicker and texture, construct a dynamic weighted self-balancing joint loss function that incorporates L2 constraints; train, accept, and package the neural network, specifically including the following steps:

[0185] Step 3.1: Sample preparation and data alignment to establish an accurate correspondence between the signals acquired by the multi-angle spectrophotometer of this invention and the industry standard measured values.

[0186] To ensure that the neural network can accurately map the real physical optical laws, this invention constructs a physical color swatch dataset and establishes a strict truth alignment mechanism with industry standard instruments:

[0187] (1) Prepare 1000 solid color swatches for distribution:

[0188] Standard baseline area (60%): covers mainstream metallic paints and pearlescent paints on the market to lay the foundation for the model's basic generalization ability;

[0189] Boundary extension area (30%): includes extreme cross combinations such as high scintillation (gloss level greater than 80), large-particle aluminum powder, and coarse sand texture, to challenge and extend the resolution limits of the model;

[0190] Extremely scarce zone (10%): Contains solid color paint and aluminum-free / pearl-like samples to provide a baseline anchor point for the network with a base color and a flicker-free state.

[0191] (2) Strict data alignment: For the above 1000 samples, the system of this invention (extracting multimodal features of 14 sets of spectra and 67 coded images) and standard equipment industry standard instruments were used for synchronous measurement. The sparkle value S and texture value G output by the standard equipment were used as the absolute truth for model supervised learning.

[0192] (3) Subset partitioning: The dataset is divided into training set (800 blocks), validation set (100 blocks) and test set (100 blocks) in a ratio of 8:1:1 using a stratified random sampling algorithm.

[0193] Step 3.2: Construct a parallel bi-branch regression topology to predict flicker and texture;

[0194] After extracting the 96-dimensional joint feature tensor X cat Subsequently, two completely independent fully connected regression branches are constructed in the terminal system. Each branch adopts a funnel-shaped dimensionality reduction structure (96→64→32→1) to transform high-dimensional abstract features into one-dimensional concrete physical quantities. Among them, the flicker prediction branch outputs the predicted value. Used to accurately fit the standard scintillation value S of industry-standard instruments; texture prediction branch output predicted value. , used to accurately fit the standard texture value G of industry standard instruments.

[0195] Step 3.3: Based on the predicted and true values ​​of flicker and texture, construct a dynamic weighted self-balancing joint loss function that incorporates L2 constraints;

[0196] With 14 sets of spectra and 67 high-dimensional images as input, the network has a huge number of parameters, making it prone to overfitting on the dataset. Furthermore, in multi-task regression, the convergence speeds and magnitudes of "flicker" and "texture" are inconsistent, easily leading to gradient interference. Therefore, this invention introduces a dynamically weighted loss function based on homoscedastic uncertainty and explicitly adds an L2 norm regularization term. The formula for the joint loss function is as follows:

[0197]

[0198] in, , These are dynamic parameters that the neural network learns automatically during training, representing the observation uncertainties of the flickering and texture tasks, respectively, and are used for dynamic adjustment of the tasks. The basic loss function for the flicker task is represented by Smooth L1 Loss. The basic loss function for the texture degree task is represented by... and Weighting terms for multi-task uncertainty; , These are adaptive regularization terms for the flickering and texture tasks, respectively, used to penalize excessively large values. This value prevents the network from increasing indefinitely in an attempt to reduce losses. That is, to prevent neural networks from "escaping" the learning of a difficult task; The preset L2 regularization coefficient (weight decay factor) is used to control the intensity of weight penalty; This represents the learnable weight parameter matrix of the i-th layer in a neural network. The L2 norm (i.e., the sum of squares of all weights) of all convolutional kernel weights and fully connected layer weights is used to measure the complexity of a network model.

[0199] Using the joint loss function described above, the network will automatically increase the value when it detects that the flicker task has high noise and is difficult to converge. The value of increases the denominator of the first term in the formula, thereby reducing the weight of this task in the total gradient, effectively preventing it from causing destructive gradient impacts on the texture branch, and achieving task weight self-balancing; while the L2 regularization constraint mechanism, during backpropagation gradient calculation, This step imposes a penalty, forcing all weights to tend towards extremely small and smooth values. This allows for more stringent suppression of the model's over-reliance on specific single illumination channels (such as a random reflection angle), ensuring that the network extracts universal physical laws under the global lighting field, rather than local image noise. The system simultaneously updates the weight matrix in each iteration. and uncertainty parameters This approach achieves a high degree of alignment with the true values ​​of industry standard instruments while maintaining the smoothness and stability of the parameter space, thus forming a dynamic collaborative game.

[0200] Step 3.4: Training, acceptance, and packaging of the neural network;

[0201] The AdamW optimizer is used to perform backpropagation updates. The trained model is then blind-tested on a completely isolated test set of 1000 blocks. The core acceptance criterion is that the prediction bias must be stably controlled within the ±1% acceptable range to achieve accuracy acceptance. Finally, the clean network structure and fixed weight parameters that have passed the acceptance are exported as a computation graph and burned into the local embedded motherboard computing chip of the instrument to complete the engineering deployment and achieve offline real-time inference.

[0202] This invention breaks through the conventional mindset of unidirectional illumination and proposes a composite square-coded illumination strategy that integrates "unidirectional basic illumination" with "multi-source combined illumination" (such as directional illumination source 1 + directional illumination source 3 lighting simultaneously, directional illumination source 2 + directional illumination source 4 lighting simultaneously, etc., specific coding patterns). This strategy, through the cooperation of underlying optical hardware and the training of a post-processor neural network, corrects the surface effect of the sample under test, making the predicted flicker and texture consistent with the measurement results of industry standard instruments, thus achieving a leapfrog performance improvement.

[0203] 1. A leap at the physical level: Reconstructing light and shadow interaction and breaking through the signal-to-noise ratio bottleneck;

[0204] The exponential improvement in signal-to-noise ratio (SNR): The combined illumination of multiple light sources multiplies the total radiant energy projected onto the sample surface. For dark paints or dark micro-textured materials with extremely high light absorption, this mechanism can effectively suppress the shot noise of the image sensor 10 at the source, and significantly improve the quality of the underlying data input to the neural network.

[0205] High-fidelity capture of nonlinear optical interaction features: The combined lighting mode realistically reproduces the microscopic shadows and multiple scattering features of special effect pigments in metallic paint under the interweaving of multidimensional light. This enables the neural network to effectively capture the "light and shadow superposition compensation effect" that is completely lost in the single light source mode, and to reproduce the highly realistic depth distribution and spatial layers of the paint surface with high fidelity.

[0206] Highly realistic simulation of actual observed light field: The composite light field model constructed by combined coded illumination closely resembles the real physical visual scene of humans. This strategy enables the high-dimensional semantic results output by the neural network when fitting the texture (graininess) and sparkle (sparkle) indicators to perfectly match the subjective visual evaluation benchmark of humans.

[0207] 2. Leap in algorithm and network layers: enhancing feature extraction efficiency and global robustness;

[0208] Deeply enhanced model robustness against interference: At the input of a multimodal heterogeneous fusion network (such as a spatial weight attention module), the data is upgraded from single-dimensional "independent angle features" to "lighting scene features" rich in environmental context interaction. This allows the network to learn more continuous and robust spatial distribution patterns of texture gradients.

[0209] Significantly suppresses hardware mechanical assembly tolerance sensitivity: Through a high-dimensional, strongly coupled feature joint representation mechanism, the local sensitivity fluctuations caused by minute mechanical assembly tolerances in individual LED light source positions in mass-produced instruments are greatly reduced. From the underlying algorithm mechanism, the inter-instrument agreement of mass-produced hardware equipment is significantly improved.

[0210] Experimental results show that training a neural network using image data acquired with multi-angle composite light sources combined with diffuse lighting significantly outperforms traditional neural networks trained using image data acquired with sequentially lit single-angle light sources. For example... Figure 4 As shown, the model trained on composite lighting data exhibits a very high linear correlation in predicting surface effects (taking the comparison between the texture parameter G calculated by the model and the measured G value of the BYK.Mac multi-angle effect colorimeter as an example), with a correlation coefficient R. 2Up to 0.907. In contrast, models using traditional multi-angle lighting data, such as Figure 5 As shown, the R value of its prediction results 2 The difference is only 0.846. This significant difference strongly demonstrates that training neural networks with image data acquired using multi-angle composite light sources combined with diffuse lighting can capture richer and more realistic surface effect information, thereby greatly improving the modeling accuracy and generalization ability of neural networks.

[0211] In the flicker parameter (S) G In terms of prediction, training neural networks with image data acquired using multi-angle composite light sources combined with diffused light illumination also shows significant advantages. Figure 6 and Figure 7 The flash parameter S calculated by the model is shown. G Flash parameter S compared to BYK.mac's measured value G Correlation comparison. A neural network was trained using image data acquired with multi-angle composite light sources and diffuse illumination. The linear correlation coefficient R of its prediction results was compared. 2 It reached 0.851, such as Figure 6 As shown. In contrast, training a neural network with image data acquired by sequentially illuminating individual angle light sources yielded a correlation coefficient of only 0.740, as... Figure 7 As shown in the figure, this comparative data further confirms that composite lighting technology can significantly improve the accuracy of network models in inferring and predicting surface scintillation characteristics.

[0212] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for measuring the surface of an object based on spectral imaging, characterized in that: Collect color spectral data of object surfaces under multiple angles of individual illumination to extract spectral features; Collect object surface image data under multi-angle illumination, different from the spectral data acquisition angle, to extract the first image features; Collect object surface image data under single diffuse lighting and under diffuse lighting combined with the multi-angle lighting, at angles different from the spectral data acquisition angles, to extract second image features; A multimodal heterogeneous fusion neural network is constructed. Through the spatial weight attention mechanism, the cursor scalar of each image feature is extracted to perceive the overall energy state of each image feature, locate image features with abnormal energy, and then assign weights to the reflectivity of image features under different lighting conditions. The weights are then combined with the image features to obtain new image features. Spatial flicker features are extracted from the new image features to generate image spatial features. Semantic features coupling color and flicker are extracted from the spectral features. Texture features are extracted from the new image features. Based on the joint features obtained by spatial concatenation of image spatial features, semantic features, and texture features, the flicker and texture of the object surface are predicted respectively. The prediction results are compared with the measurement results of standard instruments to construct a joint loss training neural network so that the predicted flicker and texture of the object surface are consistent with the measurement results of standard instruments. The trained neural network is used to predict the flicker and texture of the surface of the object being tested.

2. The object surface measurement method based on spectral imaging according to claim 1, characterized in that: The multi-angle combined lighting refers to multi-angle individual lighting and / or multi-angle composite lighting. The multi-angle composite lighting is based on a sequence of light sources at different angles. A sliding window is constructed by two light sources at different angles, and the light is slidably lit in the light source sequence with one or more light sources as the step size.

3. The object surface measurement method based on spectral imaging according to claim 2, characterized in that: The multi-angle composite illumination includes two or more adjacent light sources, which slide and illuminate in the light source sequence with the interval between adjacent light sources as the step size. This allows the extracted first image features to capture the continuity of surface reflection of the object when deflected at a small angle, and the extracted second image features to be used to evaluate the interference and enhancement effect of the macroscopic ambient light background on the physical fusion effect of adjacent flashing points.

4. The object surface measurement method based on spectral imaging according to claim 2, characterized in that: The multi-angle composite illumination includes two non-adjacent light sources spanning one or more light sources. The light sources are illuminated in a sliding manner in the light source sequence with one light source as the step size. This allows the extracted first image features to obtain the nonlinear optical features of the object surface, and the extracted second image features to be used to simulate the depth visual response under the combination of real diffuse illumination and the multi-angle illumination.

5. The object surface measurement method based on spectral imaging according to claim 2, characterized in that: The multi-angle combined illumination includes multi-angle individual illumination and multi-angle total illumination. Multi-angle individual illumination enables the extracted first image features to obtain the image representation sequence under basic unidirectional illumination, while multi-angle total illumination enables the extracted first image features to increase the total luminous flux for dark surfaces. Based on multi-angle individual illumination under diffuse lighting, the extracted second image features are used to obtain the visual saliency of flashing points at each angle under background ambient light interference. Based on multi-angle total illumination under diffuse lighting, the extracted second image features are used to obtain the limit response boundary of the object surface under all ambient lighting, which is used to correct the system bias of acquisition from different angles.

6. The object surface measurement method based on spectral imaging according to claim 1, characterized in that: A homography matrix is ​​constructed to map the physical plane of the original image acquisition to a virtual orthogonal plane. Based on the homography matrix, the acquired image data is mapped to the pixel coordinates on the virtual orthogonal plane to obtain new image data.

7. The object surface measurement method based on spectral imaging according to claim 6, characterized in that: Using the number of new image data as the number of channels, a second-order statistical model is performed on multiple channels in the new image data using the gray-level co-occurrence matrix (GLCM). The multi-channel energy, contrast, correlation, and entropy of each element in the GLCM are integrated and vectorized and concatenated, transforming the pixel spatial distribution into a high-dimensional feature descriptor with clear physical meaning. The element represents the probability density of the adjacent occurrence of the first and second gray levels under a specific spatial direction and step size. Energy is used to measure the uniformity of the image gray-level distribution and the coarseness of the texture. A more uniform texture distribution results in a larger set of elements in the GLCM. In this context, the higher the energy value, the lower it becomes when there is a complex and coarse particle distribution. Contrast reflects the drastic changes in local gray levels and the clarity of texture. Dramatic jumps in gray levels between adjacent pixels on the surface will increase the contrast value. Correlation measures the linear similarity of elements in the spatial gray-level co-occurrence matrix in the row or column direction to quantify the extensibility of texture in a specific direction and the regularity of local structure. Entropy quantifies the complexity and randomness of texture information. When the surface contains a large number of irregularly arranged special effect pigments, the spatial distribution of pixels is highly random, and the entropy value reaches its peak at this time.

8. The object surface measurement method based on spectral imaging according to claim 1, characterized in that: The mean and standard deviation of the spectral data are obtained as physical components characterizing the intensity of energy reflected from the surface of an object. The difference between the spectral data and its mean is then divided by the standard deviation to obtain the semantic components of the spectral data, which characterize the intrinsic color-changing properties as the angle changes. The physical components and the semantic components are used as spectral feature tensors.

9. A surface measurement system for an object based on spectral imaging, comprising a spectral acquisition device, an image acquisition device, a diffuse illumination source, and a set of multi-angle illumination sources arranged relative to the object surface, characterized in that: The system further includes a neural network module, which, according to any one of claims 1 to 8, acquires spectral features and image features to train the neural network module, and then uses the trained neural network module to measure the object surface.

10. A surface measurement system for an object based on spectral imaging according to claim 9, comprising a multi-angle spectrophotometer, characterized in that: The multi-angle spectrophotometer includes a fan-shaped body with a measurement hole (12) at the bottom. The arc surface of the fan-shaped body is provided with a spectral acquisition device surrounding the measurement hole (12), an image acquisition device at an angle opposite to the spectral acquisition device, and a set of multi-angle illumination sources. A diffuse illumination source is also provided above the measurement hole (12).

Images