Continuous spectrum space reconstruction method based on depth generation model

Abstract

The invention discloses a continuous spectrum space reconstruction method based on a depth generation model. The method comprises the following steps: step 1, data organization and problem formalization; 2, performing robust normalization and scale alignment; step 3, anchoring end points and parameterizing continuous conditions; step 4, sample pairing and trajectory construction; step 5, carrying out structural modeling on the endpoint condition variational auto-encoder; 6, physical consistency constraint and joint loss optimization are carried out; 7, missing interval generation and structured completion are carried out; and step 8, unified verification of generation quality and task availability is carried out, and a generation model which explicitly responds to continuous physical parameters and can maintain the consistency of a spectrum peak form and spatial statistics can be constructed, so that structured completion and continuous reconstruction of a missing interval are realized.

Description

Technical Field

[0001] This invention relates to the technical field of spectral modeling, and in particular to the technical field of a continuous spectral space reconstruction method based on a deep generative model. Background Technology

[0002] Spatially Offset Raman Spectroscopy (SORS) can obtain deep Raman responses under conditions of surface obstruction or multilayer structures, and therefore has important value in quantitative analysis and composition inversion of complex systems. Quantitative modeling of spatially offset Raman spectroscopy in engineering practice relies heavily on a standard sample database with sufficient coverage, but the construction of such a database often faces many bottleneck problems.

[0003] Chinese patent application number CN201910446804.7 discloses a quantitative Raman spectroscopy analysis technique based on the half-peak height distance method. This technique uses the ratio of the two strongest characteristic peaks in the Raman spectrum of the sample as the basis for quantitative analysis, minimizing interference from other substances. It fully considers the errors caused by peak shape and intensity, as well as the changes in spectral peaks with varying concentrations, maximizing the elimination of errors caused by human factors and the objective environment during the detection process. However, this method still has limitations. It is only suitable for samples with a single component or where the concentration of the target substance is much higher than that of other components, and is not suitable for Raman spectroscopy analysis of multi-component samples.

[0004] Chinese patent application CN202111546557.1 discloses a spectral data augmentation method based on conditional variational autoencoders, which generates virtual spectra with the same distribution as component concentrations to enhance the training set and facilitate the development of calibration models. To use the generated virtual spectra for modeling, a regression learning model based on a semi-supervised ladder network is used. This method combines the generated virtual unlabeled spectra with real labeled samples, which can ensure that the distribution of the generated virtual spectra and the actual labeled spectra are the same, thus ensuring the effectiveness of semi-supervised learning. However, the method still has shortcomings, namely that semi-supervised learning is difficult and the data augmentation effect is limited.

[0005] In existing technologies, the construction of standard sample databases often faces the following common bottlenecks: The contradiction between sampling discreteness and physical continuity is prominent: Concentration gradient or thickness parameters are essentially continuous physical quantities, but due to constraints such as sample preparation cost, collection time and stability, actual data can only be sampled discretely at a limited number of concentration points. In particular, continuous intervals are often missing: There are no intermediate state samples in the parameter interval, which causes the model training to only cover the endpoints or some sparse nodes, making the prediction of the missing interval closer to extrapolation than interpolation. Traditional interpolation / mixing methods struggle to preserve the physical morphology of spectral peaks: linear interpolation, weighted mixing, and other methods perform pixel-level averaging in the observation space, making it difficult to characterize the nonlinear evolution of Raman spectra with concentration changes. This often leads to blurred peak shapes, submerged weak peaks, and even non-physical phenomena such as "phantoms / double peaks," resulting in a lack of scientific validity in the generated results. Conventional generative models lack generalization ability for missing intervals: Typical conditional generative adversarial networks and standard variational autoencoders often follow the paradigm of "dense supervision and coverage learning." When there are insufficient or completely invisible samples in the missing interval, the model is prone to degenerate into endpoint replication, detail collapse, or condition insensitivity, making it difficult to form a stable "endpoint-intermediate state" continuous evolution trajectory. This problem is even more prominent for spatial displacement Raman spectroscopy, because the data is not only highly structured in the band dimension, but also subject to statistical morphological constraints in the spatial displacement dimension. Due to the high cost of sample preparation and collection, and the fact that concentration gradients or thickness parameters can often only be sampled discretely, the actual data exhibits significant sparsity on the physical parameter axis, especially with the common occurrence of missing continuous intervals: the lack of intermediate state samples within the parameter interval leads to problems such as incomplete database, learning bias, and extrapolation degradation in the modeling task. Summary of the Invention

[0006] The purpose of this invention is to solve the problems in the prior art and propose a continuous spectral spatial reconstruction method based on a deep generative model. This method can construct a generative model that explicitly responds to continuous physical parameters and maintains the consistency of spectral peak morphology and spatial statistics, thereby achieving structured completion and continuous reconstruction of missing intervals.

[0007] To achieve the above objectives, this invention proposes a continuous spectral space reconstruction method based on a deep generation model, comprising the following steps: Step 1: Data organization and problem formalization; Step 2: Robust normalization and scale alignment; Step 3: Endpoint anchoring and continuity condition parameterization; Step 4: Sample pairing and trajectory construction; Step 5: Endpoint-conditional variational autoencoder structure modeling; Step 6: Physical consistency constraints and joint loss optimization; Step 7: Generation and structured completion of missing intervals; Step 8: Generate a unified verification of quality and task availability.

[0008] Preferably, step 1 includes the following steps: 1-1: Each spatial displacement Raman spectrum sample is represented as a two-dimensional spectral image x∈RH×W; where For Raman shift / band dimension, Let the spatial displacement dimension be denoted by a corresponding physical parameter label for each sample. And record them according to discrete levels or continuous values; 1-2: Specify two endpoints on the parameter axis and Define the missing interval as The goal is to generate intermediate state samples that satisfy physical laws within this interval. .

[0009] Preferably, step 2 includes the following steps: 2-1: Standardize the dimensions of the original matrix; 2-2: Robust normalization using quantile truncation: Calculate the upper and lower bounds of the truncation on the training visible sample set. The data is truncated and linearly mapped to... ; 2-3: In the missing interval evaluation scenario, calculations are based only on the visible training samples, so that the missing interval samples will not indirectly participate in model fitting in a global statistical form.

[0010] Preferably, step 3 includes the following steps: 3-1: For any target parameter Construct normalized continuous condition variables ; This enables the model to perceive positional relationships "from the endpoints to the intermediate states" in a continuous manner; 3-2: During training and inference, endpoint samples As a global context anchor Continuous conditional coordinates used to control the generation of intermediate states.

[0011] Preferably, step 4 establishes an "endpoint-aligned" trajectory structure for samples under different parameters, including the following steps: 4-1: Using the left endpoint sample index as a reference, determine a fixed index for each trajectory. ; 4-2: Sample the right endpoint Compared with other parameters, the sample and this Perform one-to-one matching, employing sequential alignment or spectral feature-based matching strategies to enhance the consistency of endpoint context; 4-3: In each training iteration, randomly sample from this trajectory and... Corresponding target parameters Form training tuples .

[0012] Preferably, step 5 employs an endpoint-conditional variational autoencoder for generative learning, and its key modules include: Endpoint context encoder: As input, multi-scale contextual features are extracted to determine the global boundary conditions for intermediate state generation; Posterior encoder: Combined inputs to estimate the posterior distribution parameters of latent variables. It is used for variational inference and uncertainty modeling during the training phase; Decoder: based on endpoint context features and latent variables With conditions Input: Intermediate state prediction ; FRaman modulation: conditional Affine modulation parameters mapped to multi-layer feature channels ,by Modulate the multi-scale features at the decoding end.

[0013] Preferably, step 6, which involves constructing a joint loss for training and optimization, includes the following steps: 6-1: Robust Reconstruction Term: Huber / SmoothL1 constraint is used to constrain the overall spectral shape and spatial structure; 6-2: Peak-sensitive weighted reconstruction: Based on the rate of change of the mean spectrum of the training visible samples, a band weight map is constructed to give the peak region a higher weight in the optimization, which is used to improve the fidelity of the feature peak; 6-3: Spectral gradient consistency: constrains the first-order difference consistency of the band direction to maintain the local spectral shape change rate and peak edge structure; 6-4: Statistical morphological constraints: Constraint statistical consistency terms to maintain the stability of the intensity morphology of the spatial displacement dimension; 6-5: Overshoot penalty: Suppresses the occurrence of systematically high non-physical enhancements in predictions during the peak region; 6-6: KL Regularization and Annealing: Apply KL regularization to the distribution of latent variables and use the Warmup strategy to gradually increase the KL weights, thereby improving the structuring of the latent space and training stability.

[0014] Preferably, step 7 includes the following steps: 7-1: Given the target missing parameters Calculate the corresponding ; 7-2: Selecting the Endpoint Context latent variables Set it as a zero vector to obtain deterministic output, or sample from a standard normal distribution to obtain diverse output; 7-3: Generated via decoder The normalization result is then inversely transformed back to the original scale to obtain the completed samples for the missing intervals. 7-4: The generated samples are saved in a structured manner according to the parameter order and quantity for subsequent database amplification and downstream quantitative modeling.

[0015] Preferably, step 8 verifies the following aspects at the output level: Spectral consistency includes mean spectrum consistency, peak region error, gradient consistency, and spatial statistical consistency. Task-driven consistency: Using "missing training - missing + generation completion - full-real training" as a unified control framework, we examine whether generation completion can approach the performance upper bound of full-real data in quantitative regression tasks, and verify the effectiveness and usability of generated data in downstream modeling.

[0016] Beneficial effects of the present invention: The present invention proposes FRaman (Feature-wise) This invention employs a Raman-adaptive modulation (RAM) mechanism to enable the network to produce explicit and controllable dynamic responses to continuous physical parameters. It characterizes intermediate state generation as a reconstruction problem induced by endpoint constraints and physical consistency regularization, achieving a methodological leap from "discrete sampling" to "continuous physical field reconstruction," providing a new path for building usable databases for spatially shifted Raman spectroscopy when intermediate standard samples are difficult to obtain. The invention proposes an endpoint-anchored generation strategy, transforming the discrete spectrum generation task into a continuous manifold traversal problem: using low-concentration and high-concentration endpoints as topological anchors and continuous concentration variables as traversal coordinates, an endpoint-conditional variational autoencoder is constructed. This structured representation of the latent space carries the nonlinear changes in the spectral shape, especially under conditions of missing continuous intervals and extremely sparse endpoints. This invention characterizes intermediate state generation as a reconstruction problem induced by endpoint constraints and physical consistency regularization, achieving a methodological leap from "discrete sampling" to "continuous physical field reconstruction," providing a new path for building usable databases for spatially shifted Raman spectroscopy when intermediate standard samples are difficult to obtain. The invention also designs a FRaman (Feature-wise) system. The Raman-adaptive Modulation (FRaman) algorithm simulates the multiplicative enhancement of Raman scattering intensity and the additive effect of baseline drift caused by concentration changes on Raman scattering intensity through channel-level affine modulation within the network. Unlike traditional splicing and fusion methods, FRaman learns nonlinear mappings, enabling the network to form a controllable response to physical conditions in a multi-scale representation space. This improves the model's ability to express local peak morphology, detailed changes, and dynamic trends, as well as its interpretability. This invention constructs a set of physical consistency constraints and innovatively introduces peak-sensitive weighting. The system employs a weighting and spectral gradient constraint mechanism. By dynamically calculating the second-order difference map of the real spectrum, it constructs an attention weight matrix for Raman characteristic peak regions. This forces the model to prioritize physically meaningful spectral fingerprint regions over background noise regions during optimization. This innovation effectively suppresses the smoothing effect of the generated spectrum and ensures a high degree of consistency between the synthesized data and real experimental data in key physical indicators such as peak intensity ratio and full width at half maximum (FWHM). The endpoint condition generation framework proposed in this invention uses sparse endpoints as anchors and generates intermediate spectra in missing intervals through continuous condition modulation. This achieves low-cost expansion from a small number of experimental points to a large number of usable samples, improving the coverage and usability of the spectral database.This invention learns the spectral evolution manifold in the latent space through endpoint-anchored conditional generation and FRaman dynamic modulation. It enhances the ability to reconstruct peak region details by leveraging peak sensitivity and gradient consistency constraints, thereby improving the fitting ability to nonlinear spectral evolution laws. This invention provides boundary conditions through endpoint anchoring, endows the network with an explicit response mechanism to continuous physical parameters through FRaman, and introduces physical consistency regularizations such as peak sensitivity weighting, spectral gradient consistency, statistical morphological constraints, and overshoot penalties to enhance the identifiability and output stability under under-constraint conditions. This improves the model's continuous generation ability and physical rationality within missing regions, providing more reliable supplementary data support for subsequent quantitative analysis.

[0017] The features and advantages of the present invention will be described in detail through embodiments and in conjunction with the accompanying drawings. Attached Figure Description

[0018] Figure 1 This is a flowchart of a continuous spectral space reconstruction method based on a deep generation model according to the present invention. Figure 2 This is an experimental comparison diagram of a continuous spectral spatial reconstruction method based on a deep generation model according to the present invention; Figure 3 This is a method architecture diagram of a continuous spectral space reconstruction method based on a deep generative model according to the present invention. Detailed Implementation

[0019] See Figure 1 , Figure 2 and Figure 3 This invention discloses a continuous spectral space reconstruction method based on a deep generative model, comprising the following steps: Step 1: Data organization and problem formalization; Step 2: Robust normalization and scale alignment; Step 3: Endpoint anchoring and continuity condition parameterization; Step 4: Sample pairing and trajectory construction; Step 5: Endpoint-conditional variational autoencoder structure modeling; Step 6: Physical consistency constraints and joint loss optimization; Step 7: Generation and structured completion of missing intervals; Step 8: Generate a unified verification of quality and task availability.

[0020] This invention constructs an Endpoint-Conditioned Conditional Variational Autoencoder (Endpoint-Conditioned cVAE) framework to systematically characterize the nonlinear evolution of spatially shifted Raman spectra with continuous concentration / thickness variations. This framework uses endpoint samples as global context anchors and continuous variables obtained by mapping the target concentration as generation coordinates, formalizing intermediate state generation as a conditional generation problem from a probabilistic modeling perspective. ; in As a latent random variable, it is used to characterize the random perturbations, unobservable factors and complex noise structures of intermediate states under endpoint constraints; The parameters of the generator network are represented, and the corresponding variational inference is achieved through posterior approximation: ; This constitutes a structured generative learning framework with endpoint context as a condition and intermediate state as the target. Unlike linear interpolation which performs a simple weighted average in the observation space, the above modeling interprets the generative process as a continuous traversal of the spectral manifold under the joint drive of the latent space and conditional coordinates, thereby establishing a mapping relationship from discrete experimental observation points to the continuous spectral evolution trajectory. In terms of data representation, this invention represents each spatial displacement Raman spectrum sample as a two-dimensional spectral image. :in Corresponding band dimension Corresponding to the spatial displacement dimension, thus simultaneously constraining "spectral evolution" and "spatial distribution statistics" within the same framework. To reduce the impact of extreme noise and outliers on training stability, this invention introduces a robust normalization strategy based on quantile truncation p-clip, mapping the data to a unified scale before generating learning. This invention proposes an FRaman feature modulation mechanism, which uses "conditional-channel response" as the basic modeling unit. It explicitly maps continuous physical parameters to affine modulation coefficients of multi-scale features within the network, allowing conditional information to permeate the generation process in an interpretable form, thereby achieving a dynamic response to spectral evolution. The specific mechanism is as follows: (1) Continuous conditional parameterization and embedding representation First, the target physical parameters Mapping to normalized continuous variables : ; The scalar is then processed through a conditional embedding network (typically a two- or multi-layer MLP). Mapped to conditional embedding vectors: ; in As an embedding dimension, this embedding vector is used to carry the "nonlinear modulation law corresponding to the concentration / thickness change"; (2) Generation of channel-level affine coefficients For feature maps of any scale in the generative network FRaman does not directly Instead of concatenating with feature maps, it is based on embedding vectors. Generate the channel-level scaling and translation factors for this layer: ; in and By characterizing the sensitivity direction and response amplitude of each characteristic channel at this scale to physical conditions, it is possible to express the "difference in response of different channels to different spectral structures"; (3) FRaman characteristic modulation Perform channel-level affine modulation at the feature map level:

[0021] in This represents channel-by-channel broadcast multiplication, a form with clear physical interpretability; Corresponding to the multiplicative intensity modulation (scattering intensity enhancement / attenuation) caused by changes in concentration / thickness, the response amplitude can be changed while maintaining the spectral structure; Corresponding to additive modulation such as baseline drift, background trend changes can be compensated at the feature level; since the modulation acts on intermediate features rather than just the input, the network can learn the conditional response decomposition of "global trend - local details" at different semantic levels; (4) Multi-scale hierarchical injection To ensure that the conditional response covers both the global spectral shape and local peak details, FRaman is embedded into multi-scale modules of the generative network, such as multiple residual / upsampled blocks in the decoder. At low-resolution scales, the modulation is more focused on learning the overall intensity and background trends; at high-resolution scales, the modulation is more focused on learning the local morphology and detailed changes in the peaks. Through this hierarchical injection, the model can form a layer-by-layer response path to the conditional variables from coarse to fine, thereby significantly reducing the problem of conditional information attenuation caused by simply splicing the inputs. (5) Synergistic effect with endpoint condition generation framework In an endpoint conditional variational encoder, endpoint samples FRaman provides global boundary constraints, while FRaman provides constraints on continuous variables. The explicit dynamic response mechanism, combined with the other two, ensures that the model neither deviates from the endpoint topology when generating missing intervals, nor does it follow the trend. Continuously adjust the spectral details and peak intensity distribution to better reflect the real physical evolution logic and improve the usability and interpretability of the generated results in downstream quantitative modeling tasks; In summary, the FRaman modulation mechanism, with channel-level affine modulation as its core, transforms continuous physical conditions into a controllable hierarchical response within the network. This effectively alleviates the shortcomings of traditional condition splicing in terms of peak detail representation and condition sensitivity, and provides key structural support for high-fidelity continuous reconstruction of spatially shifted Raman spectra in missing regions. This invention proposes an endpoint-anchored generation strategy, transforming the discrete spectrum generation task into a continuous manifold traversal problem: using low-concentration and high-concentration endpoints as topological anchors and continuous concentration variables as traversal coordinates, an endpoint-conditional variational autoencoder is constructed. This allows the structured representation of the latent space to carry nonlinear changes in the spectral shape, thereby achieving continuous reconstruction of intermediate states in missing intervals. The specific construction steps are as follows: Step ①: Endpoint Determination and Continuous Coordinate Parameterization Select two endpoints on the physical parameter axis and arbitrary target parameters Mapped to normalized traversal coordinates ; This mapping unifies "discrete concentration levels" into continuous variables, making the generation process viewable as following a continuous path. Continuous traversal in direction; Step 2: Endpoint Context Anchoring For each generation / reconstruction task, endpoint samples are used. As a global context anchor, The parameter is Endpoint spectral images, The parameter is The endpoint spectral image, endpoint anchoring provides boundary conditions for the evolution of spectral shape within the missing interval, enabling the model to have topological constraints "starting from the endpoint"; Step 3: Trajectory Sample Construction and Pairing Alignment In each "endpoint trajectory", different parameter samples anchored to the same endpoint are aligned into training tuples: ; in For target parameters The real samples are generated only during inference if they are missing. To reduce trajectory noise introduced by random pairing, sequential alignment or matching strategies based on spectral feature similarity can be used to ensure... To be as consistent as possible with intermediate samples in terms of "sample identity"; Step 4: Endpoint Condition Generation Modeling The generation of intermediate states is modeled as a conditional probability distribution: ; in As latent variables, characterizing the sources of unobservable disturbances, structural noise, and diversity, we assume prior... ; Step 5: Variational Inference and Training Objective Using posterior approximation Perform variational learning to maximize the lower bound of evidence: ; The first term corresponds to reconstruction consistency, and the second term corresponds to latent space regularization. A warmup strategy can be used to gradually increase the amount of warmup data to improve stability. This invention proposes the FRaman dynamic modulation algorithm, which enables the network to form an explicit and controllable response mechanism to continuous physical conditions in a multi-scale representation space, thereby improving the ability to express local morphology, detailed changes and dynamic trends of spectral peaks and enhancing the interpretability of the generated data. The specific steps are as follows: Step a: Conditional embedding continuity condition Inputting the conditional coding network (MLP) yields the conditional embedding vector: ; Step b: Generation of channel-level affine parameters Feature map of a certain layer of the network Channel-level scaling and translation coefficients are generated from the embedding vector: ; Step c: FRaman modulation Channel-level affine modulation of the feature map: ; in This represents channel-by-channel broadcast multiplication, which can be physically interpreted as: Characterizing the multiplicative enhancement / attenuation trend of scattering intensity with concentration changes, Characterize additive effects such as baseline drift; Step d: Multi-scale injection Embedding FRaman into multiple resolution levels of the decoder (such as UNet multi-scale blocks / residual blocks) enables the network to respond to changes in conditions at both the global structural level and the local detail level, avoiding insufficient utilization of conditions caused by "only concatenating conditions at the input end". This invention overcomes the optimization bottleneck of single-pixel-level errors by constructing a physical consistency constraint system. This system, based on the core principles of "peak region priority + local rate of change consistency + spatial statistical morphology consistency," jointly constrains the consistency of the generated spectrum in key spectral fingerprint regions and spatial displacement statistics, thereby improving the physical reliability and downstream usability of the generated results. The specific steps for constructing this system are as follows: Step I: Robust Normalization and Training Visible Statistics Robust normalization is performed on the visible training samples, and in the missing scenario, only the visible training set is used to calculate the normalized statistics to avoid leakage of missing intervals in the form of statistics. Step II: Construction of Peak Sensitive Weight Map Calculate the mean spectrum of the training samples within the normalization domain: ; And construct a peak sensitivity index, a common approach is to base it on the first-order rate of change: ; This leads to the one-dimensional weights: ; Then, by copying along the spatial dimension, a two-dimensional weighted graph is obtained: ; If it is necessary to emphasize the "peak curvature", a second-order difference can be used further: ; And Incorporate weights to enhance peak sensitivity; Step 3: Design of Joint Loss Term Based on the reconstruction terms, peak region weighting and shape constraints are introduced to construct a joint optimization objective: ; Each item can be specifically written as: Robust reconstruction loss ; Peak-sensitive weighted reconstruction ; Spectral gradient consistency ; in ; Spatial statistical morphological constraints ; Overshoot penalty ; KL regular expression ; To verify the effectiveness and transferability of the proposed endpoint condition generation method under different data scales and analyte properties, this invention selects two representative Raman spectroscopy datasets for experimental evaluation: one is the white sugar dataset, which contains... One sample, the physical parameter being the percentage of white sugar content, labeled according to parameter level. The first dataset contains 100 samples for each parameter level; the second dataset contains... One sample, with the physical parameter being butter concentration, also labeled according to parameter level. Each parameter level corresponds to 20 samples. The two datasets represent typical cases of "larger sample size / dense parameter sampling" and "smaller sample size / higher underconstraint risk," respectively, to comprehensively evaluate the stability and robustness of the method. Using machine learning models with PLSR, RR, SVR, etc., the effectiveness of the relationship between the spectral response and the analytical properties of the generated samples is evaluated by regression prediction using R2 and RMSE.

[0022] The effectiveness of a generative model should not only be reflected in visual or statistical similarity at the spectral level, but also in its ability to support downstream quantitative modeling tasks. In other words, the generated samples should statistically maintain a "spectral response." "and properties of the analyte" To address the mapping relationship between these parameters, this invention employs downstream regression prediction performance as a task-oriented evaluation criterion. Partial Least Squares Regression (PLSR), Ridge Regression (RR), and Support Vector Regression (SVR) models are selected as validators, and prediction evaluations are performed on the same test set. The coefficient of determination is used as the evaluation metric. With the root mean square error RMSE, where The model's explanatory power for the target variable was characterized, and RMSE was used to measure the magnitude of the prediction error. The two indicators were used together to determine the contribution of the generated complete samples to the recoverability of the "spectral-property relationship". All experiments were performed in a server environment with NVIDIA GeForce GTX 4090 and Python 3.9 as the hardware and software environment.

[0023] This invention uses a butter dataset as the primary validation object. First, it constructs a true reference dataset covering the complete concentration range. This data is obtained from a mixture of gradients A1V9–A9V1 and labeled as 1–9 according to concentration levels, thus covering the complete parameter axis from the endpoints to intermediate states, and can serve as a true distribution. The reference benchmark is used to evaluate the consistency of generated samples in terms of spectral details, peak structure, and spatial statistical characteristics.

[0024] To simulate the typical scenario of "missing intermediate samples" in practical applications, this invention employs a continuous interval missing strategy to construct missing training conditions, assuming the parameter set is... Select continuous missing intervals ,For example Then the training visible parameter set is Under this setting, the training phase only uses Corresponding real sample, missing interval The corresponding samples are not used in training, thus forming an under-constrained completion task.

[0025] To quantitatively characterize the performance degradation caused by missing values ​​and the performance recovery brought about by generation and completion, this invention constructs three sets of control conditions and evaluates them uniformly on the same real test set: No missing upper bound, maximum coverage: using the complete real training set. Train the regression model and evaluate it on the real test set; the result is denoted as... It is used to characterize the upper bound of performance achievable under "omniscient data conditions"; Missing data—Lower Bound: masked during training. Corresponding to real samples, only using The regression model is trained and evaluated on the same real test set; the result is denoted as... , used to characterize the performance lower bound caused by missing values; Missing + Generation-Completion Scheme: Based on the "missing" condition, the endpoint condition cVAE proposed in this invention is used to complete the process. Perform generative completion to obtain the generated sample set. To ensure fairness in the comparison, the number of generated samples at each missing parameter level is consistent with the number of samples at the corresponding level in the test set. and The real training samples are merged to form an enhanced training set, the regression model is then retrained, and evaluated on the same real test set. This result is denoted as... ; Experimental results: At the spectral level, the samples generated by the conditional variational autoencoder are highly consistent with the real samples in terms of overall spectral trend, main peak structure and spatial intensity distribution statistics; at the same time, under fixed endpoint conditions and the same target concentration, sampling the latent variables can produce sample instances with reasonable differences, indicating that the model has not experienced obvious mode collapse and has usable stochastic expression capabilities. At the task-driven evaluation level, the results of PLSR, RR, and SVR show a consistent pattern: compared to "missing" results, "missing + generated" results perform better on the real test set. Significant improvement and a substantial reduction in RMSE; more importantly, its performance level is highly close to that of the "no missing" result. This phenomenon indicates that the generated completed samples not only match the true samples in terms of spectral distribution but also can statistically recover the key information corresponding to the missing intervals. This makes the downstream regression model's predictive ability for the properties of the analyte approaching that of the fully real training conditions, thus proving the effectiveness and usability of the method of this invention in missing interval completion and database expansion. See [reference needed]. Figure 2 And Table 1: Table 1: Comparison of Regression Prediction Performance between Generated Spectral Samples and Real Spectral Samples

[0026]

[0027] To characterize the predictive ability of the model under different data availability conditions, Table 1 sets three control scenarios: "Missing" represents the result of training the regression model using only visible real samples and predicting the target test set under the condition that the real samples in the middle interval are missing, which serves as the lower limit of performance; "Missing + Generated" represents the result of adding the missing interval samples generated by the conditional variational autoencoder into the training set and then predicting the model based on the "Missing" condition, which is used to measure the gain of the generated data on downstream tasks; "No Missing" represents the prediction result of training with complete real data without missing data and evaluating it on the same test set, which serves as the upper limit of performance / reference upper bound.

[0028] Conclusion: The endpoint-conditional variational autoencoder proposed in this invention demonstrates strong effectiveness and usability in spectral sample generation tasks. On the one hand, the model can generate spectral images that are highly consistent with real samples in terms of spectral statistics and key structural features within missing intervals, indicating that it has good distribution fitting ability and generation fidelity. On the other hand, and more importantly, in the evaluation of downstream regression tasks represented by PLSR and RR, after introducing missing samples generated by the variational autoencoder, the model's prediction performance on the real test set is significantly improved from the "lower limit of missing training" to close to the "upper limit of full real data". This result shows that the variational autoencoder not only reproduces the apparent similarity of the spectral response, but also maintains the mapping relationship between the spectral response and the properties of the analyte in a statistical sense. It can effectively learn and reconstruct the nonlinear evolution law and detailed change mechanism contained in this relationship. Therefore, the samples generated by the variational autoencoder can serve as effective supplementary data for missing intervals, providing reliable technical support for constructing a more complete spatial displacement Raman spectral database and improving the robustness of quantitative modeling.

[0029] This invention uses a reproducible engineering mechanism to form a closed loop between endpoint constraints, continuous conditional responses, and physical consistency regularization in the training and inference stages, thereby significantly improving the stability, controllability, and downstream usability of missing interval generation. Specifically: (1) The endpoint conditional variational autoencoder explicitly models the generation process as And infer from the posterior perspective The KL annealing strategy suppresses posterior collapse, gradually structuring the latent space and ensuring deterministic reconstruction during inference in missing intervals. Reasonable diversity can also be obtained through sampling: (2) FRaman injects continuous conditions into the multi-scale decoding path using channel-level affine modulation. This enables the model to respond explicitly and hierarchically to changes in concentration / thickness, avoiding the conditions being diluted in the deeper layers when they are only spliced ​​at the input end, thereby improving the sensitivity of peak details to changes in conditions and the interpretability of generation; (3) Peak-sensitive weight map, spectral gradient consistency, spatial statistical morphology constraints and overshoot penalty together constitute a physical consistency constraint system, shifting the optimization focus from "overall average error" to "key regions and local change rates of spectral peaks", effectively suppressing excessive smoothing and non-physical enhancement at characteristic peaks, and enhancing the consistency of generated spectra in peak intensity ratio, peak edge and spatial displacement statistics; (4) Training visible statistics further avoid the leakage of implicit information generated by normalization and weight construction of missing intervals, ensuring the fairness and credibility of missing completion evaluation. Combining the above technologies, the code implementation can generate complete samples that are highly consistent with the real distribution under conditions of missing continuous intervals or even extremely sparse endpoints, and maintain the "spectral-property mapping relationship" in regression tasks, providing a feasible technical support for building a more complete spatial displacement Raman spectral database and improving the robustness of quantitative modeling.

[0030] The above embodiments are illustrative of the present invention and are not intended to limit the present invention. Any simple modifications to the present invention are within the scope of protection of the present invention.

Claims

1. A continuous spectral space reconstruction method based on a deep generative model, characterized in that: Includes the following steps: Step 1: Data organization and problem formalization; Step 2: Robust normalization and scale alignment; Step 3: Endpoint anchoring and continuity condition parameterization; Step 4: Sample pairing and trajectory construction; Step 5: Endpoint-conditional variational autoencoder structure modeling; Step 6: Physical consistency constraints and joint loss optimization; Step 7: Generation and structured completion of missing intervals; Step 8: Generate a unified verification of quality and task availability.

2. The continuous spectral spatial reconstruction method based on a deep generative model as described in claim 1, characterized in that: Step 1 includes the following steps: 1-1: Each spatial displacement Raman spectrum sample is represented as a two-dimensional spectral image x∈RH×W; where For Raman shift / band dimension, Let the spatial displacement dimension be denoted by a corresponding physical parameter label for each sample. And record them according to discrete levels or continuous values; 1-2: Specify two endpoints on the parameter axis and Define the missing interval as The goal is to generate intermediate state samples that satisfy physical laws within this interval. .

3. The continuous spectral space reconstruction method based on a deep generative model as described in claim 1, characterized in that: Step 2 includes the following steps: 2-1: Standardize the dimensions of the original matrix; 2-2: Robust normalization using quantile truncation: Calculate the upper and lower bounds of the truncation on the training visible sample set. The data is truncated and linearly mapped to... ; 2-3: In the missing interval evaluation scenario, calculations are based only on the visible training samples, so that the missing interval samples will not indirectly participate in model fitting in a global statistical form.

4. The continuous spectral space reconstruction method based on a deep generative model as described in claim 1, characterized in that: Step 3 includes the following steps: 3-1: For any target parameter Construct normalized continuous condition variables ； This enables the model to perceive positional relationships "from the endpoints to the intermediate states" in a continuous manner; 3-2: During training and inference, endpoint samples As a global context anchor Continuous conditional coordinates used to control the generation of intermediate states.

5. The continuous spectral spatial reconstruction method based on a deep generative model as described in claim 1, characterized in that: Step 4 establishes an "endpoint-aligned" trajectory structure for samples under different parameters, including the following steps: 4-1: Using the left endpoint sample index as a reference, determine a fixed index for each trajectory. ; 4-2: Sample the right endpoint Compared with other parameters, the sample and this Perform one-to-one matching, employing sequential alignment or spectral feature-based matching strategies to enhance the consistency of endpoint context; 4-3: In each training iteration, randomly sample from this trajectory and... Corresponding target parameters Form training tuples .

6. The continuous spectral space reconstruction method based on a deep generative model as described in claim 1, characterized in that: Step 5 employs an endpoint-conditional variational autoencoder for generative learning, and its key modules include: Endpoint context encoder: As input, multi-scale contextual features are extracted to determine the global boundary conditions for intermediate state generation; Posterior encoder: Combined inputs to estimate the posterior distribution parameters of latent variables. It is used for variational inference and uncertainty modeling during the training phase; Decoder: based on endpoint context features and latent variables With conditions Input: Intermediate state prediction ; FRaman modulation: conditional Affine modulation parameters mapped to multi-layer feature channels ,by Modulate the multi-scale features at the decoding end.

7. The continuous spectral space reconstruction method based on a deep generative model as described in claim 1, characterized in that: Step 6, constructing the joint loss for training and optimization, includes the following steps: 6-1: Robust Reconstruction Term: Huber / SmoothL1 constraint is used to constrain the overall spectral shape and spatial structure; 6-2: Peak-sensitive weighted reconstruction: Based on the rate of change of the mean spectrum of the training visible samples, a band weight map is constructed to give the peak region a higher weight in the optimization, which is used to improve the fidelity of the feature peak; 6-3: Spectral gradient consistency: constrains the first-order difference consistency of the band direction to maintain the local spectral shape change rate and peak edge structure; 6-4: Statistical morphological constraints: Constraint statistical consistency terms to maintain the stability of the intensity morphology of the spatial displacement dimension; 6-5: Overshoot penalty: Suppresses the occurrence of systematically high non-physical enhancements in predictions during the peak region; 6-6: KL Regularization and Annealing: Apply KL regularization to the distribution of latent variables and use the Warmup strategy to gradually increase the KL weights, thereby improving the structuring of the latent space and training stability.

8. The continuous spectral spatial reconstruction method based on a deep generative model as described in claim 1, characterized in that: Step 7 includes the following steps: 7-1: Given the target missing parameters Calculate the corresponding ; 7-2: Selecting the Endpoint Context latent variables Set it as a zero vector to obtain deterministic output, or sample from a standard normal distribution to obtain diverse output; 7-3: Generated via decoder The normalization result is then inversely transformed back to the original scale to obtain the completed samples for the missing intervals. 7-4: The generated samples are saved in a structured manner according to the parameter order and quantity for subsequent database amplification and downstream quantitative modeling.

9. The continuous spectral spatial reconstruction method based on a deep generative model as described in claim 1, characterized in that: Step 8 verifies the following aspects at the output level: Spectral consistency includes mean spectrum consistency, peak region error, gradient consistency, and spatial statistical consistency. Task-driven consistency: Using "missing training - missing + generation completion - full-real training" as a unified control framework, we examine whether generation completion can approximate the performance upper bound of full-real data in quantitative regression tasks, and verify the effectiveness and usability of generated data in downstream modeling.

Images