A handheld spectral sensor signal fidelity method and related apparatus
By constructing a sensor offset feature training set and a machine learning model, the offset signal of the handheld spectral sensor is identified and removed, solving the signal offset problem caused by hardware limitations and environmental interference, and achieving high-precision spectral signal correction and improved detection accuracy.
Patent Information
- Application Number
- CN202610523085.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-20
- Publication Date
- 2026-07-14
AI Technical Summary
Handheld spectral sensors suffer from severe signal shifts in complex environments, resulting in inaccurate spectral signals. Traditional methods struggle to effectively correct these shifts caused by hardware limitations and environmental interference, thus affecting detection accuracy.
By constructing a training set of sensor offset features, a machine learning model is used to identify and remove offset signals. A one-dimensional U-Net model combined with channel and spatial attention mechanisms is adopted to learn and remove the offset of handheld spectral sensors in different environments.
It improves the signal fidelity accuracy of handheld spectral sensors, enables automatic real-time correction of spectral signals in complex environments, enhances detection accuracy and reliability, and reduces hardware costs and operational complexity.
Smart Images
Figure CN122385497A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of spectral detection technology, and in particular to a method and related apparatus for signal fidelity preservation of a handheld spectral sensor. Background Technology
[0002] Handheld spectroscopic sensors, with their small size, low cost, and ease of integration, have propelled spectral detection from the laboratory to field applications. However, in practical applications, signal fidelity of handheld spectroscopic sensors faces severe challenges from multiple aspects. These include inherent biases caused by hardware limitations: to achieve miniaturization, handheld spectroscopic sensors typically simplify the spectroscopic and detector components and lack sophisticated temperature control and shock protection systems, leading to baseline drift, wavelength shift, and stray light interference under different environments, resulting in the acquired spectral signal not representing the true response of the sample; dynamic biases caused by environmental sensitivity: in complex field environments, external temperature fluctuations, light source aging, and electromagnetic interference further exacerbate spectral signal shifts. These shifts, superimposed on the sample's characteristic signals, severely mask the sample's effective physicochemical information; and the fragility of calibration models: currently, most mainstream chemometric models are based on laboratory standard instruments. Since handheld spectroscopic sensors acquire contaminated signals with "true biases," directly applying these models often leads to a significant decrease in prediction accuracy, or even model failure.
[0003] To correct the aforementioned biases, traditional methods often employ physical calibration (such as frequent calibration using standard reference materials or wavelength calibration lamps) or spectral preprocessing (such as multivariate scattering correction, first derivatives, etc.). Physical calibration increases the complexity of on-site operations, cannot track instantaneous dynamic shifts in real time, and is more difficult to overcome inherent shifts caused by hardware limitations. Traditional spectral preprocessing algorithms are often based on specific mathematical assumptions, making it difficult to handle the complex shifts of handheld spectral sensors in complex backgrounds, and they are prone to losing sample features contained in the spectral signal during depolarization. Summary of the Invention
[0004] The purpose of this application is to provide a method and related apparatus for signal fidelity preservation of handheld spectral sensors, which can improve the signal fidelity accuracy of handheld spectral sensors.
[0005] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a method for signal fidelity preservation of a handheld spectral sensor, including: Acquire the measured spectral signal output by the handheld spectral sensor after measuring the target analyte.
[0006] Based on the measured spectral signal and the offset signal identification model, the offset prediction signal is identified.
[0007] Based on the measured spectral signal and the offset prediction signal, the spectral signal after removing the offset prediction signal is determined.
[0008] The offset signal recognition model is a model obtained by training a machine learning model based on a sensor offset feature training set. The sensor offset feature training set includes m×n information pairs, each of which includes synthetic spectral information and label information. The synthetic spectral information is obtained by combining an unbiased reference information from the unbiased reference information set and an offset information from the offset information set. m is the number of information in the unbiased reference information set, and n is the number of information in the offset information set. The unbiased reference information set is a set of information obtained by measuring the sample to be tested using a benchtop near-infrared spectrometer in a calibration environment, where the calibration environment is a reference environment for calibrating the benchtop near-infrared spectrometer. The offset information set is a set of information obtained by measuring a standard substance using a handheld spectrometer under certain environmental and operating conditions, and processing the measured information. The offset information includes offset information caused by environmental and hardware influences. The label information is the offset information in the synthetic spectral information.
[0009] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the steps of the handheld spectral sensor signal fidelity method described above.
[0010] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the handheld spectral sensor signal fidelity method described above.
[0011] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the handheld spectral sensor signal fidelity method described above.
[0012] Fifthly, this application provides a handheld spectral sensor, including a handheld spectral sensor body and a processing unit.
[0013] The handheld spectral sensor body is used to measure the target analyte and output the measured spectral signal.
[0014] The processing unit is used to execute the handheld spectral sensor signal fidelity method described above.
[0015] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a method and related apparatus for signal fidelity preservation of a handheld spectral sensor. First, an unbiased reference information set is formed by measuring m unbiased reference information values of the sample under test using a benchtop near-infrared spectrometer in a calibration environment. Simultaneously, a handheld spectral sensor measures a standard substance under specific environmental and operational conditions, and the measured information is processed to obtain a set of n offset information values. Then, one unbiased reference information value from the unbiased reference information set and one offset information value from the offset information set are synthesized to obtain m×n synthesized spectral information values. The synthesized spectral information values and the offset information values within them constitute a sensor offset feature training set. Since the unbiased reference information is only collected in a calibration environment, it only includes the spectral characteristics of the sample under test. The offset information includes offset information caused by environmental and hardware influences corresponding to the environment and operating conditions, such as environmental noise, baseline drift, and wavelength offset characteristics (excluding the characteristics of the target analyte or sample). Therefore, the synthesized spectral information is information synthesized from the spectral characteristics of the sample to be tested and the offset information caused by environmental and hardware influences of the environment and operating conditions. It is equivalent to the actual measurement information of the sample to be tested by the handheld spectrometer under the corresponding environment and operating conditions. On this basis, the offset signal recognition model trained by the sensor offset feature training set only learns and recognizes the offset caused by the handheld spectrometer itself and the operating environment of the data acquisition, which can avoid the loss of sample features contained in the spectral signal during the signal fidelity process and improve the signal fidelity accuracy of the handheld spectrometer. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is an application environment diagram of a handheld spectral sensor signal fidelity method according to an embodiment of this application; Figure 2 A schematic flowchart illustrating a signal fidelity method for a handheld spectral sensor provided in an embodiment of this application; Figure 3 A flowchart illustrating a method for preserving the signal fidelity of a handheld spectral sensor, provided in another embodiment of this application; Figure 4 This is a schematic diagram of the structure of a machine learning model in one embodiment of this application; Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application.
[0018] Attached image labels: 102 terminal, 104 server. Detailed Implementation
[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0020] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0021] The handheld spectral sensor signal fidelity method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send the measured spectral signal to server 104. After receiving the measured spectral signal, server 104 identifies the offset prediction signal based on the measured spectral signal and the offset signal recognition model; based on the measured spectral signal and the offset prediction signal, it determines the spectral signal after removing the offset prediction signal. Server 104 can feed back the obtained spectral signal after removing the offset prediction signal to terminal 102. Furthermore, in some embodiments, the handheld spectral sensor signal fidelity method can also be implemented independently by server 104 or terminal 102. For example, terminal 102 can directly perform handheld spectral sensor signal fidelity processing on the measured spectral signal, or server 104 can obtain the measured spectral signal from the data storage system and perform handheld spectral sensor signal fidelity processing on the measured spectral signal.
[0022] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.
[0023] In one exemplary embodiment, such as Figure 2As shown, a method for signal fidelity preservation of a handheld spectral sensor is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 201 to 203. Wherein: Step 201: Obtain the measured spectral signal output by the handheld spectral sensor after measuring the target analyte.
[0024] Step 202: Based on the measured spectral signal and the offset signal identification model, identify the offset prediction signal.
[0025] Step 203: Based on the measured spectral signal and the offset prediction signal, determine the spectral signal after removing the offset prediction signal.
[0026] The offset signal recognition model is a model obtained by training a machine learning model based on a sensor offset feature training set. The sensor offset feature training set includes m×n information pairs, each of which includes synthetic spectral information and label information. The synthetic spectral information is obtained by combining an unbiased reference information from the unbiased reference information set and an offset information from the offset information set. m is the number of information in the unbiased reference information set, and n is the number of information in the offset information set. The unbiased reference information set is a set of information obtained by measuring the sample to be tested using a benchtop near-infrared spectrometer in a calibration environment, where the calibration environment is a reference environment for calibrating the benchtop near-infrared spectrometer. The offset information set is a set of information obtained by measuring a standard substance using a handheld spectrometer under certain environmental and operating conditions, and processing the measured information. The offset information includes offset information caused by environmental and hardware influences. The label information is the offset information in the synthetic spectral information.
[0027] Implementing steps 201 to 203 above, firstly, an unbiased reference information set consisting of m unbiased reference information obtained by measuring the sample to be tested using a benchtop near-infrared spectrometer in a calibration environment is formed. Simultaneously, a standard substance is measured using a handheld spectrometer under certain environmental and operational conditions, and the measured information is processed to obtain a set of n offset information. Then, one unbiased reference information from the unbiased reference information set and one offset information from the offset information set are synthesized to obtain m×n synthesized spectral information. The sensor offset feature training set is constructed using the synthesized spectral information and the offset information in the synthesized spectral information. Since the unbiased reference information is only collected in a calibration environment, the unbiased reference information only includes the spectral characteristics of the sample to be tested. Meanwhile, the offset information includes the offset information caused by the influence of the environment and hardware on the corresponding environment and operating conditions (excluding the characteristics of the target analyte or sample). Therefore, the synthesized spectral information is the information synthesized by the spectral characteristics of the sample to be tested and the offset information caused by the influence of the environment and hardware on the environment and operating conditions. It is equivalent to the actual measurement information of the sample to be tested by the handheld spectrometer under the corresponding environment and operating conditions. On this basis, the offset signal recognition model trained by the sensor offset feature training set only learns and recognizes the offset caused by the handheld spectrometer itself and the operating environment when it collects the signal. This can avoid the loss of the sample features contained in the spectral signal during the signal fidelity process and improve the signal fidelity accuracy of the handheld spectrometer.
[0028] In one exemplary embodiment, such as Figure 3 As shown, a method for preserving the signal fidelity of a handheld spectral sensor is provided. The method involves determining the true offset data (corresponding to the offset information set in the above embodiment) using the handheld spectral sensor; constructing a handheld spectral dataset with true offset features (corresponding to the sensor offset feature training set in the above embodiment) based on the true offset data and unbiased spectral signal data (corresponding to the unbiased reference information set in the above embodiment); and training a machine learning model for extracting the offset signal from the handheld spectral sensor spectrum based on the handheld spectral dataset with true offset features (the trained model corresponds to the offset signal recognition model in the above embodiment). When the handheld spectral sensor measures a sample (corresponding to the target analyte in the above embodiment), the original spectral signal of the sample is obtained (corresponding to the measured spectral signal in the above embodiment). The offset signal (corresponding to the offset prediction signal in the above embodiment) in the original spectral signal is parsed using the trained machine learning model. Then, based on the original spectral signal and the offset signal in the original spectral signal, the unbiased spectral signal of the sample (corresponding to the spectral signal after removing the offset prediction signal in the above embodiment) is determined.
[0029] In another exemplary embodiment of this application, the machine learning model is a one-dimensional U-Net model with superimposed channel attention and spatial attention mechanisms, and the machine learning model includes a shrinking path, a bottleneck layer, and an expanding path. Figure 4 The diagram shown is a schematic representation of the structure of a machine learning model.
[0030] The contraction path (corresponding to) Figure 4 The left-hand structure is used to encode the synthetic spectral information to obtain synthetic spectral encoded information.
[0031] The bottleneck layer (corresponding to) Figure 4 The connection structure between the left and right structures is used to perform deep feature extraction on the synthesized spectral encoding information to obtain bottleneck layer information.
[0032] The expansion path (corresponding to) Figure 4 The right-hand structure is used to decode the fused information, obtain the decoded information, and output it; the fused information is the information obtained by feature fusion of the upsampled information of the synthetic spectral coding information and the bottleneck layer information.
[0033] In one implementation, the contraction path includes a plurality of contraction modules connected in series, the expansion path includes a plurality of expansion modules connected in series, and the expansion modules are configured in a one-to-one correspondence with the contraction modules.
[0034] Each of the contraction modules includes a first contraction submodule, a second contraction submodule, and a first channel attention and spatial attention module; the input terminal of the first contraction submodule is the input terminal of the corresponding contraction module; the output terminal of the first contraction submodule is connected to the input terminal of the second contraction submodule; the second contraction submodule is connected to the first channel attention and spatial attention module; and the output terminal of the second contraction submodule is the output terminal of the corresponding contraction module.
[0035] Each of the expansion modules includes a feature fusion module, a decoding module, and a second channel attention and spatial attention module; the input of the feature fusion module is the input of the corresponding expansion module; the output of the feature fusion module is connected to the input of the decoding module, and the decoding module is connected to the second channel attention and spatial attention module; the output of the decoding module is the output of the corresponding expansion module.
[0036] The output of each shrinking module is connected to the first input of the feature fusion module in the corresponding expansion module; the second input of the first feature fusion module is connected to the output of the bottleneck layer, and the second input of the other feature fusion modules is connected to the output of the previous expansion module.
[0037] The first shrinkage submodule is used to perform one-dimensional convolution operation on the input spectral information and extract features; the second shrinkage submodule is used to filter the feature information output by the first shrinkage submodule; and the first channel attention and spatial attention modules are used to perform weighted processing on the feature information transmitted by the second shrinkage submodule.
[0038] The feature fusion module is used to upsample the spectral information of its second input terminal and fuse the upsampled information with the spectral information of its first input terminal; the decoding module is used to perform feature filtering processing on the fused features output by the feature fusion module; and the second channel attention and spatial attention modules are used to perform weighted processing on the fused features transmitted by the decoding module.
[0039] In another exemplary embodiment of this application, the method for constructing the sensor offset feature training set includes: Step 100: Obtain the standard substance measurement information set; the standard substance measurement information set is a collection of information obtained by measuring the standard substance under multiple environments and operating conditions using the handheld spectral sensor, with each environment and operating condition corresponding to a standard substance measurement information set.
[0040] As an example, a handheld spectrometer is used to sample known standard materials (such as standard reflectors or dark background states) in various complex field environments (environment and working conditions) at multiple frequencies to obtain a set of standard material measurement information affected by the environment and hardware. The hardware refers to the handheld spectrometer used to measure the standard material measurement information.
[0041] Step 200: Based on the reflectance value of the standard substance, determine the offset information corresponding to the measurement information of each standard substance, and construct an offset information set based on all offset information; the measurement information of the standard substance is any information in the measurement information set of the standard substance.
[0042] Since the reflectance values of the standard materials are known, it can be determined that each standard material measurement information in the standard material measurement information set contains only the offset information caused by environmental and hardware influences.
[0043] Step 300: Determine the set of samples to be tested; the set of samples to be tested includes samples of the test object at any stage of its entire life cycle.
[0044] In one example, when the sample to be tested is crop leaves, crop leaves at different growth stages (200 groups) are selected as the sample set to be tested.
[0045] Step 400: Obtain sample measurement information by measuring any sample in the set of samples under test using a benchtop near-infrared spectrometer in a calibration environment, and determine all measurement information as unbiased reference information.
[0046] In one example, the benchtop near-infrared spectrometer uses a high-performance laboratory-grade spectrometer or an ideal spectrum generator. The calibration environment refers to the environment in which the benchtop near-infrared spectrometer is calibrated, and the specific environment should be determined according to the spectrometer manufacturer's instructions. For example, if the calibration environment is a temperature of 25°C (relative error 2%) and humidity of 30% (relative error 5%), then the spectral information measured under this environment is considered to be high signal-to-noise ratio spectral information and can be used as unbiased information.
[0047] Step 500: For any offset information, the offset information is fused with any unbiased reference information in the unbiased reference information set to obtain the corresponding synthetic spectral information, and a subset of sample information is determined based on the offset information and all the synthetic spectral information.
[0048] Step 600: Determine the sensor offset feature training set based on all the sample information subsets.
[0049] In one example, there are 200 sets of unbiased reference information (i.e., m=200) and 200,000 sets of offset information (i.e., n=200,000). Each set of unbiased reference information can be expanded to obtain 200,000 sets of spectral information. The 200,000 sets of spectral information and the 200,000 sets of offset information constitute 200,000 information pairs (as a subset of sample information), totaling 200 subsets of sample information, which constitute the sensor offset feature training set (m×n in total).
[0050] The fusion of offset information and unbiased reference information can be used to simulate the measured spectral signals of a handheld spectral sensor under various complex environments and operating conditions. For example, the first set of unbiased reference information corresponds to the first crop leaf sample at the first growth stage. This first set of unbiased reference information is fused with 200,000 sets of offset information to obtain 200,000 sets of spectral information. In this way, the measured spectral signals of the first crop leaf sample at the first growth stage under 200,000 different environments and operating conditions can be obtained.
[0051] In one example, both the offset information and the unbiased reference information include multiple wavelength values and the signal intensity corresponding to each wavelength value, and the range and spacing of the wavelength values of the offset information and the unbiased reference information are the same; then in step 500, the offset information is fused with any unbiased reference information in the set of unbiased reference information to obtain the corresponding synthetic spectral information, specifically including: For any unbiased reference information, the offset information and the unbiased reference information are synthesized based on a mathematical fusion operator to obtain the corresponding synthesized spectral information; the mathematical fusion operator is used to add the signal intensities corresponding to the same wavelength values in the offset information and the unbiased reference information point by point.
[0052] In another example, both the offset information and the unbiased reference information include multiple wavelength values and the signal intensity corresponding to each wavelength value, but the range or spacing of the wavelength values of the offset information and the unbiased reference information are different; then, before step 500, the method further includes preprocessing the offset information and the unbiased reference information, for example, interpolating the offset information or the unbiased reference information to make the wavelength values in the offset information and the unbiased reference information correspond (the correspondence here can be a one-to-one correspondence of all wavelength values or a one-to-one correspondence of some wavelength values); step 500 performs a fusion process on the preprocessed offset information and the unbiased reference information.
[0053] In another exemplary embodiment of this application, when constructing the sensor offset feature training set, the offset information set in step 200 can be expanded by information augmentation, and the expanded offset information set can be used as the offset information set finally determined in step 200. This can effectively enhance the robustness of the trained model.
[0054] As an example, the offset information set determined by sampling and processing in step 200 is marked as the initial offset information set. Part or all of the initial offset information in the initial offset information set is enhanced sequentially, and the initial offset information set is expanded based on all the enhanced initial offset information.
[0055] Among them, enhancement processing can be full-wavelength signal strength amplification, full-wavelength signal strength reduction, partial-wavelength signal strength amplification, partial-wavelength signal strength reduction, addition of random noise to partial-wavelength signals, addition of random noise to full-wavelength signals, or a combination of these processing methods.
[0056] In another exemplary embodiment of this application, the environment and operating conditions include temperature environment, humidity environment and operating condition status; the operating condition status includes the continuous working time and heat generation status of the handheld spectral sensor.
[0057] In another exemplary embodiment of this application, the detection of nutrient components in crop leaves using a miniature near-infrared spectrometer (a type of handheld spectral sensor) is used as an example to illustrate the specific process of acquiring high-fidelity leaf spectral signals using a miniature near-infrared spectrometer under fluctuating natural light, temperature, and humidity conditions in the field.
[0058] Step 1: Experimental preparation and information set construction.
[0059] Sample preparation: 200 groups of crop leaves at different growth stages were selected.
[0060] Construction of unbiased reference information set: Under calibration environment (temperature 25℃, humidity 30%, constant temperature and humidity), standard reflectance spectral signals of each crop leaf were acquired using a high-precision research-grade near-infrared spectrometer. Construct an unbiased benchmark information set ;in, This is the standard reflectance spectral signal of the first group of crop leaves. This is the standard reflectance spectral signal of the second group of crop leaves. This is the standard reflectance spectral signal of the 200th group of crop leaves.
[0061] Offset Information Extraction: A miniature near-infrared spectrometer was placed in a field environment and scanned a standard polytetrafluoroethylene (PTFE) reference plate under different environmental and operating conditions. The environment could be early morning (low temperature and high humidity), noon (high temperature and low humidity), or evening. The operating conditions could be different continuous operating times of the miniature near-infrared spectrometer or different heating states (heating levels). The scanned information was processed to obtain offset information that only included those affected by environmental and hardware factors. ; in, The spectral information of a standard polytetrafluoroethylene reference plate was collected by a miniature near-infrared spectrometer under certain environmental and operating conditions. The reflectance value is that of a standard polytetrafluoroethylene (PTFE) reference plate. for The corresponding offset information.
[0062] Constructing an offset information set : ; in The number of offset information, n=200000. This is the first offset information. This is the second offset information. This is the nth offset information.
[0063] Step 2, model training.
[0064] Sensor offset feature training set preparation: The unbiased reference information set constructed in step 1 and the spectral information in the offset information set are mathematically synthesized to generate tens of millions of synthetic spectral information with "virtual field features".
[0065] The expression for the synthesized spectral information is: The corresponding sample information is Sensor offset feature training set for: .in, This represents a synthesized spectral information. This indicates an offset information. This represents the spectral information of the i-th combination. This represents the offset information in the spectral information of the i-th combination.
[0066] Machine learning model construction: One-dimensional U-Net was selected as the backbone network, and channel attention and spatial attention mechanisms were introduced to improve the network's learning ability.
[0067] Machine learning model training: Training set based on sensor offset features The mean squared error (MSE) is used as the loss function to train the network, stochastic gradient descent is used as the optimizer, and cosine annealing is used as the learning rate strategy to adjust the machine learning model. This enables the trained machine learning model to separate "instrument response bias" and "sample feature signal" from composite signals. As an offset signal recognition model, the trained machine learning model can accurately identify offset information (i.e., distortion features, as offset prediction signals) in synthetic spectral information.
[0068] Step 3: In-situ field detection and signal restoration.
[0069] Measured spectral signal acquisition: Researchers directly scanned crop leaves using a miniature near-infrared spectrometer to obtain the measured spectral signals. Due to environmental influences, the dark current of the miniature near-infrared spectrometer increased overall and fluctuated significantly, resulting in noticeable baseline rise and noise distortion in the measured spectral signals.
[0070] Offset signal extraction: The measured spectral signal with baseline lift and noise distortion is input into the machine learning model trained in step 2. The trained machine learning model automatically identifies the offset information (offset prediction signal) contained in the measured spectral signal.
[0071] Differential subtraction and restoration: Subtract the identified offset information from the measured spectral signal.
[0072] When a new sample needs to be tested, repeat step 3.
[0073] The beneficial effects of this embodiment include: This breakthrough overcomes the trade-off between hardware cost and signal fidelity: by using machine learning models to analyze instrument biases at the algorithmic level, handheld spectroscopic sensors can acquire high-fidelity spectral signals approaching those of high-specification laboratory-grade instruments, even with simplified spectroscopic elements and calibration units. This significantly improves the detection accuracy and reliability of handheld spectroscopic sensors without increasing hardware costs.
[0074] This invention achieves adaptive compensation in complex and variable environments: Compared to traditional physical calibration methods, it constructs a sensor offset feature training set containing real offset characteristics, enabling machine learning models to identify and remove dynamic environmental interference (such as temperature drift and background light fluctuations). In complex field environments such as fields and greenhouses, automatic real-time correction of spectral signals can be achieved without frequent manual reference calibration.
[0075] The invention boasts high versatility and low deployment threshold due to its "sample decoupling" mechanism: The machine learning model trained in this invention aims to identify response biases of the handheld spectral sensor itself, rather than sample features. Therefore, this fidelity-preserving method possesses extremely high versatility. Once established, the performance of the machine learning model is solely dependent on the handheld spectral sensor and independent of the type of sample being tested. This means that the machine learning model can be pre-installed at the factory, eliminating the need for end-users to retrain the model when testing different substances (such as leaves, grains, and soil), achieving true "plug and play."
[0076] Significantly improves the robustness and transferability of downstream analysis models: Since the output is a spectral signal after removing the offset prediction signal, that is, the final output is an unbiased spectral signal that eliminates instrument and environmental interference, it is beneficial to improve the accuracy of subsequent qualitative or quantitative analysis results. It can effectively solve the problem of calibration model failure caused by poor equipment consistency and environmental interference, greatly improve the transfer efficiency of chemometrics models between different micro devices, and reduce the model maintenance cost after the large-scale commercial application of handheld spectral sensors.
[0077] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 5As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores the measured spectral signal and the spectral signal after removing the offset prediction signal. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the handheld spectral sensor signal fidelity method described in the above embodiment.
[0078] Those skilled in the art will understand that Figure 5 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0079] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0080] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0081] In one exemplary embodiment, a handheld spectral sensor is provided, including a handheld spectral sensor body and a processing unit.
[0082] The handheld spectral sensor body is used to measure the target analyte and output the measured spectral signal.
[0083] The processing unit is used to execute the handheld spectral sensor signal fidelity method described in the above embodiments.
[0084] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of the relevant data are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.
[0085] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).
[0086] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0087] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0088] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for signal fidelity preservation of a handheld spectral sensor, characterized in that, The handheld spectral sensor signal fidelity method includes: Acquire the measured spectral signal output by the handheld spectral sensor after measuring the target analyte; Based on the measured spectral signal and the offset signal identification model, the offset prediction signal is identified; Based on the measured spectral signal and the offset prediction signal, determine the spectral signal after removing the offset prediction signal; The offset signal recognition model is a model obtained by training a machine learning model based on a sensor offset feature training set. The sensor offset feature training set includes m×n information pairs, each of which includes synthetic spectral information and label information. The synthetic spectral information is obtained by combining an unbiased reference information from the unbiased reference information set and an offset information from the offset information set. m is the number of information in the unbiased reference information set, and n is the number of information in the offset information set. The unbiased reference information set is a set of information obtained by measuring the sample to be tested using a benchtop near-infrared spectrometer in a calibration environment, where the calibration environment is a reference environment for calibrating the benchtop near-infrared spectrometer. The offset information set is a set of information obtained by measuring a standard substance using a handheld spectrometer under certain environmental and operating conditions, and processing the measured information. The offset information includes offset information caused by environmental and hardware influences. The label information is the offset information in the synthetic spectral information.
2. The handheld spectral sensor signal fidelity method according to claim 1, characterized in that, The machine learning model is a one-dimensional U-Net model with channel attention mechanism and spatial attention mechanism superimposed. The machine learning model includes a shrinking path, a bottleneck layer and an expanding path. The contraction path is used to encode the synthesized spectral information to obtain synthesized spectral encoded information; The bottleneck layer is used to extract deep features from the synthesized spectral encoding information to obtain bottleneck layer information; The expansion path is used to decode the fused information, obtain the decoded information, and output it. The fusion information is obtained by feature fusion of the upsampled information of the synthetic spectral coding information and the bottleneck layer information.
3. The handheld spectral sensor signal fidelity method according to claim 2, characterized in that, The contraction path includes multiple contraction modules connected in series, and the expansion path includes multiple expansion modules connected in series, with each expansion module corresponding to a contraction module. Each of the contraction modules includes a first contraction submodule, a second contraction submodule, and a first channel attention and spatial attention module; the input terminal of the first contraction submodule is the input terminal of the corresponding contraction module; the output terminal of the first contraction submodule is connected to the input terminal of the second contraction submodule. The second contraction submodule is connected to the first channel attention and spatial attention module; The output terminal of the second contraction submodule is the output terminal of the corresponding contraction module; Each of the expansion modules includes a feature fusion module, a decoding module, and a second channel attention and spatial attention module; the input of the feature fusion module is the input of the corresponding expansion module; the output of the feature fusion module is connected to the input of the decoding module, and the decoding module is connected to the second channel attention and spatial attention module; the output of the decoding module is the output of the corresponding expansion module. The output of each of the contraction modules is connected to the first input of the feature fusion module in the corresponding expansion module; The second input of the first feature fusion module is connected to the output of the bottleneck layer, and the second input of the other feature fusion modules is connected to the output of the previous expansion module.
4. The handheld spectral sensor signal fidelity method according to claim 3, characterized in that, The first contraction submodule is used to perform one-dimensional convolution operations on the input spectral information and extract features; The second contraction submodule is used to filter the feature information output by the first contraction submodule, and the first channel attention and spatial attention module is used to perform weighted processing on the feature information transmitted by the second contraction submodule. The feature fusion module is used to upsample the spectral information of its second input terminal and fuse the upsampled information with the spectral information of its first input terminal. The decoding module is used to perform feature filtering processing on the fused features output by the feature fusion module, and the second channel attention and spatial attention modules are used to perform weighted processing on the fused features transmitted by the decoding module.
5. The handheld spectral sensor signal fidelity method according to claim 1, characterized in that, The method for constructing the sensor offset feature training set includes: A set of standard substance measurement information is obtained; the set of standard substance measurement information is a collection of information obtained by measuring the standard substance under multiple environments and operating conditions using the handheld spectrometer sensor, and each environment and operating condition corresponds to a standard substance measurement information; the environment and operating condition include temperature environment, humidity environment and operating condition status; the operating condition status includes the continuous working time and heating status of the handheld spectrometer sensor. Based on the reflectance value of the standard material, the offset information corresponding to the measurement information of each standard material is determined, and an offset information set is constructed based on all the offset information. Determine the set of samples to be tested; the set of samples to be tested includes samples of the analyte at any stage of its entire life cycle. After measuring any sample in the set of samples to be tested with a benchtop near-infrared spectrometer in a calibration environment, the sample measurement information is obtained, and all the measurement information is determined as unbiased reference information. For any offset information, the offset information is fused with any unbiased reference information in the unbiased reference information set to obtain the corresponding synthetic spectral information, and a subset of sample information is determined based on the offset information and all the synthetic spectral information. The sensor offset feature training set is determined based on all subsets of the sample information.
6. The handheld spectral sensor signal fidelity method according to claim 5, characterized in that, Both the offset information and the unbiased reference information include multiple wavelength values and the signal strength corresponding to each wavelength value, and the range and spacing of the wavelength values of the offset information and the unbiased reference information are the same. The offset information is fused with any unbiased reference information in the unbiased reference information set to obtain the corresponding synthetic spectral information, specifically including: For any unbiased reference information, the offset information and the unbiased reference information are synthesized based on a mathematical fusion operator to obtain the corresponding synthesized spectral information; The mathematical fusion operator is used to add the signal intensities corresponding to the same wavelength values in the offset information and the unbiased reference information point by point.
7. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the method for training a recognition model of the spectral sensor offset signal according to any one of claims 1-6.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for training a recognition model of the spectral sensor offset signal as described in any one of claims 1-6.
9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the method for training a recognition model of the spectral sensor offset signal as described in any one of claims 1-6.
10. A handheld spectral sensor, characterized in that, The handheld spectral sensor includes a handheld spectral sensor body and a processing unit; The handheld spectral sensor body is used to measure the target analyte and output the measured spectral signal. The processing unit is used to execute the handheld spectral sensor signal fidelity method according to any one of claims 1-6.