Micro-calcification recognition method and system based on multi-band spectrum and time-series deep network

Abstract

This invention provides a method and system for identifying minute lesions based on multi-band spectroscopy and temporal deep networks, relating to the fields of medical spectral imaging and intelligent diagnostic technology. The method includes: simultaneously acquiring visible and near-infrared multi-channel spectral data and preprocessing it to obtain standardized absorbance data; and weighting and fusing the absorbance of each channel using pixel-level spectral spatial attention weights to generate a high-contrast SpO2. 2 Mapping; Extracting consecutive T-frame SpO of candidate ROI regions 2 The sequence is input into a multi-head temporal attention BiGRU network for deep feature extraction; an adaptive covariance Kalman filter is used to fuse IMU and ambient light sensor data for SpO. 2 The measured values ​​of the sequence are compensated in real time to obtain probability prediction results; the lesion location and confidence level are output according to a preset threshold. This invention utilizes multi-source biospectral data acquisition and precise SpO2 analysis. 2 Mapping and deep learning methods enable the automatic identification and localization of minute lesions in tissues.

Description

A method and system for identifying minute lesions based on multi-band spectroscopy and temporal deep networks Technical Field

[0001] This invention relates to the field of medical spectral imaging and intelligent diagnostic technology, and in particular to a method and system for identifying minute lesions based on multi-band spectroscopy and temporal deep networks. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] In the field of medical diagnostics, the accurate identification of early, small lesions (such as early tumors with a diameter of 0.5-1.5 mm, microvascular stenosis, etc.) is key to improving cure rates. Traditional invasive testing methods (such as biopsy) have problems such as high operational risks and poor patient compliance.

[0004] Medical spectral imaging technology analyzes the absorption characteristics of biological tissues to specific wavelengths of light, providing a non-invasive detection method for early lesion diagnosis. Therefore, non-invasive detection technologies based on biospectroscopy have become a research hotspot.

[0005] However, existing technologies suffer from several bottlenecks:

[0006] (1) Traditional visible light imaging can only reflect the macroscopic structure of tissues and is not sensitive enough to physiological parameters, while common simple near-infrared detectors have low resolution and are difficult to capture the spectral details of micro-regions.

[0007] (2) Conventional dual-wavelength ratio method is not good at contrasting local lesions and fails to make full use of dynamic changes in blood oxygen saturation, making it difficult to detect subtle fluctuations in blood oxygen.

[0008] (3) Existing segmentation and classification algorithms mostly rely on static grayscale or light intensity features, lack in-depth mining of continuous temporal information, and have limited ability to suppress interference from ambient light and physiological movement (such as breathing and hand shaking), resulting in unsatisfactory automatic identification of minor lesions. Summary of the Invention

[0009] To address at least one of the technical problems mentioned in the background, this invention proposes a method and system for identifying minute lesions based on multi-band spectroscopy and a temporal deep network. The method first simultaneously acquires multi-band spectral images in visible light (400–700 nm) and near-infrared (700–1000 nm). After radiometric correction, filtering, denoising, and normalization, pixel-level SpO2 mappings are generated based on the Beer-Lambert law and a dynamic weighted fusion algorithm. Subsequently, continuous SpO2 sequences are input into an attention-enhanced bidirectional GRU network to mine the temporal dynamics of blood oxygenation, achieving highly sensitive lesion classification and localization. Finally, Kalman filtering is used to fuse IMU and ambient light data, compensating for motion and light intensity artifacts in real time, greatly improving detection stability.

[0010] The first aspect of this invention provides a method for identifying minute lesions based on multi-band spectroscopy and temporal deep networks, comprising:

[0011] Simultaneously acquire visible light and near-infrared multi-channel spectral data, preprocess the images of each channel to obtain standardized absorbance data;

[0012] Based on the standardized absorbance data, the absorbance of each channel is weighted and fused by pixel-level spectral band-spatial attention weight to generate a high-contrast SpO2 map.

[0013] Extract continuous T-frame SpO2 sequences of candidate ROI regions from the high-contrast SpO2 map, and input them into a multi-head temporal attention BiGRU network for deep feature extraction and small lesion probability prediction.

[0014] An adaptive covariance Kalman filter is used to fuse IMU and ambient light sensor data to perform real-time compensation on the measured values ​​of the SpO2 sequence, resulting in a compensated SpO2 sequence and the corresponding probability prediction results.

[0015] Based on the compensated probability prediction results, a binarization decision is made according to a preset threshold, and the lesion location and confidence level are output.

[0016] Furthermore, the preprocessing includes:

[0017] Dark current correction, flat field correction, noise reduction filtering, and normalization were performed on the images of each channel.

[0018] Furthermore, the weighted fusion of pixel-level spectral band-spatial attention weights includes:

[0019] Calculate the residual of each band in the current pixel and its spatial neighborhood, and generate spatiotemporal attention energy through a bi-branch projection function;

[0020] The spatiotemporal attention energy is normalized using softmax to obtain pixel-level spectral-spatial attention weights;

[0021] The high-contrast SpO2 mapping is generated by weighting and summing the absorbance of each channel according to the pixel-level spectral band-spatial attention weight.

[0022] Furthermore, the update of the pixel-level spectral segment-spatial attention weights introduces residual regularization, including:

[0023] The weights are updated by jointly optimizing the global residual response and the weight norm to suppress the excessive dominance of a single band. The update formula is as follows:

[0024] ;

[0025] In the formula, The residual between the current frame and the exponential moving average. This is the residual response rate coefficient. is the weight regularization coefficient.

[0026] Furthermore, the multi-head temporal attention BiGRU network includes:

[0027] Input embedding module: Maps a SpO2 sequence of length T to the feature subspace through a two-layer fully connected network;

[0028] Bidirectional GRU layer: learns the forward and backward temporal dependencies of the SpO2 sequence in parallel and outputs the hidden state sequence;

[0029] Multi-head attention module: Independently calculates attention weights for each hidden state and aggregates them into a context vector;

[0030] Output module: Outputs the probability of minute lesions through a fully connected layer and a Sigmoid activation function.

[0031] Furthermore, the training of the multi-head temporal attention BiGRU network includes:

[0032] We set the number of multi-head attention heads H, the number of GRU hidden units, and the embedding dimension, and jointly optimized them using cross-entropy loss and AUC metric.

[0033] Dropout and noise perturbation are introduced during training to suppress overfitting, and the learning rate is dynamically adjusted using the validation set loss.

[0034] Furthermore, the Kalman filter employs an adaptive covariance update strategy:

[0035] The online estimation of observation noise variance and dynamic adjustment of filter gain are achieved using the following update formula:

[0036] ;

[0037] ;

[0038] in, For the noise at the current moment, This is the noise from the previous moment. For filter gain, For smoothing coefficients, For the observed values, For the observation matrix, To predict the state, This is the covariance matrix of the prior state estimate, reflecting the uncertainty of the prediction.

[0039] A second aspect of the present invention provides a system for identifying minute lesions based on multi-band spectroscopy and temporal deep networks, comprising:

[0040] Multi-band spectral acquisition module: used to simultaneously acquire visible light and near-infrared multi-channel spectral data, preprocess the images of each channel to obtain standardized absorbance data;

[0041] SpO2 mapping module: Based on the standardized absorbance data, the absorbance of each channel is weighted and fused by pixel-level spectral band-spatial attention weight to generate a high-contrast SpO2 mapping;

[0042] Temporal inference module: used to extract continuous T-frame SpO2 sequences of candidate ROI regions from the high-contrast SpO2 mapping, and input them into a multi-head temporal attention BiGRU network for deep feature extraction and small lesion probability prediction;

[0043] Kalman filter compensation module: used to fuse IMU and ambient light sensor data using an adaptive covariance Kalman filter to compensate the measured values ​​of the SpO2 sequence in real time, and obtain the compensated SpO2 sequence and the corresponding probability prediction results;

[0044] Threshold decision and display output module: Based on the compensated probability prediction results, it performs binarization decision according to the preset threshold and outputs the lesion location and confidence level.

[0045] A third aspect of the present invention provides a device for identifying minute lesions based on multi-band spectroscopy and temporal depth networks. The device includes a memory and a processor. The memory is used to store a computer program. The processor is used to implement the above-described method for identifying minute lesions based on multi-band spectroscopy and temporal depth networks when the computer program is executed.

[0046] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for identifying minute lesions based on multi-band spectroscopy and temporal deep networks.

[0047] Compared with existing technologies, the present invention provides a method and system for identifying minute lesions based on multi-band spectroscopy and temporal deep networks, which has the following advantages:

[0048] (1) This invention uses multi-channel spectral coverage of the visible and near-infrared bands to simultaneously capture macroscopic tissue structure (visible light) and micro-area blood oxygenation information (near-infrared); preprocessing reduces the interference of system noise on micro-area details by correcting sensor errors and standardizing data distribution. It achieves high-precision acquisition of multi-dimensional spectral information, provides high signal-to-noise ratio data for subsequent SpO2 mapping, and improves the ability to capture details in small lesion areas.

[0049] (2) This invention dynamically adjusts the weights of each band (based on pixel and neighborhood residuals) to highlight the differences in blood oxygenation in the lesion area at different wavelengths, thereby improving the contrast of SpO2 mapping; and mines the temporal dependence of SpO2 in continuous T frames based on the BiGRU network, and combines the attention mechanism to focus on key fluctuation moments to capture subtle dynamic anomalies in blood oxygenation. This improves the contrast of SpO2 mapping for small lesions, and the temporal feature extraction significantly improves the sensitivity of lesion identification.

[0050] (3) This invention integrates IMU (motion data) and ambient light sensing data, and dynamically adjusts the filter gain by estimating the observation noise variance online, thereby compensating for motion artifacts and light intensity fluctuations in real time. It significantly reduces the fluctuation amplitude of SpO2 measurement and maintains a high recognition accuracy in scenarios with handshake and ambient light changes. Attached Figure Description

[0051] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0052] Figure 1 is a flowchart of the method for identifying minute lesions based on multi-band spectroscopy and temporal deep networks provided in Embodiment 1 of the present invention;

[0053] Figure 2 is a flowchart of the overall process of the method for identifying minute lesions based on multi-band spectroscopy and temporal deep network provided in Embodiment 1 of the present invention.

[0054] Figure 3 is a data processing flowchart of the microlesion identification method based on multi-band spectroscopy and temporal deep network provided in Embodiment 1 of the present invention;

[0055] Figure 4 is a network structure diagram of the micro lesion identification method based on multi-band spectroscopy and temporal deep network provided in Embodiment 1 of the present invention;

[0056] Figure 5 is a schematic diagram of the microlesion identification system based on multi-band spectroscopy and temporal depth network provided in Embodiment 2 of the present invention. Detailed Implementation

[0057] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0058] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0059] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0060] All data acquisition in this embodiment is carried out in accordance with laws and regulations and with user consent, and the data is used legally.

[0061] Example 1

[0062] As shown in Figure 1, this embodiment provides a method for identifying minute lesions based on multi-band spectroscopy and temporal deep networks, including:

[0063] Simultaneously acquire visible light and near-infrared multi-channel spectral data, preprocess the images of each channel to obtain standardized absorbance data;

[0064] Based on the standardized absorbance data, the absorbance of each channel is weighted and fused by pixel-level spectral band-spatial attention weight to generate a high-contrast SpO2 map.

[0065] Extract continuous T-frame SpO2 sequences of candidate ROI regions from the high-contrast SpO2 map, and input them into a multi-head temporal attention BiGRU network for deep feature extraction and small lesion probability prediction.

[0066] An adaptive covariance Kalman filter is used to fuse IMU and ambient light sensor data to perform real-time compensation on the measured values ​​of the SpO2 sequence, resulting in a compensated SpO2 sequence and the corresponding probability prediction results.

[0067] Based on the compensated probability prediction results, a binarization decision is made according to a preset threshold, and the lesion location and confidence level are output.

[0068] This invention provides a comprehensive workflow for identifying minute lesions based on multi-band spectroscopy and temporal deep networks, including multi-band data acquisition and preprocessing, SpO2 mapping generation, temporal feature extraction, Kalman filter compensation, and decision output. By integrating multi-source data with deep learning technology, a complete "data-feature-decision" solution is formed, overcoming the limitations of traditional methods in terms of sensitivity and anti-interference capabilities, and achieving automatic identification and localization of minute lesions.

[0069] Specifically, the preprocessing includes:

[0070] Dark current correction, flat field correction, noise reduction filtering, and normalization were performed on the images of each channel.

[0071] The normalization process maps the absorbance data to a standard normal distribution by calculating the mean and standard deviation of each channel.

[0072] Based on preprocessing steps (dark current correction, flat-field correction, noise reduction filtering, and normalization), sensor system errors (such as dark current offset and light source inhomogeneity) are eliminated, and the data distribution of different bands is standardized. This reduces noise in the raw data, makes absorbance data of each band comparable, provides stable input for subsequent weighted fusion, and improves the accuracy of SpO2 calculation.

[0073] Specifically, the weighted fusion of pixel-level spectral band-spatial attention weights includes:

[0074] Calculate the residual of each band in the current pixel and its spatial neighborhood, and generate spatiotemporal attention energy through a bi-branch projection function;

[0075] The spatiotemporal attention energy is normalized using softmax to obtain pixel-level spectral-spatial attention weights;

[0076] The high-contrast SpO2 mapping is generated by weighting and summing the absorbance of each channel according to the pixel-level spectral band spatial attention weight. This dynamically highlights the blood oxygenation differences in the lesion area at different wavelengths (such as abnormal near-infrared absorption in the lesion area), solving the problem of insufficient contrast in the traditional dual-wavelength method and making it easier to distinguish small lesion areas.

[0077] Specifically, the update of the pixel-level spectral segment-spatial attention weights introduces residual regularization, including:

[0078] The weights are updated by jointly optimizing the global residual response and the weight norm to suppress the excessive dominance of a single band. The update formula is as follows:

[0079] ;

[0080] In the formula, The residual between the current frame and the exponential moving average. This is the residual response rate coefficient. is the weight regularization coefficient.

[0081] Residual regularization is introduced into the weight update (through joint optimization of global residual and weight norm) to suppress the excessive dominance of a single band in the fusion process. This avoids interference from noise or outliers in a certain band on SpO2 mapping, improving mapping stability; and reducing adaptive weight adjustment can minimize mapping error.

[0082] Specifically, the multi-head temporal attention BiGRU network includes:

[0083] Input embedding module: Maps a SpO2 sequence of length T to the feature subspace through a two-layer fully connected network;

[0084] Bidirectional GRU layer: learns the forward and backward temporal dependencies of the SpO2 sequence in parallel and outputs the hidden state sequence;

[0085] Multi-head attention module: Independently calculates attention weights for each hidden state and aggregates them into a context vector;

[0086] Output module: Outputs the probability of minute lesions through a fully connected layer and a Sigmoid activation function.

[0087] A multi-head temporal attention BiGRU network structure is defined, including input embedding, bidirectional GRU (learning temporal dependencies), multi-head attention (focusing on keyframes), and an output module (predicting lesion probabilities). It captures the dynamic fluctuation features of SpO2 sequences (such as periodic abnormalities in blood oxygenation in lesion areas), addressing the insensitivity of traditional static features to subtle changes and improving classification sensitivity.

[0088] Specifically, the training of the multi-head temporal attention BiGRU network includes:

[0089] We set the number of multi-head attention heads H, the number of GRU hidden units, and the embedding dimension, and jointly optimized them using cross-entropy loss and AUC metric.

[0090] Dropout and noise perturbation are introduced during training to suppress overfitting, and the learning rate is dynamically adjusted using validation set loss.

[0091] A limited network training strategy was implemented (cross-entropy loss combined with AUC optimization, Dropout to suppress overfitting, and dynamic adjustment of the learning rate). This improved the model's generalization ability, avoided overfitting to the training data, and reduced the differences in recognition accuracy among different patients and different lesion types.

[0092] Specifically, the Kalman filter employs an adaptive covariance update strategy:

[0093] The online estimation of observation noise variance and dynamic adjustment of filter gain are achieved using the following update formula:

[0094] ;

[0095] ;

[0096] in, For the noise at the current moment, This is the noise from the previous moment. For filter gain, For smoothing coefficients, For the observed values, For the observation matrix, To predict the state, This is the covariance matrix of the prior state estimate, reflecting the uncertainty of the prediction.

[0097] By fusing IMU and ambient light data through adaptive covariance Kalman filtering, motion and light intensity artifacts in SpO2 measurements are compensated for in real time. This improves the stability of SpO2 measurements in scenarios such as breathing and hand tremors, providing reliable data for subsequent feature extraction and reducing misjudgments.

[0098] In one specific embodiment, the system hardware is first initialized and calibrated in a darkroom environment, and the overall flowchart is shown in Figure 2.

[0099] The system consists of a high-frame-rate visible light camera (60fps, 1920×1080), a multi-channel near-infrared (NIR) spectrometer array, a blood oxygen detection array, a three-axis accelerometer / gyroscope (IMU), and an ambient light sensor. All sensors are interconnected through an embedded processing unit (including CPU+GPU). After power-on, the VIS and NIR channels acquire images of a standard black and white board to calculate dark current. and draw reference The optical response curves of each channel are determined through linear fitting. The IMU and ambient light sensor record data for a period of time while stationary, and the zero bias and scaling factor are calculated to compensate for motion artifacts and light intensity fluctuations later. At this point, the system enters the data acquisition state, and its respective memory buffers and thread priorities are allocated within the processing unit.

[0100] When the user triggers the "Start Acquisition" command through the terminal interface, the VIS and NIR sensors simultaneously respond to the hardware trigger signal, capturing multi-band image frames synchronously at the microsecond level, and storing each frame of data in a circular buffer according to its timestamp. The original grayscale value of each pixel is denoted as... ,in, Indicates the first Each spectral segment This is a frame counter. The circular buffer uses a double-buffering mechanism, with one side used for continuous acquisition and the other for parallel preprocessing, ensuring a real-time data throughput of 60fps.

[0101] In image preprocessing, radiometric correction is first applied to each original image: using the formula...

[0102] ;

[0103] The grayscale values ​​are mapped to absorbance, while simultaneously eliminating errors in the light source and sensor system. Subsequently, the system... Perform three median filters (3×3) to eliminate isolated noise, then perform 5×5... Gaussian smoothing is applied to suppress high-frequency details. Simultaneously, the average value of each channel is calculated online based on historical frame data. with standard deviation And perform normalization.

[0104] ;

[0105] This makes the data distribution of different spectral bands tend to a standard normal distribution, laying a numerical foundation for subsequent fusion.

[0106] Based on absorbance, the system uses a dynamic weighted fusion algorithm to generate pixel-level blood oxygen saturation mapping. The data processing flowchart is shown in Figure 3.

[0107] For each pixel location, the normalized absorbance of consecutive frames is weighted by the band residual, and spatial neighborhood information is introduced for each pixel. Define the spectral band-spatial attention weights:

[0108] ;

[0109] in, For band residuals; This indicates that the band is in pixels Spatial Neighborhood The mean of the residuals; This is a learnable linear projection function. Final mapping:

[0110] ;

[0111] Calculate the current blood oxygen level.

[0112] Initially, the weights Equal distribution; then based on the current frame and the exponential moving average. residual ;

[0113] according to ;in, Controlling the residual response speed This is the weighted regularization coefficient, used to suppress excessive dominance of a single band.

[0114] renew, A value of 0.9 is typically chosen to balance smoothness and response speed. After mapping is complete, the system uses a preset threshold. The SpO2 map is initially screened to form a binary mask, and a Region of Interest (ROI) is defined for each candidate region.

[0115] The core module of the temporal feature extraction network is a deep classification network based on a bidirectional gated recurrent unit (BiGRU) and incorporating an attention mechanism, as shown in Figure 4. This network accepts continuous blood oxygen saturation sequences from dynamic Regions of Interest (ROIs) as input: each ROI region at time... Previous consecutive The SpO2 values ​​of the frame are organized into a length of One-dimensional vector time series vector To improve the model's compatibility with signals of different dimensions, this one-dimensional sequence is first mapped to a high-dimensional feature space through two fully-connected embedding layers. Specifically, the first embedding layer... The input is linearly mapped to 256 dimensions and ReLU activation is applied with Dropout of 0.2. The second embedding layer reduces the 256 dimensions to 128 dimensions to match the number of subsequent GRU units, and also uses ReLU and Dropout to achieve a balance between fitting ability and resistance to overfitting.

[0116] After the embedding transformation, the aforementioned 128-dimensional feature sequence is fed into a bidirectional GRU layer. This layer consists of 128 GRU units each in the forward and backward directions, totaling 256 units, which can effectively capture the forward and backward dependencies of the blood oxygenation time-series signal. The GRU units employ reset and update gate structures to mitigate the gradient vanishing problem and maintain and update the hidden state at each time step. The output of the bidirectional processing is a sequence of length... Each frame contains a 256-dimensional hidden state sequence. .

[0117] To further focus this temporal feature on the keyframes that best represent the blood oxygenation fluctuations in minute lesions, this implementation adds an attention mechanism after the GRU output. Multi-head temporal attention is employed, with each head... Calculate independent weights:

[0118] ;

[0119] in, Given a trainable projection matrix, Both are attention vectors, with a dimension of 256. The energy is normalized to weights using a softmax operation.

[0120] ;

[0121] Then, concatenate the context of each head:

[0122] ;

[0123] in, For the number of heads, This represents vector concatenation. The multi-head mechanism allows the model to focus on blood oxygen fluctuations at different time scales in parallel.

[0124] This vector incorporates the most diagnostically valuable moments in the sequence into the model's decision with higher weight.

[0125] In obtaining the context vector The network is then connected to a fully connected layer and outputs the lesion probability via a sigmoid activation layer. :

[0126] ;

[0127] in, Its dimension is 1×256. This is a scalar bias.

[0128] During the training phase, this implementation employs an end-to-end supervised learning process with binary cross-entropy loss. Time-series SpO2 sequence data containing labeled lesion ROIs and normal ROIs are used as samples, with a batch size of 64 and an initial learning rate of 1e-3. The Adam optimizer is used, and AUC (AreaUnderROCCurve) is monitored on the validation set. After each epoch, the learning rate is adjusted by a decay factor of 0.5 based on the validation set loss, and training is stopped early when the validation AUC stagnates for more than 5 epochs. To improve model generalization, Dropout of 0.2 is uniformly inserted into the GRU hidden layers and fully connected layers throughout the training process, and a normalized noise perturbation of 0.1 is added before attention calculation.

[0129] During the inference phase, the network model is exported in ONNX format, deployed on an embedded GPU, and subjected to tensor fusion and operator optimization via TensorRT, resulting in an inference latency of less than 10ms for a single ROI.

[0130] To suppress measurement errors caused by heartbeat, respiratory movements, and hand tremors, this implementation introduces Kalman filtering compensation before the classification results. The system uses the corrected SpO2 and its first-order difference to form a state vector. ,use:

[0131] ;

[0132] ;

[0133] Define the state transition and observation equations, where, Predicted by IMU and ambient light model. Adaptive updates based on measurement noise are added to the standard Kalman filter. :

[0134] ;

[0135] And with the updated Calculate gain :

[0136] ;

[0137] The updated state Provides smooth and accurate SpO2 values . The updated formula is:

[0138] ;

[0139] Obtain the corrected blood oxygen probability for each region Then, the system uses the optimal threshold based on the ROC curve generated during the training phase. right Binarization is performed, and a 3×3 closing operation (dilation followed by erosion) is applied to the resulting binary image to remove isolated noise points and fill micro-cracks. Then, connected component analysis is used to remove pixels with areas smaller than [a certain value]. The detection results are then overlaid onto the original image as a red mask or selection box, displayed in real time on the terminal screen. The original spectral data, SpO2 mapping, classification results, and timestamps are sequentially stored in a database or log file for subsequent review and diagnostic reference.

[0140] This implementation method achieved excellent results in both in vitro vascular models and human fingertip microcirculation experiments. In simulated micro-narrowing (0.5–1.5 mm) channels, the average sensitivity and specificity both exceeded 95%. In real volunteer fingertip imaging, the system was able to accurately capture subtle fluctuations in blood oxygenation and maintained a recognition accuracy of over 90% under various hand-held shaking and ambient light changes. The system's end-to-end response time was less than 50 ms, fully meeting the needs of early clinical diagnosis and real-time monitoring.

[0141] Example 2

[0142] As shown in Figure 5, this embodiment provides a small lesion identification system based on multi-band spectroscopy and temporal deep networks, including:

[0143] Multi-band spectral acquisition module: used to simultaneously acquire visible light and near-infrared multi-channel spectral data, preprocess the images of each channel to obtain standardized absorbance data;

[0144] SpO2 mapping module: Based on the standardized absorbance data, the absorbance of each channel is weighted and fused by pixel-level spectral band-spatial attention weight to generate a high-contrast SpO2 mapping;

[0145] Temporal inference module: used to extract continuous T-frame SpO2 sequences of candidate ROI regions from the high-contrast SpO2 mapping, and input them into a multi-head temporal attention BiGRU network for deep feature extraction and small lesion probability prediction;

[0146] Kalman filter compensation module: used to fuse IMU and ambient light sensor data using an adaptive covariance Kalman filter to compensate the measured values ​​of the SpO2 sequence in real time, and obtain the compensated SpO2 sequence and the corresponding probability prediction results;

[0147] Threshold decision and display output module: Based on the compensated probability prediction results, it performs binarization decision according to the preset threshold and outputs the lesion location and confidence level.

[0148] The timing inference module is deployed on an embedded GPU and accelerates inference through the TensorRT engine, achieving a single-frame inference latency of ≤10ms.

[0149] Example 3

[0150] This embodiment provides a device for identifying minute lesions based on multi-band spectroscopy and temporal depth networks. The device includes a memory and a processor. The memory is used to store a computer program. The processor is used to implement the above-mentioned method for identifying minute lesions based on multi-band spectroscopy and temporal depth networks when the computer program is executed.

[0151] The processor is connected to the memory, and one or more computer programs are stored in the memory. When the electronic device is running, the processor executes one or more computer programs stored in the memory to cause the electronic device to perform the method described in Embodiment 1.

[0152] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0153] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0154] In the implementation process, each step of the above method can be completed by the integrated logic circuits in the processor hardware or by software instructions.

[0155] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0156] Those skilled in the art will recognize that the units and algorithm steps described in connection with the various examples of this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0157] Example 4

[0158] In another embodiment of the present invention, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method for identifying minute lesions based on multi-band spectroscopy and temporal deep networks as described above.

[0159] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. In this application, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention according to actual needs. Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units can be implemented in hardware or as software functional units.

[0160] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A method for identifying minute lesions based on multi-band spectroscopy and temporal deep networks, characterized in that, include: Simultaneously acquire visible light and near-infrared multi-channel spectral data, preprocess the images of each channel to obtain standardized absorbance data; Based on the standardized absorbance data, the absorbance of each channel is weighted and fused by pixel-level spectral band-spatial attention weight to generate a high-contrast SpO2 map. The weighted fusion of pixel-level spectral band-spatial attention weights includes: calculating the residual of each band in the current pixel and its spatial neighborhood, generating spatiotemporal attention energy through a bi-branch projection function; performing softmax normalization on the spatiotemporal attention energy to obtain pixel-level spectral band-spatial attention weights; and weighted summing of the absorbance of each channel according to the pixel-level spectral band-spatial attention weights to generate the high-contrast SpO2 mapping. The update of the pixel-level spectral band-spatial attention weights introduces residual regularization, including: updating the weights through joint optimization of the global residual response and the weight norm to suppress excessive dominance of a single band. The update formula is as follows: In the formula, The residual between the current frame and the exponential moving average. This is the residual response rate coefficient. The weight regularization coefficient is used. A continuous T-frame SpO2 sequence of candidate ROI regions is extracted from the high-contrast SpO2 mapping and input into a multi-head temporal attention BiGRU network for deep feature extraction and small lesion probability prediction. An adaptive covariance Kalman filter is used to fuse IMU and ambient light sensor data to compensate the measured values ​​of the SpO2 sequence in real time, obtaining the compensated SpO2 sequence and the corresponding probability prediction result. Based on the compensated probability prediction result, a binarization decision is made according to a preset threshold, and the lesion location and confidence level are output.

2. The method as described in claim 1, characterized in that, The preprocessing includes performing dark current correction, flat field correction, noise reduction filtering, and normalization on each channel image.

3. The method as described in claim 1, characterized in that, The multi-head temporal attention BiGRU network includes: an input embedding module that maps a SpO2 sequence of length T to a feature subspace through a two-layer fully connected network; a bidirectional GRU layer that learns the forward and backward temporal dependencies of the SpO2 sequence in parallel and outputs a hidden state sequence; a multi-head attention module that independently calculates attention weights for each hidden state and aggregates them into a context vector; and an output module that outputs the probability of minor lesions through a fully connected layer and a Sigmoid activation function.

4. The method as described in claim 3, characterized in that, The training of the multi-head temporal attention BiGRU network includes: setting the number of multi-head attention heads H, the number of GRU hidden units and the embedding dimension, and jointly optimizing them through cross-entropy loss and AUC metric; introducing Dropout and noise perturbation during training to suppress overfitting, and dynamically adjusting the learning rate through validation set loss.

5. The method as described in claim 1, characterized in that, The Kalman filter employs an adaptive covariance update strategy: it estimates the observation noise variance online and dynamically adjusts the filter gain, with the update formula being: ； ;in, For the noise at the current moment, This is the noise from the previous moment. For filter gain, For smoothing coefficients, For the observed values, For the observation matrix, To predict the state, This is the covariance matrix of the prior state estimate, reflecting the uncertainty of the prediction.

6. A system for identifying minute lesions based on multi-band spectroscopy and temporal deep networks, characterized in that, include: Multi-band spectral acquisition module: used to simultaneously acquire visible light and near-infrared multi-channel spectral data, preprocess the images of each channel to obtain standardized absorbance data; SpO2 mapping module: used to generate high-contrast SpO2 mapping by weighting and fusing the absorbance of each channel with pixel-level spectral band-spatial attention weight based on the standardized absorbance data. The weighted fusion of pixel-level spectral band-spatial attention weights includes: calculating the residual of each band in the current pixel and its spatial neighborhood, generating spatiotemporal attention energy through a bi-branch projection function; performing softmax normalization on the spatiotemporal attention energy to obtain pixel-level spectral band-spatial attention weights; and weighted summing of the absorbance of each channel according to the pixel-level spectral band-spatial attention weights to generate the high-contrast SpO2 mapping. The update of the pixel-level spectral band-spatial attention weights introduces residual regularization, including: updating the weights through joint optimization of the global residual response and the weight norm to suppress excessive dominance of a single band. The update formula is as follows: In the formula, The residual between the current frame and the exponential moving average. This is the residual response rate coefficient. The weight regularization coefficient is used for the following modules: Temporal Inference Module: This module extracts a continuous T-frame SpO2 sequence of candidate ROI regions from the high-contrast SpO2 mapping and inputs it into a multi-head temporal attention BiGRU network for deep feature extraction and small lesion probability prediction; Kalman Filter Compensation Module: This module uses an adaptive covariance Kalman filter to fuse IMU and ambient light sensor data to compensate the measured values ​​of the SpO2 sequence in real time, obtaining the compensated SpO2 sequence and the corresponding probability prediction result; Threshold Decision and Display Output Module: Based on the compensated probability prediction result, this module performs binarization decision according to a preset threshold and outputs the lesion location and confidence level.

7. A device for identifying minute lesions based on multi-band spectroscopy and temporal deep networks, characterized in that, The device includes a memory and a processor; the memory is used to store a computer program; the processor is used to implement, when executing the computer program, the method for identifying minute lesions based on multi-band spectroscopy and temporal deep networks as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method for identifying minute lesions based on multi-band spectroscopy and temporal deep networks as described in any one of claims 1 to 5.

Images