Gastric cancer detection system, method and equipment based on hyperspectral imaging and medium

Abstract

The invention belongs to the technical field of image processing, and provides a gastric cancer detection system, method, equipment and medium based on hyperspectral imaging in order to solve the problem that an existing scheme is inaccurate in early lesion area detection. Simulating structural occlusion or abnormal information elimination in the hyperspectral stomach image, and extracting spatial features of the hyperspectral stomach image; performing frequency domain feature extraction on the hyperspectral stomach image through Fourier transform and a multi-scale average pooling network; performing spectral reconstruction on correlation and background consistency between hyperspectral stomach image modeling wavebands by using image convolution, and performing spectral feature extraction; then the extracted spatial features, frequency features and spectral features are fused, high-sensitivity and high-specificity detection of a focus area can be achieved, and the method is suitable for precise recognition tasks such as early canceration.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field related to image processing, and particularly relates to a gastric cancer detection system, method, device and medium based on hyperspectral imaging. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] The current endoscopic imaging system mainly uses white light image technology for cavity observation, which has good real-time performance and navigation ability, but cannot obtain the spectral information of the tissue. In recent years, some studies have introduced multispectral imaging or near-infrared spectral imaging to enhance the lesion recognition ability, but due to the low spectral dimension and insufficient waveband information, it is still difficult to meet the accurate detection needs of early lesion areas. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the present application provides a gastric cancer detection system, method, device and medium based on hyperspectral imaging, which fuses the extracted spatial features, frequency features and spectral features, and can realize high sensitivity and high specificity detection of the lesion area, and is particularly suitable for early cancer, micro-lesion and other precise identification tasks.

[0005] In order to achieve the above purpose, the present application adopts the following technical scheme: In a first aspect, the present application provides a gastric cancer detection system based on hyperspectral imaging, comprising: A spatial feature extraction unit configured to extract spatial features of the hyperspectral stomach image by adjusting the convolution receptive field to exclude the center pixel, simulating structural occlusion or abnormal information exclusion in the hyperspectral stomach image based on the hyperspectral stomach image. A frequency feature extraction unit configured to extract frequency domain features of the hyperspectral stomach image through Fourier transform and multi-scale average pooling network. A spectral feature extraction unit configured to model the correlation and background consistency between wavebands of the hyperspectral stomach image using graph convolution to realize spectral reconstruction and spectral feature extraction. An anomaly detection unit configured to weight and fuse the extracted spatial features, frequency features and spectral features of the hyperspectral stomach image, and realize auxiliary detection of gastric cancer according to the fusion result.

[0006] In a second aspect, the present application provides a gastric cancer detection method based on hyperspectral imaging, comprising: Based on the hyperspectral stomach image, the spatial features of the hyperspectral stomach image are extracted by adjusting the convolution receptive field to exclude the center pixel, simulating structural occlusion or abnormal information exclusion in the hyperspectral stomach image. The Fourier transform and the multi-scale average pooling network are used for frequency domain feature extraction of the hyperspectral stomach image. The graph convolution is used for modeling the correlation between bands and the background consistency of the hyperspectral stomach image to realize spectrum reconstruction and spectrum feature extraction. The spatial feature, the frequency feature and the spectrum feature extracted from the hyperspectral stomach image are weightedly fused, and the fusion result is used for realizing auxiliary detection of gastric cancer.

[0007] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, and computer instructions stored in the memory and running on the processor, when the computer instructions are run by the processor, the following steps are completed: Based on the hyperspectral stomach image, the spatial feature of the hyperspectral stomach image is extracted by adjusting the convolution receptive field to exclude the central pixel, simulating the structural occlusion or abnormal information exclusion in the hyperspectral stomach image. The Fourier transform and the multi-scale average pooling network are used for frequency domain feature extraction of the hyperspectral stomach image. The graph convolution is used for modeling the correlation between bands and the background consistency of the hyperspectral stomach image to realize spectrum reconstruction and spectrum feature extraction. The spatial feature, the frequency feature and the spectrum feature extracted from the hyperspectral stomach image are weightedly fused, and the fusion result is used for realizing auxiliary detection of gastric cancer.

[0008] In a fourth aspect, the present application provides a computer readable storage medium for storing computer instructions, when the computer instructions are executed by a processor, the following steps are completed: Based on the hyperspectral stomach image, the spatial feature of the hyperspectral stomach image is extracted by adjusting the convolution receptive field to exclude the central pixel, simulating the structural occlusion or abnormal information exclusion in the hyperspectral stomach image. The Fourier transform and the multi-scale average pooling network are used for frequency domain feature extraction of the hyperspectral stomach image. The graph convolution is used for modeling the correlation between bands and the background consistency of the hyperspectral stomach image to realize spectrum reconstruction and spectrum feature extraction. The spatial feature, the frequency feature and the spectrum feature extracted from the hyperspectral stomach image are weightedly fused, and the fusion result is used for realizing auxiliary detection of gastric cancer.

[0009] The above one or more technical solutions have the following beneficial effects: In the present application, based on the hyperspectral stomach image, the central pixel is excluded by adjusting the convolution receptive field, the structural occlusion or abnormal information exclusion in the hyperspectral stomach image is simulated, and the spatial features of the hyperspectral stomach image are extracted; the frequency domain features of the hyperspectral stomach image are extracted through Fourier transform and multi-scale average pooling network; the correlation and background consistency between the bands of the hyperspectral stomach image are modeled by using graph convolution to realize spectral reconstruction, and spectral feature extraction is performed; then the spatial features, frequency features and spectral features extracted are fused, the high sensitivity and high specificity detection of the lesion area can be realized, and it is especially suitable for early cancer, microlesion and other precise identification tasks.

[0010] Advantages of the additional aspects of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings, which form a part of this specification, are included to provide a further understanding of the present application, and are incorporated by reference herein. The embodiments illustrated in the drawings are presented by way of example in explaining the present application and are not intended to limit the present application.

[0012] Figure 1 It is a schematic diagram of the overall workflow in the embodiment one of the present application; Figure 2 It is the gastric cancer detection based on hyperspectral imaging in the embodiment one of the present application; Figure 3 It is a schematic diagram of the blind spot encoder principle in the embodiment one of the present application. DETAILED DESCRIPTION

[0013] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the present application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0014] It should be noted that the terms used herein are only for the purpose of describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present application.

[0015] The embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0016] Embodiment one The present embodiment discloses a gastric cancer detection system based on hyperspectral imaging, comprising: The spatial feature extraction unit is configured to exclude the central pixel by adjusting the convolution receptive field based on the hyperspectral stomach image, simulate the structural occlusion or abnormal information exclusion in the hyperspectral stomach image, and extract the spatial features of the hyperspectral stomach image; a frequency feature extraction unit configured to perform frequency domain feature extraction on the hyperspectral stomach image through Fourier transform; a spectral feature extraction unit configured to model the correlation between wavebands and the background consistency of the hyperspectral stomach image using graph convolution to realize spectral reconstruction and perform spectral feature extraction; an anomaly detection unit configured to perform weighted fusion on the extracted spatial features, frequency features and spectral features of the hyperspectral stomach image, and realize auxiliary detection of gastric cancer according to the fusion result.

[0017] First, the hardware involved in this embodiment is described. This embodiment extends the traditional gastroscope structure in a modular manner, and adds halogen light source, multi-mode optical fiber channel, hyperspectral camera and real-time imaging module and other key components.

[0018] Among them, the halogen lamp light source is installed externally to provide continuous and stable wide-band spectral illumination (400-2500nm); the multi-mode optical fiber channel includes an incident light channel and a back-reflection light channel, the incident light channel is used to conduct halogen lamp illumination light to the ROI; the back-reflection light channel is used to collect the diffuse reflection light from the tissue surface to the imaging module; in this embodiment, the design of the optical fiber end is as follows: the inner end of the optical fiber body is equipped with a miniature focusing lens, which can select a microscopic or wide-angle field of view. This lens assembly can select different focal lengths and field angles according to the actual application scenario to realize scene-adaptive light field regulation. The lens can adopt spherical microlens, cylindrical lens or GRIN lens structure, and the integration method can be independent configuration or shared with a group of aligner structure. This design significantly improves the illumination uniformity, reduces the interference of the incident angle on the tissue reflection, optimizes the convergence efficiency of the reflected light, enhances the signal strength and spectral consistency. In addition, this micro-lens has detachable or automatic focusing characteristics, which facilitates local enhancement observation at the microscopic level for different lesion sites, provides stable and repeatable light field conditions for subsequent high-precision spectral analysis, and becomes an important structural basis for accurate hyperspectral detection of this invention.

[0019] The hyperspectral camera is located outside the body and is connected to the back-reflection light channel to obtain high-dimensional spectral data of the reflected light from the ROI; the real-time imaging module is used for navigation and target area confirmation.

[0020] The illumination and imaging module of the system adopts an external structure, significantly reducing the volume of the front-end probe, reducing patient discomfort, and improving the flexibility and adaptability of clinical operation; a continuous broadband halogen light source and a high-resolution hyperspectral camera are used to provide 400-2500nm wavelength coverage, making up for the lack of spectral information in traditional systems, and helping to quantitatively identify early features such as precancerous lesions and mucosal abnormalities; a standard halogen lamp and a high-coupling-efficiency optical fiber are used to reduce energy consumption and light loss, avoiding the redundant design and wavelength overlap problems in multi-band LED systems; a miniature focusing lens assembly is provided at the end of the multi-mode optical fiber, which has high optical coupling efficiency, can effectively suppress system light loss and background interference, and improve signal quality and spectral accuracy, providing a stable detection basis for deep tissue.

[0021] Optionally, a tungsten lamp or a wide-spectrum LED array with smoothing filter can also achieve partial wide-spectrum illumination effect.

[0022] Optionally, a variable-focus lens or a MEMS micro-reflector is integrated into the probe tip instead of an external focusing lens.

[0023] As Figure 1 shown is a flowchart of the embodiment, using a traditional imaging module to navigate and lock the region of interest (ROI) in the stomach cavity; turning off the traditional white light source and starting the external halogen lamp and hyperspectral acquisition module; the halogen light illuminates the ROI through a multi-mode optical fiber, and the reflected signal is guided to the external camera by another optical fiber; the hyperspectral camera completes data acquisition and generates a hyperspectral stomach image.

[0024] The following will be combined Figure 2 to make a detailed description of a gastric cancer detection system based on hyperspectral imaging proposed in the embodiment: In the spatial feature extraction unit, the blind point encoder excludes the center pixel by controlling the convolution receptive field to avoid abnormal pixels directly participating in reconstruction, thereby reducing the reconstruction error of abnormal areas.

[0025] The input hyperspectral stomach image is , and the blind point convolution output is: , wherein represents a blind point encoder, and the blind point encoder is used to simulate structural occlusion or abnormal information exclusion in medical images, thereby better simulating the statistical difference between lesions and backgrounds.

[0026] As Figure 3 shown, blind point coding is a special convolution method, which designs the receptive field of the convolution kernel to make the convolution center pixel, i.e., the target pixel itself, not involved in the calculation of the output. This structure can deliberately enhance the "reconstruction difficulty" of abnormal areas, making lesions such as tumors, ulcers, and necrotic tissue stand out in the residual graph, providing high-sensitivity abnormal features for subsequent analysis.

[0027] The blind point encoder of the embodiment adopts a convolution mechanism with adaptive center weight suppression, and is trained by using an adversarial training strategy, as follows. To overcome the rigidity caused by the fixed center weight of 0 in the standard convolution kernel, the embodiment proposes a convolution mechanism with adaptive center weight suppression. The standard convolution operation can be expressed as where W is the convolution kernel and X is the input feature map. In the blind point encoding, the weight W (0,0) of the center position of the convolution kernel is no longer forced to be 0, but is suppressed by a learnable parameter α, which is dynamically generated by a small attention network.

[0028] Specifically, for each center pixel position (i,j) to be calculated, the attention network analyzes the statistical characteristics of its neighborhood pixel block P (i,j), such as mean, variance, local gradient, etc. Based on these characteristics, the network outputs a suppression coefficient α (i,j) ∈ [0,1]. This coefficient is used to adjust the on-off degree of the center pixel information flow.

[0029] In the embodiment, the attention network is a lightweight feedforward neural network composed of multiple fully connected layers, and the specific implementation steps are as follows: Step 1: Neighborhood feature extraction. For any waveband of the input hyperspectral image, a neighborhood pixel block P(i,j) of size k x k is extracted with the center pixel (i,j) to be calculated as the core, for example, k=5.

[0030] Step 2: Feature vectorization. A series of pre-defined statistical and structural features are calculated from the neighborhood pixel block P(i,j), such as pixel mean, pixel variance, Laplacian operator response, and these features are concatenated with the flattened original pixel values to form a one-dimensional feature vector v(i,j).

[0031] Step 3: Network forward propagation. The feature vector v(i,j) is input to the attention network. The attention network can be composed of 2 to 3 fully connected layers, and the intermediate layers use nonlinear activation functions such as ReLU to enhance the expression ability of the model.

[0032] Step 4: Generate suppression coefficient.

[0033] The last output layer of the attention network contains only one neuron and uses the Sigmoid activation function to ensure that its output value α(i,j) falls within the interval [0, 1]. This α(i,j) is the dynamic suppression coefficient for the center pixel (i,j).

[0034] Through end-to-end training, the attention network can learn the complex mapping relationship from local image features to the optimal suppression coefficient. When the neighborhood features indicate that the region is smooth and continuous normal tissue, the attention network tends to output α(i,j) close to 1; when the neighborhood appears a sharp change, indicating that it may be a lesion boundary or abnormal structure, the network tends to output α(i,j) close to 0, thereby realizing the intelligent and adaptive control of the convolution blind spot operation.

[0035] In this embodiment, the convolution operation is adjusted accordingly:

[0036] wherein Ω represents a set of relative coordinates covered by the convolution window, is an adaptive suppression coefficient, represents a weight value of the center position of the convolution kernel W, represents a weight value of the convolution kernel W at the relative coordinates (u,v), X (i,j) represents the pixel value of the input feature map X at the spatial coordinates (i,j).

[0037] When the attention network judges that the center pixel (i,j) and its neighborhood are extremely likely to be normal background tissue, α(i,j) tends to 1, allowing the blind spot encoder to use the center information for reconstruction; otherwise, if it is judged to be a potential abnormality such as texture mutation or color anomaly, α(i,j) tends to 0, thereby realizing the "blind spot" processing of the point, forcing the blind spot encoder to rely on the surrounding background information for prediction. This adaptive mechanism enables the blind spot encoder to dynamically adjust the information flow according to the local complexity of the image content, making it more flexible and robust than the fixed weight strategy.

[0038] To improve the reconstruction ability of the blind spot encoder for complex and irregular lesions, this embodiment introduces an adversarial training framework, which includes two core modules: a reconstruction network and a mask generator.

[0039] The reconstruction network is the blind spot encoder of the adaptive convolution described above, and its goal is to accurately reconstruct the normal background features of the occluded region based on spatial information according to the input mask M and the original image X .

[0040] The mask generator is a small generative network, whose goal is to generate the "most difficult" binary mask M, so that the reconstruction error of the reconstruction network under this mask is maximized. The input of the mask generator can be the original image X or its latent features, and the output is a mask of the same size as the original image X.

[0041] The training process is divided into the following alternating steps: Step 1: Introduce consistency loss to train the reconstruction network.

[0042] In this step, the parameters of the mask generator are fixed. The process of generating the occluded image X masked is consistent with the original scheme.

[0043] The goal of the reconstruction network is not only to accurately repair the occluded area, but also to ensure that the unoccluded area remains unchanged. To this end, the original loss function L R is upgraded to a composite loss, which includes a repair loss and a consistency loss.

[0044]

[0045] where γ is a hyperparameter, for example, taking the value of 0.5, used to balance the weights of the two losses.

[0046] Repair loss:

[0047] where R() represents the output of the reconstruction network, X is the original image, X masked is the occluded image, M g is the mask, ⊙ is element-wise multiplication, and ||·||1 is the L1 norm.

[0048] Consistency loss:

[0049] The consistency loss is a newly added loss term, which is used to punish any unnecessary changes to the unoccluded area by the reconstruction network. The consistency loss calculates the difference between the reconstruction output and the original image in the unoccluded background area.

[0050] The newly added consistency loss L identity term can effectively prevent the reconstruction network from "overdoing" when repairing the lesion area, which destroys the surrounding healthy background texture. This makes the background reconstruction more faithful, providing a more stable and reliable benchmark for subsequent calculation of the residual image, i.e., the anomaly score.

[0051] Step 2: Train the mask generator.

[0052] In this step, the parameters of the reconstruction network are fixed. The goal of the mask generator is to maximize the reconstruction difficulty of the reconstruction network. To make the "imaginary enemy" lesion generated by it more realistic, the simple sparsity regularization term in the existing technology is upgraded to a structured regularization term.

[0053]

[0054] Maximize the reconstruction error term L R : minimize -LR to maximize the reconstruction difficulty of the reconstruction network.

[0055] replace the simple L1 sparsity regularizer in the prior art with a total variation loss The total variation loss is used to measure the smoothness of the image and penalize the sharp change of pixel values.

[0056]

[0057] where M g(i,j) is the pixel value of the mask at coordinates (i,j), is a hyperparameter that controls the strength of the regularizer.

[0058] The total variation loss can effectively encourage the mask generator G to generate mask regions that are spatially continuous and edge-smooth, i.e., more like "massive" real lesions, rather than a bunch of meaningless, scattered noise points. Such morphologically more realistic "simulated lesions" are crucial for training the reconstruction network and improving its generalization ability.

[0059] This embodiment adds a consistency loss in the training of the reconstruction network, ensuring the fidelity of the background. In the training of the mask generator, the total variation loss is used to replace the simple sparsity constraint, improving the realism of the simulated lesions.

[0060] The entire training process is more antagonistic and the goal is more clear. The final model will have stronger recognition ability and robustness for complex and morphologically diverse real gastric cancer lesions. At the same time, these changes do not overturn the original core framework.

[0061] In the frequency feature extraction unit, the input hyperspectral stomach image is analyzed in the frequency domain by Fourier transform, and high-frequency information is filtered out, mainly learning and reconstructing low-frequency components. In this way, the high-frequency abnormal areas such as lesions are difficult to be accurately restored and will appear as high residuals in the subsequent residual map, realizing anomaly detection.

[0062] Specifically, it includes: performing one-dimensional or two-dimensional discrete Fourier transform on each spectral channel / pixel block X: ; Constructing a low-pass mask , only retaining low-frequency components and suppressing high-frequency content:

[0063] Inverse transforming the low-frequency content back to the spatial domain to obtain the spectral frequency domain reconstruction result:

[0064] The frequency components can highlight the high-frequency information of abnormal areas such as tissue boundaries, inflammatory exudation, and bleeding.

[0065] At the same time, a multi-scale average pooling network is used in the spatial dimension to suppress the high-frequency information of the original image. In the image, the edge area of the abnormal tissue has a sharp color change. By using an average pooling network of different scales, suppression of the abnormal area in the background reconstruction network can be achieved.

[0066] The implementation is a three-branch network: 1. After downsampling, average pooling is performed, and then upsampling is performed to restore the original scale.

[0067]

[0068] where B1 represents the features of branch 1, denotes upsampling, denotes average pooling, denotes downsampling.

[0069] 2. Directly performing average pooling operation on the original image.

[0070]

[0071] 3. Deconvolution is performed on the image to expand the receptive field, and then average pooling and upsampling are used. In this way, frequency elimination can be performed in a larger context.

[0072]

[0073] where, denotes deconvolution.

[0074] The three-branch features are fused through a 1*1 convolution. After adding the spectral high-frequency suppression information, the frequency domain branch is obtained.

[0075]

[0076] where C 1×1 denotes 1*1 convolution, and Concat denotes feature concatenation.

[0077] Through the extraction of frequency domain features, the perception ability of microstructures such as ulcers, mucosal damage, lesion boundaries, etc. can be improved, and it is suitable for clinical scenarios such as minimally invasive detection and intraoperative auxiliary judgment.

[0078] In the spectral feature extraction unit, each spectral band is regarded as a node, a graph structure of the spectral domain is constructed, and the correlation and background consistency between bands are modeled by methods such as graph convolution, realizing spectral reconstruction. This method is suitable for hyperspectral data, especially when the spectral consistency of the background tissue is very strong and the spectrum of the abnormal area is mutated, the abnormality will be highlighted.

[0079] Specifically, in order to make the construction of the graph structure more in line with the actual physical and biological characteristics of the hyperspectral stomach image, the definition of the node weight and the edge weight in this embodiment is specifically designed: Each node in the graph structure strictly corresponds to a spectral band c in the hyperspectral data cube. In order to make the graph structure not only contain the information of the spectral dimension, but also perceive the spatial characteristics of each band image, this embodiment initializes a rich feature vector hc(0) for each node. The feature vector is not a single numerical value, but is composed of a variety of statistical and texture descriptors of the band image, which can better reflect its state in the actual application image.

[0080] Specifically, it can include: Statistical features: average pixel intensity, standard deviation, and energy (sum of squares of pixel values) of the band image.

[0081] Texture features: key indicators of the gray level co-occurrence matrix calculated from the band image, such as contrast, correlation, energy, and homogeneity.

[0082] Entropy features: information entropy of the band image, used to measure the complexity of the amount of information.

[0083] Therefore, the initial feature of node c is:

[0084] Wherein, represents the initial feature vector of the cth node in the graph, i.e. the cth spectral band. The superscript (0) indicates that this is the feature of the 0th layer. represents the average value of the pixels of the cth band image. represents the standard deviation of the pixels of the cth band image. 、 、 、 respectively represent various texture, statistical, information theory and entropy features calculated from the cth band image.

[0085] This design makes each node of the graph structure carry rich spatial context information of its corresponding spectral band, providing more discriminative input for subsequent graph convolution operations.

[0086] Edge weight construction strategy: the edge weight A of the graph pq represents the correlation between the spectral band p and the spectral band q. In order to accurately model the spectral consistency of normal gastric mucosa, the calculation of the edge weight should exclude the interference of potential lesion areas as much as possible. This embodiment proposes a weighting strategy based on healthy prior: Step 1: Healthy area estimation. Before calculating the correlation, a simple preprocessing is first performed on the input hyperspectral stomach image to identify the healthy background area with a high probability. This can be done by a simple threshold method (e.g., in a specific band, the reflectance of normal mucosal tissue is usually within a stable range) or a lightweight pre-trained segmentation model to generate a healthy area mask. M healthy .

[0087] Step 2: Calculate weighted correlation.

[0088] Only in M healthy On the marked pixels, the correlation between different spectral bands is calculated.

[0089] This embodiment preferably adopts two complementary similarity metrics: The Pearson correlation coefficient was used to measure the linear correlation between two bands within the healthy area.

[0090]

[0091] Among them, ρ pq represents the Pearson correlation coefficient between band p and band q, with a value between [-1, 1]; Cov() represents the calculation of the covariance of two variables; σ() represents the calculation of the standard deviation of a variable; X p , X q Represent the image data of the pth and qth spectral bands respectively, which are usually flattened into vectors; [M healthy ] represents an index operation, that is, only the healthy area mask M is selected healthy The pixel values ​​marked as true are used for calculation.

[0092] Mutual information is also estimated on healthy regions to capture nonlinear statistical dependencies, which is particularly important for analyzing complex spectral signals of biological tissues.

[0093]

[0094] Among them, I (p,q) represents the mutual information between bands p and q; P(x p , x q ) indicates that the pixel value of band p is x p And the pixel value of band q is x q The joint probability distribution of P(x p ), P(x q ) represent the edge probability distribution of pixel values ​​of bands p and q respectively. It means to sum all possible combinations of pixel values.

[0095] Step 3: Construct the adjacency matrix. Final edge weight A pq is a weighted combination of the above two correlations, and is nonlinearly transformed by a Gaussian kernel function to highlight strong correlations:

[0096] where A pq represents the element in the pth row and qth column of the adjacency matrix A, representing the edge weight between nodes p and q; exp() represents the natural exponential function; , is a hyperparameter set to balance the importance of Pearson correlation and mutual information; P pq , I(p, q) are the Pearson correlation coefficient and mutual information, respectively; τ represents the temperature coefficient, which is also a hyperparameter, used to adjust the sensitivity of the similarity measure. A smaller τ will make the similarity difference more significant, and a larger τ will make the weight distribution smoother.

[0097] The edge weight of the graph structure constructed by this method accurately reflects the internal correlation between different spectral bands under the normal background tissue. When the graph convolution network uses the constructed graph structure for spectral reconstruction, it will predict the spectral curve of each pixel based on this "healthy spectral template". Therefore, when encountering a cancerous region whose spectral features deviate from the normal mode, the model will not be able to effectively reconstruct, thus producing significant abnormal signals in the residual graph, greatly improving the specificity and sensitivity of detection.

[0098] The forward propagation of the graph convolution network is the core of the entire process. A and A are input into the GCN model for calculation. GCN is usually stacked by one or more graph convolution layers. is stacked by the initial feature vector constructed for each band before, The role of a graph convolution layer is to update the features of each node to integrate the information of its "neighbor" nodes. Its calculation formula is as follows:

[0099] where W0 represents the learnable weight matrix of the 0th layer.

[0100] In order to make the information propagate further in the graph, i.e., to make each node be able to perceive the information of indirect neighbors, multiple graph convolution layers can be stacked: the output of the first layer is used as the input of the second layer, and the output of the second layer is used as the input of the third layer.

[0101] Finally, the output of the Lth layer is obtained:

[0102] In the above formula, each layer has its own independent learnable weight matrix .

[0103] Finally, we obtain the ideal features learned using To directly generate the weight W for image reconstruction spec .

[0104] ; ; in, For the adjacency matrix constructed above, is the spectral convolution weight, is a nonlinear activation function, and the superscript T represents transposition.

[0105] Graph convolution reconstruction can effectively model the internal structural topology of tissues and enhance the expression of lesion boundaries.

[0106] Comprehensive spatial, frequency domain, and spectral reconstruction, and gated weighted fusion:

[0107] in, , by learning adaptive weight allocation, Represents spatial features, Represents frequency domain features.

[0108] The gating mechanism adaptively allocates attention weights according to the image frequency information, enhancing the ability to jointly model local mutations and regional coherence.

[0109] Abnormal score graph:

[0110] or:

[0111] Where Score represents the anomaly score; i and j represent the spatial coordinate index of the image, that is, the row and column positions of the pixel on the two-dimensional plane; k represents the spectral band index of the hyperspectral image. Hyperspectral images are composed of multiple images of different wavelengths superimposed, and k is used to identify the specific band. Represents the pixel intensity value of the original input hyperspectral image at coordinate (i, j) and the kth spectral band; It represents the pixel intensity value of the reconstructed image at coordinate (i, j) and the kth spectral band after integrating the spatial, frequency, and spectral characteristics. This value can be understood as the pixel value that "normal healthy tissue" should have at this point based on the context.

[0112] Anomaly score high score represents an anomaly, suitable for spatial positioning and visualization. The essence of the calculation is to subtract the background reconstruction image from the original image, and the residual obtained is the abnormal area, which is an abnormal area such as a lesion or inflammation in medical sense. The score map can be used as a visual reference for doctors to quickly identify lesions during surgery, and assist in lesion grading, resection boundary planning, and biopsy decision-making.

[0113] The embodiment combines blind spot structure, frequency domain analysis, graph modeling and gating mechanism to build a lesion detection system specially designed for hyperspectral medical images, which has the advantages of strong anomaly suppression, strong spatial boundary expression and high spectral differentiation, strong robustness and high differentiation, and supports accurate positioning of small lesions in complex backgrounds, and is suitable for clinical key applications such as intraoperative rapid screening, accurate positioning of lesions, and auxiliary diagnosis.

[0114] The embodiment is suitable for minimally invasive hyperspectral detection of gastrointestinal tract and other clinical sites, especially suitable for early cancer identification, intraoperative real-time navigation and lesion boundary determination, and other high-precision medical scenarios, and has good industrialization prospects and popularization value.

[0115] Embodiment two The purpose of the embodiment is to provide a gastric cancer detection method based on hyperspectral imaging, comprising: Based on the hyperspectral stomach image, the spatial features of the hyperspectral stomach image are extracted by adjusting the convolution receptive field to exclude the central pixel, simulating the structural occlusion or abnormal information exclusion in the hyperspectral stomach image; The frequency domain features of the hyperspectral stomach image are extracted by Fourier transform and multi-scale average pooling network; The spectral features are extracted by modeling the correlation and background consistency between wavebands of the hyperspectral stomach image using graph convolution for spectral reconstruction; The spatial features, frequency features and spectral features extracted from the hyperspectral stomach image are weighted and fused, and the gastric cancer auxiliary detection is realized according to the fusion result.

[0116] In more embodiments, there are also provided: An electronic device comprising a memory and a processor, and computer instructions stored on the memory and running on the processor, when the computer instructions are run by the processor, the method described in embodiment two is completed. For the sake of brevity, it will not be repeated here.

[0117] It should be understood that the processor in the embodiments can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0118] The memory can include read-only memory and random access memory, and provide instructions and data to the processor, and a portion of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0119] A computer readable storage medium for storing computer instructions, the computer instructions being executed by a processor to complete the method described in Embodiment Two.

[0120] The method in Embodiment One can be directly embodied as a hardware processor to complete, or be completed by a combination of hardware and software modules in the processor. The software modules can be located in a storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory to complete the steps of the above method in combination with the hardware. To avoid repetition, it will not be described in detail here.

[0121] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0122] The above describes the specific embodiments of the application in combination with the accompanying drawings, but is not a limitation on the protection scope of the application. Those skilled in the art should understand that various modifications or variations made by those skilled in the art on the basis of the technical solutions of the application without inventive labor are still within the protection scope of the application.

Claims

1. A gastric cancer detection system based on hyperspectral imaging, characterized in that: include: The spatial feature extraction unit is configured to extract spatial features of the hyperspectral stomach image based on the hyperspectral stomach image by adjusting the convolution receptive field and simulating structural occlusion or abnormal information exclusion in the hyperspectral stomach image; a frequency feature extraction unit configured to extract frequency domain features from the hyperspectral stomach image through Fourier transform and multi-scale average pooling network; a spectral feature extraction unit configured to utilize graph convolution to model the correlation between bands and background consistency of the hyperspectral stomach image to achieve spectral reconstruction and perform spectral feature extraction; The abnormality detection unit is configured to perform weighted fusion on the spatial features, frequency features and spectral features extracted from the hyperspectral stomach image, and implement auxiliary detection of gastric cancer based on the fusion results.

2. A gastric cancer detection system based on hyperspectral imaging according to claim 1, characterized in that: In the spatial feature extraction unit, a convolution-based blind spot encoder is used to perform a mask operation on the hyperspectral stomach image, an attention network is used to dynamically generate learnable parameters, and the learnable parameters are used to dynamically adjust the weight of the convolution kernel center position.

3. The gastric cancer detection system based on hyperspectral imaging according to claim 1, characterized in that: In the spatial feature extraction unit, a mask generator and a reconstruction network constructed by the blind spot encoder are trained based on adversarial training, a consistency loss is introduced to train the reconstruction network, and the mask generator is trained based on the consistency loss and the total variation loss, wherein the total variation loss is used to measure the smoothness of the image; the consistency loss is calculated on the unoccluded background area, and the difference between the reconstructed output and the original image.

4. The gastric cancer detection system based on hyperspectral imaging according to claim 1, characterized in that: In the spectral feature extraction unit, graph convolution is used to model the correlation between bands and background consistency of the hyperspectral stomach image to achieve spectral reconstruction and perform spectral feature extraction, specifically: Use the spectral bands in the hyperspectral data as graph nodes; Determine edge weights based on the correlation between different spectral bands and construct an adjacency matrix; The graph nodes and adjacency matrix are input into the graph convolutional network to extract spectral features.

5. The gastric cancer detection system based on hyperspectral imaging according to claim 4, characterized in that: In the spectral feature extraction unit, edge weights are determined based on the correlation between different spectral bands, and an adjacency matrix is ​​constructed, specifically: Preprocess the hyperspectral stomach image to obtain the healthy area mask; On the marked pixels of the healthy area mask, the correlation between different spectral bands is calculated based on the Pearson correlation coefficient method and mutual information respectively; The calculated Pearson correlation coefficient and mutual information between the bands are weighted and fused to determine the edge weights between the graph nodes and construct the adjacency matrix.

6. The gastric cancer detection system based on hyperspectral imaging according to claim 4, characterized in that: In the spectral feature extraction unit, the graph node includes the initial feature vector of the spectral band, the pixel average of the spectral band image, the pixel standard deviation of the spectral band image, and the texture, statistical features, information theory features and entropy features of the spectral band image.

7. The gastric cancer detection system based on hyperspectral imaging according to claim 4, characterized in that: In the abnormality detection unit, the spatial features, frequency features and spectral features extracted from the hyperspectral gastric image are weightedly fused, and the difference between the hyperspectral gastric image and the weighted fusion features is calculated to obtain the abnormal area, thereby realizing auxiliary detection of gastric cancer.

8. A gastric cancer detection method based on hyperspectral imaging, characterized in that: include: Based on hyperspectral gastric images, the spatial features of hyperspectral gastric images are extracted by adjusting the convolution receptive field to exclude the central pixel, simulating structural occlusion or abnormal information exclusion in hyperspectral gastric images; Frequency domain features of hyperspectral gastric images are extracted through Fourier transform and multi-scale average pooling network; Graph convolution is used to model the correlation between bands and background consistency of hyperspectral gastric images to achieve spectral reconstruction and extract spectral features; The spatial features, frequency features and spectral features extracted from the hyperspectral gastric image are weightedly fused, and gastric cancer auxiliary detection is achieved based on the fusion results.

9. An electronic device, characterized in that: The system comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein the computer instructions are executed by the processor to complete the following steps: Based on hyperspectral gastric images, the spatial features of hyperspectral gastric images are extracted by adjusting the convolution receptive field to exclude the central pixel, simulating structural occlusion or abnormal information exclusion in hyperspectral gastric images; Frequency domain features of hyperspectral gastric images are extracted through Fourier transform and multi-scale average pooling network; Graph convolution is used to model the correlation between bands and background consistency of hyperspectral gastric images to achieve spectral reconstruction and extract spectral features; The spatial features, frequency features and spectral features extracted from the hyperspectral gastric image are weightedly fused, and gastric cancer auxiliary detection is achieved based on the fusion results.

10. A computer-readable storage medium, characterized in that Used to store computer instructions, which, when executed by a processor, complete the following steps: Based on hyperspectral gastric images, the spatial features of hyperspectral gastric images are extracted by adjusting the convolution receptive field to exclude the central pixel, simulating structural occlusion or abnormal information exclusion in hyperspectral gastric images; Frequency domain features of hyperspectral gastric images are extracted through Fourier transform and multi-scale average pooling network; Graph convolution is used to model the correlation between bands and background consistency of hyperspectral gastric images to achieve spectral reconstruction and extract spectral features; The spatial features, frequency features and spectral features extracted from the hyperspectral gastric image are weightedly fused, and gastric cancer auxiliary detection is achieved based on the fusion results.

Images