A gastric cancer detection system, method, device and medium based on hyperspectral imaging

Abstract

The present application belongs to the technical field of image processing, in order to solve the problem that the existing scheme is not accurate in detecting early lesion area, a gastric cancer detection system, method, equipment and medium based on hyperspectral imaging are proposed, based on hyperspectral stomach image, by adjusting the convolution receptive field to exclude the center pixel, simulating the structural occlusion or abnormal information exclusion in the hyperspectral stomach image, extracting the spatial features of the hyperspectral stomach image; the frequency domain features of the hyperspectral stomach image are extracted by Fourier transform and multi-scale average pooling network; the correlation and background consistency between the bands of the hyperspectral stomach image are modeled by graph convolution to realize spectral reconstruction, and spectral feature extraction is performed; then the extracted spatial features, frequency features and spectral features are fused, which can realize high sensitivity and high specificity detection of the lesion area, and is suitable for precise identification tasks such as early canceration.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a gastric cancer detection system and method based on hyperspectral imaging, equipment and medium. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] The current endoscopic imaging system mainly uses white light image technology for gastric cavity observation, which has good real-time performance and navigation ability, but cannot obtain the spectral information of the tissue. In recent years, some studies have introduced multispectral imaging or near-infrared spectral imaging to enhance the lesion recognition ability, but due to the low spectral dimension and insufficient waveband information, it is still difficult to meet the accurate detection needs of early lesion areas. SUMMARY

[0004] To overcome the shortcomings of the prior art, the present application provides a gastric cancer detection system and method based on hyperspectral imaging, which fuses the extracted spatial features, frequency features and spectral features, and can realize high sensitivity and high specificity detection of the lesion area, and is particularly suitable for precise identification tasks such as early canceration and micro-lesions.

[0005] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a gastric cancer detection system based on hyperspectral imaging, comprising:

[0007] The spatial feature extraction unit is configured to exclude the central pixel by adjusting the convolution receptive field based on the hyperspectral gastric image, simulate the structural occlusion or abnormal information exclusion in the hyperspectral gastric image, and extract the spatial features of the hyperspectral gastric image;

[0008] The frequency feature extraction unit is configured to perform frequency domain feature extraction on the hyperspectral gastric image through Fourier transform and multi-scale average pooling network;

[0009] The spectral feature extraction unit is configured to model the correlation and background consistency between wavebands of the hyperspectral gastric image using graph convolution to realize spectral reconstruction and perform spectral feature extraction;

[0010] The anomaly detection unit is configured to weight and fuse the extracted spatial features, frequency features and spectral features of the hyperspectral gastric image, and realize auxiliary detection of gastric cancer according to the fusion result.

[0011] In a second aspect, the present application provides a gastric cancer detection method based on hyperspectral imaging, comprising:

[0012] Based on the hyperspectral stomach image, the central pixel is excluded by adjusting the convolution receptive field, structural occlusion or abnormal information exclusion in the hyperspectral stomach image is simulated, and spatial features of the hyperspectral stomach image are extracted;

[0013] Fourier transform and a multi-scale average pooling network are used for frequency domain feature extraction of the hyperspectral stomach image;

[0014] A graph convolution is used for modeling the correlation between bands and background consistency of the hyperspectral stomach image to realize spectrum reconstruction, and spectrum feature extraction is performed;

[0015] The spatial features, frequency features and spectrum features extracted from the hyperspectral stomach image are weightedly fused, and auxiliary detection of gastric cancer is realized according to the fusion result.

[0016] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, and computer instructions stored in the memory and running on the processor, when the computer instructions are run by the processor, the following steps are completed:

[0017] Based on the hyperspectral stomach image, the central pixel is excluded by adjusting the convolution receptive field, structural occlusion or abnormal information exclusion in the hyperspectral stomach image is simulated, and spatial features of the hyperspectral stomach image are extracted;

[0018] Fourier transform and a multi-scale average pooling network are used for frequency domain feature extraction of the hyperspectral stomach image;

[0019] A graph convolution is used for modeling the correlation between bands and background consistency of the hyperspectral stomach image to realize spectrum reconstruction, and spectrum feature extraction is performed;

[0020] The spatial features, frequency features and spectrum features extracted from the hyperspectral stomach image are weightedly fused, and auxiliary detection of gastric cancer is realized according to the fusion result.

[0021] In a fourth aspect, the present application provides a computer readable storage medium for storing computer instructions, when the computer instructions are executed by a processor, the following steps are completed:

[0022] Based on the hyperspectral stomach image, the central pixel is excluded by adjusting the convolution receptive field, structural occlusion or abnormal information exclusion in the hyperspectral stomach image is simulated, and spatial features of the hyperspectral stomach image are extracted;

[0023] Fourier transform and a multi-scale average pooling network are used for frequency domain feature extraction of the hyperspectral stomach image;

[0024] A graph convolution is used for modeling the correlation between bands and background consistency of the hyperspectral stomach image to realize spectrum reconstruction, and spectrum feature extraction is performed;

[0025] The spatial features, frequency features and spectral features extracted from the hyperspectral stomach image are weighted and fused, and the auxiliary detection of gastric cancer is realized according to the fusion result.

[0026] The above one or more technical solutions have the following beneficial effects:

[0027] In the application, based on the hyperspectral stomach image, the spatial features of the hyperspectral stomach image are extracted by adjusting the convolution receptive field to exclude the center pixel, simulating the structural occlusion or abnormal information exclusion in the hyperspectral stomach image; the frequency domain features of the hyperspectral stomach image are extracted by Fourier transform and multi-scale average pooling network; the spectral reconstruction is realized by modeling the correlation and background consistency between wavebands of the hyperspectral stomach image by graph convolution, and the spectral feature extraction is performed; then the extracted spatial features, frequency features and spectral features are fused, which can realize high sensitivity and high specificity detection of the lesion area, and is especially suitable for early cancer, microlesion and other precise identification tasks.

[0028] The advantages of the additional aspects of the application will be partially given in the following description, partially become obvious from the following description, or be known by the practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0029] The drawings constituting a part of the specification of the application are used to provide further understanding of the application, the illustrative embodiments of the application and the description thereof are used to explain the application, and do not constitute improper limitation on the application.

[0030] Figure 1 It is a schematic diagram of the overall workflow in embodiment one of the application;

[0031] Figure 2 It is gastric cancer detection based on hyperspectral imaging in embodiment one of the application;

[0032] Figure 3 It is a schematic diagram of the blind spot encoder principle in embodiment one of the application. DETAILED DESCRIPTION

[0033] It should be pointed out that the following detailed description is exemplary and is intended to provide further explanation of the application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the application belongs.

[0034] It should be noted that the terms used herein are only for the purpose of describing the specific embodiments, and are not intended to limit the exemplary embodiments according to the application.

[0035] The embodiments in the application and the features in the embodiments can be combined with each other without conflict.

[0036] Embodiment one

[0037] The embodiment discloses a gastric cancer detection system based on hyperspectral imaging, comprising:

[0038] The spatial feature extraction unit is configured to exclude central pixels by adjusting a convolution receptive field based on a hyperspectral stomach image, simulate structural occlusion or abnormal information exclusion in the hyperspectral stomach image, and extract spatial features of the hyperspectral stomach image;

[0039] The frequency feature extraction unit is configured to perform frequency domain feature extraction on the hyperspectral stomach image through Fourier transform;

[0040] The spectral feature extraction unit is configured to model the correlation and background consistency between wavebands of the hyperspectral stomach image by using graph convolution, realize spectral reconstruction, and perform spectral feature extraction;

[0041] The anomaly detection unit is configured to perform weighted fusion on the extracted spatial features, frequency features and spectral features of the hyperspectral stomach image, and realize auxiliary detection of gastric cancer according to a fusion result.

[0042] First, the hardware involved in the embodiment is described. The embodiment modularly extends the traditional gastroscope structure, and adds key components such as a halogen lamp light source, a multimode optical fiber channel, a hyperspectral camera and a real-time imaging module.

[0043] The halogen lamp light source is externally mounted to provide continuous and stable wide-band spectral illumination (400-2500 nm); the multimode optical fiber channel includes an incident light channel and a back-reflection light channel, the incident light channel is used to conduct halogen lamp illumination light to the ROI, and the back-reflection light channel is used to collect tissue surface diffuse reflection light to the imaging module; in the embodiment, the design of the optical fiber end is as follows: a miniature focusing lens is arranged at the inner end of the optical fiber body, and a microscopic or wide-angle field of view can be selected, the lens assembly can select different focal lengths and field angles according to actual application scenarios to realize scene-adaptive light field regulation and control. The lens can adopt a spherical microlens, a cylindrical lens or a GRIN lens structure, and the integration mode can be independent configuration or sharing of an aligner structure. This design significantly improves illumination uniformity, reduces the interference of the incident angle on tissue reflection, optimizes the convergence efficiency of reflected light, enhances signal strength and spectral consistency. In addition, the micro-lens has detachable or automatic focusing characteristics, which facilitates microscopic local enhancement observation for different lesion sites, provides stable and repeatable light field conditions for subsequent high-precision spectral analysis, and becomes an important structural basis for realizing accurate hyperspectral detection.

[0044] The hyperspectral camera is located outside the body, connected to the back-reflection light channel, and acquires high-dimensional spectral data of the ROI reflected light; the real-time imaging module is used for navigation and target area confirmation.

[0045] The illumination and imaging module of the system adopts an external structure, significantly reducing the volume of the front-end probe, reducing patient discomfort, and improving the flexibility and adaptability of clinical operation; a continuous broadband halogen light source and a high-resolution hyperspectral camera are used to provide 400-2500nm wavelength coverage, making up for the lack of spectral information in traditional systems, and helping to quantitatively identify early features such as precancerous lesions and mucosal abnormalities; a standard halogen lamp and a high-coupling-efficiency optical fiber are used to reduce energy consumption and light loss, avoiding the redundant design and wavelength overlap problems in multi-band LED systems; a miniature focusing lens assembly is provided at the end of the multi-mode optical fiber, which has high optical coupling efficiency, can effectively suppress system light loss and background interference, and improve signal quality and spectral accuracy, providing a stable detection basis for deep tissue.

[0046] Optionally, a tungsten lamp or a wide-spectrum LED array with smoothing filter can also achieve partial wide-spectrum illumination effect.

[0047] Optionally, a variable-focus lens or a MEMS micro-reflector is integrated into the probe tip instead of an external focusing lens.

[0048] As Figure 1 shown is a flowchart of the embodiment, using a traditional imaging module to navigate and lock the region of interest (ROI) in the stomach cavity; turning off the traditional white light source and starting the external halogen lamp and hyperspectral acquisition module; the halogen light illuminates the ROI through a multi-mode optical fiber, and the reflected signal is guided to the external camera by another optical fiber; the hyperspectral camera completes data acquisition and generates a hyperspectral stomach image.

[0049] The following will be combined Figure 2 to make a detailed description of a gastric cancer detection system based on hyperspectral imaging proposed in the embodiment:

[0050] In the spatial feature extraction unit, the blind spot encoder excludes the center pixel by controlling the convolution receptive field to avoid abnormal pixels directly participating in reconstruction, thereby reducing the reconstruction error of abnormal areas.

[0051] The input hyperspectral stomach image is , and the blind spot convolution output is: , where represents the blind spot encoder, which is used to simulate structural occlusion or abnormal information exclusion in medical images, thereby better simulating the statistical difference between lesions and background.

[0052] As Figure 3 shown, blind spot coding is a special convolution method that designs the receptive field of the convolution kernel to make the convolution center pixel, i.e., the target pixel itself, not involved in the calculation of the output. This structure can deliberately enhance the "reconstruction difficulty" of abnormal areas, making lesions such as tumors, ulcers, and necrotic tissue stand out in the residual graph, providing high-sensitivity abnormal features for subsequent analysis.

[0053] The blind spot encoder of the embodiment adopts an adaptive center weight suppression convolution mechanism and is trained by an adversarial training strategy, as follows:

[0054] To overcome the rigidity caused by the fixed center weight of 0 in the standard convolution kernel, the embodiment proposes an adaptive center weight suppression convolution mechanism. The standard convolution operation can be expressed as where W is the convolution kernel and X is the input feature map. In blind spot encoding, the weight W (0,0) at the center position of the convolution kernel is no longer forced to be 0, but is suppressed by a learnable parameter α, which is dynamically generated by a small attention network.

[0055] Specifically, for each center pixel position (i,j) to be calculated, the attention network analyzes the statistical characteristics of its neighborhood pixel block P (i,j), such as mean, variance, local gradient, etc. Based on these characteristics, the network outputs a suppression coefficient α (i,j) ∈ [0,1]. This coefficient is used to adjust the degree of opening and closing of the center pixel information flow.

[0056] In the embodiment, the attention network is a lightweight feedforward neural network composed of multiple fully connected layers, and the specific implementation steps are as follows:

[0057] Step 1: Neighborhood feature extraction.

[0058] For any waveband of the input hyperspectral image, take the center pixel (i,j) to be calculated as the core, and extract a neighborhood pixel block P(i,j) of size k x k, for example, k=5.

[0059] Step 2: Feature vectorization.

[0060] Calculate a series of predefined statistical and structural features from the neighborhood pixel block P(i,j), such as pixel mean, pixel variance, Laplacian operator response, and concatenate these features with the flattened original pixel values to form a one-dimensional feature vector v(i,j).

[0061] Step 3: Network forward propagation.

[0062] Input the feature vector v(i,j) into the attention network. The attention network can be composed of 2 to 3 fully connected layers, with ReLU or other nonlinear activation functions in the middle layer to enhance the model's expression ability.

[0063] Step 4: Generate suppression coefficient.

[0064] The output layer of the attention network contains only one neuron and adopts a Sigmoid activation function, ensuring that the output value a(i,j) falls within the interval [0, 1]. This a(i,j) is the dynamic suppression coefficient for the center pixel (i,j).

[0065] Through end-to-end training, the attention network can learn the complex mapping relationship from local image features to the optimal suppression coefficient. When the neighborhood features indicate that the region is smooth and continuous normal tissue, the attention network tends to output a(i,j) close to 1; when the neighborhood shows a sharp change, indicating that it may be a lesion boundary or abnormal structure, the network tends to output a(i,j) close to 0, thereby achieving intelligent and adaptive control of the convolution blind spot operation.

[0066] In this embodiment, the convolution operation is adjusted accordingly:

[0067]

[0068] where Ω represents the set of relative coordinates covered by the convolution window, is the adaptive suppression coefficient, represents the weight value of the center position of the convolution kernel W, represents the weight value of the convolution kernel W at the relative coordinates (u,v), X (i,j) represents the pixel value of the input feature map X at the spatial coordinates (i,j).

[0069] When the attention network determines that the center pixel (i,j) and its neighborhood are extremely likely to be normal background tissue, a(i,j) approaches 1, allowing the blind spot encoder to use the center information for reconstruction; otherwise, if it is determined to be a potential abnormality such as texture mutation or color anomaly, a(i,j) approaches 0, thereby implementing "blind spot" processing of the point and forcing the blind spot encoder to rely on surrounding background information for prediction. This adaptive mechanism enables the blind spot encoder to dynamically adjust the information flow according to the local complexity of the image content, making it more flexible and robust than the fixed weight strategy.

[0070] To improve the reconstruction ability of the blind spot encoder for complex and irregular lesions, this embodiment introduces an adversarial training framework, which includes two core modules: a reconstruction network and a mask generator.

[0071] The reconstruction network is the blind spot encoder of the adaptive convolution described above, and its goal is to accurately reconstruct the normal background features of the occluded region based on spatial information according to the input mask M and the original image X .

[0072] Mask generator. This is a small generator network, whose goal is to generate the "hardest" binary mask M, such that the reconstruction error of the reconstruction network under this mask is maximized. The input of the mask generator can be the original image X or its latent feature, and the output is a mask of the same size as the original image X.

[0073] The training process is divided into the following steps of alternating iterations:

[0074] Step one: Introduce consistency loss to train the reconstruction network.

[0075] In this step, the parameters of the mask generator are fixed. The process of generating the occluded image X masked is consistent with the original scheme.

[0076] The goal of the reconstruction network is not only to accurately repair the occluded area, but also to ensure that the unoccluded area remains unchanged. To this end, the original loss function L R is upgraded to a composite loss, which includes the repair loss and the consistency loss.

[0077]

[0078] where γ is a hyperparameter, for example, taking the value of 0.5, used to balance the weights of the two losses.

[0079] Repair loss:

[0080]

[0081] where R() represents the output of the reconstruction network, X is the original image, X masked is the occluded image, M g is the mask, ⊙ is the element-wise multiplication, and ||·||1 is the L1 norm.

[0082] Consistency loss:

[0083]

[0084] The consistency loss is a newly added loss term, which is used to punish any unnecessary changes to the unoccluded area by the reconstruction network. The consistency loss calculates the difference between the reconstruction output and the original image in the unoccluded background area.

[0085] The newly added consistency loss L identity term can effectively prevent the reconstruction network from "overdoing" when repairing the lesion area, which destroys the healthy background texture around it. This makes the background reconstruction more faithful, providing a more stable and reliable benchmark for subsequent calculation of the residual map, i.e., the anomaly score.

[0086] Step two: Train the mask generator.

[0087] In this step, the parameters of the reconstruction network are fixed. The goal of the mask generator is to maximize the reconstruction difficulty of the reconstruction network. In order to make the "imaginary enemy" lesions it generates more realistic, the simple sparsity regular term in the prior art is upgraded to a structured regular term.

[0088]

[0089] Maximize the reconstruction error term L R : maximize the reconstruction difficulty of the reconstruction network by minimizing -L R

[0090] Replace the simple L1 sparsity regular term in the prior art with the total variation loss . The total variation loss is used to measure the smoothness of the image and penalize the sharp change of pixel values.

[0091]

[0092] where M g(i,j) is the pixel value of the mask at coordinates (i,j), is a hyperparameter that controls the strength of the regular term.

[0093] The total variation loss can effectively encourage the mask generator G to generate mask regions that are spatially continuous and edge-smooth, i.e., more like "massive" real lesions, rather than a bunch of meaningless, scattered noise points. Such morphologically more realistic "simulated lesions" are crucial for training the reconstruction network and improving its generalization ability.

[0094] This embodiment adds a consistency loss in the training of the reconstruction network, ensuring the fidelity of the background. In the training of the mask generator, the simple sparsity constraint is replaced with the total variation loss, improving the realism of the simulated lesions.

[0095] The entire training process is more antagonistic and the goal is more clear. The final model will have stronger recognition ability and robustness for complex and morphologically diverse real gastric cancer lesions. At the same time, these changes do not overturn the original core framework.

[0096] In the frequency feature extraction unit, the input hyperspectral stomach image is analyzed in the frequency domain by Fourier transform, and high-frequency information is filtered out, mainly learning and reconstructing low-frequency components. In this way, the high-frequency abnormal areas such as lesions are difficult to be accurately restored, and will appear as high residuals in the subsequent residual map, realizing anomaly detection.

[0097] Specifically, it includes: performing one-dimensional or two-dimensional discrete Fourier transform on each spectral channel / pixel block X: ;

[0098] ​Constructing low-pass mask Only low-frequency components are reserved, and high-frequency content is suppressed:

[0099]

[0100] Inverse transform the low-frequency content back to spatial domain to get the spectral frequency domain reconstruction result:

[0101]

[0102] The frequency components can highlight the high-frequency information of abnormal areas such as tissue boundaries, inflammatory exudation, and bleeding.

[0103] At the same time, a multi-scale average pooling network is used in the spatial dimension to suppress the high-frequency information of the original image. In the image, the edge area of the abnormal tissue has a sharp color change. By using an average pooling network of different scales, suppression of the abnormal area in the background reconstruction network can be realized.

[0104] Specifically, it is a three-branch network:

[0105] 1. After downsampling, average pooling is performed, and then upsampling is performed to restore the original scale.

[0106]

[0107] wherein B1 represents the features of branch 1, denotes upsampling, denotes average pooling, denotes downsampling.

[0108] 2. Directly perform average pooling operation on the original image.

[0109]

[0110] 3. Deconvolution is performed on the image to expand the receptive field, and then average pooling and upsampling are used. In this way, frequency elimination can be performed in a larger context.

[0111]

[0112] wherein, denotes deconvolution.

[0113] The three-branch features are fused through a 1*1 convolution. After adding the spectral high-frequency suppression information, the frequency domain branch is obtained.

[0114]

[0115] wherein, C 1×1 denotes 1*1 convolution, and Concat denotes feature concatenation.

[0116] The extraction of the frequency domain features can improve the perception ability of microstructures such as ulcers, mucosal lesions, lesion boundaries and the like, and is suitable for clinical scenes such as minimally invasive detection and intraoperative auxiliary judgment.

[0117] In the spectral feature extraction unit, each spectral band is regarded as a node, a graph structure of the spectral domain is constructed, the correlation and background consistency between the bands are modeled by using a graph convolution method, and spectral reconstruction is realized. This method is suitable for hyperspectral data, and in particular, when the spectral consistency of the background tissue is very strong and the spectrum of the abnormal area is mutated, the abnormality will be highlighted.

[0118] Specifically, in order to make the construction of the graph structure more in line with the actual physical and biological characteristics of the hyperspectral stomach image, the definition of the node weight and the edge weight is specifically designed in this embodiment:

[0119] Each node in the graph structure strictly corresponds to a spectral band c in the hyperspectral data cube. In order to make the graph structure not only contain the information of the spectral dimension, but also perceive the spatial characteristics of each band image, this embodiment initializes a rich feature vector hc(0) for each node. The feature vector is not a single numerical value, but is composed of a plurality of statistical and texture descriptors of the band image, and can better reflect its state in the actual application image.

[0120] Specifically, it can include:

[0121] Statistical features: average pixel intensity, standard deviation, and energy (sum of squares of pixel values) of the band image.

[0122] Texture features: key indicators of the gray level co-occurrence matrix calculated from the band image, such as contrast, correlation, energy, and homogeneity.

[0123] Entropy features: information entropy of the band image, used to measure the complexity of the amount of information.

[0124] Therefore, the initial feature of the node c is:

[0125]

[0126] wherein, represents the initial feature vector of the cth node in the graph, i.e., the cth spectral band. The superscript (0) indicates that this is the 0th layer feature. represents the average value of the pixels of the cth band image. represents the standard deviation of the pixels of the cth band image. , , , respectively represent various texture, statistical, information theory and entropy features calculated from the cth band image.

[0127] This design makes each node of the graph structure carry its corresponding spectral band rich spatial context information, providing more discriminative input for subsequent graph convolution operations.

[0128] Edge weight construction strategy: the edge weight A of the graph pq represents the correlation between spectral band p and spectral band q. In order to accurately model the spectral consistency of normal gastric mucosa tissue, the calculation of edge weight should exclude the interference of potential lesion areas as much as possible. However, the correlation between two spectral bands is not only related to the spectral consistency of normal tissue, but also related to the spectral heterogeneity of the lesion area. Therefore, the correlation between two spectral bands should be calculated on the basis of the health area mask M

[0129] The embodiment proposes a weighting strategy based on health prior:

[0130] Step one: health area estimation.

[0131] Before calculating the correlation, a simple preprocessing is first performed on the input hyperspectral gastric image to identify the health background area with high probability. This can be completed by a simple threshold method, for example, in a specific band, the reflectivity of normal mucosa tissue is usually within a stable range, or a lightweight pre-trained segmentation model to generate a health area mask M M healthy .

[0132] Step two: calculate the weighted correlation.

[0133] Only on the pixels marked by M healthy , the correlation between different spectral bands is calculated.

[0134] The embodiment preferably adopts two complementary similarity measures:

[0135] Pearson correlation coefficient is used to measure the linear correlation between two bands in the health area.

[0136]

[0137] where ρ pq represents the Pearson correlation coefficient between band p and band q, the value is between [-1, 1]; Cov() represents the calculation of the covariance of two variables; σ() represents the calculation of the standard deviation of a variable; X p , X q represent the image data of the pth and qth spectral bands respectively, which are usually flattened into vectors; [M healthy ] represents an indexing operation, that is, only those pixel values marked as true by the health area mask M healthy are selected for calculation.

[0138] Mutual information is also estimated on the health area, which is used to capture the nonlinear statistical dependence, which is particularly important for analyzing complex biological tissue spectral signals.

[0139]

[0140] Among them, I (p,q) P(x) represents the mutual information between bands p and q; p x q ) indicates that the pixel value of band p is x p And the pixel value of band q is x q The joint probability distribution of P(x); p ), P(x q ) represent the marginal probability distributions of pixel values ​​for bands p and q, respectively. This represents summing over all possible combinations of pixel values.

[0141] Step 3: Construct the adjacency matrix.

[0142] Final edge weight A pq It is a weighted combination of the two types of correlation mentioned above, and a nonlinear transformation is performed using a Gaussian kernel function to highlight the strong correlation:

[0143]

[0144] Among them, A pq The element in the p-th row and q-th column of the adjacency matrix A represents the edge weight between nodes p and q; exp() represents the natural exponential function. , The hyperparameters set are used to weight and balance the importance of Pearson correlation and mutual information; P pq I(p,q) represents the Pearson correlation coefficient and mutual information, respectively; τ represents the temperature coefficient, which is also a hyperparameter used to adjust the sensitivity of the similarity measure. A smaller τ makes the similarity differences more significant, while a larger τ makes the weight distribution smoother.

[0145] The graph structure constructed using this method accurately reflects the intrinsic correlations between different spectral bands within normal background tissue in terms of its edge weights. When the graph convolutional network performs spectral reconstruction using this constructed graph structure, it predicts the spectral curve of each pixel based on this "healthy spectral template." Therefore, when encountering cancerous regions whose spectral features deviate from the normal pattern, the model will be unable to reconstruct effectively, resulting in significant abnormal signals in the residual map, greatly improving the specificity and sensitivity of the detection.

[0146] Forward propagation in graph convolutional networks is the core of the entire process. A is used as input and fed into the GCN model for computation. GCN is typically composed of one or more stacked graph convolutional layers. It is the initial feature vector previously constructed for each band. Stacked together

[0147] One graph convolution layer updates the feature of each node by fusing the information of its "neighbor" nodes. The formula is as follows:

[0148]

[0149] where, represents the learnable weight matrix of the 0th layer.

[0150] To make the information propagate further in the graph, i.e., to make each node aware of the information of indirect neighbors, multiple graph convolution layers can be stacked: the output of the first layer is taken as the input of the second layer, the output of the second layer is taken as the input of the third layer.

[0151] Finally, the output of the Lth layer is obtained:

[0152]

[0153] In the above formula, each layer has its own independent learnable weight matrix .

[0154] Finally, the learned ideal feature is used to directly generate the final weight W used for image reconstruction spec .

[0155] ;

[0156] ;

[0157] where, is the adjacency matrix constructed above, is the spectral convolution weight, is a nonlinear activation function, and the superscript T represents the transpose.

[0158] Through graph convolution reconstruction, the internal structure topology of the organization can be effectively modeled, and the expression of the lesion boundary can be enhanced.

[0159] Integrate spatial, frequency domain, and spectral three-way reconstruction, and perform gated weighted fusion:

[0160]

[0161] where, , through learning to adaptively assign weights, represents the spatial feature, represents the frequency domain feature.

[0162] The gating mechanism adaptively allocates attention weights according to the graph frequency information, and enhances the joint modeling capability of local mutation and regional coherence.

[0163] Abnormal score map:

[0164]

[0165] Or:

[0166]

[0167] Wherein, Score represents the abnormal score; i, j represents the spatial coordinate index of the image, that is, the row and column positions of the pixel in the two-dimensional plane; k represents the spectral band index of the hyperspectral image. The hyperspectral image is composed of multiple images of different wavelengths, and k is used to identify the specific band number; represents the pixel intensity value of the original input hyperspectral image at coordinates (i, j) and the kth spectral band; represents the pixel intensity value of the reconstructed image output after integrating the spatial, frequency and spectral features at coordinates (i, j) and the kth spectral band. This value can be understood as the pixel value that the "normal healthy tissue" should have at this point according to the context.

[0168] An abnormal score with a high score indicates an abnormality, which is suitable for spatial positioning and visualization. The essence of the calculation is to subtract the background reconstructed image from the original image to obtain the residual, that is, the abnormal area, which in medical terms is manifested as a lesion, inflammation or other abnormal area. The score map can be used as a visual reference for doctors to quickly identify lesions during surgery, assisting in lesion grading, resection boundary planning and biopsy decision-making.

[0169] The embodiment combines blind spot structure, frequency domain analysis, graph modeling and gating mechanism to build a lesion detection system specially designed for hyperspectral medical images, which has the advantages of strong abnormality suppression, strong spatial boundary expression and high spectral differentiation. It has strong robustness and high differentiation, supports accurate positioning of small lesions in complex backgrounds, and is suitable for key clinical applications such as intraoperative rapid screening, accurate positioning of lesions and auxiliary diagnosis.

[0170] The embodiment is suitable for minimally invasive hyperspectral detection of various clinical sites such as the gastrointestinal tract, especially suitable for early cancer identification, intraoperative real-time navigation and lesion boundary determination in high-precision medical scenarios, and has good industrialization prospects and promotional value.

[0171] Embodiment two

[0172] The purpose of the embodiment is to provide a gastric cancer detection method based on hyperspectral imaging, comprising:

[0173] Based on the hyperspectral stomach image, the spatial features of the hyperspectral stomach image are extracted by adjusting the convolution receptive field to exclude the central pixel, simulating the structural occlusion or abnormal information exclusion in the hyperspectral stomach image.

[0174] The frequency domain features of the hyperspectral stomach image are extracted by Fourier transform and multi-scale average pooling network.

[0175] The spectral features are extracted by modeling the correlation and background consistency between bands of the hyperspectral stomach image using graph convolution for spectral reconstruction.

[0176] The spatial features, frequency features and spectral features extracted from the hyperspectral stomach image are fused by weighting, and the gastric cancer auxiliary detection is realized according to the fusion result.

[0177] In more embodiments, there are also provided:

[0178] An electronic device includes a memory and a processor, and computer instructions stored in the memory and running on the processor, when the computer instructions are run by the processor, the method described in embodiment two is completed. For brevity, it will not be described here.

[0179] It should be understood that in the embodiments, the processor can be a central processing unit CPU, and the processor can also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, ready-to-program gate arrays FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0180] The memory can include read-only memory and random access memory, and provide instructions and data to the processor, and a part of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0181] A computer readable storage medium for storing computer instructions, when the computer instructions are executed by the processor, the method described in embodiment two is completed.

[0182] The method in embodiment one can be directly embodied as hardware processor execution completion, or executed by hardware and software module combination in the processor. The software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register, etc. The storage medium is mature in the art. The storage medium is located in the memory, and the processor reads the information in the memory, and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0183] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the present embodiment can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software manner depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0184] Although the specific embodiments of the present application are described above in combination with the drawings, it is not a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications or variations made by those skilled in the art on the basis of the technical solutions of the present application without creative labor are still within the scope of protection of the present application.

Claims

1. A hyperspectral imaging-based gastric cancer detection system, characterized in that, The method comprises the following steps: A spatial feature extraction unit is configured to exclude the center pixel by adjusting the convolution receptive field based on the hyperspectral stomach image, simulate structural occlusion or abnormal information exclusion in the hyperspectral stomach image, and extract the spatial features of the hyperspectral stomach image; A frequency feature extraction unit is configured to perform frequency domain feature extraction on the hyperspectral stomach image through Fourier transform and a multi-scale average pooling network; A spectral feature extraction unit is configured to model the correlation and background consistency between wavebands of the hyperspectral stomach image by using graph convolution to realize spectral reconstruction and perform spectral feature extraction; An anomaly detection unit is configured to weight and fuse the spatial features, frequency features and spectral features extracted from the hyperspectral stomach image, and realize auxiliary detection of gastric cancer according to the fusion result.

2. The hyperspectral imaging-based gastric cancer detection system of claim 1, wherein, In the spatial feature extraction unit, a blind spot encoder based on convolution is used to perform mask operation on the hyperspectral stomach image, an attention network is used to dynamically generate learnable parameters, and the learnable parameters are used to dynamically adjust the weight of the center position of the convolution kernel.

3. The hyperspectral imaging-based gastric cancer detection system of claim 1, wherein, In the spatial feature extraction unit, the mask generator and the reconstruction network constructed by the blind spot encoder are trained based on adversarial training, a consistency loss is introduced to train the reconstruction network, and the mask generator is trained based on the consistency loss and a total variation loss, wherein the total variation loss is used to measure the smoothness of the image; the consistency loss is calculated based on the difference between the reconstruction output and the original image in the unoccluded background area.

4. The hyperspectral imaging based gastric cancer detection system of claim 1, wherein, In the spectral feature extraction unit, the correlation and background consistency between wavebands of the hyperspectral stomach image are modeled by using graph convolution to realize spectral reconstruction and perform spectral feature extraction, specifically: The spectral wavebands in the hyperspectral data are taken as graph nodes; The edge weights are determined based on the correlation between different spectral wavebands to construct an adjacency matrix; The graph nodes and the adjacency matrix are input into a graph convolution network to extract spectral features.

5. The hyperspectral imaging based gastric cancer detection system as claimed in claim 4, wherein, In the spectral feature extraction unit, the edge weights are determined based on the correlation between different spectral wavebands to construct an adjacency matrix, specifically: The hyperspectral stomach image is preprocessed to obtain a healthy region mask; The correlation between different spectral wavebands is calculated based on the Pearson correlation coefficient method and mutual information respectively on the labeled pixels of the healthy region mask; The Pearson correlation coefficient and mutual information between the calculated wavebands are weighted and fused to determine the edge weights between the graph nodes and construct an adjacency matrix.

6. The hyperspectral imaging based gastric cancer detection system as claimed in claim 4, wherein, In the spectral feature extraction unit, the graph nodes include the initial feature vector of the spectral waveband, the pixel average value of the spectral waveband image, the pixel standard deviation of the spectral waveband image, and the texture, statistical feature, information theory feature and entropy feature of the spectral waveband image.

7. The hyperspectral imaging based gastric cancer detection system as claimed in claim 4, wherein, In the anomaly detection unit, the spatial features, frequency features and spectral features extracted from the hyperspectral stomach image are weighted and fused, the difference between the hyperspectral stomach image and the weighted fusion features is calculated to obtain an abnormal region, and auxiliary detection of gastric cancer is realized. 8.A gastric cancer detection method based on hyperspectral imaging, characterized in that, The method comprises the following steps: Based on the hyperspectral stomach image, the spatial features of the hyperspectral stomach image are extracted by adjusting the convolution receptive field to exclude the central pixel, simulating structural occlusion or abnormal information exclusion in the hyperspectral stomach image; The frequency domain features of the hyperspectral stomach image are extracted through Fourier transform and a multi-scale average pooling network; The spectral features are extracted by modeling the correlation between bands and background consistency of the hyperspectral stomach image using graph convolution to realize spectral reconstruction; The spatial features, frequency features and spectral features extracted from the hyperspectral stomach image are weighted and fused, and the auxiliary detection of gastric cancer is realized according to the fusion result.

9. An electronic device, comprising: The computer instructions stored on the memory and run on the processor, when the processor runs, complete the following steps: Based on the hyperspectral stomach image, the spatial features of the hyperspectral stomach image are extracted by adjusting the convolution receptive field to exclude the central pixel, simulating structural occlusion or abnormal information exclusion in the hyperspectral stomach image; The frequency domain features of the hyperspectral stomach image are extracted through Fourier transform and a multi-scale average pooling network; The spectral features are extracted by modeling the correlation between bands and background consistency of the hyperspectral stomach image using graph convolution to realize spectral reconstruction; The spatial features, frequency features and spectral features extracted from the hyperspectral stomach image are weighted and fused, and the auxiliary detection of gastric cancer is realized according to the fusion result.

10. A computer-readable storage medium, characterized in that, The computer instructions for storing are executed by the processor, and the following steps are completed: Based on the hyperspectral stomach image, the spatial features of the hyperspectral stomach image are extracted by adjusting the convolution receptive field to exclude the central pixel, simulating structural occlusion or abnormal information exclusion in the hyperspectral stomach image; The frequency domain features of the hyperspectral stomach image are extracted through Fourier transform and a multi-scale average pooling network; The spectral features are extracted by modeling the correlation between bands and background consistency of the hyperspectral stomach image using graph convolution to realize spectral reconstruction; The spatial features, frequency features and spectral features extracted from the hyperspectral stomach image are weighted and fused, and the auxiliary detection of gastric cancer is realized according to the fusion result.

Images