A tissue lesion recognition method and device based on hyperspectral imaging and electronic equipment
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIAN INST OF OPTICS & PRECISION MECHANICS CHINESE ACAD OF SCI
- Filing Date
- 2026-04-02
- Publication Date
- 2026-07-03
Smart Images
Figure CN122336486A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical imaging diagnostic technology, and in particular to a method, device, and electronic device for identifying tissue lesions based on hyperspectral imaging. Background Technology
[0002] In surgical procedures and clinical diagnosis, rapid and accurate assessment of the nature of tissue lesions is crucial for developing surgical plans, determining the extent of resection, and reducing patient risk. With the development of optical imaging technology and computer-aided intelligent analysis, hyperspectral imaging (HIS) has gradually been introduced into the field of medical testing. Hyperspectral imaging is an imaging method that acquires target reflection or transmission information across multiple continuous or quasi-continuous spectral bands. The imaging result is typically a three-dimensional data cube containing two-dimensional spatial information and one-dimensional spectral information. Because different biological tissues differ in chemical composition, cell structure, and metabolic state, they exhibit different spectral response characteristics at different wavelengths. Therefore, hyperspectral imaging is considered to provide a "spectral fingerprint" reflecting the essential characteristics of tissues. In hyperspectral medical imaging applications, traditional machine learning or deep learning methods are often combined to extract and classify features from the acquired hyperspectral data to achieve automatic identification of tissue lesions. However, its accuracy and generalization ability are relatively low. Summary of the Invention
[0003] The purpose of this invention is to provide a method, device, and electronic device for identifying tissue lesions based on hyperspectral imaging, so as to improve the accuracy and generalization ability of tissue lesion identification. The specific technical solution is as follows:
[0004] In a first aspect of this application, a method for identifying tissue lesions based on hyperspectral imaging is provided, the method comprising:
[0005] Acquire hyperspectral data of the tissue to be tested;
[0006] Feature extraction is performed along the spectral dimension of the hyperspectral data to obtain the pure spectral features of the hyperspectral data;
[0007] Feature extraction is performed on the hyperspectral data or pure spectral features along the spatial dimension to obtain the pure spatial features of the hyperspectral data;
[0008] The pure spectral features and the pure spatial features are embeddedly fused to obtain fused features;
[0009] Based on the fusion features, tissue lesions are identified in the tissue to be detected.
[0010] In one possible embodiment, the embedded fusion of the pure spectral features and the pure spatial features to obtain fused features includes:
[0011] Pixel sequences of the hyperspectral data are constructed along at least one spatial direction, wherein each pixel sequence consists of pixels in the hyperspectral data located in the same spatial direction;
[0012] Based on the pixel sequences, the contribution of each pixel in the hyperspectral data to the identification of tissue lesions is determined; wherein, the contribution is positively correlated with the correlation degree, and the correlation degree is the degree of correlation between the pixel value of each pixel in the hyperspectral data and the tissue lesion;
[0013] The pure spectral features and the pure spatial features are embedded and fused according to the degree of contribution to obtain fused features.
[0014] In one possible embodiment, the embedded fusion of the pure spectral features and the pure spatial features according to the degree of contribution to obtain fused features includes:
[0015] Calculate the joint expectation of the pure spectral features and the pure spatial features;
[0016] The fused features are obtained by weighting and summing the joint expectation and the pure spectral features according to their respective contributions.
[0017] In one possible embodiment, the at least one spatial direction is any of the following directions or combinations thereof: pixel row direction, pixel column direction, pixel scan direction.
[0018] In one possible embodiment, the step of identifying tissue lesions in the tissue to be detected based on the fusion features includes:
[0019] The contribution of each feature unit in the fusion feature to the identification of tissue lesions is determined respectively;
[0020] According to the degree of contribution, feature enhancement is performed on each feature unit in the fusion feature to obtain the enhanced fusion feature;
[0021] The enhanced fusion features are used to identify lesions in the tissue to be detected.
[0022] In one possible embodiment, the feature extraction along the spectral dimension of the hyperspectral data to obtain the pure spectral features of the hyperspectral data includes:
[0023] The hyperspectral data is convolved along the spectral dimension to obtain the reflection, absorption, and / or response characteristics of the tissue to be detected in multiple continuous or quasi-continuous spectral bands, which are then used as pure spectral features.
[0024] In a second aspect of this application, a tissue lesion identification device based on hyperspectral imaging is also provided, the device comprising:
[0025] The data acquisition module is used to acquire hyperspectral data of the tissue to be tested;
[0026] A pure spectral feature extraction module is used to extract features from the hyperspectral data along the spectral dimension to obtain the pure spectral features of the hyperspectral data;
[0027] The pure spatial feature extraction module is used to extract features from the hyperspectral data or pure spectral features along the spatial dimension to obtain the pure spatial features of the hyperspectral data.
[0028] A spatial-spectral fusion embedding module is used to perform embedded fusion of the pure spectral features and the pure spatial features to obtain fused features;
[0029] The discrimination output module is used to identify tissue lesions in the tissue to be detected based on the fused features.
[0030] In one possible embodiment, the spatial-spectral fusion embedding module performs embedded fusion of the pure spectral features and the pure spatial features to obtain fused features, including: constructing pixel sequences of the hyperspectral data along at least one spatial direction, wherein each pixel sequence is composed of pixels in the hyperspectral data located in the same spatial direction; determining the contribution degree of each pixel in the hyperspectral data to tissue lesion identification based on each pixel sequence; and performing embedded fusion of the pure spectral features and the pure spatial features according to the contribution degree to obtain fused features.
[0031] In one possible embodiment, the space-spectral fusion embedding module performs embedded fusion of the pure spectral features and the pure spatial features according to the degree of contribution to obtain fused features, including: calculating the joint expectation of the pure spectral features and the pure spatial features; and performing a weighted summation of the joint expectation and the pure spectral features according to the degree of contribution to obtain fused features.
[0032] In one possible embodiment, the at least one spatial direction is any of the following directions or combinations thereof: pixel row direction, pixel column direction, pixel scan direction.
[0033] In one possible embodiment, the device further includes an attention enhancement module, configured to determine the contribution degree of each feature unit in the fusion feature to tissue lesion identification; enhance each feature unit in the fusion feature according to the contribution degree to obtain enhanced fusion features; and the discrimination output module is specifically configured to identify lesions in the tissue to be detected based on the enhanced fusion features.
[0034] In one possible embodiment, the pure spectral feature extraction module performs feature extraction on the hyperspectral data along the spectral dimension to obtain pure spectral features of the hyperspectral data, including: performing convolution operations on the hyperspectral data along the spectral dimension to obtain the reflection, absorption and / or response features of the tissue to be detected in multiple continuous or quasi-continuous spectral bands, as pure spectral features.
[0035] In a third aspect of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;
[0036] Memory, used to store computer programs;
[0037] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect above.
[0038] This invention also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the tissue lesion identification method based on hyperspectral imaging as described in any of the first aspects above.
[0039] Beneficial effects of the embodiments of the present invention:
[0040] The present invention provides a method, device and electronic device for tissue lesion identification based on hyperspectral imaging, which can achieve spatial-spectral decoupled feature modeling by embedding pure spectral features and pure spatial features, avoid mutual interference between spatial information and spectral information, and improve the stability and generalization ability of tissue lesion identification.
[0041] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.
[0043] Figure 1 A schematic flowchart of the tissue lesion identification method based on hyperspectral imaging provided in this application;
[0044] Figure 2 Another flowchart illustrating the tissue lesion identification method based on hyperspectral imaging provided in this application;
[0045] Figure 3 This is another flowchart illustrating the tissue lesion identification method based on hyperspectral imaging provided in this application.
[0046] Figure 4 A schematic diagram of a tissue lesion identification device based on hyperspectral imaging provided in this application;
[0047] Figure 5 Another schematic diagram of the tissue lesion identification device based on hyperspectral imaging provided in this application;
[0048] Figure 6 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation
[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.
[0050] To improve the accuracy of tissue lesion identification, this invention provides a tissue lesion identification method based on hyperspectral imaging. In one possible embodiment, see... Figure 1 The methods include:
[0051] Step S101: Acquire hyperspectral data of the tissue to be detected; Step S102: Extract features from the hyperspectral data along the spectral dimension to obtain pure spectral features of the hyperspectral data; Step S103: Extract features from the hyperspectral data or pure spectral features along the spatial dimension to obtain pure spatial features of the hyperspectral data; Step S104: Perform embedded fusion of pure spectral features and pure spatial features to obtain fused features; Step S105: Identify tissue lesions based on the fused features. This embodiment, by embedding the fusion of pure spectral features and pure spatial features, achieves spatial-spectral decoupled feature modeling, avoiding mutual interference between spatial and spectral information, and improving the stability and generalization ability of tissue lesion identification.
[0052] The following will explain steps S101-S105:
[0053] In step S101 of this embodiment, hyperspectral data of the tissue to be detected can be acquired using a push-broom hyperspectral imaging method, an area array hyperspectral imaging method, or other equivalent hyperspectral imaging methods. This application does not impose any limitations on these methods. Furthermore, the hyperspectral data can be the raw data acquired or obtained after preprocessing the raw data.
[0054] Hyperspectral data can be considered as three-dimensional data. For ease of description, hyperspectral data will be denoted as X∈R. HxWxB Where H and W are the dimensions of the hyperspectral data in the pixel height and pixel width directions, respectively, and B is the number of spectral bands in the hyperspectral data.
[0055] In step S102 of this embodiment, the feature extraction method is not limited, but it should satisfy the following: the spatial position correspondence remains basically unchanged during the feature extraction process, so that the output feature corresponding to each spatial position mainly reflects the spectral fingerprint characteristics of that spatial position, rather than the spatial texture characteristics. In other words, the output feature corresponding to each spatial position is calculated only from the spectral vector of that spatial position, without involving any spatial neighborhood information.
[0056] Since the spatial correspondence remains essentially unchanged, the dimensions of the pure spectral feature in the pixel height and pixel width directions are the same as those of the hyperspectral data. Therefore, the pure spectral feature can be denoted as F. s ∈R HxWxCs , where C s This represents the number of pure spectral feature channels. Since the output feature corresponding to each spatial location is calculated solely from the spectral vector of that spatial location, for pure spectral features, the elements from (i, j, 1) to (i, j, C) are... s The value of ) is determined entirely by the elements (i, j, 1) to (i, j, B) in the hyperspectral data, and is independent of other elements, where i ∈ [1, H] and j ∈ [1, W].
[0057] Since the spatial dimension is avoided in the process of extracting pure spectral features, the independence of pure spectral features can be guaranteed. That is, pure spectral features can simply characterize spectral fingerprint characteristics without characterizing spatial texture characteristics.
[0058] The specific method for extracting pure spectral feature data will be described in the following example, and will not be repeated here.
[0059] In step S103 of this embodiment, the pure spatial features are obtained by feature extraction in the spatial dimension. For the case where the pure spatial features are obtained by feature extraction from hyperspectral data, the pure spatial features can be denoted as F. P ∈R HpxWpxB , where H P W P These represent the dimensions of the pure spatial feature along the pixel height and pixel width, respectively. For the case where the pure spatial feature is obtained by extracting pure spectral features, the pure spatial feature can be denoted as F. P ∈R HpxWpxCs .
[0060] Pure spatial features can be extracted using any of the following three methods:
[0061] Method 1: Use a convolution kernel of size K1xK1 to slide along the spatial dimensions (i.e., H and W dimensions) to convolve hyperspectral data or pure spectral features to obtain pure spatial features. K1 is any positive integer less than H and less than W.
[0062] Method 2: Extract edge features from hyperspectral data or pure spectral features using spatial edge extraction operators as pure spatial features. These operators can be the Sobel operator, the Laplacian operator, or an expression like G. x =S x Canny operator for xI.
[0063] Method 3: Region consistency modeling, which captures the region structure through dilated convolution or multi-scale convolution.
[0064] Alternatively, pure spatial features can be extracted and fused using the methods described in Method 1 to Method 3, and the fusion result can be used as the pure spatial features of the hyperspectral data.
[0065] In step S104 of this embodiment, embedded fusion refers to any fusion method that satisfies the following conditions: achieving structured fusion of spectral and spatial features while maintaining their relative independence. How to perform embedded fusion will be described exemplarily below and will not be repeated here.
[0066] In step S105 of this embodiment, the fused features may be input into any network model with the ability to identify tissue lesions for identification. The identification result may include only the discrimination result used to characterize whether the tissue to be detected is a lesion tissue, or it may include the discrimination result and the classification result used to characterize the possible lesion type of the tissue to be detected. It may also include the discrimination result, the classification result, and the spatial distribution of the lesion area and / or the probability information of the lesion.
[0067] In one possible embodiment, see Figure 2 Compared to Figure 1 The example shown, Figure 2 Step S104 is further refined into steps S1041 to S1043. Figure 2 The methods shown include:
[0068] Step S101: Acquire hyperspectral data of the tissue to be detected; Step S102: Extract features from the hyperspectral data along the spectral dimension to obtain pure spectral features of the hyperspectral data; Step S103: Extract features from the hyperspectral data or pure spectral features along the spatial dimension to obtain pure spatial features of the hyperspectral data; Step S1041: Construct pixel sequences of the hyperspectral data along at least one spatial direction, wherein each pixel sequence consists of pixels in the hyperspectral data located in the same spatial direction; Step S1042: Based on each pixel sequence, determine the contribution of each pixel in the hyperspectral data to the identification of tissue lesions; wherein the contribution is positively correlated with the correlation, and the correlation is the degree of correlation between the pixel value of each pixel in the hyperspectral data and the tissue lesion; Step S1043: Perform embedded fusion of pure spectral features and pure spatial features according to the contribution, to obtain fused features; Step S105: Identify tissue lesions in the tissue to be detected based on the fused features. In this embodiment, a direction-aware embedding mechanism is used to enable the embedded fusion process. It has clear structural semantics, which improves the interpretability of tissue lesion identification and thus improves the robustness of tissue lesion identification.
[0069] Because steps S101 to S103 and step S105 are... Figure 1 The example shown is the same, so please refer to [link / reference]. Figure 1 The relevant explanations will not be repeated here; the following text will only address... Figure 2 The detailed steps S1041 to S1043 are explained below:
[0070] In step S1041 of this embodiment, a pixel sequence can be constructed along only one spatial direction, or pixel sequences can be constructed along multiple spatial directions respectively. Spatial directions include, but are not limited to: pixel row direction, pixel column direction, and pixel scan direction. Among them, the pixel row direction corresponds to the pixel height direction, the pixel column direction corresponds to the pixel width direction, and the pixel scan direction corresponds to the diagonal direction, which are used to characterize the traversal path direction when traversing each pixel in the hyperspectral data.
[0071] When constructing a pixel sequence along the width of a pixel, a pixel sequence S is obtained. row，1 S row，2 S row，H , among which, S row，i={f(i, 1), f(i, 2), ..., f(i, W)}, where f(i, j) is the spectral data in the i-th row and j-th column of the hyperspectral data. Constructing the pixel sequence along the pixel height direction yields the pixel sequence S. col，1 S col，2 S col，H , among which, S col，i ={f(1,i), f(2,i), ..., f(H,i)}. Constructing the pixel sequence along the diagonal direction will yield the pixel sequence S. dia，1 S dia，2 S dia，W Where, when i≤W-H+1, S col，i ={f(1,i),f(2,i+1),f(3,i+2),…,f(H,i+H-1)}, when i>W-H+1, S col，i ={f(1,i),f(2,i+1),f(3,i+2),…,f(W-i+1,W)}.
[0072] In step S1042 of this embodiment, feature extraction can be performed on each pixel sequence to obtain features that characterize the contribution of different pixels in tissue lesion identification. The feature extraction method can be weighting the pixel sequence or projecting the pixel sequence.
[0073] Among them, the degree of contribution is positively correlated with the degree of correlation, which is the degree of correlation between the pixel value of each pixel in the hyperspectral data and the tissue lesion.
[0074] The higher the correlation, the better the pixel value reflects the true state of tissue lesions and the more effective the information it can provide for lesion identification.
[0075] For example, the contribution of a pixel can be judged by an attention mechanism. The contribution of a pixel can be defined by analyzing the feature consistency between a pixel and other surrounding pixels: if the feature consistency between a pixel and its surrounding pixels is higher, it indicates that the lesion-related information carried by the pixel is more representative and reliable, and thus the pixel can be considered to have a higher contribution to tissue lesion identification and feature extraction; conversely, if the consistency between a pixel and its surrounding pixels is low, it carries less effective information and its contribution is relatively low.
[0076] For weighted summation, it can be achieved through convolution. For example, for the pixel sequence obtained by constructing along the pixel width direction, a convolution kernel of size K2x1xK2 can be used to perform convolution processing on the pixel sequence to achieve weighted summation of the pixel sequence. For the pixel sequence obtained by constructing along the pixel height direction, a convolution kernel of size 1xK3xK3 can be used to perform convolution processing on the pixel sequence to achieve weighted summation of the pixel sequence.
[0077] For projection, a directional projection can be used with the formula Z=PS, where P is the direction-aware projection matrix, S is a matrix composed of pixel sequences, and Z is the projection result.
[0078] In step S1043 of this embodiment, fusion can be performed using any one of the following three methods:
[0079] Method 1: Fusion according to formula (1):
[0080] F fusion =F S +βE(F S F P … (1)
[0081] Among them, F fusion For fusion features, F S For pure spectral features, β is the weighting coefficient, and F P For pure spatial features, E() is a function representing the relationship between F and F. S and F P To perform some kind of combination, interaction, or enhancement operation.
[0082] Method 1 involves calculating the joint expectation of pure spectral features and pure spatial features, and then weighting and summing the joint expectation and pure spectral features according to their contribution to obtain the fused features. Choosing Method 1 can further improve the structural semantics of the fusion process, thereby further enhancing the robustness of tissue lesion identification.
[0083] Method 2: Fusion according to formula (2):
[0084] F fusion =Concat(F S , DirectionalEmbedding)… (2)
[0085] Among them, F fusion To fuse features, Concat() is a function that directly concatenates two feature tensors end-to-end along the channel dimension. SFor pure spectral features, DirectionalEmbedding is a feature vector representing direction, angle, and spatial orientation information.
[0086] Method 3: Fusion according to formula (3):
[0087] F fusion =Fs⊙σ(WF P … (3)
[0088] Among them, F fusion For fusion features, F S For pure spectral features, ⊙ is the operator, σ() is the activation function, W is the learning weight, and F P It is a purely spatial feature.
[0089] In one possible embodiment, see Figure 3 Compared to Figure 1 The example shown, Figure 3 The example shown refines step S105 into steps S1051 to S1053. Figure 3 The methods shown include:
[0090] Step S101: Acquire hyperspectral data of the tissue to be detected; Step S102: Extract features from the hyperspectral data along the spectral dimension to obtain pure spectral features of the hyperspectral data; Step S103: Extract features from the hyperspectral data or pure spectral features along the spatial dimension to obtain pure spatial features of the hyperspectral data; Step S104: Perform embedded fusion of pure spectral features and pure spatial features to obtain fused features; Step S1051: Determine the contribution of each feature unit in the fused features to tissue lesion identification; Step S1052: Enhance the feature units in the fused features according to their contribution levels to obtain enhanced fused features; Step S1053: Identify lesions in the tissue to be detected based on the enhanced fused features. Using this embodiment, the importance of different feature units can be adjusted, and globally associated modeling units can be established to create dependencies between different spatial locations or feature units. This adaptively focuses on feature regions and spectral segments that contribute more to lesion identification, thereby improving the accuracy of tissue lesion identification.
[0091] Because steps S101 to S104 and Figure 1 The example shown is the same, so please refer to [link / reference]. Figure 1 The relevant explanations will not be repeated here; the following text will only address... Figure 3 The detailed steps S1051 to S1053 are explained below:
[0092] In step S1051 of this embodiment, the contribution of each feature unit to the identification of tissue lesions can be determined by introducing an attention mechanism. In this application, the feature unit refers to the feature vector at a single spatial location, the feature map of a single channel, and the basic vector elements in the fused features.
[0093] In step S1052 of this embodiment, each feature unit is enhanced by attention enhancement.
[0094] In step S1053 of this embodiment, the fused features after feature enhancement can focus more on the feature regions and spectral segments that contribute more to lesion identification.
[0095] In one possible embodiment, the pure spectral features can be extracted in step S102 using any of the following four methods:
[0096] Method 1: Perform convolution operations along the spectral dimension on the hyperspectral data to obtain the reflection, absorption, and / or response features of the tissue to be detected in multiple continuous or quasi-continuous spectral bands, which are then used as pure spectral features. For example, a convolution kernel of size 1X1XK3 can be used to slide along the spectral dimension to perform convolution operations on the hyperspectral data to obtain pure spectral features.
[0097] Method 2: Use spectral difference operators to calculate the hyperspectral data to obtain pure spectral data, for example, using formula d. i，j (b) = x i，j (b+1)-x i，j (b) The pure spectral characteristics are calculated, where d i，j (b) represents the values of the pure spectral features located at (i, j, b), x i，j Let x be the spectral vector located in the i-th row and j-th column of the hyperspectral data. i，j (b) is the b-th element in the spectral vector, that is, the element located at (i, j, b+1) in the hyperspectral data.
[0098] Method 3: Utilize spectral gradients or second derivative operators to calculate enhanced absorption peaks and / or reflection valleys as pure spectral features.
[0099] Method 4: Project the hyperspectral data using a spectral projection operator to obtain pure spectral features. The projection operator can be: f i，j =Px i，j .
[0100] Using the above methods to extract pure spectral features can ensure the purity and independence of pure spectral features from a structural perspective, thereby improving the accuracy and generalization ability of tissue lesion identification.
[0101] Corresponding to the aforementioned tissue lesion identification method based on hyperspectral imaging, this application also provides a tissue lesion identification device based on hyperspectral imaging. In one possible embodiment, such as... Figure 4 As shown, the device includes:
[0102] The data acquisition module 401 is used to acquire hyperspectral data of the tissue to be detected; the pure spectral feature extraction module 402 is used to extract features from the hyperspectral data along the spectral dimension to obtain the pure spectral features of the hyperspectral data; the pure spatial feature extraction module 403 is used to extract features from the hyperspectral data or pure spectral features along the spatial dimension to obtain the pure spatial features of the hyperspectral data; the spatial-spectral fusion embedding module 404 is used to perform embedded fusion of the pure spectral features and the pure spatial features to obtain fused features; and the discrimination output module 405 is used to identify tissue lesions in the tissue to be detected based on the fused features.
[0103] In one possible embodiment, the spatial-spectral fusion embedding module performs embedded fusion of the pure spectral features and the pure spatial features to obtain fused features, including: constructing pixel sequences of the hyperspectral data along at least one spatial direction, wherein each pixel sequence consists of pixels in the hyperspectral data located in the same spatial direction; determining the contribution degree of each pixel in the hyperspectral data to tissue lesion identification based on each pixel sequence; wherein the contribution degree is positively correlated with the correlation degree, the correlation degree being the degree of correlation between the pixel value of each pixel in the hyperspectral data and the tissue lesion; and performing embedded fusion of the pure spectral features and the pure spatial features according to the contribution degree to obtain fused features.
[0104] In one possible embodiment, the space-spectral fusion embedding module performs embedded fusion of the pure spectral features and the pure spatial features according to the degree of contribution to obtain fused features, including: calculating the joint expectation of the pure spectral features and the pure spatial features; and performing a weighted summation of the joint expectation and the pure spectral features according to the degree of contribution to obtain fused features.
[0105] In one possible embodiment, the at least one spatial direction is any of the following directions or combinations thereof: pixel row direction, pixel column direction, pixel scan direction.
[0106] In one possible embodiment, the device further includes an attention enhancement module, configured to determine the contribution degree of each feature unit in the fusion feature to tissue lesion identification; enhance each feature unit in the fusion feature according to the contribution degree to obtain enhanced fusion features; and the discrimination output module is specifically configured to identify lesions in the tissue to be detected based on the enhanced fusion features.
[0107] In one possible embodiment, the pure spectral feature extraction module performs feature extraction on the hyperspectral data along the spectral dimension to obtain pure spectral features of the hyperspectral data, including: performing convolution operations on the hyperspectral data along the spectral dimension to obtain the reflection, absorption and / or response features of the tissue to be detected in multiple continuous or quasi-continuous spectral bands, as pure spectral features.
[0108] In one possible embodiment, such as Figure 5 As shown, the device includes: a hyperspectral data acquisition module 4011, used to acquire raw hyperspectral data of the tissue to be detected through push-broom, array, or other equivalent methods; a data preprocessing module 4012, used to preprocess the raw hyperspectral data to obtain hyperspectral data of the tissue to be detected; a pure spectral feature extraction module 402; a pure spatial feature extraction module 403; a spatial-spectral fusion embedding module 404; an attention enhancement module 406; and a discrimination output module 405; wherein, the hyperspectral data acquisition module 4011 and the data preprocessing module 4012 constitute... Figure 4 Data acquisition module 401 in the middle.
[0109] This invention also provides an electronic device, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.
[0110] Memory 603 is used to store computer programs;
[0111] When processor 601 executes a program stored in memory 603, it performs the following steps:
[0112] Acquire hyperspectral data of the tissue to be tested;
[0113] Feature extraction is performed along the spectral dimension of the hyperspectral data to obtain the pure spectral features of the hyperspectral data;
[0114] Feature extraction is performed on the hyperspectral data or pure spectral features along the spatial dimension to obtain the pure spatial features of the hyperspectral data;
[0115] The pure spectral features and the pure spatial features are embeddedly fused to obtain fused features;
[0116] Based on the fusion features, tissue lesions are identified in the tissue to be detected.
[0117] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.
[0118] The communication interface is used for communication between the aforementioned electronic devices and other devices.
[0119] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.
[0120] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0121] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described methods for identifying tissue lesions based on hyperspectral imaging.
[0122] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the tissue lesion identification methods based on hyperspectral imaging in the above embodiments.
[0123] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0124] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0125] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, embodiments of devices, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0126] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.
Claims
1. A method for tissue lesion identification based on hyperspectral imaging, characterized in that, The method includes: Acquire hyperspectral data of the tissue to be tested; Feature extraction is performed along the spectral dimension of the hyperspectral data to obtain the pure spectral features of the hyperspectral data; Feature extraction is performed on the hyperspectral data or pure spectral features along the spatial dimension to obtain the pure spatial features of the hyperspectral data; The pure spectral features and the pure spatial features are embeddedly fused to obtain fused features; Based on the fusion features, tissue lesions are identified in the tissue to be detected.
2. The method according to claim 1, characterized in that, The embedded fusion of the pure spectral features and the pure spatial features to obtain fused features includes: Pixel sequences of the hyperspectral data are constructed along at least one spatial direction, wherein each pixel sequence consists of pixels in the hyperspectral data located in the same spatial direction; Based on the pixel sequences, the contribution of each pixel in the hyperspectral data to the identification of tissue lesions is determined; wherein, the contribution is positively correlated with the correlation degree, and the correlation degree is the degree of correlation between the pixel value of each pixel in the hyperspectral data and the tissue lesion; The pure spectral features and the pure spatial features are embedded and fused according to the degree of contribution to obtain fused features.
3. The method according to claim 2, characterized in that, The embedded fusion of the pure spectral features and the pure spatial features according to the degree of contribution to obtain fused features includes: Calculate the joint expectation of the pure spectral features and the pure spatial features; The fused features are obtained by weighting and summing the joint expectation and the pure spectral features according to their respective contributions.
4. The method according to claim 2, characterized in that, The at least one spatial direction is any of the following directions or combinations thereof: pixel row direction, pixel column direction, pixel scan direction.
5. The method according to claim 1, characterized in that, The step of identifying tissue lesions in the tissue to be detected based on the fusion features includes: The contribution of each feature unit in the fusion feature to the identification of tissue lesions is determined respectively; According to the degree of contribution, feature enhancement is performed on each feature unit in the fusion feature to obtain the enhanced fusion feature; The enhanced fusion features are used to identify lesions in the tissue to be detected.
6. The method according to claim 1, characterized in that, The step of extracting features from the hyperspectral data along the spectral dimension to obtain the pure spectral features of the hyperspectral data includes: The hyperspectral data is convolved along the spectral dimension to obtain the reflection, absorption, and / or response characteristics of the tissue to be detected in multiple continuous or quasi-continuous spectral bands, which are then used as pure spectral features.
7. A tissue lesion identification device based on hyperspectral imaging, characterized in that, The device includes: The data acquisition module is used to acquire hyperspectral data of the tissue to be tested; A pure spectral feature extraction module is used to extract features from the hyperspectral data along the spectral dimension to obtain the pure spectral features of the hyperspectral data; The pure spatial feature extraction module is used to extract features from the hyperspectral data or pure spectral features along the spatial dimension to obtain the pure spatial features of the hyperspectral data. A spatial-spectral fusion embedding module is used to perform embedded fusion of the pure spectral features and the pure spatial features to obtain fused features; The discrimination output module is used to identify tissue lesions in the tissue to be detected based on the fused features.
8. The apparatus according to claim 7, characterized in that, The spatial-spectral fusion embedding module performs embedded fusion of the pure spectral features and the pure spatial features to obtain fused features, including: constructing pixel sequences of the hyperspectral data along at least one spatial direction, wherein each pixel sequence consists of pixels in the hyperspectral data located in the same spatial direction; determining the contribution degree of each pixel in the hyperspectral data to tissue lesion identification based on each pixel sequence; wherein the contribution degree is positively correlated with the correlation degree, the correlation degree being the degree of correlation between the pixel value of each pixel in the hyperspectral data and the tissue lesion; performing embedded fusion of the pure spectral features and the pure spatial features according to the contribution degree to obtain fused features; and / or, The space-spectral fusion embedding module performs embedded fusion of the pure spectral features and the pure spatial features according to the contribution level to obtain fused features, including: calculating the joint expectation of the pure spectral features and the pure spatial features; performing a weighted summation of the joint expectation and the pure spectral features according to the contribution level to obtain fused features; and / or, The at least one spatial direction is any or a combination of the following directions: pixel row direction, pixel column direction, pixel scan direction; and / or, The device further includes an attention enhancement module, used to determine the contribution degree of each feature unit in the fusion feature to tissue lesion identification; to enhance each feature unit in the fusion feature according to the contribution degree, thereby obtaining enhanced fusion features; the discrimination output module is specifically used to identify lesions in the tissue to be detected based on the enhanced fusion features; and / or, The pure spectral feature extraction module extracts features from the hyperspectral data along the spectral dimension to obtain pure spectral features of the hyperspectral data, including: performing convolution operations on the hyperspectral data along the spectral dimension to obtain the reflection, absorption, and / or response features of the tissue to be detected in multiple continuous or quasi-continuous spectral bands, as pure spectral features.
9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-6.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-6.