Gastroscope image processing method and system based on tensor decomposition
Through the tensor decomposition method, the multi-modal features of gastroscopy images are extracted using the high-order singular value decomposition algorithm and local maximum pooling, which solves the problem of poor image processing effect in traditional methods and improves the type of gastric lesion Diagnostic accuracy and computational efficiency.
Patent Information
- Application Number
- CN202411595314.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-11
- Publication Date
- 2024-12-06
AI Technical Summary
Traditional gastroscopic image processing methods have limited effects when processing high-noise images, which may amplify noise, reduce image quality, and make it difficult to extract representative and distinctive features, resulting in low diagnostic accuracy.
Using a tensor decomposition method, the gastroscopy image is converted into a three-dimensional tensor through a high-order singular value decomposition algorithm, low-rank approximate tensors in different modes are extracted, and local maximum pooling and multi-modal feature fusion are performed to generate fusion Feature tensor input classifier to identify gastric lesion type.
Effectively extracting representative and distinctive features, improving the accuracy of identification of gastric lesion types, reducing data dimensions, improving computing efficiency, and enhancing the richness and robustness of features.
Smart Images

Figure CN119090894A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a gastroscopic image processing method and system based on tensor decomposition. Background Art
[0002] Gastroscopy is an important means of screening gastric diseases. There are mainly the following traditional gastroscopy image processing methods: First, grayscale and histogram equalization, that is, by converting color images into grayscale images and adjusting the histogram distribution of the image to improve the contrast of the image. The technical problem with this method is that it has limited effect when processing high-noise images, and may amplify the noise and reduce the image quality; second, filtering technology, that is, using various filters (such as Gaussian filtering, median filtering, etc.) to reduce the noise in the image. The technical problem with this method is that the filter may blur the details and edge information in the image while removing the noise, affecting the extraction of features; third, edge detection algorithm: such as the Canny edge detection algorithm, which is used to extract edge information in the image for analysis. The technical problem with this method is that under high noise and low contrast conditions, the edge detection algorithm is prone to produce false edges or miss real edges.
[0003] In addition, when analyzing the processed images, manual feature extraction is mainly used, that is, by designing specific feature descriptors (such as SIFT, HOG, etc.), the key features in the image are manually extracted. The technical problem with this method is that the manual feature extraction method relies on preset feature templates or rules, and it is difficult to adapt to the complex and changeable background and noise in gastroscopic images; there is also a template matching method, that is, using a pre-defined template to match the image to identify specific structures or lesions. The technical problem with this method is that the template matching method is sensitive to image rotation, scaling and deformation, and it is difficult to handle diverse lesion morphologies.
[0004] In summary, due to the presence of a large amount of visual noise, illumination changes, and complex tissue structures in gastroscopic images, traditional methods often encounter challenges in feature extraction and find it difficult to effectively distinguish normal tissue from lesion areas. In addition, image features of different modalities (such as position information, color information, etc.) are usually processed independently and lack an effective integration mechanism, resulting in limited feature expression capabilities. It is often difficult to extract representative and discriminative features, resulting in low diagnostic accuracy. Summary of the invention
[0005] Based on this, the present application proposes a gastroscopic image processing method and system based on tensor decomposition, aiming to extract representative and discriminative features to improve the accuracy of diagnosis.
[0006] The first aspect of the present application provides a gastroscopic image processing method based on tensor decomposition, the method comprising: Performing a preprocessing operation on the target gastroscopic image, and converting the preprocessed target gastroscopic image into a three-dimensional tensor; Based on the three-dimensional tensor, a high-order singular value decomposition algorithm is used to obtain low-rank approximate tensors in different modes, wherein the different modes are different dimensions in the three-dimensional tensor, and the high-order singular value decomposition algorithm includes obtaining and adaptively truncating a core tensor and a factor matrix, wherein the obtaining of the core tensor and the factor matrix is used to obtain a core tensor in each mode, and the adaptive truncation is used to extract the main characteristic components in each mode; Performing local maximum pooling on each of the low-rank approximate tensors to obtain an enhanced feature vector in each mode; Performing multi-mode feature fusion on the plurality of enhanced feature vectors to obtain a fused feature tensor; The fused feature tensor is input into a trained classifier to obtain a gastric lesion type recognition result corresponding to the target gastroscopic image.
[0007] As an optional implementation manner of the first aspect, the process of obtaining the core tensor and the factor matrix includes: Expanding the three-dimensional tensor into a matrix form along different modes to calculate the covariance matrix in each mode; Performing SVD decomposition on the covariance matrix to obtain an orthogonal basis matrix and a diagonal matrix containing singular values in each mode; Based on the orthogonal basis matrix, the three-dimensional tensor is projected to obtain the core tensor in each mode.
[0008] As an optional implementation manner of the first aspect, the processing process of the adaptive truncation includes: According to the singular values in each mode, an adaptive threshold selection strategy is used to determine the main characteristic components to be retained; The adaptive threshold selection strategy is specifically as follows: calculating the cumulative contribution rate of the singular values, and retaining the singular values whose cumulative contribution rate reaches a preset threshold as the retained main characteristic components; Wherein, a threshold is set according to the core tensor to obtain the preset threshold.
[0009] As an optional implementation of the first aspect, the calculation formula of the local maximum pooling is: in, represents the coordinates in the feature map after pooling, Represents a position in the feature map after pooling The value of Represents a position in the original feature map The value of Indicates by location The pooling window centered at Represents the operation of taking the maximum value, which is used to obtain the maximum value from the pooling window All locations in Select the largest value.
[0010] As an optional implementation manner of the first aspect, the multimodal feature fusion includes a fusion model based on an attention mechanism, and the processing process of the fusion model based on the attention mechanism includes: Inputting the enhanced feature vectors in different modes into the fusion model based on the attention mechanism to calculate the attention score; Normalizing the attention scores to obtain the weights of each mode; The weights of the various modes are weighted summed to obtain the fused feature tensor.
[0011] As an optional implementation of the first aspect, the formula for calculating the attention score is: in, represents the attention score of the mth pattern, M represents the number of patterns, represents the feature tensor, It means expanding the three-dimensional feature tensor into a one-dimensional vector. represents the weight matrix, represents the bias term, ReLU represents the activation function; The formula for normalizing the attention score is: in, represents the normalized attention score of the m-th mode, Represents the attention score Perform exponential operations, represents the exponential sum of attention scores of all modes; The formula for the weighted summation is: in, Represents the fused feature tensor.
[0012] As an optional implementation of the first aspect, the step of performing a preprocessing operation on the target gastroscopic image includes: Converting the target gastroscope image into a grayscale image, and performing denoising on the grayscale image using a median filter to remove salt and pepper noise; The denoised grayscale image is smoothed using Gaussian filtering, and then the contrast is enhanced using adaptive histogram equalization; The grayscale image after contrast enhancement is converted back into a color image, and the color image is the preprocessed gastroscopic image.
[0013] A second aspect of the present application provides a gastroscopic image processing system based on tensor decomposition, the system comprising: A tensor building module, used for performing a preprocessing operation on the target gastroscopic image, and converting the preprocessed target gastroscopic image into a three-dimensional tensor; A tensor decomposition module is used to obtain low-rank approximate tensors in different modes based on the three-dimensional tensor using a high-order singular value decomposition algorithm, wherein the different modes are different dimensions in the three-dimensional tensor, and the high-order singular value decomposition algorithm includes obtaining and adaptively truncating a core tensor and a factor matrix, wherein the obtaining of the core tensor and the factor matrix is used to obtain a core tensor in each mode, and the adaptive truncation is used to extract the main characteristic components in each mode; A feature extraction module, used for performing local maximum pooling on each of the low-rank approximate tensors to obtain an enhanced feature vector in each mode; A feature fusion module, used for performing multi-mode feature fusion on a plurality of the enhanced feature vectors to obtain a fused feature tensor; The classification and recognition module is used to input the fused feature tensor into a trained classifier to obtain a gastric lesion type recognition result corresponding to the target gastroscopic image.
[0014] The third aspect of the present application provides a computer device, which includes a memory, a processor, and a processing program stored in the memory and executable on the processor, wherein the processing program implements the above-mentioned gastroscopic image processing method based on tensor decomposition when executed by the processor.
[0015] A fourth aspect of the present application provides a storage medium, on which a processing program is stored. When the processing program is executed by a processor, the above-mentioned gastroscopic image processing method based on tensor decomposition is executed.
[0016] Compared with the prior art, the beneficial effects of the present invention include: 1. The target gastroscopic image usually has the characteristics of high-dimensional data. The three-dimensional tensor and high-order singular value decomposition algorithm can be used to effectively process high-dimensional data and extract meaningful features; 2. By converting the target gastroscopic image into a three-dimensional tensor and using a high-order singular value decomposition algorithm for dimensionality reduction processing, the dimension of the data can be reduced, which helps to improve computing efficiency and reduce storage requirements; 3. By using a high-order singular value decomposition algorithm and a local maximum pooling method, the main feature components of the target gastroscopic image in different modes can be extracted, and these features are fused into a fused feature tensor. The classifier can better learn and understand these features, thereby improving the accuracy of gastric lesion type recognition; 4. Adaptive truncation is used to extract the main feature components in each mode, and a suitable truncation point can be automatically selected according to the characteristics of the data, so as to better capture the main information in the data; 5. By fusing the enhanced feature vectors in different modes, a fused feature tensor that integrates multi-dimensional information can be obtained. This fusion helps to improve the classifier's ability to recognize gastric lesion types because it integrates feature information from different dimensions and increases the richness and robustness of the features.
[0017] Additional aspects and advantages of the present application will be given in part in the following description, and in part will become apparent from the following description, or will be understood through the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 A flowchart of a gastroscopic image processing method based on tensor decomposition proposed in the first embodiment of the present application; Figure 2 A schematic diagram of the structure of a gastroscopic image processing system based on tensor decomposition proposed in the second embodiment of the present application.
[0019] The following specific implementation methods will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION
[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0021] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.
[0022] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.
[0023] See also Figure 1 , which is a flow chart of a gastroscopic image processing method based on tensor decomposition proposed in the first embodiment of the present application. When implemented, the present invention mainly includes six steps: image preprocessing, tensor construction, tensor decomposition, feature extraction, multimodal feature fusion, and classification and recognition. Each step is explained in detail as follows.
[0024] S01: Preprocessing the target gastroscope image.
[0025] Specifically, the steps of preprocessing the target gastroscope image include: Converting the target gastroscope image into a grayscale image, and performing denoising on the grayscale image using a median filter to remove salt and pepper noise; The denoised grayscale image is smoothed using Gaussian filtering, and then the contrast is enhanced using adaptive histogram equalization; The grayscale image after contrast enhancement is converted back into a color image, and the color image is the preprocessed gastroscopic image.
[0026] Among them, image preprocessing includes denoising, contrast enhancement and other preprocessing operations on the target gastroscopic image, mainly to remove obviously problematic photos to reduce the impact of erroneous data, and to enhance the characteristics of the photos by enhancing the contrast so as to better perform the next step of tensor construction and tensor decomposition. In this specific embodiment, a multi-step denoising and enhancement scheme is adopted, that is, firstly, a median filter is used to remove salt and pepper noise, then a Gaussian filter is applied to smooth the image, and finally an adaptive histogram equalization (CLAHE) is performed to enhance the contrast. Such a design can better retain image details while removing noise, and significantly improve the contrast, which is very suitable for the preprocessing of gastroscopic images. The median filter, Gaussian filter, and adaptive histogram equalization mentioned in the above scheme are all existing technologies in image preprocessing, which are widely used in various image preprocessing and will not be repeated here.
[0027] S02: Convert the preprocessed target gastroscopic image into a three-dimensional tensor.
[0028] It can be understood that in medical image processing, especially for gastroscopic images, constructing a three-dimensional tensor can effectively retain and utilize the multi-dimensional information of the image. In the S02 tensor construction process of the present invention, the preprocessed image is converted into a three-dimensional tensor representation, wherein the first two dimensions represent spatial coordinates, and the third dimension can be different color channels (RGB color image). Assuming there is a preprocessed gastroscopic image with a size of 256×256 pixels (i.e., width W=256, height H=256), and an RGB color mode, when the tensor is constructed, since the image is in RGB format, it naturally has three color channels: red (R), green (G), and blue (B). Considering each color channel as a two-dimensional matrix, the entire image can be represented as a three-dimensional tensor: ; Where C=3 represents the number of color channels, the three-dimensional tensor Can write , where i=1,2,...,H; j=1,2,...,W; k=1,2,3; i, j, k correspond to height, width and color channel respectively; for a specific position The pixel point has its value on the red channel represented by X(i,j,1), its value on the green channel represented by X(i,j,2), and its value on the blue channel represented by X(i,j,3).
[0029] For ease of understanding, an example is given below: Here we select a simplified 2×2 pixel RGB image as a demonstration. In actual applications, a larger resolution (256×256) range will be used. In this example, the image is as follows: Pixel (1,1): R=200, G=100, B=50; Pixel (1,2): R=150, G=75, B=25; Pixel (2,1): R=100, G=50, B=25; Pixel (2,2): R=50, G=25, B=10.
[0030] The corresponding three-dimensional tensor X is expressed as: In this way, we get a tensor of shape 2×2×3, which contains the color information of all pixels. When analyzing gastroscopy images, such a three-dimensional tensor structure allows us to consider both spatial position information (through i and j) and color information (through different k) at the same time. During lesion detection, certain types of lesions may be more obvious in a certain color channel; or different types of tissues may have unique texture features, which can be further mined through tensor decomposition technology. In this way, doctors or algorithms can more accurately distinguish healthy tissue from abnormal areas, thereby improving the accuracy of diagnosis.
[0031] S03: Based on the three-dimensional tensor, a high-order singular value decomposition algorithm is used to obtain low-rank approximate tensors in different modes.
[0032] It should be noted that the different modes are different dimensions in the three-dimensional tensor, and the high-order singular value decomposition algorithm includes the acquisition and adaptive truncation of core tensors and factor matrices, wherein the acquisition of the core tensor and factor matrix is used to obtain the core tensor under each mode, and the adaptive truncation is used to extract the main characteristic components under each mode.
[0033] After completing the above S02 tensor construction, this application uses an improved version of the high-order singular value decomposition (HOSVD) tensor decomposition technology to perform S03 tensor decomposition, decomposing a high-dimensional data set into the product of several low-rank tensors to reduce the data dimension and extract key features, and better retain important information. The improved HOSVD algorithm steps include: S031: Acquisition of core tensors and factor matrices.
[0034] The three-dimensional tensor is expanded into a matrix form along different modes to calculate the covariance matrix under each mode; the covariance matrix is decomposed by SVD to obtain an orthogonal basis matrix and a diagonal matrix containing singular values under each mode; based on the orthogonal basis matrix, the three-dimensional tensor is projected to obtain a core tensor under each mode.
[0035] It can be understood as: given a three-dimensional tensor , that is, the three-dimensional tensor obtained in S02 above ; Calculate the covariance matrix for each mode , , (It indicates three different expansion operations, which are abbreviated symbols to distinguish them, corresponding to the specific expansion forms that appear below: ) and perform standard SVD decomposition to obtain the orthogonal basis matrix under each mode , , ; Through these orthogonal basis matrices, the original three-dimensional tensor Perform projection to obtain the core tensor .
[0036] S032: Adaptive truncation.
[0037] According to the singular values in each mode, an adaptive threshold selection strategy is adopted to determine the main characteristic components to be retained; wherein, the adaptive threshold selection strategy is specifically: calculating the cumulative contribution rate of the singular values, and retaining the singular values whose cumulative contribution rate reaches a preset threshold as the main characteristic components to be retained; wherein, the threshold is set according to the core tensor to obtain the preset threshold.
[0038] In this step, for each pattern's singular value, an adaptive threshold selection method is used to decide how many main characteristic components to retain. This step is an optimization for the characteristics of medical images, with the goal of reducing computational complexity while maintaining sufficient information.
[0039] S033: Reconstruction and dimensionality reduction.
[0040] The tensor is reconstructed using the selected main eigencomponents to achieve dimensionality reduction.
[0041] For ease of understanding, an example is given below: First, in the above steps, regarding the calculation of the covariance matrix, for a three-dimensional tensor , we first need to expand it into matrix form along each mode: Mode-1 expansion: , that is, to expand the tensor along the first dimension; Mode-2 expansion: , that is, to expand the tensor along the second dimension; Mode-3 expansion: , that is, expanding the tensor along the third dimension.
[0042] Next, calculate the covariance matrix for each mode: Mode-1 covariance matrix: ; Mode-2 covariance matrix: ; Mode-3 covariance matrix: .
[0043] Next, perform SVD decomposition on each covariance matrix to obtain an orthogonal basis matrix and singular value vectors: For A, we have ,in is an orthogonal basis matrix, is a diagonal matrix containing singular values; For B, we have ,in is an orthogonal basis matrix, is a diagonal matrix containing singular values; For C, we have ,in is an orthogonal basis matrix, is a diagonal matrix containing the singular values.
[0044] , , They are , , The transposed matrix of , , are orthogonal basis matrices, and their transpose is also their inverse matrix, that is, , , .
[0045] Next, the core tensor is calculated by the following formula : here represents tensor multiplication along the nth dimension. Specifically, if is an N-order tensor, is a matrix, then The result is a new N-th order tensor whose nth dimension is , the other dimensions remain unchanged.
[0046] The specific calculation process of the core tensor G is as follows: Project along mode-1: , this step transforms X along the first dimension with Perform tensor multiplication; Project along mode-2: This step combines the result of the previous step along the second dimension with Perform tensor multiplication; Project along mode-3: , this step combines the result of the previous step along the third dimension with Perform tensor multiplication and finally obtain the core tensor G.
[0047] In this way, the core tensor G retains the original three-dimensional tensor The main characteristic components of the image are usually of small size, which is convenient for subsequent feature extraction and analysis.
[0048] Next, we perform an adaptive truncation step, which is to decide which components to retain based on the size of the singular values. Usually, we choose to retain the first few singular values whose cumulative contribution rate reaches a certain threshold (such as 90% or 95%). Suppose the selected ranks are , , , we can construct a low-rank approximate tensor as follows: here is the front part taken from G , , A subtensor consisting of elements.
[0049] As a brief example, consider the simplified 2×2×3 tensor constructed in the previous step: : After HOSVD decomposition, we assume that we get the following result (we can use the corresponding existing numerical software MATLAB for actual calculation), that is, we calculate the orthogonal basis matrix , , , the diagonal matrix of singular values , , , and then calculate the core tensor G value. To simplify the explanation, it is assumed that only the first two main components of each mode are retained after analysis (that is, the two largest singular values are selected), that is = = =2, then a new low-rank approximate tensor can be reconstructed In practical applications, through HOSVD decomposition, we can effectively separate the most important feature components from gastroscopy images. For example, if the color distribution of the lesion area has a specific pattern, then this pattern is likely to be included in the main feature components. In the subsequent feature extraction process, these components can help doctors or automatic diagnosis systems to more accurately identify abnormal areas. In addition, dimensionality reduction can also accelerate the subsequent processing process and improve the efficiency of the overall system.
[0050] S04: Perform local maximum pooling on each of the low-rank approximate tensors to obtain an enhanced feature vector in each mode.
[0051] After completing the above S03 tensor decomposition, enter the S04 feature extraction stage to process the tensor based on HOSVD decomposition, determine the important features by using an adaptive threshold selection strategy, and strengthen these feature points through the local maximum pooling technology. The following are detailed steps and formula descriptions: S041: Adaptive threshold selection strategy.
[0052] The threshold is set according to the contribution rate of the singular values in each mode, and the feature components whose cumulative contribution rate reaches a certain proportion (for example, 90%) are retained.
[0053] Calculate the cumulative contribution rate ( ) is as follows: in, is the i-th singular value, k is the number of selected features, and N is the total number of singular values.
[0054] S042: Local maximum pooling.
[0055] Local maximum pooling can enhance feature points and improve the discriminative power of feature vectors. Apply local maximum pooling operations on selected important feature maps to highlight the strongest response in the local area. Define the pooling window: usually a small fixed-size window, such as 2×2 or 3×3.
[0056] Local maximum pooling: in, represents the coordinates in the feature map after pooling, Represents a position in the feature map after pooling The value of Represents a position in the original feature map The value of Indicates by location The pooling window centered at Represents the operation of taking the maximum value, which is used to obtain the maximum value from the pooling window All locations in Select the largest value.
[0057] For ease of understanding, an example is given below: First, assume that we have performed HOSVD decomposition on a 2×2×3 simplified gastroscopy image tensor X and obtained the orthogonal basis matrix , , , the diagonal matrix of singular values , , And the core tensor G, as obtained in the previous example = = =2 low-rank approximation tensor ; Assume that the singular value vector obtained after SVD decomposition is as follows (the actual value needs to be obtained by calculation): Singular value vector of mode-1: Σ1=[400,100,50]; Singular value vector of mode-2: Σ2=[200,75,25]; Singular value vector of mode-3: Σ3=[100,50,25].
[0058] Calculate the cumulative contribution of each mode: Mode-1: 0.98 The first two features are selected because their cumulative contribution is already over 90%.
[0059] Mode-2: 0.96 Select the first two features.
[0060] Mode-3: 0.96 Select the first two features.
[0061] Next, local maximum pooling is performed. Suppose we construct a new low-rank approximation tensor from the features selected above , whose shape is 2×2×2. Now we perform local maximum pooling on each feature map. Select a 2×2 pooling window and calculate the local maximum pooling result for each feature map. For example, for the first feature map (assuming its value is , the result after local maximum pooling is: Similarly, the same pooling operation is performed on other feature maps. In gastroscopy image analysis, through adaptive threshold selection and local maximum pooling, we can effectively extract the most representative features, namely, enhanced feature vectors. These features may include color changes and texture patterns in the lesion area. In this way, doctors or automatic diagnosis systems can more accurately identify abnormal areas and improve the accuracy of diagnosis. For example, in practical applications, if the color distribution or texture pattern of a certain area is significantly different from that of normal tissue, these features will be enhanced and used in subsequent classification and recognition tasks. This not only helps to quickly locate potential lesions, but also reduces the misdiagnosis rate and improves the overall diagnosis and treatment effect.
[0062] S05: Perform multi-mode feature fusion on the multiple enhanced feature vectors to obtain a fused feature tensor.
[0063] After completing the above S04 feature extraction, we enter the S05 multi-modal feature fusion stage. We will use a fusion model based on the attention mechanism to combine feature vectors from multiple modes such as spatial coordinates and color channels. This method not only considers the correlation between the modes, but also learns a weight matrix for optimal combination, which includes calculating the attention score: for each mode's feature map, calculate its attention score; normalize the attention score: normalize the attention score to obtain the weight of each mode; weighted sum: perform weighted summation on the feature map of each mode according to the normalized attention score to generate the final fused feature tensor. The following are detailed steps and formula descriptions: Assume we have a feature tensor with M modes , ,…, , the shape of each feature tensor is H×W×C, where H and W are the height and width respectively, and C is the number of color channels.
[0064] S051: Calculate attention scores: For each feature tensor , we can use a simple fully connected layer to calculate its attention score : in, represents the attention score of the mth pattern, M represents the number of patterns, represents the feature tensor, It means expanding the three-dimensional feature tensor into a one-dimensional vector. represents the weight matrix, represents the bias term, and ReLU represents the activation function.
[0065] S052: Normalize the attention score and use the softmax function to normalize the attention score: in, represents the normalized attention score of the m-th mode, Represents the attention score Perform an exponential operation, which converts the fractions into positive numbers and amplifies the differences between the fractions. Represents the exponential sum of the attention scores of all modes. This step ensures that the sum of the normalized attention scores of all modes is 1, thus achieving normalization.
[0066] S053: Weighted sum, based on the normalized attention score Perform weighted summation on the feature tensors of each mode to generate the final fused feature tensor : For ease of understanding, an example is given below: Assume that after processing multiple gastroscopy photos for feature extraction, we obtain the following three feature tensors at the same position (simplified example): The first feature tensor: ; The second feature tensor: ; The third feature tensor: .
[0067] These feature tensors are the result of local maximum pooling, and now we need to fuse them together.
[0068] First, calculate the attention score: Assume we have a simple fully connected layer to compute the attention score, the weight matrix and the bias term Already acquired through training.
[0069] for : ; for : ; for : .
[0070] Assume that the calculated attention score is as follows (the actual value needs to be obtained by calculation): , ; Next, normalize the attention scores and use the softmax function to normalize the attention scores: Finally, weighted summation is performed to weight the feature tensors of each mode according to the normalized attention score to generate the final fused feature tensor : Assume that the specific values are as follows (for simplicity, only some values are shown here).
[0071] In gastroscopic image analysis, through multimodal feature fusion based on attention mechanism, we can obtain a new gastroscopic image by organically combining the spatial coordinates and color channels of relevant features in multiple related gastroscopic images. Through this new gastroscopic image, we can finally obtain a variety of information that can reflect position, color, texture, shape, etc. This fusion method not only enhances the expressive ability of features, but also improves the accuracy of identifying abnormal areas. For example, in practical applications, if the color distribution of a certain area is significantly different from that of normal tissue, and the texture and shape also show abnormalities, then these features will be strengthened and used in subsequent classification and recognition tasks. This can not only improve the sensitivity and specificity of lesion detection, but also reduce the misdiagnosis rate. Doctors or automatic diagnosis systems can more accurately locate potential lesions through the fused feature tensor, thereby improving the overall diagnosis and treatment effect, so as to enhance the early detection and diagnosis capabilities of the disease.
[0072] S06: Inputting the fused feature tensor into a trained classifier to obtain a gastric lesion type recognition result corresponding to the target gastroscopic image.
[0073] After completing the above S05 multi-modal feature fusion, enter the S06 classification and recognition stage. A trained classifier can be used to process the comprehensive features finally fused to achieve automatic recognition of gastric lesion types. In this application, a deep learning gastroscopy image recognition model can be selected (this model is widely used in the field of image recognition and will not be explained in detail in this application). It can directly input the final fusion feature tensor If the neural network predicts that a certain area is a lesion, the system can mark the area and provide further visual auxiliary information to help doctors make a more accurate diagnosis. In addition, this automatic recognition system can also be used for large-scale screening to improve work efficiency and reduce missed diagnoses and misdiagnoses caused by human factors.
[0074] It is to be noted that, considering that the deep learning gastroscopy image recognition model in S06 requires a large amount of data when building and training, and cannot be directly used in the short term, this application aims at the final fusion feature tensor obtained after the multi-modal feature fusion in S05. It can generate fused gastroscopy images, making it easier for doctors to view gastroscopy photos that better reflect symptoms. This method of extracting and classifying gastroscopy images based on tensor decomposition technology can help doctors quickly and accurately detect early gastric cancer or other lesions, and by improving the performance of computer-aided diagnosis systems, it reduces the workload of manual film reading and improves patients' clinical outcomes. In addition, the doctor's diagnostic conclusions on the generated fused gastroscopy images can also be input into the deep learning-based gastroscopy image recognition model in S06 for training, which can help quickly complete the model training and also improve the accuracy of the model's self-judgment.
[0075] In summary, from the above implementation methods, the present application can reduce the dimension of the data by converting the target gastroscopic image into a three-dimensional tensor and using a high-order singular value decomposition algorithm for dimensionality reduction processing, which helps to improve computing efficiency and reduce storage requirements; the target gastroscopic image usually has the characteristics of high-dimensional data, and the use of a three-dimensional tensor and a high-order singular value decomposition algorithm can effectively process high-dimensional data and extract meaningful features; by using a high-order singular value decomposition algorithm and a local maximum pooling method, the main feature components of the target gastroscopic image in different modes can be extracted, and these features are fused into a fused feature tensor, and the classifier can better learn and understand these features, thereby improving the accuracy of gastric lesion type recognition; adaptive truncation is used to extract the main feature components in each mode, and a suitable truncation point can be automatically selected according to the characteristics of the data, so as to better capture the main information in the data; by fusing the enhanced feature vectors under different modes, a fused feature tensor that integrates multi-dimensional information can be obtained. This fusion helps to improve the classifier's ability to recognize gastric lesion types because it integrates feature information from different dimensions and increases the richness and robustness of the features.
[0076] See also Figure 2 , which is a schematic diagram of the structure of a gastroscopic image processing system based on tensor decomposition proposed in the second embodiment of the present application, and the system includes: A tensor construction module 10 is used to perform a preprocessing operation on the target gastroscopic image and convert the preprocessed target gastroscopic image into a three-dimensional tensor; A tensor decomposition module 20 is used to obtain low-rank approximate tensors in different modes based on the three-dimensional tensor using a high-order singular value decomposition algorithm, wherein the different modes are different dimensions in the three-dimensional tensor, and the high-order singular value decomposition algorithm includes obtaining and adaptively truncating a core tensor and a factor matrix, wherein the obtaining of the core tensor and the factor matrix is used to obtain a core tensor in each mode, and the adaptive truncation is used to extract the main characteristic components in each mode; A feature extraction module 30, configured to perform local maximum pooling on each of the low-rank approximate tensors to obtain an enhanced feature vector in each mode; A feature fusion module 40 is used to perform multi-mode feature fusion on the plurality of enhanced feature vectors to obtain a fused feature tensor; The classification and recognition module 50 is used to input the fused feature tensor into a trained classifier to obtain a gastric lesion type recognition result corresponding to the target gastroscopic image.
[0077] On the other hand, the present application also proposes a computer device, which includes a memory, a processor, and a processing program stored in the memory and executable on the processor, and when the processing program is executed by the processor, the above-mentioned gastroscopic image processing method based on tensor decomposition is implemented.
[0078] On the other hand, the present application further proposes a storage medium, on which a processing program is stored. When the processing program is executed by a processor, the above-mentioned gastroscopic image processing method based on tensor decomposition is executed.
[0079] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0080] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0081] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A gastroscopic image processing method based on tensor decomposition, characterized in that: The method comprises: Performing a preprocessing operation on the target gastroscopic image, and converting the preprocessed target gastroscopic image into a three-dimensional tensor; Based on the three-dimensional tensor, a high-order singular value decomposition algorithm is used to obtain low-rank approximate tensors in different modes, wherein the different modes are different dimensions in the three-dimensional tensor, and the high-order singular value decomposition algorithm includes obtaining and adaptively truncating a core tensor and a factor matrix, wherein the obtaining of the core tensor and the factor matrix is used to obtain a core tensor in each mode, and the adaptive truncation is used to extract the main characteristic components in each mode; Performing local maximum pooling on each of the low-rank approximate tensors to obtain an enhanced feature vector in each mode; Performing multi-mode feature fusion on the plurality of enhanced feature vectors to obtain a fused feature tensor; The fused feature tensor is input into a trained classifier to obtain a gastric lesion type recognition result corresponding to the target gastroscopic image.
2. The method according to claim 1, characterized in that: The process of obtaining the core tensor and factor matrix includes: Expanding the three-dimensional tensor into a matrix form along different modes to calculate the covariance matrix in each mode; Performing SVD decomposition on the covariance matrix to obtain an orthogonal basis matrix and a diagonal matrix containing singular values in each mode; Based on the orthogonal basis matrix, the three-dimensional tensor is projected to obtain the core tensor in each mode.
3. The method according to claim 2, characterized in that: The processing process of the adaptive truncation includes: According to the singular values in each mode, an adaptive threshold selection strategy is used to determine the main characteristic components to be retained; The adaptive threshold selection strategy is specifically as follows: calculating the cumulative contribution rate of the singular values, and retaining the singular values whose cumulative contribution rate reaches a preset threshold as the retained main characteristic components; Wherein, a threshold is set according to the core tensor to obtain the preset threshold.
4. The method according to claim 1, characterized in that: The calculation formula of the local maximum pooling is: in, represents the coordinates in the feature map after pooling, Represents a position in the feature map after pooling The value of Represents a position in the original feature map The value of Indicates by location The pooling window centered at Represents the operation of taking the maximum value, which is used to obtain the maximum value from the pooling window All locations in Select the largest value.
5. The method according to claim 1, characterized in that: The multi-modal feature fusion includes a fusion model based on an attention mechanism, and the processing process of the fusion model based on the attention mechanism includes: Inputting the enhanced feature vectors in different modes into the fusion model based on the attention mechanism to calculate the attention score; Normalizing the attention scores to obtain the weights of each mode; The weights of the various modes are weighted summed to obtain the fused feature tensor.
6. The method according to claim 5, characterized in that: The formula for calculating the attention score is: in, represents the attention score of the mth pattern, M represents the number of patterns, represents the feature tensor, It means expanding the three-dimensional feature tensor into a one-dimensional vector. represents the weight matrix, represents the bias term, ReLU represents the activation function; The formula for normalizing the attention score is: in, represents the normalized attention score of the m-th mode, Indicates the attention score Perform exponential operations, represents the exponential sum of attention scores of all modes; The formula for the weighted summation is: in, Represents the fused feature tensor.
7. The method according to claim 1, characterized in that: The steps of preprocessing the target gastroscope image include: Converting the target gastroscope image into a grayscale image, and performing denoising on the grayscale image using a median filter to remove salt and pepper noise; The denoised grayscale image is smoothed using Gaussian filtering, and then the contrast is enhanced using adaptive histogram equalization; The grayscale image after contrast enhancement is converted back into a color image, and the color image is the preprocessed gastroscopic image.
8. A gastroscopic image processing system based on tensor decomposition, characterized in that: The system comprises: A tensor building module, used for performing a preprocessing operation on the target gastroscopic image, and converting the preprocessed target gastroscopic image into a three-dimensional tensor; A tensor decomposition module is used to obtain low-rank approximate tensors in different modes based on the three-dimensional tensor using a high-order singular value decomposition algorithm, wherein the different modes are different dimensions in the three-dimensional tensor, and the high-order singular value decomposition algorithm includes obtaining and adaptively truncating a core tensor and a factor matrix, wherein the obtaining of the core tensor and the factor matrix is used to obtain a core tensor in each mode, and the adaptive truncation is used to extract the main characteristic components in each mode; A feature extraction module, used for performing local maximum pooling on each of the low-rank approximate tensors to obtain an enhanced feature vector in each mode; A feature fusion module, used for performing multi-mode feature fusion on a plurality of the enhanced feature vectors to obtain a fused feature tensor; The classification and recognition module is used to input the fused feature tensor into a trained classifier to obtain a gastric lesion type recognition result corresponding to the target gastroscopic image.
9. A computer device, characterized in that: The computer device comprises a memory, a processor, and a processing program stored in the memory and executable on the processor, wherein the processing program implements the method according to any one of claims 1 to 7 when executed by the processor.
10. A storage medium, characterized in that: The storage medium stores a processing program, and the processing program implements the method according to any one of claims 1 to 7 when executed by a processor.
Citation Information
Patent Citations
Anisotropic data compression method based on tensor approximation
CN107515843A
Patent information retrieval system and method based on big data
CN117349406A
Breast cancer early screening diagnosis method based on attention mechanism enhanced CNN (Convolutional Neural Network)
CN117898767A
Magnetocardiogram feature extraction method based on high-order tensor decomposition
CN118734042A
KR20210128887A
Cited By
Microteaching actual effect evaluation system based on artificial intelligence
CN119740916A
Brain tumor automatic segmentation method and system based on deep learning
CN119942107A
