Early prostate tumor identification method based on hyperspectral image

Abstract

The invention discloses an early-stage prostate tumor recognition method based on a hyperspectral image, and belongs to the technical field of image recognition, and the method comprises the steps: collecting early-stage prostate tumor samples with different lesion degrees; performing hyperspectral scanning on the prostate tumor early tissue sample to generate corresponding hyperspectral image data and performing preprocessing; extracting features of the preprocessed hyperspectral image data; organizing feature data and label data, wherein the label data classifies and marks a sample based on a known pathological diagnosis result; constructing an early prostate tumor recognition model based on the feature data and the label data, and performing optimization training; and performing early prostate tumor recognition on a new sample to be detected by using the trained recognition model. The method has the beneficial effects that the actual canceration condition can be easily mastered, and compared with an existing tissue slice inspection mode, the method is easier to operate, the detection result can be obtained more quickly, the result accuracy is higher, and the objectivity is higher.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to an early prostate tumor recognition method based on hyperspectral images. Background Art

[0002] In the field of modern medical diagnosis, prostate cancer is a common malignant tumor, and its early detection and treatment are crucial to improving the survival rate of patients. However, existing prostate cancer detection technology has significant limitations. One of the main screening methods is to use blood samples to detect prostate-specific antigen (PSA), but PSA is a tissue-specific marker rather than a tumor-specific marker, and its specificity and sensitivity in prostate cancer detection are limited, which leads to many false positive and false negative results, which cannot accurately reflect the actual situation of canceration, and thus affects subsequent clinical judgment and treatment decisions.

[0003] In addition, traditional pathological analysis relies on the preparation of tumor tissue slices and morphological evaluation under a microscope. This process is not only complicated and time-consuming, but also affected by the subjective judgment of pathologists, resulting in limited accuracy and consistency of test results. These pain points highlight the shortcomings of existing technologies in the early identification and diagnosis of prostate cancer, and also indicate the urgent need for a new, efficient and accurate detection method. Summary of the invention

[0004] In order to identify prostate cancer earlier and more accurately, the present invention provides an early prostate tumor identification method based on hyperspectral images.

[0005] In order to achieve the above-mentioned object of the invention, the present invention provides a method for identifying early prostate tumors based on hyperspectral images, and the identification method comprises the following steps: Step S1: collecting early prostate tumor samples of different lesion degrees for cryopreservation, and obtaining ambient light spectral features, wherein the ambient light spectral features are used to correct subsequent hyperspectral image data; Step S2: performing a hyperspectral scan on the early prostate tumor tissue sample to generate corresponding hyperspectral image data and perform preprocessing, wherein the hyperspectral image data covers multiple bands from visible light to near-infrared light; the preprocessing includes background noise removal, spectral correction, image registration and normalization processing; Step S3: extracting features from the preprocessed hyperspectral image data, wherein the feature data includes spectral features and texture features; Step S4: Organizing characteristic data and label data, wherein the label data classifies and labels the samples based on known pathological diagnosis results; Step S5: constructing an early prostate tumor recognition model based on the feature data and label data and optimizing the training; Step S6: Use the trained recognition model to identify early prostate tumors in new samples to be tested.

[0006] Wherein, in the step S2, the preprocessing is specifically: Step S201: read hyperspectral image data and load the .mat file containing the hyperspectral data; Step S202: Acquire image data of a specified wavelength band, wherein the wavelength range of the R channel is [630nm, 650nm], the wavelength range of the G channel is [530nm, 550nm], and the wavelength range of the B channel is [450nm, 470nm]; Step S203: splicing the acquired red, green and blue bands of image data into an RGB image through a cat function; Step S204: obtaining one-dimensional spectrum data of each pixel in the RGB image, and subtracting the spectrum characteristics of the ambient light to remove the interference of the ambient light; Step S205: normalizing the one-dimensional spectrum data after subtracting the ambient light spectrum characteristics, and compressing the data range to [-1, 1]; Step S206: performing data expansion on the normalized one-dimensional spectral data, and copying 32 rows of data of each pixel point to generate a single-channel grayscale image; Step S207: multiply each pixel data of the grayscale image by 255 and save the data as a grayscale image to generate a two-dimensional grayscale image data set.

[0007] In step S3, the method for extracting tumor spectral features comprises the following steps: Step S3011: Select a region of interest containing a tumor on the two-dimensional grayscale image; use manual annotation or threshold segmentation method to identify the region of interest where the tumor is located; select target pixels in the identified region of interest for subsequent texture feature extraction.

[0008] Step S3012: extracting one-dimensional normalized spectral data corresponding to each pixel point in the region of interest, where the one-dimensional normalized spectral data of each pixel point is a normalized value at different wavelengths; Step S3013: Calculating the statistical features of the one-dimensional normalized spectral data, the statistical features including the mean value, variance, energy, spectrum slope and kurtosis of the spectrum; Step S3014: storing the statistical features at corresponding pixel positions to generate a feature map, wherein each pixel in the feature map corresponds to a feature vector containing spectral statistical feature information.

[0009] In step S3, the method for extracting tumor texture features comprises the following steps: S3021: constructing a gray level co-occurrence matrix, and using the gray level co-occurrence matrix to extract contrast, energy, homogeneity and correlation of the region of interest; S3022: constructing a grayscale difference matrix, and extracting contrast and roughness of the region of interest using the grayscale difference matrix; S3023: Construct a binary image, and extract the frequency and distribution of the texture of the region of interest from the binary image.

[0010] Wherein, the step S3021 specifically includes: Define the distance and direction of neighboring pixels, including the choice of horizontal, vertical, or diagonal direction; quantifying the grayscale of the image in the region of interest, and discretizing the grayscale value into a preset grayscale level; Constructing a gray level co-occurrence matrix by counting gray value pairs between pixels in the region of interest and their neighboring pixels at a selected distance and direction to generate a gray level co-occurrence matrix; Extract the statistical features of the gray-level co-occurrence matrix, including contrast, energy, homogeneity and correlation. This feature can be obtained by calculating different combinations of gray-level co-occurrence matrices. Contrast can calculate the sum of squares of gray-level differences of adjacent pixels, and energy can calculate the sum of squares of elements in gray-level co-occurrence matrices.

[0011] The step S3022 specifically includes: Use the same neighboring pixel distance and direction as the gray-level co-occurrence matrix; Calculating the grayscale difference between the pixel in the region of interest and its neighboring pixels to quantify the grayscale change between the pixels; Construct a grayscale difference matrix by counting the frequency of grayscale difference values ​​of adjacent pixels to generate a grayscale difference matrix; Extract the statistical features of the grayscale difference matrix, including contrast and roughness; this feature can be obtained by calculating different combinations of the grayscale difference matrix. Contrast can calculate the average value of the grayscale difference of adjacent pixels, and roughness can calculate the standard deviation of the grayscale difference matrix.

[0012] The step S3023 specifically includes: Use the same neighboring pixel distance and direction as the gray-level co-occurrence matrix; Compare the grayscale values ​​of each pixel with those of its neighboring pixels and generate a binary code; Construct a binary image and convert the binary code of each pixel into a local texture pattern; Extract statistical features of binary images, including texture pattern frequency and texture pattern distribution; the statistical features can be obtained by calculating the histogram, mean, variance, etc. of the texture pattern.

[0013] The early prostate tumor recognition model in step S5 is a convolutional neural network model based on ResNet50, and introduces residual connection and attention mechanism; The loss function is the cross entropy loss function for the multi-classification task, and the early detection penalty factor is defined as 1.5; Select the Adam optimizer and set the initial learning rate to 0.001, and use the defined weighted loss function to train and optimize the model; Use the test set to evaluate the trained model, calculate the accuracy, precision, recall and F1-score, and analyze the model performance through confusion matrix and ROC curve; Model fusion is performed by training multiple convolutional neural network models with different initial parameters, and the prediction results of each model are merged using a voting strategy; The performance of the model is evaluated through cross-validation, the average performance index is calculated, and the model parameters are adjusted according to the evaluation results to optimize the model performance.

[0014] The input layer of the early prostate tumor recognition model sets the input dimension according to the number of columns of the feature matrix. The recognition model includes a convolution layer, a pooling layer, a fully connected layer and an output layer in sequence, wherein the convolution layer uses a 3×3 or 5×5 convolution kernel, the number of convolution kernels is set according to actual conditions, the step size is set to 1, the filling method can be selected as "valid" or "same", and the activation function is ReLU; the pooling layer uses a 2x2 pooling kernel, the step size is set to 2, and the filling method is "valid" or "same".

[0015] The beneficial effects of the present invention are as follows: the present invention uses the hyperspectral image information of early prostate tumor samples as training data. After the recognition model is trained, it can identify the early prostate tumor results in the hyperspectral image information of the patient's tissue samples. Compared with the existing method of detecting PSA, it is easier to grasp the actual situation of canceration. Moreover, compared with the existing method of testing tissue sections, it is easier to operate, the test results are obtained faster, the results are more accurate, and the objectivity is stronger.

[0016] The advantages of the present invention are explained in more detail: the method of the present invention can realize rapid image acquisition and processing, significantly shorten the time from sample acquisition to result analysis, and speed up the detection process. Due to the simplification of the operation process and the reduction of dependence on professional technology, the method of the present invention reduces the manpower and material costs required for detection. Through the design of a specific loss function and attention mechanism, the method of the present invention pays more attention to the detection of early tumors and improves the accuracy and sensitivity of early diagnosis. The present invention not only analyzes spectral features, but also combines texture features, and improves the accuracy and robustness of recognition through the integration of multi-dimensional information. Through the introduction of model fusion technology and custom attention mechanism, the present invention further improves the performance of the model and the recognition ability of key features; the designed loss function imposes a greater penalty on early tumor detection errors, encouraging the model to capture early tumor features more sensitively; the custom attention mechanism enables the model to dynamically adjust according to the characteristics of different samples, improving the robustness and generalization ability of recognition. The method of the present invention has the potential for wide application in the clinical diagnosis and treatment of early prostate tumors, and is expected to become an important auxiliary tool for doctors and patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of Embodiment 1 and Embodiment 2 of the present invention.

[0018] Figure 2 It is a flowchart of data preprocessing in Embodiment 1 and Embodiment 2 of the present invention.

[0019] Figure 3 It is a schematic diagram of a specific loss function flow chart designed in Embodiment 1 and Embodiment 2 of the present invention.

[0020] Figure 4 It is a flowchart of introducing a custom attention mechanism in Embodiment 1 and Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0021] Existing technologies usually use blood samples for prostate-specific antigen (PSA) screening to complete the detection of prostate cancer, because PSA is a tissue-specific marker, not tumor-specific. This leads to reduced specificity and sensitivity of PSA for prostate cancer. Pathological analysis using tumor tissue requires tissue sections to be made into specimens, which is complicated and time-consuming, and the test results lack objectivity and accuracy.

[0022] Hyperspectral imaging technology is based on image data technology with a lot of narrow and long bands. It combines imaging technology to detect the two-dimensional geometric space and one-dimensional spectral information of the target, thereby obtaining continuous, narrow-band high-spectral resolution image data. Common hyperspectral imaging technologies include grating spectrometry, acousto-optic tunable filtering spectrometry, and chip coating.

[0023] Therefore, in order to clearly illustrate the technical features of this solution, this solution is described below through specific implementation methods. The following embodiment provides a prostate tumor recognition method and system based on hyperspectral images, using the hyperspectral image information of early prostate tumor samples as training data. After the recognition model is trained, the hyperspectral image information of the patient's tissue samples can be recognized to obtain the recognition result of the early prostate tumor.

[0024] Embodiment 1 See also Figure 1 The embodiment of the present invention provides a method for identifying early prostate tumors based on hyperspectral images, which specifically includes the following steps: Step S1: collecting early prostate tumor samples of different lesion degrees for cryopreservation, and obtaining ambient light spectral features, which are used to correct subsequent hyperspectral image data; Step S2: Performing hyperspectral scanning on early-stage prostate tumor tissue samples, generating corresponding hyperspectral image data and performing preprocessing, wherein the hyperspectral image data covers multiple bands from visible light to near-infrared light; the preprocessing includes background noise removal, spectral correction, image registration and normalization processing; Step S3: extracting features from the preprocessed hyperspectral image data, the feature data including spectral features and texture features; Step S4: Organizing feature data and label data, the label data classifies and labels the samples based on known pathological diagnosis results; Step S5: constructing an early prostate tumor recognition model based on the feature data and label data and optimizing the training; Step S6: Use the trained recognition model to identify early prostate tumors in new samples to be tested.

[0025] Among them, in step S2, see Figure 2 , the preprocessing is as follows: Step S201: read hyperspectral image data and load the .mat file containing the hyperspectral data; Step S202: Acquire image data of a specified wavelength band, wherein the wavelength range of the R channel is [630nm, 650nm], the wavelength range of the G channel is [530nm, 550nm], and the wavelength range of the B channel is [450nm, 470nm]; Step S203: splicing the acquired red, green and blue bands of image data into an RGB image through the cat function; Step S204: obtaining one-dimensional spectrum data of each pixel in the RGB image, and subtracting the spectrum characteristics of the ambient light to remove the interference of the ambient light; Step S205: normalizing the one-dimensional spectrum data after subtracting the ambient light spectrum characteristics, and compressing the data range to [-1, 1]; Step S206: performing data expansion on the normalized one-dimensional spectral data, and copying 32 rows of data of each pixel point to generate a single-channel grayscale image; Step S207: multiply each pixel data of the grayscale image by 255 and save it as a grayscale image to generate a two-dimensional grayscale image data set.

[0026] In step S3, the method for extracting tumor spectral features includes the following steps: Step S3011: selecting a region of interest containing a tumor on the two-dimensional grayscale image; Step S3012: extracting one-dimensional normalized spectral data corresponding to each pixel point in the region of interest, where the one-dimensional normalized spectral data of each pixel point is a normalized value at different wavelengths; Step S3013: Calculate the statistical characteristics of the one-dimensional normalized spectrum data, the statistical characteristics including the mean value, variance, energy, spectrum slope and kurtosis of the spectrum; Step S3014: Store the statistical features at the corresponding pixel positions to generate a feature map, in which each pixel corresponds to a feature vector containing spectral statistical feature information.

[0027] In step S3, the method for extracting tumor texture features includes the following steps: S3021: construct a gray-level co-occurrence matrix, and use the gray-level co-occurrence matrix to extract contrast, energy, homogeneity and correlation of the region of interest; S3022: constructing a grayscale difference matrix, and using the grayscale difference matrix to extract contrast and roughness of the region of interest; S3023: Construct a binary image, and extract the frequency and distribution of texture in the region of interest of the binary image.

[0028] Wherein, step S3021 specifically includes: Define the distance and direction of neighboring pixels, including the choice of horizontal, vertical, or diagonal direction; Quantify the grayscale of the image within the region of interest and discretize the grayscale value into a preset grayscale level; Construct a gray-level co-occurrence matrix by counting the gray-level value pairs of pixels in the region of interest and their neighboring pixels at the selected distance and direction to generate a gray-level co-occurrence matrix; Extract the statistical features of the gray-level co-occurrence matrix, including contrast, energy, homogeneity and correlation. This feature can be obtained by calculating different combinations of gray-level co-occurrence matrices. Contrast can calculate the sum of squares of gray-level differences of adjacent pixels, and energy can calculate the sum of squares of elements in gray-level co-occurrence matrices.

[0029] Step S3022 specifically includes: Use the same neighboring pixel distance and direction as the gray-level co-occurrence matrix; The grayscale difference between the pixel in the region of interest and its neighboring pixels is calculated to quantify the grayscale change between pixels; Construct a grayscale difference matrix by counting the frequency of grayscale difference values ​​of adjacent pixels to generate a grayscale difference matrix; Extract the statistical features of the grayscale difference matrix, including contrast and roughness; this feature can be obtained by calculating different combinations of the grayscale difference matrix. Contrast can calculate the average value of the grayscale difference of adjacent pixels, and roughness can calculate the standard deviation of the grayscale difference matrix.

[0030] Step S3023 specifically includes: Use the same neighboring pixel distance and direction as the gray-level co-occurrence matrix; Compare the grayscale values ​​of each pixel with those of its neighboring pixels and generate a binary code; Construct a binary image and convert the binary code of each pixel into a local texture pattern; Extract statistical features of binary images, including texture pattern frequency and texture pattern distribution; the statistical features can be obtained by calculating the histogram, mean, variance, etc. of the texture pattern.

[0031] The early prostate tumor recognition model in step S5 is a convolutional neural network model based on ResNet50, and introduces residual connection and attention mechanism; The loss function is the cross entropy loss function for the multi-classification task, and the early detection penalty factor is defined as 1.5; Select the Adam optimizer and set the initial learning rate to 0.001, and use the defined weighted loss function to train and optimize the model; Use the test set to evaluate the trained model, calculate the accuracy, precision, recall and F1-score, and analyze the model performance through confusion matrix and ROC curve; Model fusion is performed by training multiple convolutional neural network models with different initial parameters, and the prediction results of each model are merged using a voting strategy; The performance of the model is evaluated through cross-validation, the average performance index is calculated, and the model parameters are adjusted according to the evaluation results to optimize the model performance.

[0032] The input dimension of the input layer of the early prostate tumor recognition model is set according to the number of columns of the feature matrix. The recognition model includes a convolutional layer, a pooling layer, a fully connected layer, and an output layer in sequence. The convolutional layer uses a 3×3 or 5×5 convolution kernel. The number of convolution kernels is set according to actual conditions, the step size is set to 1, the padding method can be selected as "valid" or "same", and the activation function is ReLU; the pooling layer uses a 2x2 pooling kernel, the step size is set to 2, and the padding method is "valid" or "same".

[0033] Embodiment 2 The present invention provides a prostate tumor recognition method based on hyperspectral imaging, see Figure 1 , mainly including: collecting early prostate tumor samples, obtaining the spectral characteristics of ambient light, scanning early prostate tumor samples based on hyperspectral, and preprocessing the collected hyperspectral image data; extracting features of early prostate tumors based on hyperspectral images, collating feature data and label data, and establishing an early prostate tumor identification model; model fusion, introducing attention mechanism, model evaluation and optimization, and implementing prediction.

[0034] Specific: Step S1: Collect tumor samples from patients undergoing surgery for early-stage prostate tumors, and store the samples at low temperatures.

[0035] Hyperspectral image information of early prostate tumor samples was obtained as training data. The early prostate tumor samples were derived from early prostate tumor tissues of patients who had undergone prostate puncture or prostate tumor resection surgery. After informing the patient of the purpose of the research experiment and possible risks, and with the patient's consent, the early prostate tumor samples were graded according to the Gleason scoring system (a commonly used histological grading method for prostate cancer in the industry). The samples with the same score were grouped together to form early prostate tumor samples of different grades, and all of them were placed at low temperature.

[0036] Step S2: Obtaining the spectral characteristics of ambient light.

[0037] In this embodiment, a white diffuse reflection reference object without fluorescent characteristics is placed on the shooting plane, a hyperspectral image of the reference object area is obtained, and one-dimensional spectral data is extracted for each pixel in the area. The data is stored in a csv file, and the sum and average are taken by column to obtain a row of spectral data as the spectral characteristics of the ambient light.

[0038] Step S3: scanning early prostate tumor samples based on the hyperspectral spectrum.

[0039] After the patient's pre-treated specimen is thawed, it is photographed by a hyperspectral camera to obtain a hyperspectral original image. In this embodiment, the hyperspectral image includes a spectral dimension and a three-dimensional space dimension, and the SpecView-F software is used to collect hyperspectral data. The SpecView-F acquisition software is mainly used to realize functions such as spectral imaging system control and image acquisition.

[0040] Step S4: preprocessing the collected hyperspectral image data.

[0041] Get the image data of the specified band, splice the data of multiple bands into an RGB image to generate a one-dimensional data set, then subtract the ambient light spectral characteristics (environmental noise reduction), draw a grayscale image after data normalization, and generate a two-dimensional data set.

[0042] In this embodiment, the original image data collected by the spectrometer is processed. Figure 2 , first read the data from the specified file, load the specified .mat file, obtain the image data of the specified band, and display it on the screen. Use the cat function to splice the red, green, and blue bands into an RGB image. The R channel wavelength selection range is [630nm, 650nm], the G channel wavelength selection range is [530nm, 550nm], and the B channel wavelength selection range is [450nm, 470nm] to obtain the one-dimensional spectral data of each pixel in the image. Then subtract the ambient light spectral characteristics (environmental noise reduction), that is, subtract the one-dimensional spectral data of each pixel in the tumor hyperspectral image from the average spectral characteristics of the ambient light. Then perform data normalization, compress the spectral data range to [-1, 1], and obtain the normalized one-dimensional data of each pixel, that is, the one-dimensional data of each pixel at different wavelengths. This step can shield the interference of ambient light and suppress the white noise interference in the data, compress the spectral characteristic range, but can completely retain the spectral characteristics, and prevent the problem of excessive gradient during model training. The normalized one-dimensional data is expanded to generate a single-channel grayscale image, specifically including: copying the normalized one-dimensional data, after copying 32 rows, multiplying each pixel data by 255 and saving it as a grayscale image. The grayscale images of all pixels are obtained through the above scheme as two-dimensional data.

[0043] Step S5: Extracting features of early prostate tumors based on hyperspectral images, including spectral features and texture features: Step S501: Extract tumor spectral features: Select the tumor region of interest: Select the region of interest containing the tumor on the two-dimensional grayscale image; manual annotation, threshold segmentation and other methods can be used to obtain the tumor region.

[0044] Extracting one-dimensional normalized spectral data of each pixel: For the selected tumor area, extract the one-dimensional normalized spectral data corresponding to each pixel. According to the previous data processing steps, the one-dimensional data of each pixel is the normalized value at different wavelengths.

[0045] Calculate statistical features: For the one-dimensional spectral data of each pixel, calculate statistical features to describe the distribution and changes of the spectrum. Calculate the average value of the one-dimensional spectral data to reflect the central trend of the spectrum. Calculate the variance of the one-dimensional spectral data to reflect the degree of dispersion of the spectrum. Calculate the energy of the one-dimensional spectral data to reflect the amplitude information of the spectrum. Calculate the spectral tilt of the one-dimensional spectral data to reflect the asymmetry of the spectral distribution. Calculate the kurtosis of the one-dimensional spectral data to reflect the sharpness of the spectral distribution.

[0046] For each pixel, the calculated statistical features are stored in the corresponding position to form a feature map. Each pixel corresponds to a feature vector, which contains the statistical feature information of the spectrum.

[0047] Step S502: Extracting tumor texture features: First, the texture features are extracted using the gray-level symbiosis matrix (GLDM): Define the distance and direction of neighboring pixels: According to the characteristics and requirements of the image, select the appropriate distance and direction of neighboring pixels, such as horizontal, vertical, diagonal, etc.

[0048] Grayscale quantization: discretize the grayscale of an image into a set of discrete values, using methods such as grayscale layering or histogram equalization for grayscale quantization.

[0049] Construct a gray-level co-occurrence matrix: For each pixel, calculate the gray-level value pairs of its neighboring pixels at the specified distance and direction. Based on the gray-level pairs of neighboring pixels, count their occurrence times in the gray-level co-occurrence matrix.

[0050] Extract various statistical characteristics of the gray-level co-occurrence matrix: By calculating different combinations of the gray-level co-occurrence matrix, we can obtain contrast, energy, homogeneity, and correlation characteristics; among them, contrast can be calculated by calculating the sum of squares of gray-level differences of adjacent pixels, and energy can be calculated by calculating the sum of squares of elements in the gray-level co-occurrence matrix. Specifically: Contrast: It can reflect the roughness of image texture. It reflects the grayscale difference of adjacent pixels by calculating the difference in pixel grayscale. The calculation formula is as follows:

[0051] in, P ( i,j ) is an element in GLCM, indicating that the gray level is i and jThe probability of a pixel pair occurring.

[0052] Energy: Energy reflects the uniformity of image texture and indicates the smoothness of grayscale distribution. Energy is the sum of squares of elements in GLCM. The higher the energy value, the more uniform the texture of the image. The calculation formula is as follows:

[0053] Homogeneity: Homogeneity reflects the similarity of grayscale between adjacent pixels. If the difference in grayscale values ​​of adjacent pixels is small, the homogeneity value is high. The calculation formula is as follows:

[0054] Correlation: Correlation reflects the relationship between the gray levels of pixel pairs and measures the linear dependence between two pixel values. The formula is as follows:

[0055] in, μ i and μ j The pixel values ​​are i and j The average value of σ i and σ j are the standard deviations respectively.

[0056] Secondly, the gray-level difference matrix (GLDM) is used to extract texture features: Take the same distance and direction of neighboring pixels as the gray-level co-occurrence matrix.

[0057] Calculate grayscale difference: For each pixel, calculate the grayscale difference between it and the neighboring pixels in the specified distance and direction.

[0058] Construct a grayscale difference matrix: According to the grayscale difference values ​​of neighboring pixels, count the number of times they appear in the grayscale difference matrix.

[0059] Extract various statistical features of the grayscale difference matrix: Contrast and roughness are obtained by calculating different combinations of the grayscale difference matrix. Contrast is used to calculate the average value of the grayscale difference of adjacent pixels, and roughness is used to calculate the standard deviation of the grayscale difference matrix; specifically: Contrast: It reflects the grayscale difference between adjacent pixels in the image. The grayscale difference matrix is ​​used to calculate the average grayscale difference between adjacent pixels. The formula is as follows:

[0060] in, P ( i,j ) represents the probability of occurrence in the grayscale difference matrix, i and j is the grayscale difference between adjacent pixels.

[0061] Roughness: Roughness is an important indicator that reflects the coarseness or fineness of the image surface texture. It can be measured by calculating the standard deviation of the grayscale difference value. The formula for roughness is:

[0062] Among them, μ is the mean of the grayscale difference values, P ( i,j ) is the probability value of the corresponding grayscale difference matrix.

[0063] Again, local binary pattern (LBP) is used to extract texture features: Take the same distance and direction of neighboring pixels as the gray-level co-occurrence matrix.

[0064] Comparison of the central pixel with the adjacent pixels: For each pixel, its grayscale value is compared with the grayscale values ​​of the adjacent pixels to obtain a binary code.

[0065] Construct binary image: Based on the comparison results, the binary code is converted into a binary image, where each pixel represents a local texture pattern.

[0066] Extract various statistical features from binary images: obtain texture pattern frequency and texture pattern distribution by calculating the histogram, mean, and variance of the texture pattern; specifically, 1. Construction of Local Binary Pattern (LBP) Neighboring pixel selection: Similar to the gray-level co-occurrence matrix, neighboring pixels with a distance of 1 or 2 pixels from the center pixel are selected, and values ​​are taken in the horizontal, vertical, diagonal and other directions.

[0067] Binary comparison: For each pixel in the image, compare its grayscale value with the grayscale value of the neighboring pixels. If the grayscale value of the neighboring pixel is greater than or equal to the grayscale value of the central pixel, it is assigned a value of 1; otherwise, it is assigned a value of 0. The comparison results of these 8 neighboring pixels form an 8-bit binary number.

[0068] Generate a binary image: Convert the comparison result of each pixel (i.e. binary code) into a decimal number and assign it to the corresponding pixel to form a binary image. This binary image represents the local texture pattern.

[0069] 2. Statistical feature extraction After obtaining the LBP binary image, various statistical features are extracted from it to describe the distribution of texture patterns.

[0070] 1) Histogram of texture mode: Histogram is the process of frequency statistics of LBP patterns in binary images. By counting the number of occurrences of different LBP patterns in an image, the distribution of different local texture patterns in the image can be obtained. Histogram can be used to describe the frequency of various types of textures in an image.

[0071] Histogram calculation formula:

[0072] Among them, H( k ) is the mode k The frequency of occurrence, LBP ( x,y ) is the pixel ( x,y )’s LBP value.

[0073] 2) Mean: The mean reflects the average value of the LBP mode in the entire binary image. By averaging the LBP values ​​of all pixels, the average intensity of the local texture in the image can be obtained; the mean calculation formula is:

[0074] in, N is the total number of pixels, LBP ( x,y ) is the LBP value of the pixel.

[0075] 3) Variance: It is used to measure the degree of discreteness of the LBP pattern in the image and reflect the changes of different patterns. By calculating the variance, we can understand whether the distribution of the texture pattern in the image is concentrated or dispersed; the variance calculation formula is:

[0076] Among them, μ is the mean, σ 2 is the variance.

[0077] Step S6: Feature data and label data collation Feature data organization: The extracted features (spectral features, texture features) are organized into a matrix (or feature matrix), where each row represents a sample and each column represents a feature. This matrix is ​​usually represented in the form of a numpy array.

[0078] Label data organization: The corresponding sample labels (Gleason scores) are organized into a vector (or label vector), in which each element represents the label of a sample. Since the Gleason score of early prostate tumors is 2-4 points, with a total of 3 possible values, the classification features with 3 possible values ​​are converted into 3 binary features, each of which represents a possible category, and each Gleason score can be regarded as an independent category. As follows: 2->[1,0,0] 3->[0,1,0] 4->[0,0,1] All label vectors are used to form a label matrix, where each row corresponds to a sample, each column corresponds to a Gleason score, and a value of 1 indicates that the score of the sample is the score corresponding to the column, and a value of 0 indicates that it is not.

[0079] Step S7: Establishing an early prostate tumor recognition model ResNet50 is selected and residual connections are introduced to solve the gradient vanishing and representation bottleneck problems in deep networks, making it possible to train deeper networks.

[0080] Input Layer: Input Data Shape: Set the dimension of the input layer according to the number of columns of the feature matrix.

[0081] Convolutional Layer: Convolution kernel size: You can choose a 3x3 or 5x5 convolution kernel.

[0082] Number of cores on the roll: The number of cores on the roll can be set according to the actual situation, such as 32, 64, 128, etc.

[0083] Step size: Generally set to 1.

[0084] Filling method: You can choose "valid" or "same", depending on the size of the input data and model requirements.

[0085] Activation function: ReLU activation function is generally used.

[0086] Pooling layer: Pooling kernel size: Generally, a 2x2 pooling kernel is selected.

[0087] Step size: Generally set to 2.

[0088] Filling method: You can choose "valid" or "same", depending on the size of the input data and model requirements.

[0089] Fully connected layer: Number of neurons: Set according to actual conditions. You can try different values ​​to find the best performance.

[0090] Activation function: ReLU activation function is generally used.

[0091] Output layer: Dimension of the output layer: Set the dimension of the output layer according to the number of columns of the label matrix, which is the same as the number of categories.

[0092] Activation function: Use the softmax function to convert the output into a classification result of probability distribution.

[0093] Loss function and optimizer: Loss function: For multi-classification tasks, select the cross entropy loss function (categorical_crossentropy) as the loss function. Because the Gleason score is 3 ability values. Each one can be considered an independent category. Convert the label matrix into 3 binary features. Therefore, the loss function will be calculated based on the cross entropy between the 3 categories.

[0094] In addition, in another embodiment, a specific loss function is provided to consider the importance of early tumor detection and the difference in error penalties, which is implemented in the form of a weighted loss function (see Figure 3 ): Define the early detection penalty factor: First, according to the specific requirements of the problem, define an early detection penalty factor, which represents the penalty for early tumor detection errors. This factor is set to a positive number of 1.5 to make the early detection error more costly.

[0095] Calculate the loss function: During model training, the loss function is calculated based on the difference between the predicted result and each element of the true label vector. For each sample, the loss is calculated based on the predicted category and the true category.

[0096] Weighted loss function: Based on the predicted category and the true category, a weighted loss function is calculated. For early tumor detection errors, the loss function is multiplied by the early detection penalty factor to give it a larger loss.

[0097] Loss function summation: sum the weighted loss functions of all samples to get the final loss function. Additional adjustments and optimizations can be made as needed.

[0098] Model training and optimization: Use the defined weighted loss function to train and optimize the model. Select an appropriate optimizer and learning rate, and adjust the optimization parameters based on actual conditions.

[0099] Model evaluation: Use the test set to evaluate the trained model and calculate indicators such as accuracy, precision, recall, F1-SCORE, etc. The performance of the model can be further analyzed through methods such as confusion matrix.

[0100] By designing a specific loss function, the model can pay more attention to the accuracy of early tumor detection and punish errors according to the penalty factor. This helps improve the performance of the model in early tumor detection. It is necessary to make some appropriate adjustments and optimizations based on the characteristics of the real questions and data to obtain the best performance and effect.

[0101] Among them, the optimizer selects the Adam optimizer and sets parameters such as the learning rate that suits it. The Adam optimizer combines the characteristics of momentum and adaptive learning rate, and is often used in deep learning tasks. It can automatically adjust the learning rate during training. The learning rate can be adjusted appropriately according to actual conditions. The initial learning rate is set to 0.001. And adjust the decay rate of the learning rate or other parameters according to the performance during training.

[0102] Model training: Batch size: Set according to actual situation. Usually choose a smaller batch size, such as 32, 64, etc.

[0103] Number of training rounds: set as required. Usually choose enough training rounds to achieve convergence.

[0104] Model Evaluation: In the model evaluation phase, a variety of indicators are used to evaluate the performance of the model, including accuracy, precision, recall, F1-SCORE, etc. The following is a detailed description of the calculation and analysis methods of these indicators: Accuracy: Accuracy is one of the most commonly used evaluation indicators in classification models, indicating the proportion of the total number of samples that are correctly classified. The calculation formula is as follows: Accuracy = (number of samples predicted correctly) / (total number of samples) Precision: Precision measures the proportion of samples that are actually positive among those predicted by the model to be positive. The calculation formula is as follows: Precision = (number of samples that are truly positive) / (number of samples predicted to be positive) Recall: Recall measures the proportion of samples that a model can correctly identify a category. That is, the proportion of positive category samples that the model successfully captures. The calculation formula is as follows: Recall = (Number of samples that are truly positive) / (Number of samples that are truly positive) F1-score: F1-SCORE is an evaluation indicator that takes into account both precision and recall. It is the harmonic mean of precision and recall. Its calculation formula is: F1-score = 2 * (precision * recall) / (precision + recall) These indicators can be used to further analyze the performance of the model through the confusion matrix. The confusion matrix is ​​a two-dimensional matrix that represents the correspondence between the model prediction results and the true labels. It contains four important indicators: True Positive (TP), False Positive (FP), True Negative (TN), and False Negative (FN).

[0105] TP: The number of positive samples predicted by the model as positive FP: The number of negative class samples predicted by the model as positive class samples TN: The number of negative class samples predicted by the model as negative class FN: The number of positive samples predicted by the model as negative samples The above indicators can be calculated through the confusion matrix. The following are the calculation steps: According to the model prediction results and the true labels, the samples are classified into four categories: TP, FP, TN, and FN.

[0106] Calculate the accuracy, precision, recall, and F1-SCORE based on the classification results.

[0107] The calculation and analysis of the confusion matrix and the above indicators can help understand the performance of the model on different categories, and then adjust and optimize the model. It should be noted that the above parameters are only for reference, and the specific parameter selection should be adjusted and optimized according to the actual situation and the characteristics of the data set. During the model training process, cross-validation and other techniques can be used to evaluate the performance of the model on different data subsets to select the best parameters and model configuration.

[0108] By calculating these metrics, we can evaluate and compare the performance of the models. Accuracy measures the overall accuracy of the classification. Precision and recall provide an assessment of the accuracy and completeness of the positive category predictions. F1-score combines precision and recall to more comprehensively evaluate the performance of the model.

[0109] In addition, the classification ability and discrimination of the model are evaluated by drawing the ROC curve (receiver operating characteristic curve) and calculating the AUC (area under the curve). The ROC curve is a curve with the recall rate as the horizontal axis and the 1-specificity (False Positive Rate) as the vertical axis. AUC represents the area under the ROC curve and can be used to evaluate the overall performance of the model.

[0110] Step S8: Model fusion Model fusion is the prediction result of combining multiple models. This improves the robustness of the overall forecast technology. In this embodiment, multiple convolutional neural network models with different initial parameters are trained and model fusion is performed using voting. Then, the effectiveness of the fusion model is verified by evaluating its performance indicators, and further adjustments and optimizations are performed as needed.

[0111] Among them, the specific steps of using the voting strategy in model fusion are as follows: Train multiple independent convolutional neural network models: Train multiple convolutional neural network models with different initial parameters. Make sure each model is fully trained and trained on the same training set to ensure fairness.

[0112] Use the test set to make predictions for each model. Use an independent test set (not overlapping with the training set and validation set) to make predictions for each trained model. For each test sample, get the prediction results of each model.

[0113] Voting: For each sample, vote on the prediction results of all models. You can use the majority voting principle to select the category with the highest number of votes as the final prediction result. If there is an equal number of votes, you can choose to randomly select a category or use other decision rules.

[0114] Calculate performance indicators: Use the prediction results after model fusion to calculate the performance indicators of the model, such as accuracy, precision, recall, F1-SCORE, etc. You can use the confusion matrix to further analyze the performance of the model.

[0115] Adjustment and optimization: Further adjustment and optimization of parameters are performed based on the performance indicators of model fusion. You can try to increase or decrease the number of models participating in the vote to find the best model fusion strategy.

[0116] When performing model fusion, ensure that the models participating in the voting are diverse, that is, there are certain differences between the models. This can be achieved by adjusting the model architecture, optimizing parameters, using different loss functions, etc.

[0117] Finally, model fusion can be used to comprehensively utilize the prediction results of multiple models to improve the accuracy and robustness of the overall prediction. But remember that model fusion also needs to weigh the computing resources and time costs, because training and predicting multiple models may require more computing resources and time.

[0118] Step S9: Introduce custom attention mechanism, see Figure 4 ; Define the attention module: This module accepts the feature map (the output of the middle layer of ResNet50) as input, and then outputs a weight matrix of the same size through convolution, Batch Normalization and sigmoid activation function.

[0119] Insert an attention module into ResNet: Insert the attention module defined above into the appropriate position of ResNet (e.g. after the output of a residual block). Then, use the output (weight matrix) of the attention module to weight the feature map to obtain a new feature map. This process is a "weighted attention" process, which allows the network to pay more attention to areas with larger weights (the output of the attention module is close to 1).

[0120] Model training and optimization: Use weighted feature representation for model training and optimization. Select a specific loss function and optimizer, and adjust and optimize parameters according to actual conditions.

[0121] Model evaluation: Use the test set to evaluate the trained model and calculate indicators such as accuracy, precision, recall, and F1-score.

[0122] In practical applications, attention can also be visualized to help understand where the model focuses on the image. By introducing the attention mechanism, convolutional neural networks can focus on important features in the image more accurately, improving the performance and expressiveness of the model.

[0123] Step S10: Model evaluation and optimization The performance of the model is evaluated through cross-validation and other methods. The parameters of the model are adjusted according to the evaluation results to optimize the performance of the model. The specific steps are as follows: Divide the dataset: Divide the image dataset into two parts: training set and test set. In the cross-validation process, the training set is further divided into two parts: training set and validation set. Perform 10-fold cross-validation, and divide the training data into 10 subsets. Each time, 9 subsets are used as training image data, and the remaining subsets are used as validation image data.

[0124] Train the model: For each training / validation data split, train the model using the training data and evaluate the model using the validation data. Record the results of each evaluation.

[0125] Average performance evaluation: After 10 training and validation runs, the average of all validation results is calculated to obtain the average performance evaluation of the model.

[0126] Parameter adjustment: According to the average performance evaluation of the model, adjust the parameters of the model, such as the learning rate, the size and number of convolution kernels, the number of neurons in the fully connected layer, etc. Then repeat the division of the data set, training the model, and average performance evaluation until the optimal model parameters are found.

[0127] Model optimization: After finding the optimal model parameters, use these parameters to train the model and use the test set for final performance evaluation. If the model's performance still has room for improvement, you can try to use a more complex model structure, or introduce regularization, batch normalization and other techniques to optimize the model.

[0128] Result analysis: After the model training and optimization are completed, the training process and results are analyzed. Visualization tools such as learning curves and confusion matrices can be used to help understand the performance of the model. If the model has poor prediction performance in certain categories, the data can be further analyzed to find possible reasons.

[0129] Step S11: Implement prediction / identification The new data after preprocessing is input into the optimized model. The prediction of the malignancy of early prostate tumors is obtained.

[0130] Embodiment 3 The embodiment of the present invention provides an early prostate tumor recognition system based on hyperspectral imaging, the system comprising: The sample collection module is used to collect early prostate tumor tissue samples of different lesion degrees and store them at low temperatures, while obtaining spectral characteristic data of ambient light for correcting subsequent hyperspectral image data; A hyperspectral imaging module is configured to perform hyperspectral scanning on the early prostate tumor tissue sample to generate hyperspectral image data covering multiple bands from visible light to near-infrared light, and perform background noise removal, spectral correction, image registration and normalization on the hyperspectral image data through a preprocessing module; A feature extraction module is configured to extract spectral features and texture features from the preprocessed hyperspectral image data; A data organization module is configured to associate and label the feature data with the label data according to known pathological diagnosis results, and to construct a feature database; A recognition model building module is configured to build an early prostate tumor recognition model, and train and optimize the early prostate tumor recognition model based on the feature data and label data; The recognition module is configured to recognize early prostate tumors on the sample to be tested based on the trained recognition model and output a recognition result.

[0131] Embodiment 4 This embodiment provides a computer-readable storage medium having a computer program stored thereon, which can implement the steps in the prostate tumor identification method of embodiments one and two when executed by a processor, such as the contents described in the above embodiments one and two.

[0132] Embodiment 5 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, which implements the steps in the prostate tumor identification method based on hyperspectral images described in the above-mentioned embodiments 1 and 2, and is implemented when the processor executes the program.

[0133] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying early prostate tumors based on hyperspectral images, the method comprising the following steps: Step S1: collecting early prostate tumor samples with different lesion degrees; Step S2: performing a hyperspectral scan on the early prostate tumor tissue sample to generate corresponding hyperspectral image data and perform preprocessing; Step S3: extracting spectral features and texture features from the preprocessed hyperspectral image data; Step S4: Organizing characteristic data and label data, wherein the label data classifies and labels the samples based on known pathological diagnosis results; Step S5: constructing an early prostate tumor recognition model based on the feature data and label data and optimizing the training; Step S6: Use the trained recognition model to identify early prostate tumors in new samples to be tested.

2. The method according to claim 1, characterized in that In step S2, the preprocessing is specifically: Step S201: Acquire the ambient light spectrum characteristics, read the hyperspectral image data, and load the .mat file containing the hyperspectral data; Step S202: Acquire image data of a specified band; Step S203: splicing the acquired red, green and blue bands of image data into an RGB image through a cat function; Step S204: obtaining one-dimensional spectrum data of each pixel in the RGB image, and subtracting the spectrum characteristics of ambient light; Step S205: normalizing the one-dimensional spectrum data after subtracting the ambient light spectrum characteristics, and compressing the data range to [-1, 1]; Step S206: performing data expansion on the normalized one-dimensional spectral data, and copying 32 rows of data of each pixel point to generate a single-channel grayscale image; Step S207: multiply each pixel data of the grayscale image by 255 and save the data as a grayscale image to generate a two-dimensional grayscale image data set.

3. The method according to claim 1, characterized in that In step S3, the method for extracting tumor spectral features comprises the following steps: Step S3011: selecting a region of interest containing a tumor on the two-dimensional grayscale image; Step S3012: extracting one-dimensional normalized spectral data corresponding to each pixel point in the region of interest; Step S3013: Calculating the statistical features of the one-dimensional normalized spectral data, the statistical features including the mean value, variance, energy, spectrum slope and kurtosis of the spectrum; Step S3014: Store the statistical features at corresponding pixel locations to generate a feature map.

4. The method according to claim 1, characterized in that: In step S3, the method for extracting tumor texture features comprises the following steps: S3021: constructing a gray level co-occurrence matrix, and using the gray level co-occurrence matrix to extract contrast, energy, homogeneity and correlation of the region of interest; S3022: constructing a grayscale difference matrix, and extracting contrast and roughness of the region of interest using the grayscale difference matrix; S3023: Construct a binary image, and extract the frequency and distribution of the texture of the region of interest from the binary image.

5. The method according to claim 4, characterized in that In the step S3011, a manual annotation or threshold segmentation method is used to identify the region of interest where the tumor is located; target pixels are selected in the identified region of interest for subsequent texture feature extraction.

6. The method according to claim 4, characterized in that The step S3021 specifically includes: Define the distance and direction of neighboring pixels, including the choice of horizontal, vertical, or diagonal direction; quantifying the grayscale of the image in the region of interest, and discretizing the grayscale value into a preset grayscale level; Constructing a gray level co-occurrence matrix by counting gray value pairs between pixels in the region of interest and their neighboring pixels at a selected distance and direction to generate a gray level co-occurrence matrix; Extract statistical features of the gray-level co-occurrence matrix, including contrast, energy, homogeneity, and correlation.

7. The method according to claim 4, characterized in that The step S3022 specifically includes: Use the same neighboring pixel distance and direction as the gray-level co-occurrence matrix; Calculating the grayscale difference between the pixel in the region of interest and its neighboring pixels to quantify the grayscale change between the pixels; Construct a grayscale difference matrix by counting the frequency of grayscale difference values ​​of adjacent pixels to generate a grayscale difference matrix; Extract statistical features of the grayscale difference matrix, including contrast and roughness.

8. The method according to claim 1, characterized in that The step S3023 specifically includes: Use the same neighboring pixel distance and direction as the gray-level co-occurrence matrix; Compare the grayscale values ​​of each pixel with those of its neighboring pixels and generate a binary code; Construct a binary image and convert the binary code of each pixel into a local texture pattern; Extract statistical features of binary images, including texture pattern frequency and distribution.

9. The method according to claim 1, characterized in that: The early prostate tumor recognition model in step S5 is a convolutional neural network model based on ResNet50, and introduces residual connection and attention mechanism; The loss function is the cross entropy loss function for the multi-classification task, and the early detection penalty factor is defined as 1.5; Select the Adam optimizer and set the initial learning rate to 0.001, and use the defined weighted loss function to train and optimize the model; Use the test set to evaluate the trained model, calculate the accuracy, precision, recall and F1-score, and analyze the model performance through confusion matrix and ROC curve; Model fusion is performed by training multiple convolutional neural network models with different initial parameters, and the prediction results of each model are merged using a voting strategy; The performance of the model is evaluated through cross-validation, the average performance index is calculated, and the model parameters are adjusted according to the evaluation results to optimize the model performance.

10. The method according to claim 1, characterized in that The early prostate tumor recognition model includes a convolution layer, a pooling layer, a fully connected layer and an output layer in sequence, wherein the convolution layer uses a 3×3 or 5×5 convolution kernel.

Images