Lesion area identification method, system and electronic equipment

Abstract

The present invention provides a method, system and electronic device for identifying a diseased area, including: obtaining initial image data of a region to be identified; preprocessing the initial image data to determine image data to be segmented; segmenting the image data to be segmented based on a preset grayscale threshold parameter to obtain a preset type of tissue region, and determining a grayscale digital signal corresponding to the tissue region; and, collecting Raman molecular information of the tissue region through Raman spectroscopy acquisition technology to obtain spectral data of the tissue region; mapping the spectral data to a grayscale digital signal according to the position information of the tissue region corresponding to the spectral data to obtain mapped spectral data; inputting the mapped spectral data into a pre-trained convolutional neural network model, and outputting the diagnostic result of the mapped spectral data. The method combines image segmentation, Raman spectroscopy acquisition and convolutional neural network analysis to achieve accurate positioning and efficient diagnosis of diseased tissues.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and analysis, and in particular to a lesion area recognition method, system and electronic equipment. Background Art

[0002] For existing technologies for lesion area identification, there is no mature method that can effectively combine the macroscopic features of medical images with the microscopic molecular information of Raman spectroscopy to achieve the fusion analysis of imaging omics and molecular omics, thereby providing a non-invasive, automated intelligent medical diagnosis method. In addition, although convolutional neural networks have shown superiority in image feature extraction, existing technologies have not been able to establish a unified mathematical model between image data and molecular information to achieve accurate multimodal diagnosis. Due to these technical limitations, existing methods are insufficient in terms of automation level, accuracy and consistency of diagnostic results, and it is difficult to meet the clinical needs for high-precision and high-reproducibility diagnosis. Summary of the invention

[0003] The purpose of the present invention is to provide a method, system and electronic device for identifying a lesion area, so as to alleviate the technical problems that the existing methods are insufficient in terms of automation level, accuracy and consistency of diagnostic results, and are difficult to meet the clinical needs for high-precision and high-reproducibility diagnosis, and to improve the accuracy of lesion area identification.

[0004] In a first aspect, an embodiment of the present invention provides a method for identifying a lesion area, comprising: acquiring initial image data of an area to be identified; preprocessing the initial image data to determine image data to be segmented; segmenting the image data to be segmented based on a preset grayscale threshold parameter to obtain a tissue area of ​​a preset type, and determining a grayscale digital signal corresponding to the tissue area; and, collecting Raman molecular information of the tissue area through Raman spectroscopy acquisition technology to obtain spectral data of the tissue area; mapping the spectral data to the grayscale digital signal according to position information of the spectral data corresponding to the tissue area to obtain mapped spectral data; inputting the mapped spectral data into a pre-trained convolutional neural network model, and outputting a diagnostic result of the mapped spectral data.

[0005] In a preferred embodiment of the present invention, after the step of inputting the mapped spectral data into a pre-trained convolutional neural network model and outputting the diagnostic result of the mapped spectral data, the method further includes: inputting the grayscale digital signal and the diagnostic result into a preset regression model, outputting the error value between the grayscale digital signal and the diagnostic result; and determining the accuracy of the diagnostic result based on the error value.

[0006] In a preferred embodiment of the present invention, the steps of preprocessing the above-mentioned initial image data and determining the image data to be segmented include: segmenting the above-mentioned initial image data according to a preset scale to obtain a two-dimensional matrix; each pixel point in the above-mentioned two-dimensional matrix carries a corresponding grayscale value; based on the above-mentioned two-dimensional matrix, normalizing each of the above-mentioned pixel points to obtain a normalized matrix; and determining the above-mentioned normalized matrix as the above-mentioned image data to be segmented.

[0007] In a preferred embodiment of the present invention, the steps of segmenting the above-mentioned image data to be segmented based on preset grayscale threshold parameters to obtain tissue regions of preset types, and determining the grayscale digital signals corresponding to the above-mentioned tissue regions include: segmenting the above-mentioned normalized matrix based on preset grayscale threshold parameters to obtain tissue regions of preset types, and determining the grayscale digital signals corresponding to the above-mentioned tissue regions.

[0008] In a preferred embodiment of the present invention, the preset types of tissue regions include: malignant tumor regions, benign tumor regions and necrotic tissue regions; based on preset grayscale threshold parameters, the above-mentioned normalized matrix is ​​segmented to obtain the preset types of tissue regions, and the grayscale digital signals corresponding to the above-mentioned tissue regions are determined; and the steps of collecting Raman molecular information of the above-mentioned tissue regions through Raman spectroscopy acquisition technology to obtain the spectral data of the above-mentioned tissue regions include: based on preset grayscale threshold parameters, the above-mentioned normalized matrix is ​​segmented to obtain the above-mentioned malignant tumor regions, the above-mentioned benign tumor regions and the above-mentioned necrotic tissue regions, and the grayscale digital signals corresponding to the above-mentioned malignant tumor regions, the above-mentioned benign tumor regions and the above-mentioned necrotic tissue regions are determined; and the Raman molecular information of the above-mentioned malignant tumor regions, the above-mentioned benign tumor regions and the above-mentioned necrotic tissue regions are collected through Raman spectroscopy acquisition technology to obtain the spectral data of the above-mentioned malignant tumor regions, the above-mentioned benign tumor regions and the above-mentioned necrotic tissue regions are obtained.

[0009] In a preferred embodiment of the present invention, the pre-trained convolutional neural network model includes: an input layer, multiple convolutional layers, a pooling layer and a fully connected layer; the mapped spectral data is input into the pre-trained convolutional neural network model, and the step of outputting the diagnostic result of the mapped spectral data includes: inputting the mapped spectral data through the input layer, extracting local features of the mapped spectral data through the multiple convolutional layers, performing dimensionality reduction processing on the local features through the pooling layer, mapping the reduced dimensionality local features to a specific diagnostic result classification through the fully connected layer, and calculating the probability that the reduced dimensionality local features correspond to the target classification result in the diagnostic result classification through the softmax function in the fully connected layer.

[0010] In a preferred embodiment of the present invention, the expression of the softmax function is:

[0011]

[0012] in, is the probability of the above target classification result, a is the category of the above target classification, is the preset weight matrix of the above fully connected layer, As the feature map output of the above pooling layer, K is the number of categories of the above diagnosis results.

[0013] In a preferred embodiment of the present invention, the step of collecting Raman molecular information of the above-mentioned tissue area by Raman spectroscopy acquisition technology to obtain spectral data of the above-mentioned tissue area includes: based on multimodal image registration technology, physical marking or intraoperative navigation technology, collecting Raman molecular information of the above-mentioned tissue area by Raman spectroscopy acquisition technology to obtain spectral data of the above-mentioned tissue area.

[0014] In the second aspect, an embodiment of the present invention provides a lesion area identification system, including: a data acquisition module, used to acquire initial image data of the area to be identified; a preprocessing module, used to preprocess the above initial image data and determine the image data to be segmented; a segmentation module, used to segment the above image data to be segmented based on a preset grayscale threshold parameter, to obtain a preset type of tissue area, and determine the grayscale digital signal corresponding to the above tissue area; and, through Raman spectroscopy acquisition technology, Raman molecular information of the above tissue area is collected to obtain spectral data of the above tissue area; a diagnosis result output module, used to map the above spectral data to the above grayscale digital signal according to the position information of the above tissue area corresponding to the above spectral data, to obtain the mapped spectral data; the above mapped spectral data is input into a pre-trained convolutional neural network model, and the diagnosis result of the above mapped spectral data is output.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the lesion area identification method.

[0016] The embodiments of the present invention have the following beneficial technical effects:

[0017] The embodiment of the present invention provides a method, system and electronic device for identifying a diseased area, including: obtaining initial image data of a region to be identified; preprocessing the initial image data to determine image data to be segmented; segmenting the image data to be segmented based on a preset grayscale threshold parameter to obtain a preset type of tissue region, and determining a grayscale digital signal corresponding to the tissue region; and, collecting Raman molecular information of the tissue region through Raman spectroscopy acquisition technology to obtain spectral data of the tissue region; mapping the spectral data to the grayscale digital signal according to the position information of the spectral data corresponding to the tissue region to obtain mapped spectral data; inputting the mapped spectral data into a pre-trained convolutional neural network model, and outputting the diagnostic result of the mapped spectral data. The method combines image segmentation, Raman spectroscopy acquisition and convolutional neural network analysis to achieve accurate positioning and efficient diagnosis of diseased tissues. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A schematic diagram of a flow chart of a lesion area identification method provided by an embodiment of the present invention;

[0020] Figure 2 A schematic flow chart of another lesion area identification method provided by an embodiment of the present invention;

[0021] Figure 3 A schematic diagram of the structure of a lesion area recognition system provided by an embodiment of the present invention;

[0022] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0023] Icons: 31 - data acquisition module; 32 - preprocessing module; 33 - segmentation module; 34 - diagnosis result output module; 41 - memory; 42 - processor; 43 - bus; 44 - communication interface. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0025] Among the current medical imaging diagnostic technologies, imaging technologies such as computed tomography (CT) and magnetic resonance imaging (MRI) have been widely used in clinical practice. They are not only used for the detection and evaluation of lung nodules, but also play an important role in the diagnosis of various organs and diseases, including brain lesions, bone and joint problems, abdominal organ lesions, and cardiovascular disease. Imaging images can provide macroscopic anatomical feature information such as tissue morphology, density, and texture, and assist doctors in diagnosing various lesions (such as tumors, inflammation, and pathological structures). However, existing medical imaging diagnostic methods still rely on the subjective judgment of doctors, and there are certain diagnostic errors and individual differences. At the same time, imaging examinations can only reflect the macroscopic structural characteristics of tissues, and cannot provide information about the microscopic molecular level of the lesion area, which limits its application in molecular diagnosis.

[0026] In order to solve this limitation, Raman spectroscopy has gradually been applied to the field of medical diagnosis. As a non-invasive spectroscopic technology, Raman spectroscopy can accurately identify the molecular composition and chemical structure information in tissues and cells by detecting the vibration characteristics of molecules, and has a high degree of molecular-level specificity. In the diagnosis of various diseases, especially tumors, Raman spectroscopy can distinguish normal tissue from diseased tissue by analyzing the characteristics at the molecular level, thereby helping to determine the nature of the lesion (such as benign or malignant). However, traditional Raman spectroscopy technology usually requires in vitro analysis through tissue samples, which is an invasive operation and is difficult to be widely used in clinical practice. In addition, Raman spectroscopy data is complex, the signal is weak, the processing process is cumbersome, and it is easily interfered by noise, which leads to certain limitations in its diagnostic efficiency and accuracy in practical applications.

[0027] Furthermore, Convolutional Neural Networks (CNN), as a type of deep learning technology, has shown unique advantages in Raman spectroscopy data analysis. CNN's automatic feature learning ability can extract highly relevant molecular feature information from highly complex Raman spectroscopy data with low signal-to-noise ratio, overcoming the strict dependence of traditional Raman spectroscopy analysis on signal quality. CNN effectively captures subtle differences related to specific molecular vibration modes through multi-level, nonlinear processing of spectral data, thereby achieving efficient identification and classification of target molecular features without relying on human intervention. Compared with traditional chemometric methods, CNN can significantly improve the efficiency of effective information extraction from low-quality spectral signals and maintain a high degree of diagnostic accuracy in noisy environments.

[0028] The shortcoming of the prior art is that there is no mature system that can effectively combine the macroscopic features of medical images with the microscopic molecular information of Raman spectroscopy to achieve the fusion analysis of imaging omics and molecular omics, thereby providing a non-invasive, automated intelligent medical diagnosis system. In addition, although CNN has shown superiority in image feature extraction, the prior art has not been able to establish a unified mathematical model between image data and molecular information to achieve accurate multimodal diagnosis. Due to these technical limitations, the existing system has deficiencies in the level of automation, accuracy and consistency of diagnostic results, and it is difficult to meet the clinical needs for high-precision and high-reproducibility diagnosis. In view of the above-mentioned deficiencies in the prior art, the present invention proposes a digital medical imaging diagnosis system based on Raman spectroscopy and convolutional neural network, which realizes accurate molecular diagnosis of various lesions by fusing imaging omics and molecular omics information.

[0029] Based on this, an embodiment of the present invention provides a lesion region recognition method, system and electronic device, which combines image segmentation, Raman spectrum acquisition and convolutional neural network analysis to achieve accurate positioning and efficient diagnosis of lesion tissue. For ease of understanding, the above-mentioned lesion region recognition method is first introduced in detail.

[0030] Example 1

[0031] In this embodiment, Figure 1 A schematic flow chart of a lesion area identification method provided in an embodiment of the present invention.

[0032] Depend on Figure 1 As can be seen, the method includes:

[0033] Step S101: Acquire initial image data of the area to be identified.

[0034] Step S102: pre-processing the initial image data to determine the image data to be segmented.

[0035] In this embodiment, the present invention first represents the above-mentioned initial image data as a two-dimensional matrix, assuming that the size of the image is M×N, where each pixel corresponds to a grayscale value. The grayscale value ranges from [0, 255] and is usually used to represent the brightness intensity of different pixels in the image, reflecting the density or contrast of the tissue or organ. The grayscale value of 0 represents black and 255 represents white. Normal tissue is usually presented as an isodensity image with a grayscale value close to the middle value; the tumor area with a higher degree of malignancy has a higher density, which is presented as a high-density area in the image and tends to be white; the area density of benign tumors and necrotic tissues is lower, which is presented as a low-density area and tends to be black.

[0036] In one implementation manner, the initial image data is taken as a CT image.

[0037] Step S103: based on the preset gray threshold parameter, the above-mentioned image data to be segmented is segmented to obtain a preset type of tissue area, and the gray digital signal corresponding to the above-mentioned tissue area is determined; and Raman molecular information of the above-mentioned tissue area is collected by Raman spectroscopy acquisition technology to obtain spectral data of the above-mentioned tissue area.

[0038] In actual operation, different imaging devices and conditions will lead to differences in grayscale values. To ensure that the model can stably process different data, the present invention normalizes the image grayscale values ​​and scales the grayscale values ​​to the range of [0, 1] to reduce the interference of different imaging conditions on data processing and ensure the consistency of data and the robustness of the model. In order to extract the region of interest (ROI) in medical images, the present invention adopts a threshold segmentation method and sets the grayscale threshold T. The specific segmentation formula is as follows:

[0039] C(i ,j)=

[0040] Wherein, C(i, j) is the tissue area of ​​the above preset type, is the gray value after normalization.

[0041] Here, the image matrix M×N of the preset tissue area is traversed through a nested loop, and the normalized grayscale value of each pixel is extracted row by row and column by column. During the traversal process, the normalized grayscale value of each pixel is extracted, which represents the density of different tissues. The normalized grayscale value of each pixel constitutes the grayscale digital signal corresponding to the tissue area.

[0042] Before performing Raman spectroscopy pathology analysis, in order to ensure that the spectral acquisition points in the pathological sample correspond to the specific positions in the imaging data (such as CT, MRI), a three-dimensional reconstruction method such as the Marching Cubes algorithm can be used to reconstruct a static three-dimensional structural model of the target area, and further introduce the finite element method (FEM) or adaptive voxel grid reconstruction technology (AVGR) to ensure that the three-dimensional model is always consistent with the actual anatomical structure under dynamic conditions. The spectral information of the tissue region for Raman molecular information acquisition is accurately matched with the grayscale digital signal corresponding to the above tissue region. After the registration is completed, Raman spectral acquisition is performed based on the specific area in the in vitro pathological sample. Since the imaging and pathology data have been registered, the spectral data of each acquisition point can be mapped to the corresponding pixel position in the image.

[0043] Step S104: mapping the spectral data to the grayscale digital signal according to the position information of the spectral data corresponding to the tissue region to obtain mapped spectral data.

[0044] In this embodiment, the step S104 includes: matching the spectral data with the position information of the grayscale digital signal according to the position information of the spectral data corresponding to the tissue area to obtain matched spectral data, i.e., the mapped spectral data; and then, using the matched spectral data as the mapped spectral data.

[0045] In actual operations, other spectral analysis techniques can also be used as alternatives. For example, infrared spectroscopy (IR) or ultraviolet-visible spectroscopy (UV-Vis) can also provide information at the molecular level and be combined with the above-mentioned imaging data to achieve diagnosis.

[0046] Specifically, the spectral data of each acquisition point of the tissue region may be mapped to a corresponding pixel position in the grayscale digital signal, so that the spectral data is matched with the position information of the grayscale digital signal.

[0047] Step S105: input the mapped spectral data into a pre-trained convolutional neural network model, and output the diagnostic result of the mapped spectral data.

[0048] In actual operation, the above-mentioned pre-trained convolutional neural network model can also replace other machine learning algorithms that can classify the above-mentioned mapped spectral data, such as: a pre-trained support vector machine (SVM) or a pre-trained random forest.

[0049] The embodiment of the present invention provides a method for identifying a lesion region, including: obtaining initial image data of a region to be identified; preprocessing the initial image data to determine image data to be segmented; segmenting the image data to be segmented based on a preset grayscale threshold parameter to obtain a preset type of tissue region, and determining a grayscale digital signal corresponding to the tissue region; and, collecting Raman molecular information of the tissue region through Raman spectroscopy acquisition technology to obtain spectral data of the tissue region; mapping the spectral data to the grayscale digital signal according to the position information of the spectral data corresponding to the tissue region to obtain mapped spectral data; inputting the mapped spectral data into a pre-trained convolutional neural network model, and outputting the diagnostic result of the mapped spectral data. The method combines image segmentation, Raman spectroscopy acquisition and convolutional neural network analysis to achieve accurate positioning and efficient diagnosis of lesion tissues.

[0050] Example 2

[0051] Based on the above embodiments, Figure 2 A schematic flow chart of another lesion area identification method provided in an embodiment of the present invention.

[0052] Depend on Figure 2 As can be seen, the method includes:

[0053] Step S201: Acquire initial image data of the area to be identified.

[0054] Step S202: pre-processing the initial image data to determine the image data to be segmented.

[0055] In this embodiment, the steps of preprocessing the above-mentioned initial image data and determining the image data to be segmented include: segmenting the above-mentioned initial image data according to a preset scale to obtain a two-dimensional matrix; each pixel point in the above-mentioned two-dimensional matrix carries a corresponding grayscale value; based on the above-mentioned two-dimensional matrix, normalizing each of the above-mentioned pixel points to obtain a normalized matrix; and determining the above-mentioned normalized matrix as the above-mentioned image data to be segmented.

[0056] Step S203: based on the preset gray threshold parameter, segment the above-mentioned image data to be segmented to obtain a preset type of tissue area, and determine the gray digital signal corresponding to the above-mentioned tissue area; and, collect Raman molecular information of the above-mentioned tissue area through Raman spectroscopy acquisition technology to obtain spectral data of the above-mentioned tissue area.

[0057] Here, based on the preset grayscale threshold parameters, the above-mentioned image data to be segmented is segmented to obtain a preset type of tissue area, and the step of determining the grayscale digital signal corresponding to the above-mentioned tissue area includes: based on the preset grayscale threshold parameters, the above-mentioned normalized matrix is ​​segmented to obtain a preset type of tissue area, and the grayscale digital signal corresponding to the above-mentioned tissue area is determined.

[0058] In one embodiment, the preset types of tissue regions include: malignant tumor regions, benign tumor regions and necrotic tissue regions; based on preset grayscale threshold parameters, the above-mentioned normalized matrix is ​​segmented to obtain the preset types of tissue regions, and the grayscale digital signals corresponding to the above-mentioned tissue regions are determined; and the steps of collecting Raman molecular information of the above-mentioned tissue regions through Raman spectroscopy acquisition technology to obtain the spectral data of the above-mentioned tissue regions include: based on preset grayscale threshold parameters, the above-mentioned normalized matrix is ​​segmented to obtain the above-mentioned malignant tumor regions, the above-mentioned benign tumor regions and the above-mentioned necrotic tissue regions, and the grayscale digital signals corresponding to the above-mentioned malignant tumor regions, the above-mentioned benign tumor regions and the above-mentioned necrotic tissue regions are determined; and the Raman molecular information of the above-mentioned malignant tumor regions, the above-mentioned benign tumor regions and the above-mentioned necrotic tissue regions are collected through Raman spectroscopy acquisition technology to obtain the spectral data of the above-mentioned malignant tumor regions, the above-mentioned benign tumor regions and the above-mentioned necrotic tissue regions are obtained.

[0059] In actual operation, the preset type of tissue area segmentation of the above-mentioned image data to be segmented can also be achieved through image segmentation methods based on texture features or automatic segmentation technology based on deep learning (such as U-Net or Mask R-CNN), and associated with the following spectral data.

[0060] Furthermore, the step of collecting Raman molecular information of the above-mentioned tissue area by Raman spectroscopy acquisition technology to obtain spectral data of the above-mentioned tissue area includes: based on multimodal image registration technology, physical marking or intraoperative navigation technology, collecting Raman molecular information of the above-mentioned tissue area by Raman spectroscopy acquisition technology to obtain spectral data of the above-mentioned tissue area.

[0061] Step S204: mapping the spectral data to the grayscale digital signal according to the position information of the spectral data corresponding to the tissue region to obtain mapped spectral data.

[0062] Step S205: input the mapped spectral data into a pre-trained convolutional neural network model, and output the diagnostic result of the mapped spectral data.

[0063] In this embodiment, the pre-trained convolutional neural network model includes: an input layer, multiple convolutional layers, a pooling layer and a fully connected layer; the mapped spectral data is input into the pre-trained convolutional neural network model, and the step of outputting the diagnostic result of the mapped spectral data includes: inputting the mapped spectral data through the input layer, extracting local features of the mapped spectral data through the multiple convolutional layers, performing dimensionality reduction processing on the local features through the pooling layer, mapping the reduced dimensionality local features to a specific diagnostic result classification through the fully connected layer, and calculating the probability that the reduced dimensionality local features correspond to the target classification result in the diagnostic result classification through the softmax function in the fully connected layer.

[0064] Furthermore, the expression of the above softmax function is:

[0065]

[0066] in, is the probability of the above target classification result, a is the category of the above target classification, is the preset weight matrix of the above fully connected layer, As the feature map output of the above pooling layer, K is the number of categories of the above diagnosis results.

[0067] Step S206: input the grayscale digital signal and the diagnosis result into a preset regression model, and output an error value between the grayscale digital signal and the diagnosis result.

[0068] In this embodiment, the above step S206 combines the gray value after the above normalization with the spectrum data of the Raman spectrum to perform regression analysis. Assume that the gray value of each pixel is G′(i, j) and the diagnosis result of the Raman spectrum is R. The relationship between the two can be expressed by the regression model as follows:

[0069] R=a⋅G′(i,j)+b

[0070] The goal of regression analysis is to minimize the error between the predicted value and the actual value, specifically to minimize the following objective function:

[0071]

[0072] Among them, R i，j is the diagnostic result of the Raman spectrum of pixel point (i, j).

[0073] Therefore, based on the above G′(i, j), the Raman spectrum diagnosis results of the corresponding area are inferred to achieve an effective combination of imaging features and molecular diagnosis.

[0074] In addition, the present invention introduces the molecular diagnostic information of grayscale value distribution and Raman spectrum in the image through CNN model learning, and continuously optimizes the feature extraction and classification capabilities. The trained CNN model can be used for automated diagnosis of images, further improving the accuracy and reliability of diagnosis. Finally, through the above process, the grayscale value of any target area in the image can be quickly analyzed, and the corresponding Raman spectrum diagnostic results can be calculated according to the regression model.

[0075] In actual operations, there are many options for data fusion methods. In addition to regression models, multimodal learning algorithms, joint analysis models, etc. can also be used to combine the normalized grayscale values ​​with spectral data in different ways to achieve inference and diagnosis of molecular information.

[0076] Step S207: Determine the accuracy of the diagnosis result according to the error value.

[0077] The embodiment of the present invention provides a method for identifying a lesion region, including: obtaining initial image data of a region to be identified; preprocessing the initial image data to determine image data to be segmented; segmenting the image data to be segmented based on a preset grayscale threshold parameter to obtain a preset type of tissue region, and determining a grayscale digital signal corresponding to the tissue region; and, collecting Raman molecular information on the tissue region through Raman spectroscopy acquisition technology to obtain spectral data of the tissue region; mapping the spectral data to the grayscale digital signal according to the position information of the spectral data corresponding to the tissue region to obtain the mapped spectral data; inputting the mapped spectral data into a pre-trained convolutional neural network model, and outputting the diagnosis result of the mapped spectral data; inputting the grayscale digital signal and the diagnosis result into a preset regression model, and outputting the error value between the grayscale digital signal and the diagnosis result; and determining the accuracy of the diagnosis result according to the error value. The method realizes high-precision identification and evaluation of lesion tissues by combining image segmentation, Raman spectroscopy acquisition, convolutional neural network diagnosis and regression model error analysis, and further effectively improves the accuracy and reliability of the diagnosis result.

[0078] Example 3

[0079] Based on the above embodiments, Figure 3 A structural schematic diagram of a lesion area recognition system provided in an embodiment of the present invention.

[0080] Depend on Figure 3 As can be seen, the system includes:

[0081] The data acquisition module 31 is used to acquire the initial image data of the area to be identified.

[0082] The preprocessing module 32 is used to preprocess the initial image data to determine the image data to be segmented.

[0083] The segmentation module 33 is used to segment the above-mentioned image data to be segmented based on a preset grayscale threshold parameter to obtain a preset type of tissue area and determine the grayscale digital signal corresponding to the above-mentioned tissue area; and to collect Raman molecular information on the above-mentioned tissue area through Raman spectroscopy acquisition technology to obtain spectral data of the above-mentioned tissue area.

[0084] The diagnosis result output module 34 is used to map the above-mentioned spectral data to the above-mentioned grayscale digital signal according to the position information of the above-mentioned spectral data corresponding to the above-mentioned tissue area, so as to obtain the mapped spectral data; input the above-mentioned mapped spectral data into a pre-trained convolutional neural network model, and output the diagnosis result of the above-mentioned mapped spectral data.

[0085] The data acquisition module 31 , the preprocessing module 32 , the segmentation module 33 and the diagnosis result output module 34 are connected in sequence.

[0086] In one embodiment, the diagnosis result output module 34 is further used to input the grayscale digital signal and the diagnosis result into a preset regression model, and output the error value between the grayscale digital signal and the diagnosis result; and determine the accuracy of the diagnosis result based on the error value.

[0087] In one embodiment, the preprocessing module 32 is also used to segment the above-mentioned initial image data according to a preset scale to obtain a two-dimensional matrix; each pixel point in the above-mentioned two-dimensional matrix carries a corresponding grayscale value; based on the above-mentioned two-dimensional matrix, each of the above-mentioned pixel points is normalized to obtain a normalized matrix; and the above-mentioned normalized matrix is ​​determined as the above-mentioned image data to be segmented.

[0088] In one embodiment, the segmentation module 33 is further used to segment the normalized matrix based on a preset grayscale threshold parameter to obtain a preset type of tissue region and determine the grayscale digital signal corresponding to the tissue region.

[0089] In one embodiment, the preset types of tissue regions include: malignant tumor regions, benign tumor regions and necrotic tissue regions; the segmentation module 33 is also used to segment the above-mentioned normalized matrix based on preset grayscale threshold parameters to obtain the above-mentioned malignant tumor regions, the above-mentioned benign tumor regions and the above-mentioned necrotic tissue regions, and determine the grayscale digital signals corresponding to the above-mentioned malignant tumor regions, the above-mentioned benign tumor regions and the above-mentioned necrotic tissue regions, respectively; and, through Raman spectroscopy acquisition technology, Raman molecular information of the above-mentioned malignant tumor regions, the above-mentioned benign tumor regions and the above-mentioned necrotic tissue regions is collected to obtain spectral data of the above-mentioned malignant tumor regions, the above-mentioned benign tumor regions and the above-mentioned necrotic tissue regions.

[0090] In one embodiment, the pre-trained convolutional neural network model includes: an input layer, multiple convolutional layers, a pooling layer and a fully connected layer; the diagnosis result output module 34 is also used to input the mapped spectral data through the input layer, extract the local features of the mapped spectral data through the multiple convolutional layers, perform dimensionality reduction processing on the local features through the pooling layer, map the reduced dimensionality local features to a specific diagnosis result classification through the fully connected layer, and calculate the probability that the reduced dimensionality local features correspond to the target classification result in the diagnosis result classification through the softmax function in the fully connected layer.

[0091] In one embodiment, the segmentation module 33 is also used to collect Raman molecular information of the above-mentioned tissue area through Raman spectroscopy acquisition technology based on multimodal image registration technology, physical marking or intraoperative navigation technology to obtain spectral data of the above-mentioned tissue area.

[0092] The lesion region identification system provided by the embodiment of the present invention has the same technical features as the lesion region identification method provided by the above embodiment, so it can also solve the same technical problems and achieve the same technical effects. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system described above can refer to the corresponding process in the above method embodiment, and will not be repeated here.

[0093] Example 4

[0094] This embodiment provides an electronic device, including a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the lesion area identification method.

[0095] This embodiment provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the lesion area identification method are implemented.

[0096] See also Figure 4 The schematic diagram of the structure of an electronic device shown in the figure comprises: a memory 41 and a processor 42. The memory 41 stores a computer program that can be run on the processor 42. When the processor executes the computer program, the steps provided by the above-mentioned lesion area identification method are implemented.

[0097] like Figure 4 As shown, the device further includes: a bus 43 and a communication interface 44, a processor 42, a communication interface 44 and a memory 41 are connected via the bus 43; the processor 42 is used to execute executable modules stored in the memory 41, such as computer programs.

[0098] The memory 41 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 44 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0099] The bus 43 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0100] Among them, the memory 41 is used to store the program, and the processor 42 executes the program after receiving the execution instruction. The method executed by the lesion area identification system disclosed in any embodiment of the present invention can be applied to the processor 42, or implemented by the processor 42. The processor 42 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 42. The above processor 42 can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 41, and the processor 42 reads the information in the memory 41 and completes the steps of the above method in combination with its hardware.

[0101] Furthermore, an embodiment of the present invention also provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by the processor 42, the machine-executable instructions prompt the processor 42 to implement the above-mentioned lesion area identification method.

[0102] The electronic device and computer-readable storage medium provided by the embodiments of the present invention have the same technical features, and therefore can solve the same technical problems and achieve the same technical effects.

[0103] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0104] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the system or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

Claims

1. A method for identifying a lesion area, characterized in that: include: Acquire initial image data of the area to be identified; Preprocessing the initial image data to determine the image data to be segmented; Based on a preset grayscale threshold parameter, the image data to be segmented is segmented to obtain a preset type of tissue region, and a grayscale digital signal corresponding to the tissue region is determined; And, collecting Raman molecular information of the tissue region by Raman spectrum collection technology to obtain spectrum data of the tissue region; According to the position information of the tissue region corresponding to the spectral data, the spectral data is matched with the position information of the grayscale digital signal to obtain matched spectral data; and the matched spectral data is determined as mapped spectral data; The mapped spectral data is input into a pre-trained convolutional neural network model, and the diagnosis result of the mapped spectral data is output.

2. The lesion area recognition method according to claim 1, characterized in that: After the step of inputting the mapped spectral data into a pre-trained convolutional neural network model and outputting the diagnostic result of the mapped spectral data, the method further comprises: Inputting the grayscale digital signal and the diagnosis result into a preset regression model, and outputting an error value between the grayscale digital signal and the diagnosis result; The accuracy of the diagnosis result is determined according to the error value.

3. The lesion area recognition method according to claim 1, characterized in that: The step of preprocessing the initial image data to determine the image data to be segmented includes: Segmenting the initial image data according to a preset scale to obtain a two-dimensional matrix; each pixel in the two-dimensional matrix carries a corresponding grayscale value; Based on the two-dimensional matrix, normalizing each pixel point to obtain a normalized matrix; The normalized matrix is ​​determined as the image data to be segmented.

4. The lesion area recognition method according to claim 3, characterized in that: The steps of segmenting the image data to be segmented based on a preset grayscale threshold parameter to obtain a preset type of tissue region, and determining a grayscale digital signal corresponding to the tissue region include: Based on a preset grayscale threshold parameter, the normalized matrix is ​​segmented to obtain a preset type of tissue region, and a grayscale digital signal corresponding to the tissue region is determined.

5. The lesion area recognition method according to claim 4, characterized in that: The preset types of tissue regions include: malignant tumor region, benign tumor region and necrotic tissue region; The steps of segmenting the normalized matrix based on a preset grayscale threshold parameter to obtain a preset type of tissue region and determining a grayscale digital signal corresponding to the tissue region; and acquiring Raman molecular information of the tissue region by Raman spectroscopy acquisition technology to obtain spectral data of the tissue region include: Based on preset grayscale threshold parameters, the normalized matrix is ​​segmented to obtain the malignant tumor area, the benign tumor area and the necrotic tissue area, and the grayscale digital signals corresponding to the malignant tumor area, the benign tumor area and the necrotic tissue area are determined; and Raman molecular information of the malignant tumor area, the benign tumor area and the necrotic tissue area is collected through Raman spectroscopy acquisition technology to obtain spectral data of the malignant tumor area, the benign tumor area and the necrotic tissue area.

6. The lesion area recognition method according to claim 1, characterized in that: The pre-trained convolutional neural network model includes: an input layer, multiple convolutional layers, a pooling layer, and a fully connected layer; The step of inputting the mapped spectral data into a pre-trained convolutional neural network model and outputting the diagnostic result of the mapped spectral data comprises: The mapped spectral data is input through the input layer, local features of the mapped spectral data are extracted through the multiple convolutional layers, dimension reduction is performed on the local features through the pooling layer, the local features after dimension reduction are mapped to specific diagnosis result classifications through the fully connected layer, and the probability that the local features after dimension reduction correspond to the target classification results in the diagnosis result classification is calculated through the softmax function in the fully connected layer.

7. The lesion area recognition method according to claim 6, characterized in that: The expression of the softmax function is: in, is the probability of the target classification result, a is the category of the target classification, is the preset weight matrix of the fully connected layer, As the feature map output of the pooling layer, K is the number of categories for classifying the diagnosis results.

8. The lesion area identification method according to claim 1, characterized in that: The step of collecting Raman molecular information of the tissue region by using Raman spectrum collection technology to obtain spectrum data of the tissue region includes: Based on multimodal image registration technology, physical marking or intraoperative navigation technology, Raman spectroscopy acquisition technology is used to collect Raman molecular information of the tissue area to obtain spectral data of the tissue area.

9. A lesion area recognition system, characterized in that: include: A data acquisition module, used to acquire initial image data of the area to be identified; A preprocessing module, used for preprocessing the initial image data to determine the image data to be segmented; A segmentation module, used to segment the image data to be segmented based on a preset grayscale threshold parameter, obtain a tissue region of a preset type, and determine a grayscale digital signal corresponding to the tissue region; And, collecting Raman molecular information of the tissue region by Raman spectrum collection technology to obtain spectrum data of the tissue region; The diagnosis result output module is used to match the spectral data with the position information of the grayscale digital signal according to the position information of the tissue area corresponding to the spectral data to obtain matched spectral data; determine the matched spectral data as mapped spectral data; input the mapped spectral data into a pre-trained convolutional neural network model, and output the diagnosis result of the mapped spectral data.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the lesion area identification method according to any one of claims 1 to 8.

Images