A neural tumor recognition positioning method and system based on image recognition

Abstract

The application relates to the technical field of medical image processing, and discloses a neural tumor recognition and positioning method and system based on image recognition, which acquires a neural medical image of a patient, pre-processes the image, extracts features of the pre-processed image by using a Transformer-ConvNet hybrid model, inputs the extracted features into a classifier, performs preliminary recognition of a tumor, generates a candidate region containing the tumor in the image by using an RPN network based on deformable convolution if the preliminary recognition exists the tumor, performs semantic segmentation on the candidate region by using a fully-connected neural network, determines the specific position and contour of the tumor, generates a final recognition result according to the specific position and contour of the tumor and in combination with morphological characteristics of the tumor; and the application can more efficiently and accurately recognize and position the neural tumor, thereby providing strong support for clinical diagnosis and treatment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, in particular to a neural tumor recognition and positioning method and system based on image recognition. BACKGROUND

[0002] Accurate recognition and positioning of neural tumors are crucial in clinical diagnosis and treatment. Currently, the commonly used method includes manual analysis of medical images such as CT, MRI, etc. by doctors, but this method relies on the experience of doctors, is low in efficiency and prone to subjective judgment errors. With the development of computer technology, automatic analysis methods based on image recognition have been gradually applied. However, the recognition accuracy and positioning precision of some existing algorithms in complex neural image environment still need to be improved, especially for early-stage micro-tumors or tumors with low contrast with the surrounding tissue, which are prone to missed detection or misjudgment. Therefore, there is an urgent need for a more efficient and accurate neural tumor recognition and positioning method. SUMMARY

[0003] The present application aims to solve the above problems and provides a neural tumor recognition and positioning method and system based on image recognition.

[0004] The first aspect of the present application provides a neural tumor recognition and positioning method based on image recognition, which comprises the following steps:

[0005] Obtain the neural medical image of the patient, and pre-process the image, wherein the pre-processing includes image denoising, gray scale normalization and image enhancement;

[0006] Use a Transformer-ConvNet hybrid model to extract features from the pre-processed image, input the extracted features into a classifier, and perform preliminary identification of the tumor;

[0007] If the preliminary identification exists a tumor, use a RPN network based on deformable convolution to generate a candidate region containing the tumor in the image;

[0008] Use a fully connected neural network to perform semantic segmentation on the candidate region to determine the specific position and contour of the tumor;

[0009] According to the specific position and contour of the tumor, combined with the morphological characteristics of the tumor, generate the final recognition result.

[0010] Optionally, in the first implementation manner of the first aspect of the present application, the obtaining the neural medical image of the patient and pre-processing the image comprises:

[0011] Obtain the neural medical image of the patient, calculate the weighted mean by searching all regions similar to the current pixel block in the image to perform denoising, and obtain the denoised image;

[0012] An effective region is extracted by an Otsu threshold method, 1% and 99% quantiles of gray values in the effective region are calculated, extreme values exceeding the range are truncated, and the truncated gray values are linearly scaled to a target interval to obtain a gray normalized image;

[0013] The gray normalized image is divided into overlapping sub-blocks, histogram equalization is independently performed on each sub-block, a contrast limit threshold is set to 0.01, a part exceeding the threshold is uniformly distributed to other gray levels, processing results of adjacent sub-blocks are merged through bilinear interpolation to obtain a preprocessed image.

[0014] Optionally, in a second implementation manner of the first aspect of the present application, the feature extraction on the preprocessed image by using the Transformer-ConvNet hybrid model comprises:

[0015] The preprocessed image is input into the Transformer-ConvNet hybrid model, in the ConvNet part, local feature maps with different scales are obtained through multi-layer convolution and pooling operations, and the local feature maps are flattened into sequences as inputs of the Transformer;

[0016] The attention weights of each position to other positions are calculated through the multi-head self-attention mechanism to capture long-distance dependency relationships between different regions of the image and obtain global features;

[0017] The global features output by the Transformer part are fused with the local features extracted by the ConvNet to obtain hybrid features.

[0018] The Transformer part is composed of multiple Transformer encoders, and each encoder contains a multi-head self-attention mechanism and a feedforward neural network.

[0019] Optionally, in a third implementation manner of the first aspect of the present application, the feature extraction is performed on the preprocessed image by using the Transformer-ConvNet hybrid model, and the extracted features are input into a classifier to preliminarily identify the tumor, comprising:

[0020] The features extracted by the Transformer-ConvNet hybrid model are input into three parallel branches of the classifier, different scale feature abstraction is respectively performed, attention weights are generated through a fully connected layer for an output of a global context branch, and spatial attention maps are respectively generated through a 3x3 convolution and a Sigmoid activation function for an output of a local detail branch and an output of a boundary sensitive branch;

[0021] The outputs of the three branches are fused to obtain fusion features, a pathological type of the tumor is predicted based on the fusion features to output a probability distribution, and a malignant degree of the tumor is evaluated to output a classification probability.

[0022] Optionally, in a fourth implementation form of the first aspect of the application, when the tumor is preliminarily identified, the RPN network based on deformable convolution is used to generate the tumor-containing candidate region in the image, comprising:

[0023] The image features extracted and preliminarily identified are input into the deformable convolution RPN network, deep layer features are extracted through the deformable convolution layer, and the anchor point generation module generates a plurality of anchor points at each position of the deep layer features;

[0024] The classification regression module processes each anchor point, and outputs the probability of each anchor point belonging to the tumor candidate region and the corresponding boundary box offset;

[0025] The anchor points with high probability are screened out according to the confidence threshold of 0.7, and the positions and sizes of the anchor points are adjusted through the boundary box regression to generate the tumor-containing candidate region.

[0026] Optionally, in a fifth implementation form of the first aspect of the application, the full-connection neural network is used to perform semantic segmentation on the candidate region to determine the specific position and contour of the tumor, comprising:

[0027] The image region corresponding to each generated tumor candidate region is input into the full-connection neural network, and the full-connection neural network classifies each pixel to determine whether it belongs to the tumor tissue or the normal nerve tissue;

[0028] The probability distribution of each pixel belonging to different categories is obtained through the Softmax function, the category with the maximum probability is taken as the category label of the pixel, and thus the semantic segmentation result of the candidate region is obtained, and according to the segmentation result, the specific position and contour of the tumor are determined.

[0029] Optionally, in a sixth implementation form of the first aspect of the application, the specific position and contour of the tumor are combined with the morphological features of the tumor to generate the final recognition result, comprising:

[0030] The tumor contour obtained through semantic segmentation is mapped back to the original neural medical image, a rigid registration algorithm is used to align the tumor region, and a tumor mask is generated;

[0031] The volume, surface area, sphericity and lobulation coefficient are calculated based on the tumor mask, the tumor region is divided into necrotic area, enhanced area and edema area, and the proportion and spatial distribution features of each area are calculated;

[0032] The texture features inside the tumor are extracted using the gray level co-occurrence matrix and the local binary pattern, and the color-coded tumor contour is superimposed on the original neural medical image to obtain the final recognition result.

[0033] The second aspect of the present application provides a neural tumor recognition positioning system based on image recognition, which comprises:

[0034] An image preprocessing module is configured to acquire a neural medical image of a patient, and pre-process the image, wherein the pre-processing comprises image denoising, gray scale normalization and image enhancement.

[0035] A feature extraction module is configured to extract features from the pre-processed image by using a Transformer-ConvNet hybrid model, and input the extracted features into a classifier to perform preliminary identification of the tumor.

[0036] A first generation module is configured to generate a candidate region containing the tumor in the image by using a RPN network based on deformable convolution if the preliminary identification exists the tumor.

[0037] A semantic segmentation module is configured to perform semantic segmentation on the candidate region by using a fully connected neural network to determine the specific position and contour of the tumor.

[0038] A second generation module is configured to generate a final recognition result according to the specific position and contour of the tumor in combination with morphological features of the tumor.

[0039] The third aspect of the present application provides a neural tumor recognition positioning device based on image recognition, which comprises a memory and at least one processor, and the memory stores instructions; the at least one processor invokes the instructions in the memory to enable the neural tumor recognition positioning device based on image recognition to perform each step of the neural tumor recognition positioning method based on image recognition as described in any one of the above.

[0040] The fourth aspect of the present application provides a computer readable storage medium, which stores instructions, and the instructions are executed by a processor to implement each step of the neural tumor recognition positioning method based on image recognition as described in any one of the above.

[0041] In the technical solution provided by the present application, the neural medical image of a patient is acquired, the image is pre-processed, the pre-processed image is extracted by using a Transformer-ConvNet hybrid model, the extracted features are input into a classifier to perform preliminary identification of the tumor, a candidate region containing the tumor is generated in the image by using a RPN network based on deformable convolution if the preliminary identification exists the tumor, semantic segmentation is performed on the candidate region by using a fully connected neural network to determine the specific position and contour of the tumor, and a final recognition result is generated according to the specific position and contour of the tumor in combination with morphological features of the tumor; the present application can more efficiently and accurately identify and locate the neural tumor, and provides strong support for clinical diagnosis and treatment. Attached Figure Description

[0042] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0043] Figure 1 A flowchart illustrating the image recognition-based neural tumor identification and localization method provided in this embodiment of the invention;

[0044] Figure 2 A schematic diagram of the structure of the image recognition-based neural tumor identification and localization system provided in an embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram of the structure of a neural tumor identification and localization device based on image recognition provided in an embodiment of the present invention. Detailed Implementation

[0046] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0047] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The flowchart of the image recognition-based neural tumor identification and localization method provided in this embodiment of the invention includes the following steps:

[0048] Step 101: Acquire neuromedical images of the patient and preprocess the images, including image denoising, grayscale normalization and image enhancement.

[0049] In this embodiment, the neural medical image of the patient is acquired, the weighted mean is calculated by searching all regions similar to the current pixel block in the image to reduce noise, and a denoised image is obtained; the effective region is extracted by Otsu threshold method, the 1% and 99% quantiles of the gray value in the effective region are calculated, the extreme values exceeding the range are truncated, and the truncated gray value is linearly scaled to the target interval to obtain the gray normalized image; the gray normalized image is divided into overlapping sub-blocks, and the histogram equalization is independently performed on each sub-block, the contrast limit threshold is set to 0.01, the part exceeding the threshold is uniformly distributed to other gray levels, and the processing results of adjacent sub-blocks are merged by bilinear interpolation to obtain the preprocessed image.

[0050] In this embodiment, a non-local self-similarity based denoising algorithm is used for image preprocessing, all regions with the most similar gray distribution to the current pixel block are searched in the global range of the image, the weighted mean of the similar blocks is calculated as the output value of the current pixel by using a Gaussian weight function, the Euclidean distance or structural similarity index is calculated, and the blocks with smaller distance are given higher weight. Finally, the weighted average formula is used to generate the denoised pixel value. Compared with the traditional Gaussian filter or median filter, this method can effectively preserve the tumor edge details and texture features, and the noise suppression effect on low-contrast tumor regions is remarkable.

[0051] After the effective region extraction and gray truncation normalization based on Otsu threshold are completed, the Otsu adaptive threshold segmentation algorithm is used for image binarization, the regions containing brain tissue, tumor and cerebrospinal fluid and other effective anatomical structures are automatically extracted, the background noise interference is excluded, the Otsu algorithm determines the optimal segmentation threshold by maximizing the inter-class variance, the image is divided into foreground and background two parts, and the 1% and 99% quantiles of the gray value distribution in the effective region are further calculated to eliminate extreme abnormal values caused by motion artifacts or device noise. The truncated gray value is normalized to the target interval through linear transformation, so that the images collected by different modalities and different devices have a unified gray distribution range.

[0052] The adaptive histogram equalization based on block contrast limit is used to enhance the local contrast of tumor and surrounding tissue. The gray normalized image is divided into overlapping sub-blocks, and the histogram equalization is independently performed on each sub-block. The high-frequency gray level pixels exceeding the threshold are uniformly distributed to the low-frequency gray level, the cumulative distribution function is recalculated, and the processing results of adjacent sub-blocks are fused by bilinear interpolation algorithm to eliminate the discontinuity of the block boundary. Compared with the global histogram equalization, this method can enhance the edge details of the tumor region while avoiding the excessive contrast stretching of the normal brain tissue region, and is especially suitable for processing the local gray deviation caused by the non-uniformity of the magnetic field in the MRI image.

[0053] Step 102, using the Transformer-ConvNet hybrid model to extract features from the preprocessed image, and inputting the extracted features into the classifier for preliminary identification of the tumor;

[0054] In this embodiment, the preprocessed image is input into the Transformer-ConvNet hybrid model. In the ConvNet part, multi-layer convolution and pooling operations are performed to obtain local feature maps with different scales. The local feature maps are flattened into sequences as the input of the Transformer. The attention weights between each position and other positions are calculated through the multi-head self-attention mechanism to capture the long-distance dependency relationship between different regions of the image and obtain global features. The global features output by the Transformer part are fused with the local features extracted by the ConvNet to obtain hybrid features. The Transformer part is composed of multiple Transformer encoders, and each encoder includes a multi-head self-attention mechanism and a feedforward neural network.

[0055] In this embodiment, the features extracted by the Transformer-ConvNet hybrid model are input into three parallel branches of the classifier, which respectively perform feature abstraction of different scales. The attention weights are generated by a fully connected layer for the output of the global context branch. The spatial attention maps are generated by 3x3 convolution and Sigmoid activation function for the output of the local detail branch and the output of the boundary sensitive branch, respectively. The outputs of the three branches are fused to obtain fusion features, and the fusion features are used to predict the probability distribution of the pathological type of the tumor and evaluate the classification probability of the malignant degree of the tumor.

[0056] In this embodiment, the global context branch compresses the feature map into a global vector of Cx1x1 through global average pooling to capture the macro features of the entire image. The local detail branch uses a 3x3 convolution layer to reduce the dimension of the feature map to retain local detail information and outputs a feature of C / 2xH / 2xW / 2. The boundary sensitive branch uses a Sobel operator to extract edge information of the feature map to generate a boundary enhanced feature map CxHxW to highlight the boundary features of the tumor.

[0057] In this embodiment, the fusion features are input into a double-task output head for classification prediction: the pathological type prediction branch maps the fusion features to a preset pathological category space, such as glioma, meningioma, metastatic tumor, etc., through a fully connected layer, and outputs the probability distribution of each pathological type using a Softmax activation function, and a focal loss function is introduced during training to solve the imbalance problem of rare tumor type samples; the malignant degree evaluation branch first compresses the fusion features into a fixed-length feature vector through adaptive average pooling, extracts abstract decision features through two fully connected layers, and finally outputs the malignant degree classification probability such as 0-1 continuous value through a Sigmoid function, with 0.5 as the benign-malignant dividing threshold, and a multi-modal supervision mechanism is specially designed for this branch, in addition to using pathological section annotation data, clinical imaging features are also introduced as auxiliary supervision signals, and the sensitivity of the model to early malignant features is improved through multi-task learning regularization, and in the inference stage, the prediction results of the two branches are post-processed through a Bayesian fusion algorithm, the classification probability is calibrated combined with prior medical knowledge, and finally the preliminary identification of the tumor containing the pathological type confidence and the malignant degree score is output.

[0058] Step 103, if the preliminary identification exists a tumor, a RPN network based on deformable convolution is used to generate a candidate region containing the tumor in the image;

[0059] In this embodiment, the image features extracted and preliminarily identified are input into the deformable convolution RPN network, deep features are extracted through the deformable convolution layer, and the anchor point generation module generates multiple anchor points at each position of the deep features; the classification and regression module processes each anchor point to output the probability that each anchor point belongs to the tumor candidate region and the corresponding boundary box offset; according to the confidence threshold of 0.7, the anchor points with high probability are screened out, and the position and size of the anchor points are adjusted through the boundary box regression to generate the candidate region containing the tumor.

[0060] In this embodiment, on the basis of feature extraction and preliminary identification, the image feature tensor is input into the deformable convolution RPN network specially designed for neural tumors. Different from the traditional fixed grid convolution operation, the deformable convolution layer can adaptively adjust the sampling position of the convolution kernel during spatial sampling through the additional learned offset parameter, effectively capturing the irregular morphology of neural tumors, such as the edge of infiltrative glioma and the outline of lobulated meningioma. In specific implementation, the deformable convolution layer increases an offset prediction branch of the same size based on the traditional 3x3 convolution kernel, which generates a two-dimensional offset for each sampling point from the input feature map through 1x1 convolution, and then adaptively samples the feature map through bilinear interpolation to generate a deep feature map containing complex geometric features of the tumor. This process enables the network to break through the limitation of rectangular receptive field, and is more accurate in extracting tumor features in complex structures such as the hippocampus and brainstem, especially improving the sensitivity to tumors in curved anatomical parts such as the skull base;

[0061] The anchor point generation module generates multiple groups of adaptive anchor points at each spatial position based on the deep feature map output by the deformable convolution. Unlike the fixed proportion anchor point setting of the traditional RPN, the module dynamically generates anchor point templates of three basic sizes such as 16x16, 64x64, and 256x256 pixels and four adaptive ratios such as 1:1, 1:2, 2:1, and the optimal ratio of ellipse fitting by analyzing the morphological statistical characteristics of tumors in the training data, such as volume, long and short axis ratio, and adhesion to brain grooves. This mechanism enables the anchor points to better fit the variable morphology of neural tumors, especially for small tumors in narrow areas such as the cerebellar vermis, to generate more accurate initial candidate regions.

[0062] The anchor point classification regression and candidate region refinement classification regression module adopts a twin branch structure to process the classification probability and bounding box regression task of each anchor point in parallel. The classification branch extracts anchor point features through 2 layers of 3x3 deformable convolution and accesses the Softmax layer to output the probability of belonging to a tumor candidate region. The regression branch simultaneously outputs 4 bounding box offset quantities corresponding to the normalized adjustment quantities of the anchor point center coordinates, width, and height. The regression function adopts a smooth L1 loss optimization. During inference, low-probability anchor points are filtered by a confidence threshold of 0.7, and candidates with a tumor probability of ≥0.7 are retained. The position and size of the anchor points are adjusted using bounding box regression to make the candidate region closer to the actual tumor boundary.

[0063] Step 104: Perform semantic segmentation on the candidate region using a fully connected neural network to determine the specific location and contour of the tumor.

[0064] In this embodiment, the image region corresponding to each generated tumor candidate region is input into the fully connected neural network. The fully connected neural network classifies each pixel to determine whether it belongs to tumor tissue or normal neural tissue. The probability distribution of each pixel belonging to different classes is obtained through the Softmax function, and the class with the highest probability is taken as the class label of the pixel, thereby obtaining the semantic segmentation result of the candidate region. According to the segmentation result, the specific location and contour of the tumor are determined.

[0065] In this embodiment, after obtaining the tumor candidate region generated by the deformable RPN, first, the image sub-block corresponding to each candidate region is standardized and preprocessed: through bilinear interpolation, its size is uniformly adjusted to 256x256 pixels, the spatial context relationship between the tumor and the surrounding tissue is preserved, and classification bias caused by scale difference is avoided. The input image sub-block enters the fully connected neural network specially designed for neural tumors, which consists of an encoding path and a decoding path: the encoding path contains 4 residual convolution blocks, each block consists of 2 layers of 3x3 deformable convolution and layer normalization, which gradually down-samples the input feature map from 256x256x3 to 16x16x512, effectively capturing the deep semantic features of the tumor; the decoding path fuses the deep semantic features and the shallow edge features through the deconvolution layer and the skip connection, and finally outputs a feature map with the same size as the input, 256x256xC, C is the number of classes, including tumor core, edema area, normal white matter and gray matter 4 classes, this architecture breaks through the limitation of traditional fully connected layers that lose spatial position information, and realizes end-to-end pixel-level classification.

[0066] Softmax probability modeling and pixel-by-pixel decision mechanism The feature map output by the network is processed by the Softmax activation function to generate a probability distribution matrix for each pixel belonging to C classes. To address the class imbalance problem in neural tumor segmentation where tumor pixels account for a small proportion, a class-weighted Softmax is introduced, which assigns a weight of 3 times that of the normal tissue class to the tumor core class. The class with the highest probability is taken as the label for each pixel, and a semantic segmentation mask containing multiple tissue classes is generated, where the binary combination of the tumor core area and the edema area is the actual tumor occupied area.

[0067] Segmentation result post-processing and tumor contour accurate analysis Based on the semantic segmentation mask, first, morphological opening operation is used to remove isolated noise points, and the largest connected region is extracted as the effective tumor region through connected component analysis to exclude discrete misclassification caused by partial volume effect. Subsequently, the Canny edge detection algorithm is used in combination with the contour tracking technology to extract the closed contour curve on the binary tumor mask, and the contour is smoothed through the polynomial fitting algorithm to obtain the accurate contour representation of the tumor. For multi-modal images, the segmentation results of different modalities are fused, and a Markov random field model is introduced to optimize the cross-modal inconsistent regions, and finally the three-dimensional spatial position and two-dimensional contour pixel set of the tumor are determined.

[0068] Step 105, according to the specific position and contour of the tumor, combined with the morphological characteristics of the tumor, the final recognition result is generated.

[0069] In this embodiment, the tumor contour obtained by semantic segmentation is mapped back to the original neuro medical image, the tumor region is aligned using a rigid registration algorithm, and a tumor mask is generated; based on the tumor mask, the volume, surface area, sphericity and lobulation coefficient are calculated, the tumor region is divided into necrotic area, enhanced area and edema area, and the proportion and spatial distribution characteristics of each area are calculated; the texture features inside the tumor are extracted using the gray level co-occurrence matrix and the local binary pattern, and the color-coded tumor contour is superimposed on the original neuro medical image to obtain the final recognition result.

[0070] In this embodiment, after completing semantic segmentation, the tumor contour obtained by segmentation is first mapped from the candidate region coordinate system back to the original neuro medical image by a rigid registration algorithm. The registration process uses mutual information as a similarity measure, and uses an iterative closest point algorithm to optimize the rotation and translation parameters to ensure the spatial position consistency of the tumor contour in different sequence images. The image coordinate system in the preprocessing stage is marked as an anatomical landmark point as a registration reference. The voxel coordinates of the segmentation mask are converted to physical space coordinates of the original image by B-spline interpolation to generate a binary tumor mask, with the tumor region being 1 and the background being 0. For the resolution difference of multi-modal images, trilinear interpolation is used for resampling the mask to make its voxel size completely match the original image;

[0071] Based on the generated tumor mask, the three-dimensional volume is calculated by multiplying the voxel count by the volume of a single voxel, the surface area is calculated based on the surface triangle mesh extracted by the MarchingCubes algorithm, the sphericity and lobulation coefficient are calculated by the contour Fourier descriptor, the ratio of the convex hull area to the actual tumor area is calculated to reflect the irregularity of the edge, the region is divided in combination with the multi-modal image features, the enhanced region such as the blood-brain barrier destruction region is extracted by the Otsu threshold method, the edema region such as the region with abnormally high water content is segmented by fuzzy C-means clustering, and the non-enhanced and non-edematous solid part is defined as the necrotic area. The volume proportion of each region is calculated by three-dimensional connected component analysis, such as the volume of the enhanced region / the total volume of the tumor, and a spatial distribution feature vector is constructed, including the centroid coordinates of each region, the minimum distance to the ventricle / choroid, the peritumoral edema index such as the edema volume / tumor core volume, etc. These quantitative indicators provide objective basis for clinical grading;

[0072] In the texture analysis stage, a multi-feature fusion strategy combining gray level co-occurrence matrix and local binary pattern is adopted: GLCM calculates the co-occurrence matrix in 0°, 45°, 90° and 135° directions within the tumor mask area, extracts 8 texture features such as contrast, correlation, energy and entropy, and reflects the spatial correlation of pixel gray levels; the LBP operator adopts an 8-neighbor circular symmetric mode to generate a 256-dimensional texture histogram feature to capture the gray level change pattern of the microscopic structure inside the tumor.

[0073] The visualization link enhances the doctor's recognition of the tumor structure through pseudo-color coding technology: the enhanced area is mapped to red, the edema area is mapped to blue, and the necrosis area is mapped to yellow. The color contour is superimposed on the original image through alpha blending, and key indicators such as volume and malignancy score are displayed on the image edge.

[0074] Please refer to Figure 2 The structure diagram of the neural tumor recognition positioning system based on image recognition provided by the embodiment of the present application is shown in the figure, and the system comprises:

[0075] An image preprocessing module is configured to acquire a neural medical image of a patient and perform preprocessing on the image, wherein the preprocessing comprises image noise reduction, gray scale normalization and image enhancement.

[0076] A feature extraction module is configured to perform feature extraction on the preprocessed image by using a Transformer-ConvNet hybrid model, and input the extracted features into a classifier to perform preliminary identification of the tumor.

[0077] A first generation module is configured to generate a candidate region containing the tumor in the image by using an RPN network based on deformable convolution if the preliminary identification exists the tumor.

[0078] A semantic segmentation module is configured to perform semantic segmentation on the candidate region by using a fully connected neural network to determine the specific position and contour of the tumor.

[0079] A second generation module is configured to generate a final identification result according to the specific position and contour of the tumor in combination with the morphological characteristics of the tumor.

[0080] Figure 3 The structure diagram of the neural tumor recognition positioning device based on image recognition provided by the embodiment of the present application is shown in the figure. The neural tumor recognition positioning device based on image recognition 300 can have a large difference due to different configurations or performances, and can include one or more central processing units (CPUs) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage devices) storing application programs 333 or data 332. The memory 320 and the storage medium 330 can be temporary storage or persistent storage. The programs stored in the storage medium 330 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the neural tumor recognition positioning device based on image recognition 300. Further, the processor 310 can be configured to communicate with the storage medium 330 and execute a series of instruction operations in the storage medium 330 on the neural tumor recognition positioning device based on image recognition 300 to realize the method provided by the above embodiment.

[0081] The image recognition based neural tumor recognition and positioning device 300 can also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating devices 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that, Figure 3 The image recognition based neural tumor recognition and positioning device structure shown does not constitute a limitation on the computer device provided by the present application, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0082] The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium, or a volatile computer readable storage medium, and the computer readable storage medium stores instructions, which, when executed on a computer, cause the computer to perform the steps of the image recognition based neural tumor recognition and positioning method provided by each of the embodiments.

[0083] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the device or apparatus, unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0084] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each of the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0085] The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-mentioned embodiments, and the above-mentioned embodiments and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Various changes and improvements can be made to the present application without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. An image recognition-based neural tumor recognition positioning method, characterized by, The method includes the following steps: The patient's neuromedical images were acquired and preprocessed, including image denoising, grayscale normalization, and image enhancement. The Transformer-ConvNet hybrid model is used to extract features from the preprocessed image, and the extracted features are input into a classifier for preliminary tumor identification. If a tumor is initially identified, a candidate region containing the tumor is generated in the image using a deformable convolution-based RPN network. A fully connected neural network was used to perform semantic segmentation on the candidate region to determine the specific location and outline of the tumor. Based on the specific location and outline of the tumor, combined with its morphological characteristics, the final identification result is generated. Feature extraction of the preprocessed image is performed using a Transformer-ConvNet hybrid model, including: The preprocessed image is input into the Transformer-ConvNet hybrid model. In the ConvNet part, after multiple convolution and pooling operations, local feature maps with different scales are obtained. The local feature maps are flattened into a sequence and used as the input of the Transformer. By calculating the attention weights of each location with other locations through a multi-head self-attention mechanism, long-distance dependencies between different regions of the image are captured, and global features are obtained. The global features output by the Transformer are fused with the local features extracted by the ConvNet to obtain hybrid features; The Transformer part consists of multiple Transformer encoders, each of which contains a multi-head self-attention mechanism and a feedforward neural network. The final identification result is generated based on the specific location and outline of the tumor, combined with its morphological characteristics, including: The tumor contour obtained from semantic segmentation is mapped back to the original neuromedical image, and a rigid registration algorithm is used to align the tumor regions to generate a tumor mask. Based on the tumor mask, the volume, surface area, sphericity and lobulation coefficient are calculated, and the tumor region is divided into necrotic region, enhancement region and edema region. The proportion and spatial distribution characteristics of each region are also calculated. Texture features inside the tumor are extracted using gray-level co-occurrence matrix and local binary pattern. Color-coded tumor contours are then overlaid on the original neuromedical image to obtain the final recognition result.

2. The image recognition-based neural tumor recognition and positioning method according to claim 1, wherein, The acquisition of the patient's neuromedical images and the preprocessing of the images include: The patient's neuromedical images are acquired, and the weighted average is calculated to reduce noise by searching all regions in the image that are similar to the current pixel block, resulting in a denoised image. The effective region is extracted by Otsu thresholding method, the 1% and 99th percentiles of gray values ​​within the effective region are calculated, extreme values ​​outside the range are truncated, and the truncated gray values ​​are linearly scaled to the target range to obtain the gray-normalized image. The image after grayscale normalization is divided into overlapping sub-blocks. Histogram equalization is performed independently on each sub-block. A contrast limit threshold of 0.01 is set. The parts exceeding the threshold are evenly distributed to other grayscale levels. The processing results of adjacent sub-blocks are merged by bilinear interpolation to obtain the preprocessed image.

3. The image recognition-based neural tumor recognition and localization method of claim 1, wherein, The extracted features are input into a classifier for preliminary identification of the tumor, including: The features extracted by the Transformer-ConvNet hybrid model are input into three parallel branches of the classifier, which respectively perform feature abstraction at different scales, the output of the global context branch is passed through a fully connected layer to generate attention weights, and the outputs of the local detail branch and the boundary sensitive branch are respectively passed through a 3x3 convolution and a Sigmoid activation function to generate spatial attention maps; The outputs of the three branches are fused to obtain fused features, and based on the fused features, the pathological type of the tumor is predicted to output a probability distribution, and the malignant degree of the tumor is evaluated to output a classification probability.

4. The image recognition-based neural tumor recognition and localization method of claim 1, wherein, If the preliminary identification indicates the presence of a tumor, a deformable convolution-based RPN network is used to generate tumor-containing candidate regions in the image, including: The image features extracted and preliminarily identified are input into the deformable convolution RPN network, deep features are extracted through the deformable convolution layer, and the anchor point generation module generates multiple anchor points at each position of the deep features; The classification and regression module processes each anchor point to output the probability that each anchor point belongs to a tumor candidate region and the corresponding boundary box offset; High-probability anchor points are selected according to a confidence threshold of 0.7, and the positions and sizes of the anchor points are adjusted through boundary box regression to generate tumor-containing candidate regions.

5. The image recognition-based neural tumor recognition and localization method of claim 1, wherein, The system includes: An image preprocessing module for obtaining a patient's neuroimaging image, preprocessing the image, wherein the preprocessing includes image denoising, grayscale normalization and image enhancement; ​ 6. An image recognition-based neurotumor identification and localization system, characterized by, ​ ​ The feature extraction module is configured to extract features from the preprocessed image by using a Transformer-ConvNet hybrid model, and input the extracted features into a classifier to preliminarily identify the tumor; the preprocessed image is input into the Transformer-ConvNet hybrid model, in the ConvNet part, a plurality of convolution and pooling operations are performed to obtain local feature maps with different scales, the local feature maps are flattened into a sequence as the input of the Transformer; the attention weight of each position with other positions is calculated by the multi-head self-attention mechanism to capture the long-distance dependence relationship between different regions of the image and obtain global features; the global features output by the Transformer part are fused with the local features extracted by the ConvNet to obtain hybrid features; wherein the Transformer part is composed of a plurality of Transformer encoders, and each encoder includes a multi-head self-attention mechanism and a feedforward neural network; The first generation module is configured to generate a candidate region containing the tumor in the image by using a RPN network based on deformable convolution if the preliminary identification exists the tumor; The semantic segmentation module is configured to perform semantic segmentation on the candidate region by using a fully connected neural network to determine the specific position and contour of the tumor; The second generation module is configured to generate a final identification result according to the specific position and contour of the tumor and in combination with the morphological features of the tumor: the final identification result is generated according to the specific position and contour of the tumor and in combination with the morphological features of the tumor, including: mapping the tumor contour obtained by semantic segmentation back to the original neural medical image, using a rigid registration algorithm to align the tumor region, and generating a tumor mask; calculating the volume, surface area, sphericity and lobulation coefficient based on the tumor mask, dividing the tumor region into necrotic area, enhanced area and edema area, and calculating the proportion and spatial distribution characteristics of each area; extracting the texture features inside the tumor by using a gray level co-occurrence matrix and a local binary pattern, and superimposing the color-coded tumor contour on the original neural medical image to obtain the final identification result.

7. An image recognition-based neurotumor recognition and positioning device, characterized by, The image recognition-based neural tumor identification and positioning device includes a memory and at least one processor, and the memory stores instructions; the at least one processor invokes the instructions in the memory to enable the image recognition-based neural tumor identification and positioning device to perform the steps of the image recognition-based neural tumor identification and positioning method according to any one of claims 1-5.

8. A computer-readable storage medium having stored thereon instructions, the computer-readable storage medium comprising: The instructions are executed by the processor to implement the steps of the image recognition-based neural tumor identification and positioning method according to any one of claims 1-5.

Images