Medical image data processing method and system

Abstract

The present invention belongs to the technical field of medical image data processing, and relates to a medical image data processing method and system, which obtains medical image data of marked lesions, inputs them into a lesion feature extraction model for training, determines whether the lesion feature extraction model meets specified requirements, creates a three-dimensional lesion generation model and trains it, obtains medical image data of designated parts of the current patient at different periods and different angles, and sequentially inputs them into the trained lesion feature extraction model for lesion feature extraction; inputs the lesion features of the current patient at different periods into the trained three-dimensional lesion generation model, generates three-dimensional lesion model data of different periods respectively and transmits them to a disease prediction model; the disease prediction model calculates the lesion development speed and overall development speed of two adjacent periods respectively; and the lesion development speed in each direction, and predicts the lesion development. Data processing of medical image data of different periods is achieved, and doctors are assisted in diagnosing the patient's condition.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image data processing, and in particular relates to a medical image data processing method and system. Background Art

[0002] Medical imaging refers to the technology and process of obtaining images of internal tissues of the human body or a part of the human body in a non-invasive manner for medical treatment or medical research. By imaging designated internal tissues of the human body, medical imaging provides doctors with a way to "see" internal structures and lesions without direct contact with the patient's body, and greatly improves the accuracy of disease diagnosis and the effectiveness of treatment. From X-rays to MRI, to ultrasound and nuclear medicine technology, the continuous progress and innovation of imaging examinations have enabled doctors to have a deeper understanding of the nature of the disease and develop more accurate and personalized treatment plans for patients.

[0003] In the process of diagnosis and treatment, existing doctors will identify and diagnose based on the medical images of the patient's tissues, and choose treatment plans based on the identification and diagnosis results. In this process, although the existing medical images provide a visual way for the lesions of various tissues of the human body and provide a certain data basis for the doctor's diagnosis and treatment, after all, most of the current medical image data is two-dimensional. Even if some of the medical image data is three-dimensional imaging, it is also based on three-dimensional data shot at a certain angle. Due to the complexity of the internal tissues of the human body, the three-dimensional data shot from a certain angle will affect the imaging of the lesions due to the interlaced influence of the tissues, thereby reducing the accuracy of the doctor's diagnosis based on medical images. In addition, the doctor also needs to identify the medical images of the patient at different periods, and judge the development of the patient's condition based on the comparison of the current lesions with the patient's previous medical image data. When the doctor makes a diagnosis based on the medical images of the patient at different periods and different angles, it will become very complicated.

[0004] Therefore, how to process the medical imaging data of the patient's designated parts taken from various angles and the medical imaging data of the patient at different stages in advance before the doctor makes a diagnosis based on the patient's medical imaging, so as to assist the doctor in diagnosing the patient's condition, is a technical problem that urgently needs to be solved. Summary of the invention

[0005] The purpose of the present invention is to provide a medical image data processing method and system for processing the medical image data of a patient's designated parts taken at various angles in advance and combining the medical images of the patient at different stages to assist doctors in diagnosing the patient's condition.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] A medical image data processing method includes the following steps:

[0008] S1: Based on medical big data, obtain medical image data of marked lesions corresponding to the current patient, pre-process the data and input them into the created lesion feature extraction model for feature extraction training, compare the marked lesions with the extracted lesion features, and determine whether the lesion feature extraction model meets the specified requirements. If so, execute step S2; if not, continue model training and adjust model parameters until the specified requirements are met;

[0009] S2: creating a three-dimensional lesion generation model, transmitting the output lesion features to the three-dimensional lesion generation model for model training, and determining whether the trained three-dimensional lesion generation model meets specified requirements, if so, executing step S3, if not, retraining the three-dimensional lesion generation model until it meets the specified requirements;

[0010] S3: Obtain medical imaging data of designated parts of the current patient at different angles at different periods, and divide them into medical imaging data sets of different periods based on the time sequence, pre-process the medical imaging data sets of different periods and sequentially input them into the trained lesion feature extraction model, and sequentially extract lesion features from the medical imaging data of different periods;

[0011] S4: inputting the lesion features of the current patient at different stages into the trained three-dimensional lesion generation model, generating three-dimensional lesion model data at different stages respectively, and transmitting the three-dimensional lesion model data at different stages to the disease prediction model;

[0012] S5: The disease prediction model calculates the volume of the three-dimensional lesion model at different periods, calculates the lesion development speed of two adjacent periods, and calculates the overall development speed of the lesion from the earliest period to the current period;

[0013] S6: The three-dimensional lesion model is virtually cut into eight lesion areas in different directions according to the same three-dimensional coordinate system, and the lesion development speed of the lesion area in each direction in two adjacent periods and the overall development speed from the earliest period to the current period are calculated based on the lesions in the corresponding areas at different periods;

[0014] S7: The disease prediction model is based on the lesion development speed in two adjacent periods, and calculates the overall development speed of the lesion from the earliest period to the current period and the lesion development speed of the lesion area in each direction in two adjacent periods and the overall development speed from the earliest period to the current period to predict the lesion development.

[0015] Preferably, the lesion feature extraction model is a convolutional neural network model, including an input layer, a convolution layer + a pooling layer, a fully connected layer and an output layer, and the convolution layer + the pooling layer are arranged in at least multiple groups, wherein the convolution kernel size of each convolution layer is 3×3, the step size is 1, the padding is 1, the pooling layer area size is 2×2, and the step size is 2; the fully connected layer has 3 layers, wherein the first two layers are arranged with 4096 channels, and the last layer has a total of 1000 channels, representing 1000 label categories.

[0016] Preferably, in step S3, the specific process of feature extraction by the lesion feature extraction model is as follows:

[0017] S31: inputting medical image data of a specified part at different angles during a specified period through the input layer, the convolution layer extracting specified features of the medical image based on a convolution operation, the convolution operation sliding the input medical image by using convolution kernels of multiple convolution layers, calculating the weighted sum of each local area, generating a lesion feature map, and transmitting the lesion feature map to the pooling layer;

[0018] S32: performing dimensionality reduction processing on the lesion feature map through a pooling layer, retaining the most significant features in the medical image by selecting the average value of each pooling window, reducing the spatial dimension of the lesion feature map, and implementing multiple rounds of processing of step S31 and step S32 through multiple groups of convolutional layers + pooling layers;

[0019] S33: Use ReLU activation function to perform nonlinear activation on the lesion feature extraction model;

[0020] S34: The lesion feature map after multiple rounds of processing is transmitted to the fully connected layer, which flattens the lesion feature map into a vector, multiplies the vector with the specified weight matrix, adds the specified bias, and finally processes it through the Softmax activation function to obtain the final classification result.

[0021] Preferably, the specific formula for calculating the weighted sum of each local area to generate the lesion feature map in step S31 is:

[0022] h out =( h in + padding ×2- k size ) / stride +1;

[0023] w out =( w in + padding ×2- k size ) / stride +1;

[0024] in, h out is the height of the output lesion feature map, h in is the height of the input lesion feature map, w out is the width of the output lesion feature map, w in is the width of the input lesion feature map, padding is the fill value size, k size is the size of the convolution kernel, stride is the step size of the convolution kernel moving on the input lesion feature map;

[0025] Preferably, the formula for the pooling layer in S32 to perform dimensionality reduction processing on the lesion feature map is:

[0026] x j l = f ( u j l );

[0027] u j l = β j l pool ( x l- j 1 )+ b j l ;

[0028] x j l For the pooling layer l Middle j The activation of the channels, u j l is the lesion feature map passed through the previous layer x j l The value after pooling weighted bias, pool is the pooling function, β j l is the pooling weight coefficient, b j l is the bias of the pooling layer.

[0029] Preferably, in step S34, the fully connected layer flattens the lesion feature map into a vector, multiplies the vector by a specified weight matrix, and then adds a specified bias to obtain a calculation formula as follows:

[0030] y = f ( w 1* x 1+ w 2* x 2+···+ w n * x n + b );

[0031] in, y is the output value, f is the activation function, x 1、 x 2、···、 x n For input, w 1. w 2. w n for x 1、 x 2、···、 x n The corresponding weights, b is the bias value.

[0032] Preferably, the specific process in step S4 is as follows:

[0033] S41: The three-dimensional lesion generation model extracts features of the input lesion feature map of the specified period, where the features of the lesion feature map include edge information, texture features, and color features;

[0034] S42: performing feature matching on the features of the extracted lesion feature map to find the same and similar feature points in the lesion feature maps obtained based on medical imaging images at different angles;

[0035] S43: performing feature fusion on the features of the matched lesion feature maps, integrating feature information of different lesion feature maps together, generating a comprehensive feature representation, and establishing a fused three-dimensional lesion model based on the features of the lesion feature maps after feature fusion;

[0036] S44: Generate three-dimensional lesion models of other periods of the current patient according to steps S41-S43, and transmit the three-dimensional lesion models of different periods to the disease prediction model in chronological order.

[0037] Preferably, the specific process of extracting the features of the input lesion feature map of the specified period by the three-dimensional lesion generation model in step S41 is as follows:

[0038] S411: performing convolution operation on the lesion feature map using a Gaussian kernel to construct a Gaussian difference scale space, and determining local extreme points in the constructed Gaussian difference scale space by comparing each pixel point of the lesion feature map with its surrounding designated point neighbor points, where the designated point neighbor points include 8 direct neighbors and 9 neighbors at each scale;

[0039] S412: Calculate the eigenvalue of the Hessian matrix to determine whether a point is an edge point. When the ratio of the two principal curvatures of the point is less than a specified threshold, the point is identified as a feature point and retained; otherwise, it is identified as an edge point and removed;

[0040] S413: assigning a direction to each feature point by calculating the gradient direction histogram of the area around the feature point, and generating a descriptor for each feature point, wherein the descriptor is a 128-dimensional vector including the position, scale and direction information of the point.

[0041] In a second aspect, a medical image data processing system is provided, for implementing any one of the medical image data processing methods described above, comprising a data acquisition module, a preprocessing module, a model creation module, a lesion feature extraction model, a three-dimensional lesion generation model, and a disease prediction model, wherein the disease prediction model comprises a calculation module; the data acquisition module is connected to the preprocessing module, the preprocessing module is connected to the lesion feature extraction model, the lesion feature extraction model is connected to the three-dimensional lesion generation model, and the three-dimensional lesion generation model is connected to the calculation module;

[0042] The data acquisition module is used to acquire the medical image data of the marked lesions at the corresponding parts of the current patient based on the medical big data, and to acquire the medical image data of the designated parts of the current patient at different angles at different periods;

[0043] The preprocessing module is used to preprocess the medical image data of the marked lesions at the corresponding parts of the current patient and the medical image data of the designated parts of the current patient at different angles at different periods;

[0044] The model creation module is used to create a lesion feature extraction model, a three-dimensional lesion generation model and a disease prediction model;

[0045] The lesion feature extraction model is used to extract lesion features from medical imaging data of patients at different stages;

[0046] The three-dimensional lesion generation model is used to generate three-dimensional lesion model data of different periods based on the lesion characteristics of the current patient at different periods, and transmit the three-dimensional lesion model data of different periods to the disease prediction model;

[0047] The calculation module is used to calculate the volume of the three-dimensional lesion model at different periods, respectively calculate the lesion development speed of two adjacent periods, and calculate the overall development speed of the lesion from the earliest period to the current period, and calculate the lesion development speed of the lesion area in each direction in two adjacent periods and the overall development speed from the earliest period to the current period based on the lesions in the corresponding areas at different periods;

[0048] The disease prediction model is used to predict the development of lesions based on the development speed of lesions in two adjacent periods, and calculate the overall development speed of lesions from the earliest period to the current period and the lesion development speed of the lesion area in each direction in two adjacent periods and the overall development speed from the earliest period to the current period.

[0049] The beneficial effects of the present invention include:

[0050] The medical image data processing method and system provided by the present invention obtain medical image data of marked lesions, input them into a lesion feature extraction model for training, determine whether the lesion feature extraction model meets the specified requirements, create a three-dimensional lesion generation model and train it, obtain medical image data of the specified part of the current patient at different periods and different angles, and input them into the trained lesion feature extraction model in sequence to extract lesion features; input the lesion features of the current patient at different periods into the trained three-dimensional lesion generation model, respectively generate three-dimensional lesion model data of different periods and transmit them to the disease prediction model; the disease prediction model respectively calculates the lesion development speed and overall development speed of two adjacent periods; and the lesion development speed in each direction, and predicts the lesion development. Data processing of medical image data at different periods is achieved, and doctors are assisted in diagnosing the patient's condition.

[0051] First, the lesion feature extraction model sets the input layer, convolution layer + pooling layer, fully connected layer and output layer. The convolution layer + pooling layer sets at least multiple groups, where the convolution kernel size of each convolution layer is 3×3, the step size is 1, the padding is 1, and the pooling layer area size is 2×2, the step size is 2; the fully connected layer has 3 layers, where the first two layers have 4096 channels, and the last layer has 1000 channels, representing 1000 label categories. This model structure can realize the accurate extraction of lesion features of patient medical imaging pictures, and provide accurate data support for the subsequent generation of lesion three-dimensional models.

[0052] Secondly, the convolution layer is used to extract the specified features of the medical image based on the convolution operation. The convolution operation slides the input medical image using the convolution kernels of multiple convolution layers, calculates the weighted sum of each local area, generates a lesion feature map, and performs dimensionality reduction on the lesion feature map to reduce the spatial dimension of the lesion feature map. The ReLU activation function performs nonlinear activation on the lesion feature extraction model; the fully connected layer flattens the lesion feature map into a vector, multiplies the vector with the specified weight matrix and adds the specified bias, and finally processes it through the Softmax activation function to obtain the final classification result, thereby realizing the accurate extraction of lesion features in medical images.

[0053] Again, by setting and calculating the weighted sum of each local area, a specific formula for generating the size of the lesion feature map is generated. h out =( h in + padding ×2- k size ) / stride +1 and w out =( w in + padding ×2- k size ) / stride +1; Achieved accurate quantification of the size of lesion feature maps.

[0054] Again, the formula for reducing the dimension of the lesion feature map through the pooling layer is: x j l = f ( u j l )and u j l = β j l pool ( x l- j 1 )+ b j l , it realizes the dimensionality reduction processing under the accurate quantification of the lesion feature map, reduces the computational dimension of the model, and improves the efficiency of the model.

[0055] Finally, the features of the input lesion feature map of the specified period are extracted through the three-dimensional lesion generation model, and then feature matching is performed to find the same and similar feature points in the lesion feature map obtained based on medical imaging pictures at different angles; the features of the matched lesion feature map are fused to generate a comprehensive feature representation, and three-dimensional lesion models of different periods are generated based on the features of the lesion feature map after feature fusion. By visualizing the lesion data in three dimensions, we can clearly understand the overall development of the lesion and its development in various directions, providing accurate data support for the selection of subsequent treatment plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic diagram of the flow of the medical image data processing method of the present invention.

[0057] Figure 2 A schematic diagram of the process of feature extraction of the lesion feature extraction model of the present invention.

[0058] Figure 3 FIG. 4 is a schematic diagram of the architecture of the medical image data processing system of the present invention. DETAILED DESCRIPTION

[0059] The following is combined with Figures 1 - 3 The present invention is further described in detail:

[0060] Example 1

[0061] See attached Figure 1 As shown, a medical image data processing method includes the following steps:

[0062] S1: Based on medical big data, obtain the medical imaging data of the marked lesions corresponding to the current patient, pre-process them and input them into the created lesion feature extraction model for feature extraction training, compare the marked lesions with the extracted lesion features, and judge whether the lesion feature extraction model meets the specified requirements. If so, execute step S2, if not, continue model training and adjust model parameters until the specified requirements are met. The lesion feature extraction model is trained with the medical imaging data of the marked lesions, and combined with the parameter adjustment process, the lesion feature extraction model has high performance, which effectively improves the accuracy of subsequent lesion feature extraction.

[0063] S2: Create a three-dimensional lesion generation model, transmit the output lesion features to the three-dimensional lesion generation model for model training, and determine whether the trained three-dimensional lesion generation model meets the specified requirements. If so, execute step S3, if not, retrain the three-dimensional lesion generation model until it meets the specified requirements. Based on the features extracted by the lesion feature extraction model, train the three-dimensional lesion generation model to ensure the accuracy of the three-dimensional lesion model generated by the subsequent three-dimensional lesion generation model.

[0064] S3: Obtain medical imaging data of designated parts of the current patient at different angles at different periods, and divide them into medical imaging data sets of different periods based on the time sequence, pre-process the medical imaging data sets of different periods and sequentially input them into the trained lesion feature extraction model, and sequentially extract lesion features from the medical imaging data of different periods. By obtaining medical imaging data of designated parts of the current patient at different angles at different periods, the one-sidedness of medical imaging of designated parts at a single angle is avoided, and inaccurate imaging of lesions caused by the positional relationship between tissues in the patient is prevented.

[0065] S4: Input the lesion features of the current patient at different stages into the trained three-dimensional lesion generation model, generate three-dimensional lesion model data at different stages respectively, and transmit the three-dimensional lesion model data at different stages to the disease prediction model. The three-dimensional lesion generation model extracts the features of the input lesion feature map of the specified period, and then performs feature matching to find the same and similar feature points in the lesion feature map obtained based on medical imaging images from different angles; the features of the matched lesion feature map are feature fused to generate a comprehensive feature representation, and three-dimensional lesion models of different stages are generated based on the features of the lesion feature map after feature fusion. By visualizing the lesion data in three dimensions, the overall development of the lesion and its development in various directions can be clearly understood, providing accurate data support for the selection of subsequent treatment plans.

[0066] S5: The disease prediction model calculates the volume of the three-dimensional lesion model at different periods, calculates the lesion development speed of two adjacent periods, and calculates the overall development speed of the lesion from the earliest period to the current period.

[0067] S6: Virtually cut the three-dimensional lesion model into eight lesion areas in different directions according to the same three-dimensional coordinate system, and calculate the lesion development speed of the lesion area in each direction in two adjacent periods and the overall development speed from the earliest period to the current period based on the lesions in the corresponding areas in different periods. When cutting the three-dimensional lesion model, firstly take the center of gravity of the three-dimensional lesion model as the origin of the three-dimensional coordinate system, and then set the x-axis, y-axis, and z-axis of the coordinate system based on the origin, and virtually cut the three-dimensional lesion model into 8 areas through the x-axis, y-axis, and z-axis, and the 8 areas represent the 8 directions of the lesion.

[0068] S7: The disease prediction model is based on the lesion development speed in two adjacent periods, and calculates the overall development speed of the lesion from the earliest period to the current period, the lesion development speed of the lesion area in each direction in two adjacent periods, and the overall development speed from the earliest period to the current period to predict the development of the lesion. Through the above process, data processing of medical imaging data in different periods is achieved to assist doctors in diagnosing patients' conditions.

[0069] In this embodiment, the lesion feature extraction model is a convolutional neural network model, including an input layer, a convolutional layer + pooling layer, a fully connected layer and an output layer, wherein the convolutional layer + pooling layer is set to at least multiple groups, wherein the convolution kernel size of each convolutional layer is 3×3, the step size is 1, the padding is 1, and the pooling layer area size is 2×2, and the step size is 2; the fully connected layer is provided with 3 layers, wherein the first two layers are provided with 4096 channels, and the last layer has a total of 1000 channels, representing 1000 label categories. The convolutional layer is the core layer in the convolutional neural network model, and the features of the input image are extracted by convolution operation. The convolution operation performs weighted summation on the local area of ​​each medical image by sliding the convolution kernel on the input medical image to generate a feature map. Different convolution kernels can extract different features, such as edge information or texture features or color features. For each input matrix, the pooling layer cuts it into several square blocks of equal size, takes the average value for each block, and forms a new matrix with the average value result. The fully connected layer plays the role of a classifier in the entire convolutional neural network. Through the repeated processing of the previous multiple "convolution-activation-pooling" layers, the characteristics of the medical image data to be processed are effectively improved. On the one hand, the dimension of the input data has been reduced to a level that can be processed by a traditional feedforward fully connected network. On the other hand, the data input to the fully connected layer at this time is the result of repeated purification, so the output feature accuracy is much higher. ‌This model structure can achieve accurate extraction of lesion features in patient medical imaging images, providing accurate data support for the subsequent generation of lesion three-dimensional models.

[0070] Example 2

[0071] On the basis of Example 1, in step S3, the specific process of feature extraction by the lesion feature extraction model is as follows:

[0072] S31: inputting medical image data of a specified part at different angles during a specified period through the input layer, the convolution layer extracting specified features of the medical image based on a convolution operation, the convolution operation sliding the input medical image by using convolution kernels of multiple convolution layers, calculating the weighted sum of each local area, generating a lesion feature map, and transmitting the lesion feature map to the pooling layer;

[0073] S32: performing dimensionality reduction processing on the lesion feature map through a pooling layer, retaining the most significant features in the medical image by selecting the average value of each pooling window, reducing the spatial dimension of the lesion feature map, and implementing multiple rounds of processing of step S31 and step S32 through multiple groups of convolutional layers + pooling layers;

[0074] S33: Use ReLU activation function to perform nonlinear activation on the lesion feature extraction model;

[0075] S34: The lesion feature map after multiple rounds of processing is transmitted to the fully connected layer, which flattens the lesion feature map into a vector, multiplies the vector with the specified weight matrix, adds the specified bias, and finally processes it through the Softmax activation function to obtain the final classification result.

[0076] The convolution layer extracts the specified features of the medical image based on the convolution operation. The convolution operation slides the input medical image by using the convolution kernels of multiple convolution layers, calculates the weighted sum of each local area, generates a lesion feature map, performs dimensionality reduction processing on the lesion feature map, reduces the spatial dimension of the lesion feature map, and the ReLU activation function performs nonlinear activation on the lesion feature extraction model; the fully connected layer flattens the lesion feature map into a vector, multiplies the vector with the specified weight matrix and adds the specified bias, and finally processes it through the Softmax activation function to obtain the final classification result, thereby realizing the accurate extraction of lesion features in medical images.

[0077] In this embodiment, the specific formula for calculating the weighted sum of each local area to generate the lesion feature map in step S31 is:

[0078] h out =( h in + padding ×2- k size ) / stride +1;

[0079] w out =( w in + padding ×2- k size ) / stride +1;

[0080] in, h out is the height of the output lesion feature map, h in is the height of the input lesion feature map, w out is the width of the output lesion feature map, w in is the width of the input lesion feature map, padding is the fill value size, k size is the size of the convolution kernel, stride is the step size of the convolution kernel moving on the input lesion feature map.

[0081] By setting and calculating the weighted sum of each local area to generate a specific formula for the size of the lesion feature map, h out =( h in + padding ×2- k size ) / stride +1 and w out =( w in + padding ×2- k size ) / stride +1; Achieved accurate quantification of the size of lesion feature maps.

[0082] The formula for the pooling layer in S32 to reduce the dimension of the lesion feature map is:

[0083] x j l = f ( u j l );

[0084] u j l = β j l pool ( x l- j 1 )+ b j l ;

[0085] x j l For the pooling layer l Middle j The activation of the channels, u j l is the lesion feature map passed through the previous layer x j l The value after pooling weighted bias, pool is the pooling function, β j l is the pooling weight coefficient, b j lis the bias of the pooling layer. The formula for reducing the dimension of the lesion feature map through the pooling layer is: x j l = f ( u j l )and u j l = β j l pool ( x l- j 1 )+ b j l , it realizes the dimensionality reduction processing under the accurate quantification of the lesion feature map, reduces the computational dimension of the model, and improves the efficiency of the model.

[0086] In step S34, the fully connected layer flattens the lesion feature map into a vector, multiplies the vector by the specified weight matrix and adds the specified bias to calculate the formula as follows:

[0087] y = f ( w 1* x 1+ w 2* x 2+···+ w n * x n + b );

[0088] in, y is the output value, f is the activation function, x 1、 x 2、···、 x n For input, w 1. w 2. w n for x 1、 x 2、···、 x n The corresponding weights, b is the bias value. The fully connected layer combines the features extracted by the previous layer for tasks such as classification or regression. Each neuron in the fully connected layer is connected to all neurons in the previous layer and is responsible for mapping the feature map to the sample label space.

[0089] Example 3

[0090] Based on Example 1 or Example 2, the specific process in step S4 is as follows:

[0091] S41: The three-dimensional lesion generation model extracts the features of the lesion feature map of the medical image pictures at different angles during the specified period of time, and the features of the lesion feature map include edge information, texture features and color features;

[0092] S42: performing feature matching on the features of the extracted lesion feature map to find the same and similar feature points in the lesion feature maps obtained based on medical imaging images at different angles;

[0093] S43: Fusing the features of the matched lesion feature maps, integrating the feature information of different lesion feature maps together, generating a comprehensive feature representation, and establishing a fused three-dimensional lesion model based on the features of the lesion feature maps after feature fusion. The purpose of feature fusion here is to integrate the feature information of different lesion feature maps together to form a more comprehensive feature representation.

[0094] S44: Generate three-dimensional lesion models of other periods of the current patient according to steps S41-S43, and transmit the three-dimensional lesion models of different periods to the disease prediction model in chronological order.

[0095] In this embodiment, the specific process of extracting the features of the input lesion feature map of the specified period by the three-dimensional lesion generation model in step S41 is as follows:

[0096] S411: performing convolution operation on the lesion feature map using a Gaussian kernel to construct a Gaussian difference scale space, and determining local extreme points in the constructed Gaussian difference scale space by comparing each pixel point of the lesion feature map with its surrounding designated point neighbor points, where the designated point neighbor points include 8 direct neighbors and 9 neighbors at each scale;

[0097] S412: Calculate the eigenvalue of the Hessian matrix to determine whether a point is an edge point. When the ratio of the two principal curvatures of the point is less than a specified threshold, the point is identified as a feature point and retained; otherwise, it is identified as an edge point and removed;

[0098] S413: assigning a direction to each feature point by calculating the gradient direction histogram of the area around the feature point, and generating a descriptor for each feature point, wherein the descriptor is a 128-dimensional vector including the position, scale and direction information of the point.

[0099] A medical image data processing system, used to implement any one of the medical image data processing methods described above, comprising a data acquisition module, a preprocessing module, a model creation module, a lesion feature extraction model, a three-dimensional lesion generation model, and a disease prediction model, wherein the disease prediction model comprises a calculation module; the data acquisition module is connected to the preprocessing module, the preprocessing module is connected to the lesion feature extraction model, the lesion feature extraction model is connected to the three-dimensional lesion generation model, and the three-dimensional lesion generation model is connected to the calculation module.

[0100] The data acquisition module is used to acquire the medical image data of the marked lesions corresponding to the current patient's parts based on the medical big data, and to acquire the medical image data of the designated parts of the current patient at different angles at different periods. The preprocessing module is used to preprocess the medical image data of the marked lesions corresponding to the current patient's parts and the medical image data of the designated parts of the current patient at different angles at different periods. The model creation module is used to create a lesion feature extraction model, a three-dimensional lesion generation model and a disease prediction model. The lesion feature extraction model is used to extract lesion features from the medical image data of the patient at different periods. The three-dimensional lesion generation model is used to generate three-dimensional lesion model data of different periods based on the lesion features of the current patient at different periods, and transmit the three-dimensional lesion model data of different periods to the disease prediction model.

[0101] The calculation module is used to calculate the volume of the three-dimensional lesion model at different periods, calculate the lesion development speed of two adjacent periods respectively, and calculate the overall development speed of the lesion from the earliest period to the current period, and calculate the lesion development speed of the lesion area in each direction in two adjacent periods and the overall development speed from the earliest period to the current period based on the lesions in the corresponding areas of different periods. The disease prediction model is used to predict the development of the lesion based on the lesion development speed of two adjacent periods, and calculate the overall development speed of the lesion from the earliest period to the current period, the lesion development speed of the lesion area in each direction in two adjacent periods, and the overall development speed from the earliest period to the current period.

[0102] In summary, the medical image data processing method and system provided by the present invention, through the lesion feature extraction model, the input layer, convolution layer + pooling layer, fully connected layer and output layer are set, and the convolution layer + pooling layer is set at least multiple groups, wherein the convolution kernel size of each convolution layer is 3×3, the step size is 1, the padding is 1, the pooling layer area size is 2×2, and the step size is 2; the fully connected layer is provided with 3 layers, wherein the first two layers are provided with 4096 channels, and the last layer has a total of 1000 channels, representing 1000 label categories. This model structure can realize the accurate extraction of lesion features of patient medical imaging pictures, and provide accurate data support for the subsequent generation of lesion three-dimensional models. The convolution layer extracts the specified features of the medical image based on the convolution operation. The convolution operation slides the input medical image by using the convolution kernels of multiple convolution layers, calculates the weighted sum of each local area, generates a lesion feature map, performs dimensionality reduction processing on the lesion feature map, reduces the spatial dimension of the lesion feature map, and the ReLU activation function performs nonlinear activation on the lesion feature extraction model; the fully connected layer flattens the lesion feature map into a vector, multiplies the vector with the specified weight matrix and adds the specified bias, and finally processes it through the Softmax activation function to obtain the final classification result, thereby realizing the accurate extraction of lesion features in medical images.

[0103] By setting and calculating the weighted sum of each local area to generate a specific formula for the size of the lesion feature map, h out =( h in + padding ×2- k size ) / stride +1 and w out =( w in + padding ×2- k size ) / stride +1; the precise quantification of the size of the lesion feature map is achieved. The formula for reducing the dimension of the lesion feature map through the pooling layer is: x j l = f ( u j l )and u j l = β j l pool ( x l- j 1)+ b j l , it realizes the dimensionality reduction processing under the precise quantification of the lesion feature map, reduces the operation dimension of the model, and improves the efficiency of the model. The features of the input lesion feature map of the specified period are extracted through the three-dimensional lesion generation model, and then feature matching is performed to find the same and similar feature points in the lesion feature map obtained based on medical imaging pictures at different angles; the features of the matched lesion feature map are fused to generate a comprehensive feature representation, and the three-dimensional lesion model of different periods is generated based on the features of the lesion feature map after feature fusion. By visualizing the lesion data in three dimensions, we can clearly understand the overall development of the lesion and its development in various directions, providing accurate data support for the selection of subsequent treatment plans.

Claims

1. A method for processing medical image data, characterized in that: The process includes: S1: Based on medical big data, obtain medical image data of marked lesions corresponding to the current patient, pre-process the data and input them into the created lesion feature extraction model for feature extraction training, compare the marked lesions with the extracted lesion features, and determine whether the lesion feature extraction model meets the specified requirements. If so, execute step S2; if not, continue model training and adjust model parameters until the specified requirements are met; S2: creating a three-dimensional lesion generation model, transmitting the output lesion features to the three-dimensional lesion generation model for model training, and determining whether the trained three-dimensional lesion generation model meets specified requirements, if so, executing step S3, if not, retraining the three-dimensional lesion generation model until it meets the specified requirements; S3: Obtain medical imaging data of designated parts of the current patient at different angles at different periods, and divide them into medical imaging data sets of different periods based on the time sequence, pre-process the medical imaging data sets of different periods and sequentially input them into the trained lesion feature extraction model, and sequentially extract lesion features from the medical imaging data of different periods; S4: inputting the lesion features of the current patient at different stages into the trained three-dimensional lesion generation model, generating three-dimensional lesion model data at different stages respectively, and transmitting the three-dimensional lesion model data at different stages to the disease prediction model; S5: The disease prediction model calculates the volume of the three-dimensional lesion model at different periods, calculates the lesion development speed of two adjacent periods, and calculates the overall development speed of the lesion from the earliest period to the current period; S6: The three-dimensional lesion model is virtually cut into eight lesion areas in different directions according to the same three-dimensional coordinate system, and the lesion development speed of the lesion area in each direction in two adjacent periods and the overall development speed from the earliest period to the current period are calculated based on the lesions in the corresponding areas at different periods; S7: The disease prediction model is based on the lesion development speed in two adjacent periods, and calculates the overall development speed of the lesion from the earliest period to the current period, the lesion development speed of the lesion area in each direction in the two adjacent periods, and the overall development speed from the earliest period to the current period to predict the lesion development; The lesion feature extraction model is a convolutional neural network model, including an input layer, a convolution layer + a pooling layer, a fully connected layer and an output layer. The convolution layer + the pooling layer are set in at least multiple groups, wherein the convolution kernel size of each convolution layer is 3×3, the step size is 1, the padding is 1, the pooling layer area size is 2×2, and the step size is 2; the fully connected layer has 3 layers, wherein the first two layers have 4096 channels, and the last layer has a total of 1000 channels, representing 1000 label categories; In step S3, the specific process of feature extraction by the lesion feature extraction model is as follows: S31: inputting medical image data of a specified part at different angles during a specified period through the input layer, the convolution layer extracting specified features of the medical image based on a convolution operation, the convolution operation sliding the input medical image by using convolution kernels of multiple convolution layers, calculating the weighted sum of each local area, generating a lesion feature map, and transmitting the lesion feature map to the pooling layer; S32: performing dimensionality reduction processing on the lesion feature map through a pooling layer, retaining the most significant features in the medical image by selecting the average value of each pooling window, reducing the spatial dimension of the lesion feature map, and implementing multiple rounds of processing of step S31 and step S32 through multiple groups of convolutional layers + pooling layers; S33: Use ReLU activation function to perform nonlinear activation on the lesion feature extraction model; S34: transmitting the lesion feature map after multiple rounds of processing to the fully connected layer, the fully connected layer flattens the lesion feature map into a vector, multiplies the vector with a specified weight matrix, adds a specified bias, and finally processes it through a Softmax activation function to obtain a final classification result; The specific process in step S4 is as follows: S41: The three-dimensional lesion generation model extracts the features of the lesion feature map of the input specified period, and the features of the lesion feature map include edge information, texture features and color features; S42: performing feature matching on the features of the extracted lesion feature map to find the same and similar feature points in the lesion feature maps obtained based on medical imaging images at different angles; S43: performing feature fusion on the features of the matched lesion feature maps, integrating feature information of different lesion feature maps together, generating a comprehensive feature representation, and establishing a fused three-dimensional lesion model based on the features of the lesion feature maps after feature fusion; S44: generating three-dimensional lesion models of the current patient at other stages according to steps S41-S43, and transmitting the three-dimensional lesion models of different stages to the disease prediction model in chronological order; The specific process of extracting the features of the input lesion feature map of the specified period by the three-dimensional lesion generation model in step S41 is as follows: S411: performing convolution operation on the lesion feature map using a Gaussian kernel to construct a Gaussian difference scale space, and determining local extreme points in the constructed Gaussian difference scale space by comparing each pixel point of the lesion feature map with its surrounding designated point neighbor points, where the designated point neighbor points include 8 direct neighbors and 9 neighbors at each scale; S412: Calculate the eigenvalue of the Hessian matrix to determine whether a point is an edge point. When the ratio of the two principal curvatures of the point is less than a specified threshold, the point is identified as a feature point and retained; otherwise, it is identified as an edge point and removed; S413: assigning a direction to each feature point by calculating the gradient direction histogram of the area around the feature point, and generating a descriptor for each feature point, wherein the descriptor is a 128-dimensional vector including the position, scale and direction information of the point.

2. A medical image data processing method according to claim 1, characterized in that: The specific formula for calculating the weighted sum of each local area to generate the lesion feature map in step S31 is: h out =( h in + padding ×2- k size ) / stride +1； w out =( w in + padding ×2- k size ) / stride +1； in, h out is the height of the output lesion feature map, h in is the height of the input lesion feature map, w out is the width of the output lesion feature map, w in is the width of the input lesion feature map, padding is the fill value size, k size is the size of the convolution kernel, stride is the step size of the convolution kernel moving on the input lesion feature map.

3. A medical image data processing method according to claim 1, characterized in that: The formula for the pooling layer in step S32 to perform dimensionality reduction processing on the lesion feature map is: x j l = f ( u j l )； u j l = β j l pool ( x l- j 1 )+ b j l ； x j l For the pooling layer l Middle j The activation of the channels, u j l is the lesion feature map passed through the previous layer x j l The value after pooling weighted bias, pool is the pooling function, β j l is the pooling weight coefficient, b j l is the bias of the pooling layer.

4. A medical image data processing method according to claim 1, characterized in that: In step S34, the fully connected layer flattens the lesion feature map into a vector, multiplies the vector by the specified weight matrix and adds the specified bias to calculate the formula as follows: y=f ( w 1* x 1+ w 2* x 2+···+ w n * x n + b )； in, y is the output value, f is the activation function, x 1、 x 2、···、 x n For input, w 1. w 2. w n for x 1、 x 2、···、 x n The corresponding weights, b is the bias value.

5. A medical image data processing system, used to implement a medical image data processing method according to any one of claims 1 to 4, characterized in that: It includes a data acquisition module, a preprocessing module, a model creation module, a lesion feature extraction model, a three-dimensional lesion generation model, and a disease prediction model, wherein the disease prediction model includes a calculation module; the data acquisition module is connected to the preprocessing module, the preprocessing module is connected to the lesion feature extraction model, the lesion feature extraction model is connected to the three-dimensional lesion generation model, and the three-dimensional lesion generation model is connected to the calculation module; The data acquisition module is used to acquire the medical image data of the marked lesions at the corresponding parts of the current patient based on the medical big data, and to acquire the medical image data of the designated parts of the current patient at different angles at different periods; The preprocessing module is used to preprocess the medical image data of the marked lesions at the corresponding parts of the current patient and the medical image data of the designated parts of the current patient at different angles at different periods; The model creation module is used to create a lesion feature extraction model, a three-dimensional lesion generation model and a disease prediction model; The lesion feature extraction model is used to extract lesion features from medical imaging data of patients at different stages; The three-dimensional lesion generation model is used to generate three-dimensional lesion model data of different periods based on the lesion characteristics of the current patient at different periods, and transmit the three-dimensional lesion model data of different periods to the disease prediction model; The calculation module is used to calculate the volume of the three-dimensional lesion model at different periods, respectively calculate the lesion development speed of two adjacent periods, and calculate the overall development speed of the lesion from the earliest period to the current period, and calculate the lesion development speed of the lesion area in each direction in two adjacent periods and the overall development speed from the earliest period to the current period based on the lesions in the corresponding areas at different periods; The disease prediction model is used to predict the development of lesions based on the development speed of lesions in two adjacent periods, and calculate the overall development speed of lesions from the earliest period to the current period and the lesion development speed of the lesion area in each direction in two adjacent periods and the overall development speed from the earliest period to the current period.

Images