Oral stem prediction method, device, equipment and medium

Abstract

The invention relates to the technical field of risk prediction, in particular to a mouth stem prediction method and device, equipment and a medium, and the method comprises the steps: obtaining a three-dimensional dose distribution feature and a medical image feature of a target object; according to a dose weight coefficient and an image weight coefficient, performing weighted fusion on the three-dimensional dose distribution feature and the medical image feature to obtain a first mixed feature; wherein the dose weight coefficient and the image weight coefficient are obtained by adjusting the weight coefficients of the three-dimensional dose distribution feature and the medical image feature of the reference object based on an adaptive average pooling method; and performing mouth stem prediction based on the prediction data to obtain a mouth stem prediction value. According to the method, dose distribution information of a three-dimensional space is introduced, fusion analysis is performed on three-dimensional dose distribution characteristics and medical image characteristics, and a weight coefficient is adjusted based on an adaptive three-dimensional average pooling method, so that the accuracy of mouth dryness prediction is improved, and the interpretability of a mouth dryness prediction result is improved.

Description

Technical Field

[0001] The present application relates to the technical field of risk prediction, and in particular to a dry mouth prediction method, device, equipment and medium. Background Art

[0002] NPC is usually caused by abnormal proliferation and malignant changes of nasopharyngeal epithelial cells. Radiotherapy is one of the main treatments for NPC (nasopharyngeal carcinoma), but radiotherapy may damage the surrounding salivary glands, resulting in reduced saliva secretion or changes in quality, thus causing dry mouth. The dry mouth prediction method in the related art uses radiomics features combined with clinical features and dose parameters for modeling and prediction. The dose parameters are expressed in a one-dimensional discrete form of the dose-volume relationship. The extracted dose parameter data are discrete points, and the prediction accuracy is low and not accurate enough. Summary of the invention

[0003] The present application provides a dry mouth prediction method, device, equipment and medium, which solves the technical problem of inaccurate dry mouth prediction in the related art.

[0004] In order to achieve the above objectives, the main technical solutions adopted in this application include:

[0005] In a first aspect, an embodiment of the present application provides a dry mouth prediction method, comprising:

[0006] Acquire three-dimensional dose distribution characteristics and medical image characteristics of the target object; wherein the three-dimensional dose distribution characteristics are obtained by extracting the three-dimensional dose distribution data of the target object;

[0007] The three-dimensional dose distribution feature and the medical image feature are weightedly fused according to the dose weight coefficient and the image weight coefficient to obtain a first mixed feature; wherein the dose weight coefficient and the image weight coefficient are obtained by adjusting the respective weight coefficients of the three-dimensional dose distribution feature and the medical image feature of the reference object based on the adaptive average pooling method;

[0008] Residual convolution processing is performed based on the first mixed feature to obtain prediction data; dry mouth prediction is performed based on the prediction data to obtain a dry mouth prediction value.

[0009] This application introduces dose distribution information in three-dimensional space, fuses and analyzes the three-dimensional dose distribution characteristics and medical image characteristics, and adjusts the weight coefficients of the three-dimensional dose distribution characteristics and medical image characteristics based on the adaptive three-dimensional average pooling method, thereby improving the accuracy of dry mouth prediction and the interpretability of dry mouth prediction results.

[0010] Optionally, performing residual convolution processing based on the first mixed feature to obtain prediction data includes:

[0011] Performing residual convolution processing based on the first mixed feature to obtain processed feature data;

[0012] The processed feature data are fused with the clinical feature data of the target subject to obtain prediction data.

[0013] This application introduces dose distribution information in three-dimensional space, fuses and analyzes three-dimensional dose distribution characteristics, medical imaging characteristics and clinical characteristic data, and adjusts the weight coefficients of three-dimensional dose distribution characteristics and medical imaging characteristics based on an adaptive three-dimensional average pooling method, thereby improving the accuracy of dry mouth prediction and the interpretability of dry mouth prediction results.

[0014] Optionally, the three-dimensional dose distribution data of the target object is divided into a plurality of segments of three-dimensional dose segmentation data; and the step of obtaining the three-dimensional dose distribution characteristics of the target object includes:

[0015] Perform feature extraction on the three-dimensional dose segmentation data to obtain three-dimensional dose segmentation features;

[0016] The three-dimensional dose segmentation features are fused to obtain the three-dimensional dose distribution features.

[0017] This application introduces dose distribution information in three-dimensional space, extracts three-dimensional dose distribution characteristics, medical imaging characteristics and clinical characteristic data based on deep learning technology and performs fusion analysis, adjusts the weight coefficients of three-dimensional dose distribution characteristics and medical imaging characteristics based on an adaptive three-dimensional average pooling method, and adjusts the weights of each segmented dose distribution based on an adaptive three-dimensional average pooling method, thereby improving the accuracy of dry mouth prediction and the interpretability of dry mouth prediction results. The higher the weight, the greater the impact on the prediction results.

[0018] Optionally, different segments of the three-dimensional dose segmentation features correspond to different three-dimensional dose segmentation weight coefficients; the three-dimensional dose segmentation features are fused to obtain the three-dimensional dose distribution features, including:

[0019] The three-dimensional dose segmentation characteristics are weighted and fused according to the three-dimensional dose segmentation weight coefficient to obtain the three-dimensional dose distribution characteristics.

[0020] Optionally, the three-dimensional dose segmentation weight coefficient corresponding to the three-dimensional dose segmentation feature is obtained by adjusting the weight coefficient of the three-dimensional dose segmentation feature of the reference object based on an adaptive average pooling method.

[0021] Optionally, the three-dimensional dose segmentation data is obtained by dividing the three-dimensional dose distribution data of the target object according to a preset segment number N; wherein N is an integer greater than or equal to 2.

[0022] The overall dose distribution is divided into five dose segments, namely L, ML, M, MH and H. Taking M as the base and extending to both sides, it is divided into uniform intervals by equal division, or by random or normal division. The division method is not unique.

[0023] Optionally, the acquiring of three-dimensional dose distribution characteristics and medical imaging characteristics of the target object includes:

[0024] Acquire a feature matrix of the parotid gland region of the target object;

[0025] Performing convolution processing on the three-dimensional dose distribution data of the target object to obtain an intermediate dose feature, and performing convolution processing on the medical image data of the target object to obtain an intermediate image feature;

[0026] Performing a dot multiplication operation on the feature matrix and the intermediate dose feature to obtain the three-dimensional dose distribution feature;

[0027] A dot product operation is performed on the feature matrix and the intermediate image feature to obtain the medical image feature.

[0028] In a second aspect, an embodiment of the present application provides a dry mouth prediction device, including an acquisition module, a weight adjustment module, and a prediction module;

[0029] The acquisition module is used to acquire the three-dimensional dose distribution characteristics and medical image characteristics of the target object; wherein the three-dimensional dose distribution characteristics are obtained by extracting the three-dimensional dose distribution data of the target object;

[0030] The weight adjustment module is used to perform weighted fusion of the three-dimensional dose distribution feature and the medical image feature according to the dose weight coefficient and the image weight coefficient to obtain a first mixed feature; wherein the dose weight coefficient and the image weight coefficient are obtained by adjusting the respective weight coefficients of the three-dimensional dose distribution feature and the medical image feature of the reference object based on the adaptive average pooling method;

[0031] The prediction module is used to perform residual convolution processing based on the first mixed feature to obtain prediction data; and perform dry mouth prediction based on the prediction data to obtain a dry mouth prediction value.

[0032] In a third aspect, an embodiment of the present application provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the above-mentioned dry mouth prediction method by executing the computer instructions.

[0033] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the above-mentioned dry mouth prediction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 A flowchart of a dry mouth prediction method provided in an embodiment of the present application;

[0036] Figure 2 A structural diagram of a TD model provided in an embodiment of the present application;

[0037] FIG3 is a step diagram of a dry mouth prediction method provided in an embodiment of the present application;

[0038] Figure 4 A structural diagram of a TDC model provided in an embodiment of the present application;

[0039] Figure 5 A structural diagram of an SD model provided in an embodiment of the present application;

[0040] Figure 6 A structural diagram of an SDC model provided in an embodiment of the present application;

[0041] Figure 7 A comparison chart of the effects of various models provided in the embodiments of the present application;

[0042] Figure 8 A block diagram of a dry mouth prediction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0044] NPC (Nasopharyngeal Carcinoma) is a type of cancer that occurs in the nasopharynx, also known as nasopharyngeal carcinoma. It is a relatively rare but serious type of cancer that originates in the nasopharynx (nasopharynx), which is the area connecting the nasal cavity and the throat. NPC usually results from abnormal proliferation and malignant changes in nasopharyngeal epithelial cells. Its specific cause is not fully understood, but it is related to genetic factors, viral infection (such as Epstein-Barr virus) and environmental factors. Treatments usually include radiotherapy, chemotherapy and surgery.

[0045] Radiotherapy is one of the main treatments for NPC (nasopharyngeal carcinoma), but radiotherapy may damage the surrounding salivary glands, resulting in reduced saliva secretion or changes in quality, which can cause dry mouth. Dry mouth not only affects the patient's oral comfort, but may also cause difficulty in chewing, swallowing and speaking, and even increase the risk of oral infection.

[0046] Each patient has different tolerance and response to treatment. Some patients may be more sensitive to radiotherapy and more prone to dry mouth. The medical team usually takes certain measures to relieve dry mouth symptoms, such as oral moisturizers, increased water intake, food adjustments, and drug interventions when necessary. At the same time, the medical team will assess the risk of dry mouth in the early stages of diagnosis and treatment planning, and regularly monitor and adjust during treatment to provide the best treatment effect and patient quality of life.

[0047] The dry mouth prediction methods in related technologies can be summarized into the following three types: First, the modeling prediction is carried out using radiomics features combined with clinical features, and the accuracy of the dry mouth prediction results is low; second, the modeling prediction is carried out using radiomics features combined with clinical features and dose parameters, and the dose parameters are extracted from the dose volume histogram DVH. DVH represents the distribution information in a one-dimensional discrete form of the dose-volume relationship. The extracted dose parameter data (such as: Dmax) are discrete points, with low prediction accuracy and poor ability to interpret the prediction results; third, the modeling prediction is carried out using radiomics features combined with clinical features and dosimetry, and the dose data covered by dosimetry is larger in magnitude. Furthermore, dosimetry is defined as applying radiomics methods or deep learning strategies to the three-dimensional dose distribution in radiotherapy to extract dose information related to the research objectives.

[0048] The third method is a common method in related technologies. It is generally divided into two categories: one is radiomics based on manually extracted features, or dosimetrics based on manually defined features, and the other is radiomics based on deep learning. Radiomics based on manually extracted features is more commonly used in related technologies, which requires additional feature screening when predicting dry mouth, which is time-consuming and labor-intensive.

[0049] The present application embodiment provides a method for predicting dry mouth, which extracts dose features from three-dimensional dose distribution data through deep learning technology, and uses adaptive average pooling technology to adjust the weights of the extracted features to achieve dimensionality reduction and improve the accuracy and interpretability of dry mouth prediction. Figure 1 , Figure 1 A flowchart of a dry mouth prediction method provided in an embodiment of the present application includes:

[0050] S100, obtaining three-dimensional dose distribution characteristics and medical image characteristics of the target object; wherein the three-dimensional dose distribution characteristics are obtained by extracting characteristics based on the three-dimensional dose distribution data of the target object.

[0051] Among them, the three-dimensional dose distribution data is used to characterize the dose distribution information in three-dimensional space, the medical image data is CT images and ROI images, and the medical image features are obtained by feature extraction based on the medical image data of the target object. Medical image features are extracted from the region of interest (ROI) of radiological images (CT, MR, and also PET, etc.) with high throughput for automated analysis, and key information is extracted from them to accurately quantify and evaluate the lesions, which are ultimately used for auxiliary diagnosis, classification or grading of diseases.

[0052] S200. According to the dose weight coefficient, denoted as W1, and the image weight coefficient, denoted as W2, weighted fusion is performed on the three-dimensional dose distribution feature W1 and the medical image feature W2 to obtain a first mixed feature; wherein the dose weight coefficient and the image weight coefficient are obtained by adjusting the respective weight coefficients of the three-dimensional dose distribution feature and the medical image feature of the reference object based on the adaptive average pooling method.

[0053] S300, performing residual convolution processing based on the first mixed feature to obtain prediction data; performing dry mouth prediction based on the prediction data to obtain a dry mouth prediction value.

[0054] Specifically, the reference object is the sample object used in the training of the dry mouth prediction model. Figure 2 , Figure 2 This is a structural diagram of a TD model provided in an embodiment of the present application, Total Dose Distribution. The TD model is an overall dose distribution model, which is predicted based on a dry mouth prediction method provided in an embodiment of the present application. The patient's head and neck three-dimensional dose distribution data, head and neck CT, and parotid gland ROI are input into the dry mouth prediction model, and the output is the dry mouth prediction value, that is, the probability of the patient experiencing dry mouth.

[0055] The network architecture of the TD model includes a convolutional block, three residual convolutional blocks, four Dropout layers, an adaptive 3D average pooling layer, and five fully connected layers. The convolutional block contains a 3×3×3 convolutional layer with a stride of 1×1×1, a batch normalization (BN) layer, and a ReLU nonlinear activation layer. Each residual convolutional block includes a 1×1×1 convolutional layer with a stride of 1×1×1, two 3×3×3 convolutional layers with a stride of 1×1×1, two BN layers, and two ReLU layers.

[0056] The coefficients of the four Dropout layers are 0.1, 0.1, 0.08, and 0.05, respectively. After the input image passes through the last Dropout layer, it is converted into a set of feature matrices with a shape of (512, 66, 94, 52). The feature matrix is ​​then input into an adaptive three-dimensional average pooling layer with a kernel size of 4×8×4. After being downsampled, it is flattened into a one-dimensional vector M of size 65536, indicating that the TD model extracts information from the input data.

[0057] Refer to Figure 3, which is a step diagram of a dry mouth prediction method provided in an embodiment of the present application, including the steps of: data collection, data preprocessing, data partitioning, model training and model evaluation.

[0058] S110. Obtain reference data of 243 reference NPC patients, collect three-dimensional dose distribution data, parotid ROI data and CT data, and perform data preprocessing. The preprocessing includes three-dimensional dose distribution reconstruction and head and neck region interception (dose, CT, ROI). Then, perform three-dimensional convolution operations on the preprocessed three-dimensional dose distribution data, parotid ROI data and CT data, respectively, to obtain three-dimensional dose distribution features, denoted as DF, parotid ROI feature matrix and CT features, denoted as RF.

[0059] S210, obtain reference data of 243 reference NPC patients. Before training the dry mouth prediction model, divide the reference data set into 190 training sets, 23 validation sets, and 30 test sets. This application is based on the Pytorch deep learning framework, trains the dry mouth prediction model on NVIDIA RTX 3090, and uses a five-fold cross validation method for experiments. Furthermore, in order to increase the complexity of the data and reduce the overfitting degree of the dry mouth prediction model, the data set is enhanced, specifically: the original sample objects in the training set, validation set, and test set are flipped and rotated 10°, 15°, and 20° in the horizontal direction, respectively.

[0060] S220, training the initial dry mouth prediction model to obtain a trained dry mouth prediction model, including:

[0061] A loss function of the initial dry mouth prediction model is constructed, and the initial dry mouth prediction model is trained until the loss function converges to a preset value to obtain a trained dry mouth prediction model.

[0062] Specifically, when training the dry mouth prediction model, the binary cross entropy function BCELoss is used as the loss function, and the formula is as follows:

[0063]

[0064] S230. Perform dot multiplication operation on the three-dimensional dose distribution feature DF and the CT feature RF with the parotid gland ROI feature matrix respectively to obtain the three-dimensional dose distribution feature DF' and the CT feature RF' of the parotid gland; adjust the weights of the three-dimensional dose distribution feature DF' and the CT feature RF' of the parotid gland based on the adaptive three-dimensional average pooling method to obtain the weight-adjusted dose weight coefficient and the image weight coefficient; concatenate the three-dimensional dose distribution feature DF' and the CT feature RF' of the parotid gland based on the dose weight coefficient and the image weight coefficient to obtain the first mixed feature, recorded as HF.

[0065] S240, the first mixed feature HF passes through three residual convolution blocks in sequence, namely RS Block1, RS Block2 and RS Block3, and then passes through an adaptive three-dimensional average pooling layer Pooling to obtain a vector M of size 65536, and then passes through a fully connected layer (Fully Connected Layer, FC) to obtain predicted data, denoted as Q, and then after passing through the Sigmoid function, dry mouth prediction is performed based on the predicted data to obtain a dry mouth prediction value. The dry mouth prediction value is a number in the range of [0,1]. The larger the value, the greater the probability that the reference object will experience dry mouth.

[0066] S250, training the initial dry mouth prediction model to obtain a trained dry mouth prediction model, including: setting evaluation indicators of the initial dry mouth prediction model until the evaluation indicators are optimized to a preset threshold value to obtain a trained dry mouth prediction model, the evaluation indicators including an area under the curve indicator AUC (Area Under roc Curve), an accuracy indicator ACC (Accuracy), a precision indicator (Precision) and an F1 metric indicator (F1-score).

[0067] Specifically, the above indicators are calculated as follows:

[0068]

[0069] In the above formula, TP represents the number of samples correctly classified as positive, FP represents the number of samples incorrectly classified as positive, TN represents the number of samples correctly classified as negative, and FN represents the number of samples incorrectly classified as negative.

[0070] S260, obtaining an adjusted dose weight coefficient and an image weight coefficient through the trained dry mouth prediction model, and weightedly fusing the three-dimensional dose distribution characteristics and medical image characteristics of the target object according to the adjusted dose weight coefficient and the image weight coefficient to obtain a first mixed characteristic of the target object. The dose weight coefficient and the image weight coefficient are obtained by adjusting the weight coefficients of the three-dimensional dose distribution characteristics and the medical image characteristics of the reference object based on the adaptive average pooling method.

[0071] S300, performing residual convolution processing based on the first mixed feature of the target object to obtain prediction data; performing dry mouth prediction based on the prediction data to obtain a dry mouth prediction value.

[0072] This application introduces dose distribution information in three-dimensional space, fuses and analyzes the three-dimensional dose distribution characteristics and medical image characteristics, and adjusts the weight coefficients of the three-dimensional dose distribution characteristics and medical image characteristics based on the adaptive three-dimensional average pooling method, thereby improving the accuracy of dry mouth prediction and the interpretability of dry mouth prediction results.

[0073] As an embodiment of the present invention, performing residual convolution processing based on the first mixed feature to obtain prediction data includes:

[0074] Performing residual convolution processing based on the first mixed feature to obtain processed feature data;

[0075] The processed feature data are fused with the clinical feature data of the target subject to obtain the predicted data.

[0076] The clinical characteristics of the target subjects include gender data, age data, T stage data and N stage data. Figure 4 , Figure 4 A structural diagram of a TDC model provided in an embodiment of the present application. Based on the TD model, the present application constructs a TDC model (Total Dose Distribution and Clinical, TDC). The network architecture of the TDC model includes a convolution block, three residual convolution blocks, four Dropout layers, an adaptive three-dimensional average pooling layer and three fully connected layers connected in sequence.

[0077] Specifically, the workflow of the TDC model is similar to that of the TD model. Feature extraction is performed on the three-dimensional dose distribution data of the head and neck and the head and neck CT data, respectively, and then converted into a one-dimensional vector H of size 256 through the residual convolution block and the fully connected layer.

[0078] The input clinical features first pass through two fully connected layers separately to obtain a one-dimensional vector J of size 64, which is then concatenated with H to obtain a one-dimensional vector Q of size 320. Vector Q passes through subsequent fully connected layers to obtain the predicted value of the probability of dry mouth.

[0079] This application introduces dose distribution information in three-dimensional space, fuses and analyzes three-dimensional dose distribution characteristics, medical imaging characteristics and clinical characteristic data, and adjusts the weight coefficients of three-dimensional dose distribution characteristics and medical imaging characteristics based on an adaptive three-dimensional average pooling method, thereby improving the accuracy of dry mouth prediction and the interpretability of dry mouth prediction results.

[0080] As an embodiment of the present invention, the three-dimensional dose distribution data of the target object is divided into a plurality of segments of three-dimensional dose segmentation data; and the three-dimensional dose distribution characteristics of the target object are obtained, including:

[0081] Perform feature extraction on the three-dimensional dose segmentation data to obtain three-dimensional dose segmentation features;

[0082] The three-dimensional dose segmentation features are fused to obtain a three-dimensional dose distribution feature, and N three-dimensional dose segmentation features are superimposed to obtain a fused three-dimensional dose distribution feature.

[0083] Reference Figure 5 , Figure 5 A structural diagram of an SD model provided in an embodiment of the present application. Based on the TD model, the present application constructs an SD model (segmented dose distribution). Through the dose segmentation strategy, the overall three-dimensional dose distribution is divided into several three-dimensional dose segments, namely L (0-16Gy), the corresponding weights are recorded as W11, ML (16-32Gy), the corresponding weights are recorded as W12, M (32-48Gy), the corresponding weights are recorded as W13, MH (48-64Gy), the corresponding weights are recorded as W14 and H (above 64Gy), the corresponding weights are recorded as W15, and the weights of the five segments are recorded as W11, W12, W13, W14 and W15.

[0084] Convolution extraction was performed on the five segmented dose distributions respectively to obtain the corresponding dose-omics features based on the segmented dose distribution. Subsequently, each dose-omics feature was multiplied by the corresponding weight and then superimposed.

[0085] Reference Figure 6 , Figure 6A structural diagram of an SDC model provided in an embodiment of the present application, Segmented DoseDistribution and Clinical, the input of the SDC model is three-dimensional dose distribution data of the head and neck, CT data, parotid ROI data and clinical feature data, and clinical features are added on the basis of the SD model. The SDC model first segments the overall three-dimensional dose distribution, and performs separate feature extraction on each segmented dose distribution to obtain 5 segmented dose features. The segmented dose features are then fused and spliced ​​with the image features to obtain a mixed feature HF. HF obtains 256 features after subsequent residual convolution blocks and fully connected layers. The above 256 features are then spliced ​​with the clinical feature data that has passed through two fully connected layers to obtain a matrix Q. After the matrix Q passes through a fully connected layer, the predicted value of dry mouth is obtained.

[0086] It is understandable that the embodiment of the present application provides an initial model, and only clinical characteristic data is input into the initial model, and the prediction effect is poor.

[0087] Furthermore, after each model is trained, each weight coefficient is solidified for subsequent direct use, saving time and effort. Depending on whether clinical characteristic data is input and whether dose segmentation is performed, the embodiments of the present application respectively establish a TD model (overall dose distribution model), a TDC model (overall dose distribution + clinical characteristic model), a SD model (segmented dose distribution model) and a SDC model (segmented dose distribution + clinical characteristic model), referring to Figure 7 , Figure 7 This is a comparison chart of the effects of various models provided in the embodiments of the present application. The differences between the dry mouth prediction models provided in the embodiments of the present application are shown in Table 1 below:

[0088] Table 1 Input feature information of different dry mouth prediction models

[0089]

[0090] In the table, “√” indicates that the current model uses the selected features as input. Figure 7 It can be seen that the SDC model has the best prediction effect.

[0091] This application introduces dose distribution information in three-dimensional space, extracts three-dimensional dose distribution features, medical imaging features and clinical feature data based on deep learning technology and performs fusion analysis, adjusts the weight coefficients of three-dimensional dose distribution features and medical imaging features based on an adaptive three-dimensional average pooling method, adjusts the weights of each segmented dose distribution based on an adaptive three-dimensional average pooling method, and establishes multiple end-to-end dry mouth prediction models, which improves the accuracy of dry mouth prediction and the interpretability of dry mouth prediction results. The higher the weight, the greater the impact on the prediction results.

[0092] As an embodiment of the present invention, different segments of three-dimensional dose segmentation features correspond to different three-dimensional dose segmentation weight coefficients; the three-dimensional dose segmentation features are fused to obtain three-dimensional dose distribution features, including:

[0093] The three-dimensional dose segmentation characteristics are weighted and fused according to the three-dimensional dose segmentation weight coefficient to obtain the three-dimensional dose distribution characteristics.

[0094] As an embodiment of the present invention, the three-dimensional dose segmentation weight coefficient corresponding to the three-dimensional dose segmentation feature is obtained by adjusting the weight coefficient of the three-dimensional dose segmentation feature of the reference object based on the adaptive average pooling method.

[0095] As an embodiment of the present invention, the three-dimensional dose segmentation data is obtained by dividing the three-dimensional dose distribution data of the target object according to a preset segment number N; wherein N is an integer greater than or equal to 2.

[0096] Specifically, the overall dose distribution is divided into five dose segments, namely L, ML, M, MH and H. Taking M as the base and extending to both sides, it is divided into uniform intervals by equal division, or by random or normal division. The division method is not unique.

[0097] As an embodiment of the present invention, obtaining three-dimensional dose distribution characteristics and medical image characteristics of a target object includes:

[0098] Obtain a feature matrix of the parotid gland region of the target object;

[0099] Performing convolution processing on the three-dimensional dose distribution data of the target object to obtain an intermediate dose feature, and performing convolution processing on the medical image data of the target object to obtain an intermediate image feature;

[0100] Perform dot multiplication operation on the feature matrix and the intermediate dose feature to obtain the three-dimensional dose distribution feature;

[0101] Perform a dot product operation on the feature matrix and the intermediate image features to obtain the medical image features.

[0102] Reference Figure 8 , Figure 8 This is a block diagram of a dry mouth prediction device provided in an embodiment of the present application. The present application provides a dry mouth prediction device, including an acquisition module 1010 , a weight adjustment module 1020 and a prediction module 1030 .

[0103] The acquisition module 1010 is used to acquire the three-dimensional dose distribution characteristics and medical image characteristics of the target object; wherein the three-dimensional dose distribution characteristics are obtained by performing feature extraction based on the three-dimensional dose distribution data of the target object.

[0104] The weight adjustment module 1020 is used to perform weighted fusion of the three-dimensional dose distribution characteristics and the medical image characteristics according to the dose weight coefficient and the image weight coefficient to obtain a first mixed feature; wherein the dose weight coefficient and the image weight coefficient are obtained by adjusting the respective weight coefficients of the three-dimensional dose distribution characteristics and the medical image characteristics of the reference object based on the adaptive average pooling method.

[0105] The prediction module 1030 is configured to perform residual convolution processing based on the first mixed feature to obtain prediction data; and perform dry mouth prediction based on the prediction data to obtain a dry mouth prediction value.

[0106] As an embodiment of the present invention, the prediction module 1030 includes:

[0107] Performing residual convolution processing based on the first mixed feature to obtain processed feature data;

[0108] The processed feature data are fused with the clinical feature data of the target subject to obtain prediction data.

[0109] As an embodiment of the present invention, the three-dimensional dose distribution data of the target object is divided into three-dimensional dose segmented data of several segments; the acquisition module 1010 includes:

[0110] Perform feature extraction on the three-dimensional dose segmentation data to obtain three-dimensional dose segmentation features;

[0111] The three-dimensional dose segmentation features are fused to obtain the three-dimensional dose distribution features.

[0112] As an embodiment of the present invention, different three-dimensional dose segmentation features correspond to different three-dimensional dose segmentation weight coefficients; the acquisition module 1010 also includes:

[0113] The three-dimensional dose segmentation characteristics are weighted and fused according to the three-dimensional dose segmentation weight coefficient to obtain the three-dimensional dose distribution characteristics.

[0114] As an embodiment of the present invention, the three-dimensional dose segmentation weight coefficient corresponding to the three-dimensional dose segmentation feature is obtained by adjusting the weight coefficient of the three-dimensional dose segmentation feature of the reference object based on the adaptive average pooling method.

[0115] As an embodiment of the present invention, the three-dimensional dose segmentation data is obtained by dividing the three-dimensional dose distribution data of the target object according to a preset segment number N; wherein N is an integer greater than or equal to 2.

[0116] As an embodiment of the present invention, the acquisition module 1010 includes:

[0117] Acquire a feature matrix of the parotid gland region of the target object;

[0118] Performing convolution processing on the three-dimensional dose distribution data of the target object to obtain an intermediate dose feature, and performing convolution processing on the medical image data of the target object to obtain an intermediate image feature;

[0119] Performing a dot multiplication operation on the feature matrix and the intermediate dose feature to obtain the three-dimensional dose distribution feature;

[0120] A dot product operation is performed on the feature matrix and the intermediate image feature to obtain the medical image feature.

[0121] The present application provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes a dry mouth prediction method provided in the above embodiment by executing the computer instructions.

[0122] The computer device includes: one or more processors, memory, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0123] The processor may be a central processing unit, a network processor or a combination thereof. The processor may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0124] The memory stores instructions executable by at least one processor, so that the at least one processor executes the method shown in the above embodiment.

[0125] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the storage may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0126] The memory may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid state drive; the memory may also include a combination of the above types of memory.

[0127] The computer device also includes a communication interface, which is used for the computer device to communicate with other devices or a communication network.

[0128] The present application provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute a dry mouth prediction method provided in the above embodiment.

[0129] The embodiment of the present application also provides a computer-readable storage medium. The above method according to the embodiment of the present application can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0130] The embodiment of the present application provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of any embodiment of the present application.

[0131] Although the embodiments of the present application are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations are all within the scope defined by the appended claims.

[0132] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0133] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0134] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0135] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0138] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0139] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0140] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

[0141] Although the embodiments of the present application have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for predicting dry mouth, characterized in that: include: Acquire three-dimensional dose distribution characteristics and medical image characteristics of the target object; wherein the three-dimensional dose distribution characteristics are obtained by extracting the three-dimensional dose distribution data of the target object; The three-dimensional dose distribution feature and the medical image feature are weightedly fused according to the dose weight coefficient and the image weight coefficient to obtain a first mixed feature; wherein the dose weight coefficient and the image weight coefficient are obtained by adjusting the respective weight coefficients of the three-dimensional dose distribution feature and the medical image feature of the reference object based on the adaptive average pooling method; Residual convolution processing is performed based on the first mixed feature to obtain prediction data; dry mouth prediction is performed based on the prediction data to obtain a dry mouth prediction value.

2. The method according to claim 1, characterized in that The performing residual convolution processing based on the first mixed feature to obtain prediction data includes: Performing residual convolution processing based on the first mixed feature to obtain processed feature data; The processed feature data are fused with the clinical feature data of the target subject to obtain prediction data.

3. The method according to claim 1 or 2, characterized in that: The three-dimensional dose distribution data of the target object is divided into a plurality of segments of three-dimensional dose segmentation data; and the three-dimensional dose distribution characteristics of the target object are obtained, including: Perform feature extraction on the three-dimensional dose segmentation data to obtain three-dimensional dose segmentation features; The three-dimensional dose segmentation features are fused to obtain the three-dimensional dose distribution features.

4. The method according to claim 3, characterized in that Different three-dimensional dose segmentation features of different segments correspond to different three-dimensional dose segmentation weight coefficients; the three-dimensional dose segmentation features are fused to obtain the three-dimensional dose distribution features, including: The three-dimensional dose segmentation characteristics are weighted and fused according to the three-dimensional dose segmentation weight coefficient to obtain the three-dimensional dose distribution characteristics.

5. The method according to claim 4, characterized in that The three-dimensional dose segmentation weight coefficient corresponding to the three-dimensional dose segmentation feature is obtained by adjusting the weight coefficient of the three-dimensional dose segmentation feature of the reference object based on the adaptive average pooling method.

6. The method according to claim 3, characterized in that The three-dimensional dose segmentation data is obtained by dividing the three-dimensional dose distribution data of the target object according to a preset segment number N; wherein N is an integer greater than or equal to 2.

7. The method according to claim 1, characterized in that The step of acquiring the three-dimensional dose distribution characteristics and medical imaging characteristics of the target object includes: Acquire a feature matrix of the parotid gland region of the target object; Performing convolution processing on the three-dimensional dose distribution data of the target object to obtain an intermediate dose feature, and performing convolution processing on the medical image data of the target object to obtain an intermediate image feature; Performing a dot multiplication operation on the feature matrix and the intermediate dose feature to obtain the three-dimensional dose distribution feature; A dot product operation is performed on the feature matrix and the intermediate image feature to obtain the medical image feature.

8. A dry mouth prediction device, characterized in that: Includes acquisition module, weight adjustment module and prediction module; The acquisition module is used to acquire the three-dimensional dose distribution characteristics and medical image characteristics of the target object; wherein the three-dimensional dose distribution characteristics are obtained by extracting the three-dimensional dose distribution data of the target object; The weight adjustment module is used to perform weighted fusion of the three-dimensional dose distribution feature and the medical image feature according to the dose weight coefficient and the image weight coefficient to obtain a first mixed feature; wherein the dose weight coefficient and the image weight coefficient are obtained by adjusting the respective weight coefficients of the three-dimensional dose distribution feature and the medical image feature of the reference object based on the adaptive average pooling method; The prediction module is used to perform residual convolution processing based on the first mixed feature to obtain prediction data; and perform dry mouth prediction based on the prediction data to obtain a dry mouth prediction value.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes a dry mouth prediction method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute a dry mouth prediction method according to any one of claims 1 to 7.

Images