Multimodal Colorectal Cancer Pre-diagnosis Information Processing Method, System, Medium, and Device

Abstract

The present invention relates to a multi-modal colorectal cancer pre-diagnosis information processing method, system, medium, and device. By converting image information into first-modal features, first clinical information into second-modal features, and family history information into third-modal features, and then performing feature fusion and inputting it into a neural network model of deep learning. Finally, the first pre-diagnosis report output can provide information on the disease risk level, stage feature prediction, canceration probability, and treatment plan of the current user. The multi-modal technology is applied to integrate various information sources such as the patient's personal information, image information, first clinical symptoms, and family history information, improving the comprehensive analysis of the colorectal diseases of the current user, enabling doctors to obtain a more comprehensive disease assessment, and thus achieving a more accurate risk level classification, disease stage prediction, and personalized treatment plan formulation, improving the efficiency and accuracy of the diagnosis and treatment process.

Description

Technical Field

[0001] This application relates to the field of colorectal cancer, and specifically to a multimodal colorectal cancer pre-diagnosis information processing method, system, medium, and device. Background Art

[0002] Colorectal cancer is a malignant tumor that occurs in the colon and rectum. It usually originates from the abnormal proliferation and differentiation of intestinal mucosal epithelial cells and gradually develops into a tumor. Colorectal cancer includes colon cancer and rectal cancer. Colon cancer mainly occurs in different parts of the colon, such as the ascending colon, transverse colon, descending colon, and sigmoid colon. Rectal cancer occurs in the rectal segment between the anus and the S-shaped colon. Most colorectal cancers are caused by the adenoma-carcinoma sequence of intestinal epithelial cells. Colorectal cancer is also associated with a family history of the disease. Diagnostic methods include colonoscopy, imaging examinations, and clinical symptoms, combined with a doctor's specific assessment, prediction, and diagnosis of the condition.

[0003] As a malignant tumor, the inducing factors, canceration probability, and diagnosis and treatment methods of colorectal cancer vary due to the specific conditions of each patient. The information provided by the patient during the medical treatment process is also one-sided, resulting in the doctor being unable to obtain a more comprehensive understanding of the inducing factors of colorectal cancer in the current patient, thereby affecting the formulation process of the diagnosis and treatment methods. Summary of the Invention

[0004] In view of the above problems, this application provides a multimodal colorectal cancer pre-diagnosis information processing method, system, medium, and device, which solves the problem that the information provided by patients in the early diagnosis stage of existing colorectal cancer is not comprehensive enough, affecting the formulation of subsequent diagnosis and treatment methods.

[0005] To achieve the above object, in the first aspect, the present invention provides a multimodal colorectal cancer pre-diagnosis information processing method, including:

[0006] Collect the medical treatment information of the user, where the medical treatment information includes personal information, imaging information, first clinical information, and family medical history information. The personal information includes the name, permanent residence information, and user ID number of the current user. The imaging information includes multiple pathological images of the current user. The first clinical information includes the clinical symptoms, course duration, and onset node at the current stage of medical treatment. The family medical history information includes the second clinical information of the family members related to the current user under the current disease condition;

[0007] Input the imaging information into the UNET algorithm model to obtain first-modal features, where the first-modal features include at least one of a lesion area, tumor size, tumor distribution information, and the proportion of cancerous cells in the tumor;

[0008] Extract keywords from the first clinical information to obtain multiple semantic keywords, and input the multiple semantic keywords into the semantic dictionary to construct the semantic feature vector of the current first clinical information, denoted as the second modality feature;

[0009] Input the family medical history information into the random forest algorithm model to obtain the third modality feature, and the third modality feature includes the specific onset characteristics of the current user under the current disease;

[0010] Input the first modality feature, the second modality feature, and the third modality feature into the feature fusion algorithm model for feature fusion to obtain the first fusion feature;

[0011] Input the first fusion feature into the trained neural network model to obtain the first preliminary diagnosis result, and the first preliminary diagnosis result includes the disease risk level information of the current user, the predicted information of the disease stage characteristics, the canceration probability information of the disease, and the treatment plan information of the disease;

[0012] Convert the first preliminary diagnosis result into a first preliminary diagnosis report and display it.

[0013] Further, input the imaging information into the UNET algorithm model, and the obtained first modality feature includes:

[0014] Label the imaging information using a labeler to obtain multiple pieces of first image information;

[0015] Preprocess each piece of first image information one by one to obtain multiple pieces of second image information, and the preprocessing includes at least one of image enhancement, image cutting, and image normalization processing;

[0016] Arrange the multiple pieces of second image information in the acquisition order, and obtain the slice position information between two adjacent pieces of second image information. The slice position information includes the slice thickness and the slice spacing;

[0017] Stack the multiple pieces of second image information arranged in the acquisition order along a preset direction according to the slice position information to generate a first three-dimensional image matrix;

[0018] Input the first three-dimensional image matrix into the trained UNET algorithm model to obtain a second three-dimensional image matrix;

[0019] Map the second three-dimensional image matrix and the multiple pieces of second image information to obtain the first modality feature.

[0020] Further, the first three-dimensional image matrix includes multiple first voxel vectors, and the second three-dimensional image matrix includes multiple second voxel vectors. Inputting the first three-dimensional image matrix into the trained UNET algorithm model to obtain the second three-dimensional image matrix includes:

[0021] Encode the first three-dimensional image matrix layer by layer to a first preset threshold to obtain first local feature information, where the first local feature information includes the local features extracted from each first voxel vector during the layer-by-layer encoding process;

[0022] Decode the first local feature information layer by layer to obtain second local feature information, where the second local feature information includes the local features of each second voxel vector, and the local feature of the second voxel vector is the local feature of each first voxel vector fused layer by layer during the decoding process;

[0023] Perform convolution on the second local feature information to obtain a first primary three-dimensional image matrix;

[0024] Perform threshold processing on the first primary three-dimensional image matrix to obtain a first segmentation mask;

[0025] Determine whether the first segmentation mask meets a second preset threshold;

[0026] If so, output the first primary three-dimensional image matrix as the second three-dimensional image matrix;

[0027] If not, correct the first segmentation mask to obtain a second segmentation mask;

[0028] Calculate the loss function of the UNET algorithm model during the layer-by-layer decoding process according to the second segmentation mask;

[0029] Perform gradient feedback according to the loss function to update the model parameters used by the UNET algorithm model during the layer-by-layer decoding process;

[0030] Perform layer-by-layer decoding on the first local feature information in the updated UNET algorithm model to obtain third local feature information, where the third local feature information includes the local features of each first voxel vector fused during the decoding process;

[0031] Perform convolution on the third local feature information to obtain a second primary three-dimensional image matrix;

[0032] Perform threshold processing on the second primary three-dimensional image matrix to obtain a third segmentation mask;

[0033] Determine whether the third segmentation mask meets the second preset threshold;

[0034] If so, output the second primary three-dimensional image matrix as the second three-dimensional image matrix.

[0035] Furthermore, extract keywords from the first clinical information to obtain multiple semantic keywords including:

[0036] Perform data cleaning on the first clinical information to obtain a first information text;

[0037] Perform sentence segmentation on the first information text to obtain multiple segments to be processed, and each segment to be processed has multiple words;

[0038] Input each current segment to be processed into the BERT variant model one by one to obtain the first embedding vector information of each segment to be processed, and the first embedding vector information includes the first word embedding vector of each word in the same segment to be processed;

[0039] Input the first embedding vector information into the TF-IDF model to obtain the weight value corresponding to the first word embedding vector of the same segment to be processed;

[0040] Arrange the multiple first word embedding vectors in the first information text in ascending order of the weight value from left to right, and select the words corresponding to the preset number of first word embedding vectors starting from the left as semantic keywords, and denote the first word embedding vectors corresponding to the semantic keywords as the second word embedding vectors.

[0041] Furthermore, input the multiple semantic keywords into the semantic dictionary to construct the semantic feature vector of the current first clinical information, denoted as the second modality feature, including:

[0042] Obtain the semantic dictionary, which is trained according to the sample clinical information and includes multiple clinical entries;

[0043] Match the semantic keywords with the clinical entries one by one;

[0044] Fill the second word embedding vectors associated with the successfully matched semantic keywords into the semantic vectors corresponding to the clinical entries, and denote them as the first semantic vectors;

[0045] Set the semantic vectors corresponding to the clinical entries that are not successfully matched to 0;

[0046] Perform feature concatenation on the multiple first semantic vectors to obtain the semantic feature vector represented by a matrix, denoted as the second modality feature;

[0047] Matching the semantic keywords with the clinical entries one by one includes:

[0048] Obtain the second word embedding vector of each semantic keyword;

[0049] Obtain the semantic vector of each clinical entry;

[0050] Calculate the cosine similarity between the second word embedding vector and the current semantic vector one by one;

[0051] Determine whether the cosine similarity exceeds the semantic similarity threshold;

[0052] If so, record the cosine similarity as the first cosine similarity;

[0053] Determine whether the number of the first cosine similarities is greater than 1;

[0054] If the number of the first cosine similarities is greater than 1, arrange the multiple first cosine similarities in descending order according to the similarity values, and select the semantic keyword corresponding to the second word embedding vector with the highest first cosine similarity to match the current clinical entry;

[0055] If the number of the first cosine similarities is 1, match the semantic keyword associated with the first cosine similarity with the current clinical entry;

[0056] If not, it means that the current clinical entry fails to match.

[0057] Further, input the family history information into the random forest algorithm model to obtain the third modal features, including:

[0058] Taking the family members in the family history information as nodes and the kinship as the hierarchical basis, construct the family tree information, assign 1 to the family members with the second clinical information, and assign 0 to the family members without the second clinical information;

[0059] Take the family tree information as the input feature and the semantic keyword as the variable label and input them into the random forest algorithm model to obtain the influence features of each family member under the current disease condition, denoted as the first influence features;

[0060] Calculate the importance of the first influence features one by one, and denote it as the first importance, which is represented by formula (1). Formula (1) is as follows:

[0061] ;

[0062] In formula (1), is the first importance of the th first influence feature, is the reduction in impurity when splitting using the feature in tree , is the reduction in impurity, is the total number of trees in the random forest algorithm model, is the tree in the random forest algorithm model;

[0063] Normalize the multiple first importances to obtain multiple second importances, which are represented by formula (2). Formula (2) is as follows:

[0064] ;

[0065] In formula (2), is the second importance of the th first influence feature, and N is the number of nodes of the family members;

[0066] Represent multiple second importances using a numerical sequence to obtain third-modal features.

[0067] Furthermore, input the first-modal features, second-modal features, and third-modal features into a feature fusion algorithm model for feature fusion to obtain first fusion features, including:

[0068] Concatenate the first-modal features, second-modal features, and third-modal features to obtain a first concatenated feature, and represent the first concatenated feature with formula (3), where formula (3) is as follows:

[0069] ;

[0070] In formula (3), is the first-modal feature, is the second-modal feature, is the third-modal feature, is the first concatenated feature;

[0071] Input the first concatenated feature into an attention mechanism algorithm model to obtain a query vector, a key vector, and a value vector, which are represented by formula (4), where formula (4) is as follows:

[0072]

[0073] In formula (4), is the query vector, is the key vector, is the value vector, is the weight matrix of the query vector, is the weight matrix of the key vector, is the weight matrix of the value vector;

[0074] Calculate the attention weights based on the query vector, key vector, and value vector to obtain an attention weight matrix, which is represented by formula (5), where formula (5) is as follows:

[0075] ;

[0076] In formula (5), is the transposed matrix of, is the dimension of the key vector, is the attention of the first-modal feature to itself, is the attention of the first-modal feature to the second-modal feature, is the attention of the first-modal feature to the third-modal feature, is the attention of the second-modal feature to the first-modal feature, is the attention of the second-modal feature to itself, is the attention degree of the second modal feature to the third modal feature, is the attention degree of the third modal feature to the first modal feature, is the attention degree of the third modal feature to the second modal feature, is the attention degree of the third modal feature to itself;

[0077] According to the attention weight matrix, the value vectors are weighted and summed to obtain the attention fusion feature, and the attention fusion feature is represented by formula (6), and formula (6) is as follows:

[0078] ;

[0079] In formula (6), is the attention fusion feature, is the weighted summation function;

[0080] The attention fusion feature and the first splicing feature are fused to obtain the first fusion feature, and the first fusion feature is represented by formula (7), and formula (7) is as follows:

[0081] ;

[0082] In formula (7), is the first fusion feature.

[0083] In a second aspect, the present invention also provides a multi-modal colorectal cancer pre-diagnosis information processing system, which is applicable to the pre-diagnosis information processing method described in the first aspect. The pre-diagnosis information processing system includes an information collection module, a first feature conversion module, a second feature conversion module, a third feature conversion module, a feature fusion module, and a pre-diagnosis report generation module;

[0084] The information collection module is used to collect the medical information of users. The medical information includes personal information, imaging information, first clinical information, and family medical history information. Personal information includes the name of the current user, permanent residence information, and user ID number. Imaging information includes multiple pathological images of the current user. First clinical information includes clinical symptoms, disease duration, and onset nodes at the current stage of diagnosis. Family medical history information includes second clinical information of family members related to the current user under the current disease; The first feature conversion module is used to input the imaging information into the UNET algorithm model to obtain first-modal features, which include at least one of the lesion area, tumor size, tumor distribution information, and proportion of cancerous cells in the tumor; The second feature conversion module is used to extract keywords from the first clinical information to obtain multiple semantic keywords, and input the multiple semantic keywords into the semantic dictionary to construct a semantic feature vector of the current first clinical information, denoted as the second-modal feature; The third feature conversion module is used to input the family medical history information into the random forest algorithm model to obtain third-modal features, which include specific onset characteristics of the current user under the current disease; The feature fusion module is used to input the first-modal features, second-modal features, and third-modal features into the feature fusion algorithm model for feature fusion to obtain first fusion features; The preliminary diagnosis report generation module is used to input the first fusion features into the trained neural network model to obtain a first preliminary diagnosis result, which includes the disease risk level information, stage feature prediction information of the disease, canceration probability information of the disease, and treatment plan information of the disease of the current user, and convert the first preliminary diagnosis result into a first preliminary diagnosis report and display it.

[0085] In a third aspect, the present invention also provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in the first aspect is implemented.

[0086] In a fourth aspect, the present invention also provides an electronic device, including a memory and a processor. The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method described in the first aspect.

[0087] Different from the prior art, in the above technical solution, by converting the imaging information into the first modal feature, the first clinical information into the second modal feature, and the family medical history information into the third modal feature, and then performing feature fusion on the first modal feature, the second modal feature, and the third modal feature and inputting them into the neural network model of deep learning, the first preliminary diagnosis report finally output can give the disease risk level information, the stage feature prediction information of the disease, the canceration probability information of the disease, and the treatment plan information of the disease. The entire technical solution applies the multi-modal technical principle to integrate various information sources such as the personal information, imaging information, first clinical symptoms, and family medical history information of the patient, improving the comprehensive analysis of the colorectal disease of the current user, enabling doctors to obtain a more comprehensive condition assessment, and thus realizing a more accurate risk level classification, disease stage prediction, and personalized treatment plan formulation, improving the efficiency and accuracy of the diagnosis and treatment process.

[0088] The relevant records in the above invention content are only an overview of the technical solution of this application. In order to enable those of ordinary skill in the art to more clearly understand the technical solution of this application, and thus can be implemented according to the content recorded in the description and the drawings, and in order to make the above objects, other objects, features, and advantages of this application more easily understood, the following is described in conjunction with the specific embodiments of this application and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] The drawings are only used to illustrate the principles, implementation methods, applications, features, and effects of the specific embodiments of the present invention and other related contents, and should not be considered as a limitation to this application.

[0090] In the accompanying drawings of the specification:

[0091] Figure 1 It is a step schematic diagram of steps S101 to S107 of the multi-modal colorectal cancer preliminary diagnosis information processing method described in the specific embodiment;

[0092] Figure 2 It is a schematic diagram of the multi-modal colorectal cancer preliminary diagnosis information processing system described in the specific embodiment;

[0093] Figure 3 It is a step schematic diagram of steps S201 to S206 of the multi-modal colorectal cancer preliminary diagnosis information processing method described in the specific embodiment;

[0094] Figure 4 It is a step schematic diagram of steps S301 to S305 of the multi-modal colorectal cancer preliminary diagnosis information processing method described in the specific embodiment;

[0095] Figure 5Schematic diagram of steps S401 to S405 of the multi-modal colorectal cancer pre-diagnosis information processing method described in the specific implementation manner.

[0096] The descriptions of the reference numerals involved in the above-mentioned respective drawings are as follows:

[0097] 1. Multi-modal colorectal cancer pre-diagnosis information processing system;

[0098] 11. Information acquisition module;

[0099] 12. First feature conversion module;

[0100] 13. Second feature conversion module;

[0101] 14. Third feature conversion module;

[0102] 15. Feature fusion module;

[0103] 16. Pre-diagnosis report generation module. Specific implementation manner

[0104] To illustrate in detail the possible application scenarios, technical principles, implementable specific solutions, achievable objectives and effects, etc. of the present application, the following is described in detail with reference to the specific examples listed and in conjunction with the drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of the present application, so they are only examples and cannot be used to limit the protection scope of the present application.

[0105] Referring to "embodiment" in this article means that the specific features, structures or characteristics described in connection with the embodiment may be included in at least one embodiment of the present application. The term "embodiment" appearing in various positions in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or relevance to other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, the technical features mentioned in each embodiment can be combined in any way to form the corresponding implementable technical solution.

[0106] Unless otherwise defined, the meanings of the technical terms used herein are the same as those generally understood by those skilled in the technical field to which the present application belongs; the use of the relevant terms herein is only for describing specific embodiments and is not intended to limit the present application.

[0107] In the description of the present application, the phrase "and / or" is an expression used to describe the logical relationship between objects, indicating that there can be three relationships. For example, A and / or B means: there is A, there is B, and there is both A and B at the same time. In addition, the character " / " in this article generally represents an "or" logical relationship between the associated objects.

[0108] In this application, terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual quantitative, primary-secondary, or sequential relationship between these entities or operations.

[0109] Without further limitation, in this application, the open-ended expressions such as "comprising", "including", "having", or other similar expressions used in a statement are intended to cover non-exclusive inclusion. These expressions do not exclude the possibility that there may be additional elements in the process, method, or product that includes the said elements. Thus, in a process, method, or product that includes a series of elements, it may include not only those defined elements, but also other elements not explicitly listed, or elements inherent to such a process, method, or product.

[0110] Similar to the understanding in the Examination Guidelines, in this application, expressions such as "greater than", "less than", "exceeding", etc. are understood not to include the base number; expressions such as "above", "below", "within", etc. are understood to include the base number. In addition, in the description of the embodiments of this application, the meaning of "a plurality of" is two or more (including two). Similar expressions related to "many", such as "multiple groups", "multiple times", etc., are understood in the same way, unless otherwise specifically defined.

[0111] Please refer to Figure 1 , in a first aspect, this embodiment provides a multimodal colorectal cancer pre-diagnosis information processing method, including:

[0112] S101. Collect the user's medical treatment information, where the medical treatment information includes personal information, imaging information, first clinical information, and family medical history information. The personal information includes the name of the current user, the permanent residence information, and the user's ID number. The imaging information includes multiple pathological images of the current user. The first clinical information includes the clinical symptoms, the duration of the disease course, and the onset node at the current stage of medical treatment. The family medical history information includes the second clinical information of the family members related to the current user under the current disease condition;

[0113] S102. Input the imaging information into the UNET algorithm model to obtain first-modal features, where the first-modal features include at least one of the lesion area, tumor size, tumor distribution information, and the proportion of cancerous cells in the tumor;

[0114] S103. Extract keywords from the first clinical information to obtain multiple semantic keywords, and input the multiple semantic keywords into the semantic dictionary to construct a semantic feature vector of the current first clinical information, denoted as the second-modal feature;

[0115] S104. Input the family medical history information into the random forest algorithm model to obtain the third modal feature, where the third modal feature includes the specific onset characteristics of the current user under the current disease;

[0116] S105. Input the first modal feature, the second modal feature, and the third modal feature into the feature fusion algorithm model for feature fusion to obtain the first fusion feature;

[0117] S106. Input the first fusion feature into the trained neural network model to obtain the first preliminary diagnosis result, where the first preliminary diagnosis result includes the disease risk level information of the current user, the predicted information of the disease stage characteristics, the canceration probability information of the disease, and the treatment plan information of the disease;

[0118] S107. Convert the first preliminary diagnosis result into the first preliminary diagnosis report and display it.

[0119] In step S101, the medical visit information includes personal information, imaging information, first clinical information, and family medical history information. Among them, personal information can be understood as the personal identity information of the current patient, including name, permanent residence, ID number, and may also include contact phone number, eating habits, etc. Personal information mainly plays a role in identification in this step. In the database, the historical medical records of the current user can be retrieved through personal information, and the historical medical records of relatives associated with the current user can also be retrieved together through personal information, facilitating subsequent data analysis. Imaging information includes multiple pathological images taken by the current user during the medical visit. Specifically, the imaging information can be CT images, MRI images, among which CT images are preferred. The imaging information is multiple image information collected by the user in the diseased area during the current medical visit, including the tumor area, and may also include the local tumor area. Preferably, the imaging information is multiple consecutive CT slice images to facilitate more comprehensive three-dimensional modeling and analysis of the user's diseased area in the future. The first clinical information includes clinical symptoms, disease course duration, and onset node at the current medical visit stage. The clinical symptoms can be recorded by the doctor after being orally described, or can also be recorded by the doctor through corresponding simple pressing and observation on-site. The disease course duration and onset node mainly rely on the user's direct oral description. The doctor can make a written description of the clinical symptoms based on the on-site examination to obtain the first clinical information. The family medical history information includes the second clinical information of the family members related to the current user under the current disease condition. It should be noted that being related to the current user can be understood as having a genetically related family relationship with the current user, and the second clinical information is clinical information of the same category as the current disease condition. That is, if the current first clinical information of the user is information content in the field of colorectal diseases, then the second clinical information is information content in the field of colorectal cancer diseases of the family members related to the current user. The introduction of family medical history information in this embodiment is more in line with the characteristic that colorectal cancer is affected by certain genetic factors, making the subsequent feature fusion more comprehensive and closer to the actual situation of the current user.

[0120] In step S102, the UNET algorithm model is used to extract and fuse features from multiple pathological images in the image information, so as to obtain the first-modal features. The specific steps can be referred to the following description. The UNET algorithm model can perform detailed segmentation on each pathological image, so as to strip the features of the tumor area or suspected tumor area from the pathological image. The obtained first-modal features can reflect the association between the tumor area or suspected tumor area and the organ position and organ area of the current user in the form of vectors, which is convenient for subsequent feature fusion. Specifically, the first-modal features include at least one of the lesion area, tumor size, tumor distribution information, and proportion of cancerous cells in the tumor; the lesion area is the area where the user has repeated symptoms, and the tumor size and tumor distribution information can be measured according to the actual situation. For example, if some users have evolved from intestinal polyps to tumors, the first-modal features will include the tumor size and tumor distribution information. The tumor distribution information is the distribution position of the tumor in the colon and rectum. The proportion of cancerous cells in the tumor can be understood as the prediction in the evolution stage from benign tumors to malignant tumors, which is also measured according to the actual situation. If the user is currently in the stage of ordinary enteritis, there may be a situation where the proportion of cancerous cells in the tumor is 0. This embodiment does not limit this.

[0121] In step S103, keywords are extracted from the first clinical information to obtain semantic keywords, which are keywords containing semantic information related to the current disease. Moreover, in this embodiment, a semantic dictionary is also introduced as the basis for constructing the semantic feature vector, where the semantic dictionary can be generated in advance through big data information. The specific steps can be: obtaining the sample clinical information in the disease field to which colorectal cancer belongs, and classifying the similarity of multiple sample keywords in the sample clinical information to obtain multiple clinical entries. Each clinical entry includes multiple synonyms and hyponyms. Different clinical entries have corresponding association degrees with the current disease field. The association degree can be obtained by converting the occurrence probability of the clinical entry in the current disease field. For example, the user's clinical symptom includes "toothache", but in the semantic dictionary of colorectal cancer, the association degree of the clinical entry corresponding to "toothache" is low. If 1-10 is used as the measurement unit of the association degree, the association degree of "toothache" is 0.5, because this symptom will only occur when the cancer cells of colorectal cancer metastasize to the oral cavity in the later stage, and when this symptom appears, there must be other clinical entries with higher association degrees, such as "intestinal bleeding", etc. By using the sample clinical information of big data to construct the semantic dictionary, the finally obtained semantic dictionary can cover all clinical entries in the current disease field, and each clinical entry has a corresponding association degree, which is convenient for subsequent analysis.

[0122] The construction of the semantic feature vector of the current first clinical information can be understood as follows: sequentially match semantic keywords in the semantic dictionary, and when a semantic keyword is successfully matched, replace it with a clinical term, and so on, to form a semantic feature vector corresponding to the first clinical information of the current user, denoted as the second modal feature.

[0123] In step S104, a random forest algorithm model is used to perform probability analysis and prediction on the canceration situation of the current user based on the family history information. Specifically, the random forest algorithm model improves the accuracy and stability of prediction by constructing multiple decision trees and combining their results. Further steps can be understood by referring to the subsequent descriptions. The random forest can effectively handle complex feature relationships, provide interpretable results, and help identify important genetic factors. Through the random forest algorithm model, the influence of genetic factors among different kinship relationships can be clarified. For example, when the family members within three generations of the user's direct line are all diagnosed with the onset feature A associated with colorectal cancer, the probability of the current user having the onset feature A is relatively high, and then a probability range of the user suffering from colorectal cancer is formed. When some of the family members outside three generations of the user's direct line are diagnosed with the onset feature B associated with colorectal cancer, the probability of the current user having the onset feature B is relatively low, and the colorectal cancer probability of the current user fluctuates to a certain extent. The third modal feature obtained through the random forest algorithm model includes the specific onset features of the current user under the current disease condition. It can be understood that the third modal feature is a vector, that is, the onset feature includes onset symptoms and onset probability.

[0124] In step S105, after obtaining the first modal feature, the second modal feature, and the third modal feature, a feature fusion algorithm model is used for feature fusion to obtain the first fusion feature. Specifically, the feature fusion algorithm model can be obtained based on the non-linear activation function in the feature fusion network, can also be obtained by fusing based on the attention mechanism, and can also be obtained based on the domain adaptation algorithm. The feature fusion algorithm shown in this embodiment can also be understood as simple feature fusion logics such as directly splicing and directly dot-multiplying multiple features. This embodiment does not limit this. Preferably, in this embodiment, the model corresponding to the attention mechanism is selected as the feature fusion algorithm model, which can achieve the optimal fusion effect of fusing the first modal feature, the second modal feature, and the third modal feature in the intermediate stage.

[0125] In step S106, the first fusion feature is input into the trained neural network model to obtain the first preliminary diagnosis result. It should be noted that the neural network model can use a convolutional neural network model. The training process of the neural network model can be specifically as follows: construct a basic neural network model, obtain multiple sample fusion features in the sample database to train the basic neural network model, compare the output result with the sample preliminary diagnosis result for accuracy. When the accuracy of the output result and the sample preliminary diagnosis result exceeds the preset accuracy threshold, it indicates that the current neural network model training is completed. Optionally, the accuracy threshold can be set to 90% to improve the preliminary diagnosis accuracy of the model.

[0126] In this embodiment, the obtained first preliminary diagnosis result includes the user's disease risk level information, the predicted information of the disease stage characteristics, the canceration probability information of the disease, and the treatment plan information of the disease. Among them, the disease risk level information can be understood as the treatment level division of the disease area where colorectal cancer is located. For example, in the stage of not having cancer, it can be understood as the cancer risk level, and in the stage of having cancer, it can be understood as the cancer spread level, such as early stage, middle stage, and late stage. The treatment level division facilitates doctors to select a more appropriate treatment plan according to the level where the user is located. For example, when not having cancer but the cancer risk level is medium, doctors tend to adjust the user's living habits such as eating habits and work and rest rules to achieve the prevention effect on the user. Another example is that in the early stage of cancer, doctors tend to use chemotherapy, resection and other methods to eliminate cancer cells. The predicted information of the disease stage characteristics includes the evolution of the disease in the next stage and the possible specific onset characteristics generated based on the existing medical information. The predicted information of the disease stage characteristics can provide reference for doctors and users, facilitating doctors to make corresponding solution plans in advance.

[0127] In this embodiment, the canceration probability information of the disease is more applicable to users who do not have cancer but have colorectal diseases. When a certain area in the colorectum forms a lesion due to repeated attacks, the cells in this area have a certain probability of canceration. Through the canceration probability information of the disease, doctors can understand the user's current situation more accurately, and can also provide a wake-up call to the user, mobilizing the user's treatment enthusiasm and cooperation. In this embodiment, the treatment plan can be multiple or one, and can be specifically analyzed according to the actual situation.

[0128] In step S107, the first preliminary diagnosis result is converted into a first preliminary diagnosis report and presented in front of the user and the doctor in the form of a report. When the user is still in the waiting stage, the first preliminary diagnosis report can facilitate the user to understand their own status in advance and improve the user's treatment enthusiasm and cooperation during the medical treatment. The first preliminary diagnosis report also facilitates doctors to efficiently understand the user's disease state and current clinical situation, improving the consultation efficiency.

[0129] In this embodiment, the imaging information is converted into the first modality feature, the first clinical information is converted into the second modality feature, and the family medical history information is converted into the third modality feature. Then, the first modality feature, the second modality feature, and the third modality feature are fused and input into the neural network model of deep learning. Finally, the first preliminary diagnosis report output can give the disease risk level information, the stage feature prediction information of the disease, the canceration probability information of the disease, and the treatment plan information of the disease. The entire technical solution integrates multiple information sources such as the personal information, imaging information, first clinical symptoms, and family medical history information of the patient by applying the multi-modal technology principle, improving the comprehensive analysis of the colorectal disease of the current user, enabling doctors to obtain a more comprehensive disease assessment, and thus achieving a more accurate risk level classification, disease stage prediction, and personalized treatment plan formulation, improving the efficiency and accuracy of the diagnosis and treatment process.

[0130] Correspondingly, please refer to Figure 2 , in the second aspect, this embodiment further provides a multi-modal colorectal cancer preliminary diagnosis information processing system 1, which is applicable to the preliminary diagnosis information processing method described in the first aspect. The preliminary diagnosis information processing system includes an information acquisition module 11, a first feature conversion module 12, a second feature conversion module 13, a third feature conversion module 14, a feature fusion module 15, and a preliminary diagnosis report generation module 16;

[0131] The information collection module 11 is used to collect the medical information of the user. The medical information includes personal information, imaging information, first clinical information, and family medical history information. The personal information includes the name of the current user, the permanent residence information, and the user's ID number. The imaging information includes multiple pathological images of the current user. The first clinical information includes the clinical symptoms, the duration of the disease course, and the onset node at the current stage of treatment. The family medical history information includes the second clinical information of the family members related to the current user under the current disease; the first feature conversion module 12 is used to input the imaging information into the UNET algorithm model to obtain the first modal features, and the first modal features include at least one of the lesion area, tumor size, tumor distribution information, and the proportion of cancerous cells in the tumor; the second feature conversion module 13 is used to extract keywords from the first clinical information to obtain multiple semantic keywords, and input the multiple semantic keywords into the semantic dictionary to construct the semantic feature vector of the current first clinical information, denoted as the second modal feature; the third feature conversion module 14 is used to input the family medical history information into the random forest algorithm model to obtain the third modal features, and the third modal features include the specific onset characteristics of the current user under the current disease; the feature fusion module 15 is used to input the first modal features, the second modal features, and the third modal features into the feature fusion algorithm model for feature fusion to obtain the first fusion feature; the preliminary diagnosis report generation module 16 is used to input the first fusion feature into the trained neural network model to obtain the first preliminary diagnosis result. The first preliminary diagnosis result includes the disease risk level information, the stage feature prediction information of the disease, the canceration probability information of the disease, and the treatment plan information of the disease, and convert the first preliminary diagnosis result into the first preliminary diagnosis report and display it.

[0132] The execution steps in the above embodiments can be understood with reference to the method described in the first aspect above, and will not be elaborated here.

[0133] This embodiment applies the principle of multi-modal technology to integrate various information sources such as the personal information, imaging information, first clinical symptoms, and family medical history information of the patient, improving the comprehensive analysis of the colorectal diseases of the current user, enabling doctors to obtain a more comprehensive disease assessment, and thus achieving a more accurate risk level classification, disease stage prediction, and personalized treatment plan formulation, improving the efficiency and accuracy of the diagnosis and treatment process.

[0134] Please refer to Figure 3 , in some embodiments, inputting the imaging information into the UNET algorithm model to obtain the first modal features includes:

[0135] S201. Label the imaging information using a labeler to obtain multiple pieces of first image information;

[0136] S202. Preprocess each piece of first image information one by one to obtain multiple pieces of second image information. The preprocessing includes at least one of image enhancement, image cutting, and image normalization processing;

[0137] S203. Arrange the multiple pieces of second image information in the acquisition order, and obtain the slice position information between two adjacent pieces of second image information. The slice position information includes the slice thickness and the slice spacing;

[0138] S204. Stack the multiple pieces of second image information arranged in the acquisition order along a preset direction according to the slice position information to generate a first three-dimensional image matrix;

[0139] S205. Input the first three-dimensional image matrix into the trained UNET algorithm model to obtain a second three-dimensional image matrix;

[0140] S206. Map the second three-dimensional image matrix with the multiple pieces of second image information to obtain first modal features.

[0141] In step S201, the image information is labeled using a labeler. Optionally, the labeler can be a Labelbox labeler, an ITK-SNAP labeler, a 3D Slicer labeler, etc. The labeler can achieve autonomous labeling of the image information through sample training. Doctors can also manually label the image information through the labeler. Preferably, after the image information is autonomously labeled by the labeler, the doctor then makes manual adjustments to the labeling results. This can ensure the accuracy of the labeling while saving the doctor's labeling workload. Using the labeler to label the image information can be specifically understood as labeling and bounding the tumor regions, suspected tumor regions, diseased regions, and lesion regions contained in the image information. The labeled pathological image is recorded as the first image information.

[0142] In step S202, preprocessing is performed on the first image information one by one. Image enhancement may include adjusting the original image in terms of brightness, contrast, color, etc. to improve the image quality and enhance the visibility of the lesion area. Specifically, filtering algorithms such as Gaussian filtering and median filtering can be used to remove noise and artifacts in the image, further optimizing the image details; applying a sharpening algorithm to highlight the edge information of the image is beneficial for subsequent image segmentation and feature extraction. Image cutting specifically means, based on anatomical knowledge, automatically or manually locating and cutting the region of interest (ROI) of the image, removing the irrelevant background area, and ensuring that the cut image only contains the colorectal part, which is beneficial for subsequent feature analysis. Image normalization is also to perform scale standardization on the cut two-dimensional image, and uniformly adjust it to the same size. For example, it can be grayscale value normalization: mapping the image pixel values to a standard numerical range, such as [0, 1] or [-1, 1], which can eliminate the image scale and brightness differences generated under different scanning devices and conditions. This step can make the subsequent segmentation process of UNET more accurate and reduce the influence of image noise.

[0143] In step S203, multiple pieces of second image information are arranged in the acquisition order of the pathological images, and there is a certain slice thickness and slice spacing between two adjacent pieces of second image information, thus forming a continuous CT image group.

[0144] Furthermore, in step S204, multiple pieces of second image information arranged in the acquisition order are stacked according to the slice position information along a preset direction. It should be noted that the preset direction can be understood as the scanning direction of the second image information of the user. After stacking, they can be stitched together to form a first three-dimensional image matrix. Specifically, the missing values in the first three-dimensional image matrix can be filled in the form of linear interpolation between two adjacent pieces of second image information to achieve a smooth transition of the three-dimensional volume data between multiple pieces of second image information.

[0145] In step S205, the first three-dimensional image matrix is input into the trained UNET algorithm model to obtain a second three-dimensional image matrix. The UNET algorithm model is a deep learning algorithm widely used in medical image segmentation. It consists of a convolutional neural network and an upsampling layer, and can effectively extract and fuse multi-scale features of images. Further, the UNET model consists of two parts: an encoder (contraction path) and a decoder (expansion path). The encoder uses a series of convolutional and pooling operations to extract multi-scale feature representations, while the decoder fuses these features through upsampling and skip connections to achieve fine pixel-level segmentation. Before using the UNET algorithm model, it needs to be trained. End-to-end supervised learning training can be performed using a large number of sample three-dimensional image matrices. The training objective is to minimize the segmentation loss function of the model. For example, Dice loss or cross-entropy loss can be used to improve the accuracy of model segmentation recognition.

[0146] Inputting the first three-dimensional image matrix into the trained UNET algorithm model can obtain a second three-dimensional image matrix. It should be noted that the first three-dimensional image matrix contains the values corresponding to multiple pixel points, and this value represents the gray information of the pixel points. Similarly, after the first three-dimensional image matrix is converted into the second three-dimensional image matrix, the values contained in the second three-dimensional image matrix are the confidence levels of each pixel point belonging to the region of colorectal diseases. Moreover, since the first three-dimensional image matrix and the second three-dimensional image matrix include the three-dimensional volume data of the entire colorectal region, the pixel points corresponding to the values contained in the second three-dimensional image matrix are pixel points in the three-dimensional sense.

[0147] Further, in step S206, the second three-dimensional image matrix is mapped with multiple pieces of second image information, that is, the three-dimensional volume data is remapped into the second image information to achieve a three-dimensional to two-dimensional numerical mapping relationship. In this way, more refined boundary features of the colorectal disease region and the colorectal region can be obtained. After mapping the second three-dimensional image matrix with multiple pieces of second image information, the feature information containing the colorectal disease region, such as tumor volume, tumor location, and tumor shape, is combined to form a feature vector. Preferably, features of the same attribute and the mapping relationship are combined to form a vector of one dimension, and then the features of multiple attributes and the mapping relationship are combined to form a multi-dimensional vector. That is, the first modality feature is represented by a multi-dimensional vector containing the colorectal disease region.

[0148] In this embodiment, the original image information is accurately segmented by the UNET algorithm, and rich feature information of the colorectal region can be extracted therefrom. The two-dimensional CT / MRI images are stacked into a three-dimensional image matrix, making full use of the three-dimensional spatial structure information. Then, this three-dimensional image matrix is input into the trained UNET model to obtain a three-dimensional output matrix representing the probability of the colorectal region. After that, this three-dimensional output matrix is mapped with the original two-dimensional CT / MRI images to obtain a series of vector representations containing features such as the size, position, and shape of the tumor, that is, the first-modal features. This three-dimensional to two-dimensional feature mapping can make full use of the rich three-dimensional features extracted by UNET while maintaining compatibility with the two-dimensional medical image information commonly used in clinical applications. It can more accurately describe the lesion information of the colorectal part and provide high-quality input data for subsequent multi-modal feature fusion and disease prediction.

[0149] Further, in some embodiments, the first three-dimensional image matrix includes a plurality of first voxel vectors, and the second three-dimensional image matrix includes a plurality of second voxel vectors. Inputting the first three-dimensional image matrix into the trained UNET algorithm model to obtain the second three-dimensional image matrix includes:

[0150] Encoding the first three-dimensional image matrix layer by layer to a first preset threshold to obtain first local feature information, where the first local feature information includes the local features extracted by each first voxel vector during the layer-by-layer encoding process;

[0151] Decoding the first local feature information layer by layer to obtain second local feature information, where the second local feature information includes the local features of each second voxel vector, and the local feature of the second voxel vector is the local feature of each first voxel vector fused layer by layer during the decoding process;

[0152] Convolving the second local feature information to obtain a first primary three-dimensional image matrix;

[0153] Performing threshold processing on the first primary three-dimensional image matrix to obtain a first segmentation mask;

[0154] Judging whether the first segmentation mask meets a second preset threshold;

[0155] If so, outputting the first primary three-dimensional image matrix as the second three-dimensional image matrix;

[0156] If not, correcting the first segmentation mask to obtain a second segmentation mask;

[0157] Calculating the loss function of the UNET algorithm model during the layer-by-layer decoding process according to the second segmentation mask;

[0158] Perform gradient feedback according to the loss function to update the model parameters used by the UNET algorithm model during the layer-by-layer decoding process;

[0159] Perform layer-by-layer decoding on the first local feature information in the updated UNET algorithm model to obtain third local feature information, where the third local feature information includes the local features of each first voxel vector fused during the decoding process;

[0160] Perform convolution on the third local feature information to obtain a second primary three-dimensional image matrix;

[0161] Perform threshold processing on the second primary three-dimensional image matrix to obtain a third segmentation mask;

[0162] Determine whether the third segmentation mask meets the second preset threshold;

[0163] If so, output the second primary three-dimensional image matrix as the second three-dimensional image matrix.

[0164] In this embodiment, the first three-dimensional image matrix includes multiple first voxel vectors. The first voxel vector refers to the feature representation of each voxel in the first three-dimensional image matrix input to the UNET algorithm model. The first voxel vector usually contains the gray value or other feature information of the corresponding voxel in the original image. The second three-dimensional image matrix includes multiple second voxel vectors. It can be understood that after being processed by the UNET algorithm model, the second voxel vector is the feature representation output by the model and usually contains the feature information obtained after the encoding and decoding processes. The second voxel vector reflects the understanding and extracted features of the UNET algorithm model for the input data, including local spatial relationships, etc.

[0165] In this embodiment, perform layer-by-layer encoding on the first three-dimensional image matrix until the first preset threshold. Here, the first preset threshold can be understood as the number of layers of layer-by-layer encoding. The UNET algorithm model will finally reach a bottleneck layer after layer-by-layer encoding, that is, the limit of layer-by-layer encoding. Set this bottleneck layer as the first preset threshold. During the layer-by-layer encoding process, the local features of each layer will be segmented and retained to form the first local feature information. That is, the first local feature information includes the local features extracted from each first voxel vector during the layer-by-layer encoding process.

[0166] Perform layer-by-layer decoding on the first local feature information again. Layer-by-layer decoding is the reverse step of layer-by-layer encoding. If layer-by-layer encoding is understood as downsampling, then layer-by-layer decoding can be understood as upsampling. During the layer-by-layer decoding process, the local features of each layer will be fused, and finally the second local feature information will be formed. That is, the second local feature information includes the local features of the first voxel vectors fused layer by layer during the decoding process. For the convenience of description, the fused first voxel vector is denoted as the second voxel vector.

[0167] Finally, the second local feature information is convolved to form a first primary three-dimensional image matrix. Threshold processing is performed on the first primary three-dimensional image matrix to generate a segmentation mask. Optionally, the segmentation mask is denoted as the first segmentation mask, and the first segmentation mask is a binary image. Further, the first segmentation mask is recognized. If the first segmentation mask meets the second preset threshold, the first primary three-dimensional image matrix is output as the second three-dimensional image matrix. In this process, the second preset threshold can be understood as a criterion for evaluating the segmentation accuracy of the model. When the first segmentation mask meets the second preset threshold, it means that the segmentation accuracy of the current first primary three-dimensional image matrix meets the conditions, that is, the error of the output first primary three-dimensional image matrix is small, and it can be directly used as the second three-dimensional image matrix for subsequent mapping steps.

[0168] If the first segmentation mask does not meet the second preset threshold, the first segmentation mask needs to be corrected to obtain a second segmentation mask, and then the loss function in the UNET algorithm model is calculated according to the second segmentation mask. The loss function affects the model parameters used in the decoding process layer by layer of the UNET algorithm model. This step can further correct the upsampling accuracy of the UNET algorithm model to improve the feature fusion effect in the decoding process of the UNET algorithm model.

[0169] Further, in the updated UNET algorithm model, the process of decoding the first local feature information layer by layer is performed again, and finally the third local feature information is obtained. The third local feature information is convolved to obtain a second primary three-dimensional image matrix. Similarly, threshold processing is performed on the second primary three-dimensional image matrix to obtain a third segmentation mask, and then it is judged whether the third segmentation mask meets the second preset threshold. If so, the second primary three-dimensional image matrix is output as the second three-dimensional image matrix.

[0170] It should be noted that in this process, if the third segmentation mask still does not meet the second preset threshold, the foregoing steps can be repeated to correct the parameters in the UNET algorithm model until the latest segmentation mask meets the second preset threshold.

[0171] This embodiment gives the processing steps of the UNET algorithm model between the first three-dimensional image matrix and the second three-dimensional image matrix, making the local features contained in the second three-dimensional image matrix more accurate and comprehensive to improve the accuracy of the first modal features.

[0172] Please refer to Figure 4 , in some embodiments, keyword extraction is performed on the first clinical information to obtain multiple semantic keywords including:

[0173] S301. Perform data cleaning on the first clinical information to obtain a first information text;

[0174] S302. Segment the first information text into multiple segments to be processed, where each segment to be processed has multiple words;

[0175] S303. Input each current segment to be processed into the BERT variant model one by one to obtain the first embedding vector information of each segment to be processed. The first embedding vector information includes the first word embedding vectors of each word in the same segment to be processed;

[0176] S304. Input the first embedding vector information into the TF-IDF model to obtain the weight values corresponding to the first word embedding vectors of the same segment to be processed;

[0177] S305. Arrange the multiple first word embedding vectors in the first information text in ascending order of the weight values from left to right. Select the words corresponding to a preset number of the first word embedding vectors starting from the left as semantic keywords, and denote the first word embedding vectors corresponding to the semantic keywords as the second word embedding vectors.

[0178] In step S301, clean the first clinical information by removing the useless characters in the first clinical information. Since the first clinical information is presented in the form of manual input or notes by doctors, there are some useless characters in the first clinical information, such as modal particles, redundant punctuation marks, etc. Denote the cleaned first clinical information as the first information text.

[0179] In step S302, segment the first information text, that is, segment the first information text into segments one by one according to the sentence segments to obtain multiple segments to be processed. Each segment to be processed describes at least one clinical symptom. For example, "Diarrhea 5 times, blood in the stool 2 times, accompanied by pain" can be split into three segments to be processed: "Diarrhea 5 times", "Blood in the stool 2 times", and "Accompanied by pain". Each segment to be processed has multiple words.

[0180] In step S303, input each current segment to be processed into the BERT variant model one by one. The BERT variant model is preferably the ClinicalBERT model. The BERT variant model converts the text information contained in the segment to be processed into a high-dimensional vector, and stores the semantic information and the feature of the context connection relationship for each word in the high-dimensional vector. Finally, multiple first embedding vector information can be obtained.

[0181] In step S304, the TF-IDF model is a text feature weighting algorithm model. Specifically, the TF-IDF model calculates the frequency of each word in the first information text and combines it with the frequency of the word in the entire first information text set to obtain the weight value of each word. The weight value obtained by TF-IDF reflects the importance of each word in the current sentence segment to be processed. The higher the weight value, the more important the word is in the current sentence segment to be processed and the greater its semantic contribution to the sentence segment to be processed.

[0182] In step S305, multiple first word embedding vectors in the first information text are arranged in ascending order of weight value from left to right. Then, a preset number of first word embedding vectors are selected from the left, that is, the words corresponding to the first word embedding vectors with the greatest semantic contribution to the first information text are selected as semantic keywords. The preset number can be set according to actual needs, and this embodiment does not limit it. For ease of description, this embodiment also denotes the first word embedding vector corresponding to the semantic keyword as the second word embedding vector.

[0183] Further, please refer to Figure 5 , in some embodiments, after obtaining the semantic keywords and their corresponding second word embedding vectors, multiple semantic keywords are input into the semantic dictionary to construct the semantic feature vector of the current first clinical information, denoted as the second modality feature, including:

[0184] S401. Obtain the semantic dictionary, which is trained based on sample clinical information and includes multiple clinical entries;

[0185] S402. Match the semantic keywords with the clinical entries one by one;

[0186] S403. Fill the second word embedding vector associated with the successfully matched semantic keyword into the semantic vector corresponding to the clinical entry and denote it as the first semantic vector;

[0187] S404. Set the semantic vectors corresponding to the clinical entries that are not successfully matched to 0;

[0188] S405. Perform feature concatenation on multiple first semantic vectors to obtain the semantic feature vector represented by a matrix, denoted as the second modality feature;

[0189] In step S401, the semantic dictionary can capture professional terms, concepts in the clinical field, and their semantic relationships. Optionally, the semantic dictionary is trained based on sample clinical information. For ease of description, the professional terms in the semantic dictionary are denoted as clinical entries.

[0190] In step S402, the semantic keywords are matched with the clinical terms. It can be understood that this matching process includes the matching of semantic similarity. Preferably, the cosine similarity algorithm described later is used to calculate the semantic similarity between multiple semantic keywords and multiple clinical terms.

[0191] In step S403, the second word embedding vector corresponding to the successfully matched semantic keyword is filled into the semantic vector corresponding to the clinical term, so that the semantic keyword and the matched clinical term are fused into a first semantic vector, which reflects the semantic characteristics of the clinical term after the fusion of semantic keywords.

[0192] In step S404, if there is an unmatched clinical term, the semantic vector corresponding to the unmatched clinical term is set to 0 to indicate that the current patient does not have this symptom.

[0193] In step S405, the multiple first semantic vectors are subjected to feature splicing, and finally a semantic feature vector represented by a matrix is obtained, denoted as the second modality feature.

[0194] Furthermore, the process of matching the semantic keywords with the clinical terms one by one includes:

[0195] Obtain the second word embedding vector of each semantic keyword;

[0196] Obtain the semantic vector of each clinical term;

[0197] Calculate the cosine similarity between the second word embedding vector and the current semantic vector one by one;

[0198] Judge whether the cosine similarity exceeds the semantic similarity threshold;

[0199] If so, record the cosine similarity as the first cosine similarity;

[0200] Judge whether the number of the first cosine similarities is greater than 1;

[0201] If the number of the first cosine similarities is greater than 1, arrange the multiple first cosine similarities in descending order of similarity values, and select the semantic keyword corresponding to the second word embedding vector with the highest first cosine similarity to match the current clinical term;

[0202] If the number of the first cosine similarities is 1, match the semantic keyword associated with the first cosine similarity with the current clinical term;

[0203] If not, it means that the current clinical term fails to match.

[0204] In this embodiment, a clinical term has a semantic vector in the semantic dictionary, which exists during the training phase of the semantic dictionary and is used to represent the influence degree of the current clinical term in this disease. Obtain the second word embedding vector of each semantic keyword, and calculate the cosine similarity between the second word embedding vector and the semantic vector. Specifically, when there are multiple semantic keywords and multiple clinical terms, it is necessary to calculate the cosine similarity between multiple semantic keywords and each clinical term. Optionally, to reduce subsequent computing power, when a semantic keyword successfully matches a certain clinical term, the successfully matched semantic keyword does not participate in the matching process of the remaining clinical terms, and this method can improve the matching efficiency of the entire semantic keyword and clinical term.

[0205] Furthermore, in this embodiment, a semantic similarity threshold is also set, and the semantic similarity threshold can be adjusted according to actual needs. The semantic similarity threshold indicates the similarity standard that the current semantic keyword can reach. For example, the semantic similarity between hematochezia and bloody stool meets the semantic similarity threshold, while the semantic similarity between abdominal pain and bloody stool does not reach the standard set by the semantic similarity threshold. That is to say, the semantic similarity threshold is the lower limit standard for judging clinical terms and semantic keywords.

[0206] If the cosine similarity calculated between the semantic keyword and the clinical term does not exceed the semantic similarity threshold, it means that the current semantic keyword and the clinical term are not similar or have a low similarity degree, that is, the current semantic keyword and the clinical term do not match successfully.

[0207] If the cosine similarity calculated between the semantic keyword and the clinical term exceeds the semantic similarity threshold, it means that the current semantic keyword has a certain relevance to this clinical term, and then continue to calculate and judge the cosine similarity between the next semantic keyword and the current clinical term until all the remaining unmatched semantic keywords are judged. Record the cosine similarity corresponding to the semantic keyword similar to the current clinical term as the first cosine similarity.

[0208] Further, judge whether the number of the first cosine similarities is greater than 1. If it is greater than 1, it means that there are multiple first cosine similarities, that is, there are multiple semantic keywords that are similar to the current clinical term. Then arrange the multiple first cosine similarities in descending order of similarity values, and select the semantic keyword corresponding to the second word embedding vector with the highest first cosine similarity to match the current clinical term, which means that the current clinical term matches successfully.

[0209] If the number of the first cosine similarities is 1, it means that there is only one semantic keyword similar to the clinical term currently, and then just match this semantic keyword with the current clinical term.

[0210] In this embodiment, a rich semantic feature vector is constructed through semantic keywords in combination with a pre-trained semantic dictionary. This feature vector can better capture the semantic information of clinical texts and provide valuable input for subsequent disease prediction or other medical decisions.

[0211] In some embodiments, the family medical history information is input into the random forest algorithm model, and the third-modal features obtained include:

[0212] Taking the relatives in the family medical history information as nodes and the kinship as the hierarchical basis, the family tree information is constructed. The relatives with the second clinical information are assigned a value of 1, and the relatives without the second clinical information are assigned a value of 0;

[0213] Taking the family tree information as the input feature and the semantic keywords as the variable labels, they are input into the random forest algorithm model to obtain the influence features of each relative under the current disease condition, denoted as the first influence features;

[0214] Calculate the importance of the first influence features one by one and denote it as the first importance, which is represented by formula (1). Formula (1) is as follows:

[0215] ;

[0216] In formula (1), is the first importance of the th first influence feature, is the reduction in impurity when using the feature for splitting in tree , is the reduction in impurity, is the total number of trees in the random forest algorithm model, is the tree in the random forest algorithm model;

[0217] Normalize multiple first importances to obtain multiple second importances, which are represented by formula (2). Formula (2) is as follows:

[0218] ;

[0219] In formula (2), is the second importance of the th first influence feature, and N is the number of nodes of the relatives;

[0220] Represent multiple second importances in a numerical sequence to obtain the third-modal features.

[0221] In this embodiment, the family medical history information can be obtained by constructing a medical visit database and associating the medical visit database with population information. The family medical history information can also be directly provided by the user. Optionally, the user can provide the information of family members within three generations of direct blood relatives, and then the medical visit database can be used to improve and supplement the user's family medical history information.

[0222] Taking the family members in the family medical history information as nodes in the family tree information and the kinship as the hierarchical basis. For example, the level where the father is located is one level higher than the level where the user is located, and the level where the grandfather is located is one level higher than the level where the father is located, and so on. Assign the value of 1 to the family members with the second clinical information and the value of 0 to the family members without the second clinical information, so as to realize the family tracking of colorectal diseases.

[0223] Furthermore, taking the family tree information as the input feature and the semantic keywords as the variable labels, input them into the random forest algorithm model to obtain the influence features of each family member under the current disease, denoted as the first influence features. The random forest algorithm model is an ensemble learning algorithm that predicts and classifies by training multiple decision tree models. The semantic keywords can reflect the important concepts and feature information related to colorectal diseases. Through the random forest algorithm model, the influence degree of each family member in the family tree information on colorectal diseases can be obtained, and these influence degrees are recorded as the first influence features, which reflect the influence probability of each family member in the disease suffered by the current user.

[0224] Furthermore, calculate the importance of the first influence features. For the convenience of distinction, the importance of the first influence features calculated for the first time is denoted as the first importance. Specifically, reference can be made to formula (1), and formula (1) calculates the average importance of each first influence feature in the random forest algorithm model.

[0225] After obtaining the first importance, perform normalization processing on the first importance. The difference between the obtained second importance and the first importance is that the sum of multiple second importances is 1, that is, the second importance is the relative importance of each first influence feature in the random forest algorithm model.

[0226] In this embodiment, represent the calculated multiple second importances with a numerical sequence to form the third modal feature for subsequent modal fusion steps.

[0227] This embodiment quantifies the influence degree of each family member in the colorectal disease suffered by the current user and converts it into a feature representation that can be used in the machine learning model. The formation of the third modal feature helps the subsequent multi-modal fusion and the pre-diagnosis step based on multi-modal fusion.

[0228] In some embodiments, inputting the first modal feature, the second modal feature, and the third modal feature into a feature fusion algorithm model for feature fusion to obtain the first fusion feature includes:

[0229] Performing feature concatenation on the first modal feature, the second modal feature, and the third modal feature to obtain a first concatenated feature, and representing the first concatenated feature by formula (3), where formula (3) is as follows:

[0230] ;

[0231] In formula (3), is the first modal feature, is the second modal feature, is the third modal feature, is the first concatenated feature;

[0232] Inputting the first concatenated feature into an attention mechanism algorithm model to obtain a query vector, a key vector, and a value vector, which is represented by formula (4), where formula (4) is as follows:

[0233]

[0234] In formula (4), is the query vector, is the key vector, is the value vector, is the weight matrix of the query vector, is the weight matrix of the key vector, is the weight matrix of the value vector;

[0235] Calculating attention weights based on the query vector, the key vector, and the value vector to obtain an attention weight matrix, which is represented by formula (5), where formula (5) is as follows:

[0236] ;

[0237] In formula (5), is the transpose matrix of, is the dimension of the key vector, is the attention degree of the first modal feature to itself, is the attention degree of the first modal feature to the second modal feature, is the attention degree of the first modal feature to the third modal feature, is the attention degree of the second modal feature to the first modal feature, is the attention degree of the second modal feature to itself, is the attention degree of the second modal feature to the third modal feature, is the attention degree of the third - mode feature to the first - mode feature, is the attention degree of the third - mode feature to the second - mode feature, is the attention degree of the third - mode feature to itself;

[0238] According to the attention - weight matrix, a weighted sum of the value vectors is performed to obtain the attention - fused feature. The attention - fused feature is represented by formula (6), and formula (6) is as follows:

[0239] ;

[0240] In formula (6), is the attention - fused feature, is the weighted - sum function;

[0241] The attention - fused feature is fused with the first - spliced feature to obtain the first - fused feature. The first - fused feature is represented by formula (7), and formula (7) is as follows:

[0242] ;

[0243] In formula (7), is the first - fused feature.

[0244] In this embodiment, first, the first - mode feature, the second - mode feature, and the third - mode feature are feature - spliced to form the first - spliced feature. Then, the first - spliced feature is introduced into the attention - mechanism algorithm model. Through the attention mechanism, the correlation and interaction relationship between the three - mode features can be learned. The query vector, the key vector, and the value vector contain the interaction information between the three - mode features, which helps to better fuse different features. Further, according to the query vector, the key vector, and the value vector, the attention - weight matrix is calculated, and then the attention - fused feature is obtained. Finally, the attention - fused feature is fused with the first - spliced feature to obtain the first - fused feature with rich feature content, which is convenient for improving the accuracy of the subsequent first - pre - diagnosis result.

[0245] In a third aspect, this embodiment also provides a computer - readable storage medium, on which computer - program instructions are stored. When the computer - program instructions are executed by a processor, the method described in the first aspect is implemented.

[0246] The computer program involved in this embodiment can be stored in a computer-readable storage medium, which includes but is not limited to magnetic disks, magnetic tapes, magnetic cards, floppy disks, flash memories, optical discs, optical cards, read-only memories (ROMs), random access memories (RAMs), erasable programmable ROMs (EPROMs), and electrically erasable programmable ROMs (EEPROMs), etc. It also includes other biological, physical, or chemical structures that can achieve functions similar to or equivalent to those of the above-listed storage media, such as units with information storage capabilities like DNA, RNA, proteins, etc. In a specific embodiment, the storage medium involved can be one of the above medium types or a combination of the above medium types. In different embodiments, the computer program involved in the embodiment can be centrally stored in a single medium or distributedly stored in multiple media. The memory containing the computer-readable storage medium can be a non-volatile memory or a random access memory. These computer-readable storage media can be built into the device or can be an external device or a part of an external device connected to the device involved in the embodiment. In some embodiments, the memory with the computer-readable storage medium is deployed locally; in other embodiments, a scheme of deploying the memory away from the processor can also be adopted, such as a network-attached memory accessed via an RF circuit or an external port and a communication network, where the communication network can be the Internet, one or more internal networks, local area networks (LANs), wide area wireless networks (WLANs), storage area networks (SANs), etc., or a suitable combination thereof, as long as the computer device can access the memory. In addition, the computer program involved in the embodiment can be stored in plaintext / ciphertext form or can be designed as training data and be integrally reorganized and implicitly saved in the parameter states of a deep neural network or other machine learning models through model training.

[0247] In a fourth aspect, this embodiment further provides an electronic device, including a memory and a processor, where the memory is used to store one or more computer program instructions, and wherein the one or more computer program instructions are executed by the processor to implement the method described in the first aspect.

[0248] The processor described in this embodiment can be implemented by hardware, firmware, software, or a combination thereof. It can use circuits, one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), central processing units (CPUs), controllers, microcontrollers, microprocessors, or at least one of the like, and also includes other physical, biological, or chemical structures that can implement functions similar to or equivalent to those of the above-listed processors, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute some steps, all steps, or any combination of the steps mentioned in the computer programs or methods involved in the various embodiments of the present application.

[0249] In the above technical solution, by converting the image information into the first modal feature, the first clinical information into the second modal feature, and the family medical history information into the third modal feature, and then performing feature fusion on the first modal feature, the second modal feature, and the third modal feature and inputting them into the neural network model of deep learning, the first preliminary diagnosis report finally output can give the disease risk level information, the stage feature prediction information of the disease, the canceration probability information of the disease, and the treatment plan information of the disease. The entire technical solution integrates various information sources such as the patient's personal information, image information, first clinical symptoms, and family medical history information by applying the multi-modal technology principle, improves the comprehensive analysis of the current user's colorectal disease, enables doctors to obtain a more comprehensive disease assessment, and further realizes a more accurate risk level classification, disease stage prediction, and personalized treatment plan formulation, improving the efficiency and accuracy of the diagnosis and treatment process.

[0250] Finally, it should be noted that although the above embodiments have been described in the text and drawings of the specification of the present application, this does not limit the patent protection scope of the present application. Any technical solution obtained by equivalent structure or equivalent process substitution or modification using the content recorded in the text and drawings of the specification of the present application based on the essential concept of the present application, as well as any technical solution directly or indirectly implementing the above embodiments in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A multimodal colorectal cancer prognostic information processing method, characterized in that: include: Collecting the user's medical information, the medical information includes personal information, imaging information, first clinical information and family medical history information, the personal information includes the current user's name, permanent residence information, and user ID number, the imaging information includes multiple pathological images of the current user, the first clinical information includes clinical symptoms, duration of the disease course and onset node of the current medical stage, and the family medical history information includes second clinical information of family members who are related to the current user under the current disease; Inputting the image information into a UNET algorithm model to obtain a first modal feature, wherein the first modal feature includes at least one of a lesion area, a tumor size, a tumor distribution information, and a percentage of cancerous cells in the tumor; Extracting keywords from the first clinical information to obtain a plurality of semantic keywords, inputting the plurality of semantic keywords into a semantic dictionary, and constructing a semantic feature vector of the current first clinical information, recorded as a second modal feature; Inputting the family medical history information into a random forest algorithm model to obtain a third modality feature, wherein the third modality feature includes specific disease characteristics of the current user under the current disease; Inputting the first modal feature, the second modal feature and the third modal feature into a feature fusion algorithm model for feature fusion to obtain a first fusion feature; Inputting the first fusion feature into the trained neural network model to obtain a first pre-diagnosis result, the first pre-diagnosis result including the disease risk level information of the current user, the stage feature prediction information of the disease, the canceration probability information of the disease, and the treatment plan information of the disease; Converting the first pre-diagnosis result into a first pre-diagnosis report and displaying the report; The family medical history information is input into the random forest algorithm model, and the third modal features obtained include: Taking the family members in the family medical history information as nodes and the kinship as the hierarchical basis to construct family tree information, assigning a value of 1 to the family members with the second clinical information and assigning a value of 0 to the family members without the second clinical information; The family tree information is used as an input feature, and the semantic keywords are used as variable labels to input into the random forest algorithm model, so as to obtain the influencing feature of each family member under the current disease, which is recorded as the first influencing feature; The importance of the first influencing features is calculated one by one and recorded as the first importance, which is expressed by formula (1). The formula (1) is as follows: In formula (1), Im(f i ) is the first importance of the i-th first influencing feature, ΔGini(f i , t) is the use of f in tree t i The reduction in Gini impurity when the feature is split, T is the total number of trees in the random forest algorithm model, and trees is the number of trees in the random forest algorithm model; The plurality of first importances are normalized to obtain a plurality of second importances, which are expressed by formula (2). The formula (2) is as follows: In formula (2), Im(f i )′ is the second importance of the i-th first influencing feature, and N is the number of nodes of the family members; The third modal feature is obtained by representing a plurality of the second importances with a numerical sequence.

2. The multimodal colorectal cancer prognostic information processing method according to claim 1, characterized in that: The image information is input into the UNET algorithm model to obtain the first modal features including: Annotating the image information using an annotator to obtain a plurality of first image information; Preprocessing the first image information one by one to obtain a plurality of second image information, wherein the preprocessing includes at least one of image enhancement, image cutting, and image normalization processing; Arranging the plurality of second image information in a collection order, and acquiring slice position information between two adjacent second image information, wherein the slice position information includes slice thickness and slice spacing; Stacking the plurality of sequentially acquired second image information along a preset direction according to the slice position information to generate a first three-dimensional image matrix; Inputting the first three-dimensional image matrix into the trained UNET algorithm model to obtain a second three-dimensional image matrix; The second three-dimensional image matrix is ​​mapped to the plurality of second image information to obtain the first modal feature.

3. The multimodal colorectal cancer prognostic information processing method according to claim 2, characterized in that: The first three-dimensional image matrix includes a plurality of first voxel vectors, and the second three-dimensional image matrix includes a plurality of second voxel vectors. The first three-dimensional image matrix is ​​input into the trained UNET algorithm model to obtain the second three-dimensional image matrix including: Encoding the first three-dimensional image matrix layer by layer to a first preset threshold value to obtain first local feature information, where the first local feature information includes a local feature extracted from each of the first voxel vectors in the layer-by-layer encoding process; Decoding the first local feature information layer by layer to obtain second local feature information, where the second local feature information includes a local feature of each second voxel vector, where the local feature of the second voxel vector is a local feature of each first voxel vector fused layer by layer during the decoding process; Convolving the second local feature information to obtain a first primary three-dimensional image matrix; performing threshold processing on the first primary three-dimensional image matrix to obtain a first segmentation mask; Determining whether the first segmentation mask meets a second preset threshold; If yes, outputting the first primary three-dimensional image matrix as a second three-dimensional image matrix; If not, modifying the first segmentation mask to obtain a second segmentation mask; Calculating a loss function of the UNET algorithm model in a layer-by-layer decoding process according to the second segmentation mask; Performing gradient feedback according to the loss function to update the model parameters used by the UNET algorithm model in the layer-by-layer decoding process; Decoding the first local feature information layer by layer in the updated UNET algorithm model to obtain third local feature information, wherein the third local feature information includes the local features of each of the first voxel vectors fused during the decoding process; Convolving the third local feature information to obtain a second primary three-dimensional image matrix; performing threshold processing on the second primary three-dimensional image matrix to obtain a third segmentation mask; Determining whether the third segmentation mask meets a second preset threshold; If so, the second primary three-dimensional image matrix is ​​output as a second three-dimensional image matrix.

4. The multimodal colorectal cancer prognostic information processing method according to claim 1, characterized in that: The first clinical information is subjected to keyword extraction to obtain multiple semantic keywords including: Performing data cleaning on the first clinical information to obtain a first information text; Segmenting the first information text into sentences to obtain a plurality of to-be-processed sentence segments, each of which has a plurality of words; Input the current to-be-processed segments into the BERT variant model one by one to obtain first embedding vector information of each to-be-processed segment, wherein the first embedding vector information includes a first word embedding vector of each vocabulary in the same to-be-processed segment; Inputting the first embedding vector information into the TF-IDF model to obtain a weight value corresponding to the first word embedding vector of the same sentence to be processed; Arrange the multiple first word embedding vectors in the first information text from left to right according to the size of the weight value, select a preset number of words corresponding to the first word embedding vectors from the left as the semantic keywords, and record the first word embedding vectors corresponding to the semantic keywords as second word embedding vectors.

5. The multimodal colorectal cancer prognostic information processing method according to claim 4, characterized in that: Inputting the plurality of semantic keywords into the semantic dictionary to construct the semantic feature vector of the current first clinical information, which is recorded as the second modal feature, includes: Acquire a semantic dictionary, wherein the semantic dictionary is obtained by training based on sample clinical information, and the semantic dictionary includes a plurality of clinical terms; Matching the semantic keywords with the clinical terms one by one; Filling the second word embedding vector associated with the successfully matched semantic keyword into the semantic vector corresponding to the clinical term and recording it as the first semantic vector; The semantic vector corresponding to the clinical term that has not been successfully matched is set to 0; Performing feature concatenation on the plurality of the first semantic vectors to obtain a semantic feature vector represented by a matrix, which is recorded as a second modal feature; Matching the semantic keywords with the clinical terms one by one includes: Obtaining a second word embedding vector for each of the semantic keywords; Obtaining a semantic vector of each of the clinical terms; Calculate the cosine similarity between the second word embedding vector and the current semantic vector one by one; Determining whether the cosine similarity exceeds a semantic similarity threshold; If yes, the cosine similarity is recorded as the first cosine similarity; Determine whether the number of the first cosine similarities is greater than 1; If the number of the first cosine similarities is greater than 1, arranging the plurality of the first cosine similarities in descending order according to the similarity values, and selecting the semantic keyword corresponding to the second word embedding vector with the highest first cosine similarity to match with the current clinical term; If the number of the first cosine similarity is 1, matching the semantic keyword associated with the first cosine similarity with the current clinical term; If not, it means that the current clinical term matching is unsuccessful.

6. The multimodal colorectal cancer prognostic information processing method according to claim 1, characterized in that: The first modal feature, the second modal feature and the third modal feature are input into the feature fusion algorithm model for feature fusion, and the first fusion feature obtained includes: The first modal feature, the second modal feature and the third modal feature are concatenated to obtain a first concatenated feature, and the first concatenated feature is expressed by formula (3), which is as follows: F = [F1, F2, F3]; In formula (3), F1 is the first modal feature, F2 is the second modal feature, F3 is the third modal feature, and F is the first splicing feature; The first concatenated feature is input into the attention mechanism algorithm model to obtain a query vector, a key vector and a value vector, which are expressed by formula (4). Formula (4) is as follows: In formula (4), Q is the query vector, K is the key vector, V is the value vector, and W is the q is the weight matrix of the query vector, W k is the weight matrix of the key vector, W v is the weight matrix of the value vector; The attention weight is calculated according to the query vector, the key vector and the value vector to obtain an attention weight matrix, which is expressed by formula (5). The formula (5) is as follows: In formula (5), K T is the transposed matrix of K, d k is the dimension of the key vector, α 11 is the attention of the first modal feature to itself, α 12 is the attention degree of the first modal feature to the second modal feature, α 13 is the attention degree of the first modal feature to the third modal feature, α 21 is the attention degree of the second modal feature to the first modal feature, α 22 is the attention of the second modal feature to itself, α 23 is the attention degree of the second modal feature to the third modal feature, α 31 is the attention degree of the third modal feature to the first modal feature, α 32 is the attention degree of the third modal feature to the second modal feature, α 33 is the third modal feature’s attention to itself; The value vector is weighted and summed according to the attention weight matrix to obtain an attention fusion feature, and the attention fusion feature is expressed by formula (6), which is as follows: In formula (6), F at is the attention fusion feature, At(Q, K, V) is the weighted sum function; The attention fusion feature is fused with the first concatenation feature to obtain a first fusion feature, and the first fusion feature is expressed by formula (7), which is as follows: F fusion =[F,F at ]; In formula (7), F fusion is the first fusion feature.

7. A multimodal colorectal cancer prognosis information processing system, characterized in that: The method for processing prognostic information according to any one of claims 1 to 6 is applicable, wherein the prognostic information processing system comprises an information acquisition module, a first feature conversion module, a second feature conversion module, a third feature conversion module, a feature fusion module and a prognostic report generation module; The information collection module is used to collect the user's medical information, which includes personal information, imaging information, first clinical information and family medical history information. The personal information includes the current user's name, permanent residence information, and user ID number. The imaging information includes multiple pathological images of the current user. The first clinical information includes clinical symptoms, duration of the disease course, and onset node of the current medical stage. The family medical history information includes second clinical information of family members who are related to the current user under the current disease. The first feature conversion module is used to input the image information into the UNET algorithm model to obtain a first modal feature, wherein the first modal feature includes at least one of the lesion area, tumor size, tumor distribution information, and the proportion of tumor cancerous cells; The second feature conversion module is used to extract keywords from the first clinical information to obtain multiple semantic keywords, input the multiple semantic keywords into a semantic dictionary, and construct a semantic feature vector of the current first clinical information, which is recorded as a second modal feature; The third feature conversion module is used to input the family medical history information into the random forest algorithm model to obtain a third modal feature, wherein the third modal feature includes a specific onset feature of the current user under the current disease; The feature fusion module is used to input the first modal feature, the second modal feature and the third modal feature into the feature fusion algorithm model to perform feature fusion to obtain a first fusion feature; The pre-diagnosis report generation module is used to input the first fusion feature into the trained neural network model to obtain a first pre-diagnosis result, which includes the current user's disease risk level information, disease stage feature prediction information, disease cancer probability information and disease treatment plan information, and convert the first pre-diagnosis result into a first pre-diagnosis report and display it.

8. A computer-readable storage medium storing computer program instructions, characterized in that: The computer program instructions implement the method according to any one of claims 1 to 6 when executed by a processor.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 6.

Images