Colorectal cancer patient discovery system based on AI multi-mode technology

Abstract

The invention belongs to the technical field of health care and artificial intelligence, and particularly relates to a colorectal cancer patient discovery system based on an AI multi-mode technology. The system comprises a structured medical record data collection module, a patient primary screening module based on colorectal cancer keyword detection, a multi-mode accurate colorectal cancer patient secondary screening module, an intelligent multi-mode patient identification result final screening module and an intelligent colorectal cancer patient condition summarization visualization module. According to the system, multi-modal data is combined, a Pytorch framework and a multi-modal intelligent colorectal cancer screening fusion model constructed based on Transform are adopted, and three-level screening and classification of colorectal cancer patients are achieved through multi-modal recognition means such as text semantic analysis, image segmentation recognition and data weighted fusion. The accuracy and efficiency of early screening, diagnosis and risk assessment of colorectal cancer in different clinical scenes can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of health care and artificial intelligence technology, and specifically relates to a colorectal cancer patient discovery system based on AI multimodal technology. Background Art

[0002] With the continuous deepening of medical research and the rapid development of medical technology, medical data has shown explosive growth and unprecedented complexity, covering a wide range of dimensions such as clinical records, laboratory test reports, and high-resolution imaging data. At present, most hospitals have realized the digitization of medical data, that is, doctors can access and view a certain test report of a patient one by one through the local area network within the hospital, but cannot quickly analyze multiple types of medical data at the same time. Some hospitals have also begun to perform intelligent processing of medical image data, but the premise is that the doctor already has a certain estimate of the patient's condition, and then the doctor manually sends the patient's relevant data into the model for analysis. The degree of automation is low and relatively dependent on the doctor's subjective judgment.

[0003] At present, in the clinical practice of colorectal cancer, it is mainly examined and identified through colonoscopy, pathological examination, biomarker detection and other means. However, since the symptoms of diseases such as ulcerative colitis, colonic nuclear, and appendicitis are highly similar to the symptoms of a certain stage or a certain lesion site of colorectal cancer, it is difficult to directly determine the patient's condition based on the patient's clinical complaints alone. Therefore, under limited medical conditions, some grassroots hospitals will also adopt the method of doctors constantly asking patients about their specific conditions and performing digital examinations to diagnose the patient's condition, which is highly invasive and relatively difficult for patients to accept.

[0004] Currently, there is a lack of multi-level diagnostic discovery and management directly targeting patients with rectal cancer. The diagnosis of colorectal cancer mainly relies on outpatient doctors to inquire about the patient's medical history in detail, including the time of symptom onset, course of disease, and possible risk factors, and then obtain images of the patient's colorectal condition through physical examination, blood test, fecal occult blood test, colonoscopy, CT colonography, X-ray barium enema and other means, and finally diagnose the patient based on the doctor's clinical experience. When doctors need to compare and analyze various medical reports of patients at the same time, the relevant systems of the hospital are often unable to achieve this, or due to insufficient hardware configuration and other reasons, the level of human-computer interaction is low, which makes it difficult to discover and identify rectal cancer patients.

[0005] In terms of diagnostic techniques for colorectal cancer, although the overall available means at the social level are relatively diverse, it is difficult to achieve rapid diagnosis in a short period of time for a single patient in the same hospital and by the same doctor due to factors such as the size of the hospital and the economic affordability of the patient. On the one hand, after obtaining the relevant medical imaging data, it is currently mainly analyzed and processed manually by doctors, which requires high clinical experience and takes a long time. On the other hand, my country's medical resources are limited and unevenly distributed, and some patients have a low degree of compliance with invasive examinations, resulting in some patients being unable to receive a fairly comprehensive and detailed examination, leading to misdiagnosis or missed diagnosis.

[0006] In addition, the entire diagnostic process requires patients and doctors to spend a lot of time queuing and exchanging information, and the overall diagnostic efficiency is unsatisfactory. Faced with an increasing number of patients, the traditional medical diagnosis model is limited by the number of clinicians and their diagnostic experience, and is prone to a shortage of diagnostic support capabilities and insufficient diagnostic accuracy, which in turn leads to missed or misdiagnosed colorectal cancer patients, delaying precious early treatment time.

[0007] Therefore, it is very important to design a colorectal cancer patient detection system based on AI multimodal technology that can alleviate the shortage of resources for colorectal cancer patient screening and diagnosis and reduce the misdiagnosis and missed diagnosis of colorectal cancer patients. Summary of the invention

[0008] The present invention aims to overcome the problems in the prior art, such as difficulty in analyzing medical data and low level of human-computer interaction in the existing process of discovering and diagnosing colorectal cancer patients, and provides a colorectal cancer patient discovery system based on AI multimodal technology that can alleviate the shortage of resources for screening and diagnosing colorectal cancer patients and reduce the misdiagnosis and missed diagnosis of colorectal cancer patients.

[0009] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical solutions: Colorectal cancer patient discovery system based on AI multimodal technology, including: Structured medical record data collection module, used to systematically collect and manage patients' medical information using structured medical records; The patient screening module based on colorectal cancer keyword detection is used to screen potential colorectal cancer patients from the medical record database, and to form a tuple of patient-data-disease status with all relevant medical records and historical medical data of each patient to analyze potential colorectal cancer patients; The multimodal accurate secondary screening module for colorectal cancer patients is used to further extract all the corresponding medical data of potential colorectal cancer patients, and analyze the corresponding data through the AI ​​multimodal system to obtain the predicted disease results; The final screening module of intelligent multimodal patient identification results is used to compare the predicted condition results with the diagnosis results given by the clinician, and make a judgment based on the relative deviation value between the clinician's diagnosis results and the predicted condition results to obtain the final screening results; The intelligent colorectal cancer patient condition summary visualization module is used to display the disease risk grading of colorectal cancer patients, the residential distribution of colorectal cancer patients, and the follow-up records.

[0010] Preferably, in the structured medical record data collection module, the content of the structured medical record includes the patient's basic information, chief complaint, physical examination, auxiliary examination, clinical physician's diagnosis results and treatment methods; wherein, the keyword entry method is adopted in the chief complaint, and after the doctor enters the patient data, the system automatically checks and judges the entered data, and performs data cleaning and data normalization, removes irrelevant characters, and unifies the medical record text format, and stores it locally in the hospital or on the medical data cloud in accordance with the medical record confidentiality requirements.

[0011] Preferably, the patient initial screening module based on colorectal cancer keyword detection specifically includes the following process: S21, complete the initial screening of potential colorectal cancer patients through the bag-of-words model and the text medical record keyword scoring detection method; convert the text into a numerical feature vector by calculating the TF weight and the word numerical feature vector; the TF weight calculation formula and the word vector calculation formula are as follows: ; ; in, is the number of times the keyword appears, is the total number of occurrences of all keywords, is the word TF weight, is the word numerical feature vector, is the word direction vector coefficient; S22, using the bag-of-words model to quickly capture various text keywords in medical records and examination reports; after the bag-of-words model completes keyword capture, the keywords are classified and segmented, and different types of keywords are assigned corresponding colorectal cancer risk scores , and then evaluate the potential risk of colorectal cancer through the text medical record keyword scoring detection formula, and determine the range of patients entering the multimodal precise colorectal cancer patient secondary screening module; the text medical record keyword scoring detection formula is as follows: ; in, is the potential risk of colorectal cancer, Assign a score to each text medical record keyword, M is the sum of the scores of all text medical record keywords in the database, are the parameters for the incidence rate of colorectal cancer by region and patient occupation.

[0012] Preferably, the patient initial screening module based on colorectal cancer keyword detection further includes the following process: S23, set the colorectal cancer incidence rate parameters by region and patient occupation The formula is as follows: ; in, is the number of confirmed colorectal cancer medical records in each hospital in the region, The number of suspected medical records reported in the area. is the number of medical records of colorectal cancer diagnosed in the institution where the patient is treated, The number of suspected cases reported by the hospital. The area code of the hospital where the patient is treated. is the regional weight coefficient, is the proportion coefficient of the visiting unit. In principle, the proportion coefficient of the visiting unit is subject to the regional proportion coefficient, that is, ; is the occupational disease risk factor, The risk time standard is the 24-month hazard standard. The time factor for working in the profession; S24, working time coefficient The conversion is as follows: Let T be the total time spent on the job, in months: ; When the potential risk of colorectal cancer Greater than the regional colorectal cancer risk line When the patient is identified, the corresponding patient will be included in the next phase of precision colorectal cancer patient discovery operations; The calculation method is as follows: ; in, is the number of confirmed colorectal cancer medical records in each hospital in the region, is the number of suspected medical records reported in the area, P is the permanent population in the area, is the elasticity diagnostic conversion coefficient; S25, finally the potential patient discrimination formula based on colorectal cancer keyword detection is obtained: ; when When , then the corresponding medical record owner is determined to be a potential colorectal cancer patient, and the data is entered into the second-level screening.

[0013] Preferably, the multimodal precise colorectal cancer patient secondary screening module obtains a list of potential colorectal cancer patients through a patient primary screening module based on colorectal cancer keyword detection, and then extracts all medical images and biomarker detection reports related to the corresponding patients from the database, and extracts and separates the image information in the biomarker detection report, and transfers it to the U-net3+ model based on the pytorch framework for precise colorectal cancer image recognition, and the text part is handed over to the Transformer module for semantic analysis and processing.

[0014] Preferably, in the multimodal accurate colorectal cancer patient secondary screening module, the accurate colorectal cancer image recognition specifically includes the following steps: S31, performing image preprocessing on the colorectal examination image data through the torchvision visual processing package; the image preprocessing includes unifying the size and color parameters of the colorectal examination image, removing unclear, blurred, dark or bright images; and finally adjusting the image size to 256×256 for input into the deep learning model; After image preprocessing is completed, the colorectal image is enhanced to simulate colorectal images at different shooting angles and shooting distances; the image enhancement includes random rotation, shearing and scaling; After completing image data preprocessing and image enhancement, the U-net3+ model is used to perform fine segmentation of colorectal images. The encoder in the U-net3+ model is responsible for downsampling and feature extraction, the decoder is responsible for upsampling and segmentation result generation, and the full-scale skip connection realizes feature fusion between different levels. The semantic analysis process specifically includes the following steps: S32, Transformer module is responsible for analyzing the medical record text and test report text extracted from the database; Remove irrelevant characters, including special symbols and punctuation marks, through text cleaning; standardize and normalize the cleaned text to remove common words that do not contribute to classification; Through the self-attention mechanism of the Transformer module, we capture keywords related to colorectal cancer in all text reports, and widely link the keywords to obtain accurate global information, ultimately achieving detection and information extraction of various medical texts including medical records and test reports.

[0015] As a preferred method, for the accurate image recognition process of colorectal cancer, a classification strategy combining center loss + softmax loss function is used. The specific formula is as follows: Image classification function based on center loss+softmax loss : ; in, represents the softmax loss function, that is ; Represents the center loss function, that is ; It is a proportional parameter, which is used to adjust the proportion of the center loss function to the total function. It realizes the function of intra-class aggregation and inter-class separation; c is the number of samples accepted for training, is the current sample training feature, is the predicted value of the current feature in the loop, is the predicted value of the target sample; is the feature extracted by the convolutional neural network for the i-th sample, is the average characteristic of the i-th class of samples, and N represents the number of samples; In the process of semantic analysis, the cross-entropy loss function is used to predict the multi-classification probability of the text content. The specific formula is as follows: Text classification function based on cross-entropy loss : ; in, =- , is the prediction test result of each sample T; is the value of the true sample t in the i-th category, is the predicted probability of the true sample t of the i-th category; Finally add the weight coefficient and Weight the image prediction probability and text prediction probability separately to get the final synthetic distribution probability , the specific formula is as follows: .

[0016] Preferably, the intelligent multimodal patient identification result final screening module specifically includes the following process: S41, analyze the patient’s clinical diagnosis text through the Transformer network to obtain the doctor’s diagnosis result , is a logical variable, The clinician believes that the patient has colorectal cancer. The clinician believes that the patient does not have colorectal cancer; When the probability of colorectal cancer risk When the diagnosis made by the clinician and the system is consistent, the corresponding patient will be included in the colorectal cancer patient database for management. or When , it is considered that the diagnoses made by the clinician and the system are inconsistent, and the various medical records of the corresponding patient are packaged and submitted to higher-level medical experts for diagnosis, and the decision on whether to include the patient in the colorectal cancer patient database for management is made according to the diagnosis of the medical experts; if the judgment result shows that the system recognition result is incorrect, the corresponding data is back-propagated and fed back to the multimodal precise colorectal cancer patient secondary screening module, and the recognition ability of the multimodal precise colorectal cancer patient secondary screening module is optimized; when , it is considered that the potential patient does not have colorectal cancer and will not be included in the colorectal cancer patient database. The corresponding data will be fed back to the patient initial screening module based on colorectal cancer keyword detection, and the module will be optimized.

[0017] Preferably, the intelligent colorectal cancer patient condition summary visualization module specifically includes the following processes: S51, through the combination of GIS system and basic information of patients, the patient's permanent residence map is displayed, and the potential colorectal cancer environmental impact of the location of the medical institution is analyzed through the patient's permanent residence information, and the distribution density of colorectal cancer patients in the area is calculated. , analyze the survival time and treatment courses of patients after diagnosis, and calculate the estimated survival rate of patients in the region by combining the regional colorectal cancer full-cycle survival rate and late-stage diagnosis survival rate ; Regional distribution density of colorectal cancer patients The calculation formula is as follows: ; in, is the area of ​​the selected region, Total area served for patient discovery and management systems, is the number of colorectal cancer patients in the selected area, The total number of colorectal cancer patients in the patient discovery and management system service area; Estimated survival time of patients in the region The calculation formula is as follows: ; in, For selected patients, To calculate the survival time of current patients with early colorectal cancer, To count the number of patients with early colorectal cancer in the system, To calculate the survival time of patients with advanced colorectal cancer. Count the number of patients with advanced colorectal cancer in the system; Coordinate parameters for estimated survival time, parameters Adapt according to the number of treatment courses and diagnosis of the patient obtained by the system, and obey ; As the patient's condition worsens or time passes, parameters Gradually obey .

[0018] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention realizes automatic screening of colorectal cancer for patients in a designated hospital or region by constructing a three-level screening mechanism, including primary screening of patients based on colorectal cancer keyword detection, multimodal accurate secondary screening of colorectal cancer patients, and final screening of intelligent multimodal patient recognition results. This can effectively meet the medical needs of a large number of patients and reduce the possibility of misdiagnosis and missed diagnosis through standardized means; (2) The system of the present invention automatically performs multimodal fusion of various medical test data of potential patients to improve the human-computer interaction experience. Without the need to significantly improve the existing outpatient doctor information hardware configuration, effective interaction can be achieved on a single screen, reducing the data integration workload of clinical doctors. There is no need to check and switch multiple pages back and forth to have a clear understanding of the patient's condition; (3) The system automatically screens the hospital's clinical diagnosis data on the day at three levels to detect colorectal cancer in patients on the day. The system automatically scans and confirms the diagnosis decision of the day, and automatically reports suspected misdiagnoses, thereby improving the hospital's treatment quality control capabilities. At the same time, through system probability analysis and clinical diagnosis text recognition, two-way confirmation is achieved to achieve joint improvement of doctors and intelligent multimodal screening systems. It can not only optimize the screening algorithm through the feedback of medical experts in medical record processing, but also improve the data technology of clinicians by discovering clinical diagnosis problems, and assist clinicians in making decisions. (4) The system automatically integrates and manages various types of medical data of patients, and can achieve one-time examination and global sharing within the hospital or region, which is convenient for patients to obtain data during subsequent treatment, minimize invasive examinations on patients, and reduce patients' psychological and economic burdens. At the same time, the system predicts and calculates the distribution density and expected survival time of patients, which is convenient for medical units to carry out follow-up and scientific research analysis, and further ensures the discovery and early treatment of colorectal cancer patients in the region. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A process relationship diagram of three-level screening for the colorectal cancer patient discovery system based on AI multimodal technology of the present invention; Figure 2 It is an operation flow chart of the structured medical record data collection module in the present invention; Figure 3 This is an operation flow chart of a patient initial screening module based on colorectal cancer keyword detection in the present invention; Figure 4 This is an operation flow chart of the multimodal accurate secondary screening module for colorectal cancer patients in the present invention; Figure 5 An operation flow chart of the final screening module of the intelligent multimodal patient identification result in the present invention; Figure 6 A flowchart of the actual application of the colorectal cancer patient discovery system based on AI multimodal technology provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to more clearly illustrate the embodiments of the present invention, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other accompanying drawings and other implementation methods can be obtained based on these accompanying drawings without creative work.

[0021] The present invention is to solve the problems of difficulty in analyzing medical data and low level of human-computer interaction in the existing diagnosis of colorectal cancer patients, alleviate the shortage of diagnosis resources for colorectal cancer patients, and reduce misdiagnosis and missed diagnosis of colorectal cancer patients.

[0022] The system achieves automatic discovery and management of colorectal cancer patients by establishing a three-level screening system. In the first-level screening, irrelevant medical records are removed from the data through relatively simple text recognition calculations, which improves computing efficiency and reduces the computing pressure of multimodal recognition analysis in the second-level screening. The second-level screening completes the discovery of potential patients through multimodal screening based on deep learning. The third-level screening serves as a check and review to ensure that the model's erroneous prediction results will not be used directly. The specific functions of each level of screening in the system are as follows: like Figure 1As shown, in the first level of screening, the system automatically scans all outpatient and inpatient medical data of the target hospital on that day, extracts its structured medical records, and analyzes potential colorectal cancer patients. In the second level of screening, based on the potential patients obtained in the first level of screening, all medical data of such patients (including clinical medical records and various test reports) are further extracted, and the data is analyzed through the AI ​​multimodal system to obtain a predicted condition. In the third level of screening, the results are compared with the diagnosis given by the clinician. The system makes a judgment based on the relative deviation value between the clinician's diagnosis and the system's predicted data results, and obtains the final screening results (if the clinician and, as appropriate, submit it to a more senior medical expert for review, and then determine the patient's final condition). After the screening is completed, the colorectal cancer patient condition summary visualization system is used to realize the risk classification display of colorectal cancer patients, the residential distribution display of colorectal cancer patients, and the follow-up record display, which greatly improves the level of patient data visualization and helps hospitals improve their patient management capabilities.

[0023] Based on the above functional requirements, the present invention provides a colorectal cancer patient discovery system based on AI multimodal technology, including: Structured medical record data collection module (preparation before the third-level screening, to ensure that the subsequent third-level screening can proceed smoothly), used to systematically collect and manage patients' medical information using structured medical records; The patient screening module (first-level screening) based on colorectal cancer keyword detection is used to screen potential colorectal cancer patients from the medical record database, and to form a tuple of patient-data-disease status with all relevant medical records and historical medical data of each patient to analyze potential colorectal cancer patients; The multimodal accurate secondary screening module for colorectal cancer patients (second-level screening) is used to further extract all the corresponding medical data of potential colorectal cancer patients, and analyze the corresponding data through the AI ​​multimodal system to obtain the predicted disease results; The final screening module of intelligent multimodal patient identification results (third-level screening) is used to compare the predicted condition results with the diagnosis results given by the clinician, and make a judgment based on the relative deviation value between the clinician's diagnosis results and the predicted condition results to obtain the final screening results; The intelligent colorectal cancer patient condition summary visualization module (the summary stage after the third-level screening, further improving the effectiveness of the third-level screening for users) is used to display the disease risk grading of colorectal cancer patients, the residential distribution of colorectal cancer patients, and the follow-up records.

[0024] The present invention uses a Transformer model architecture based on a Pytorch framework (PyTorch is an open source machine learning library, mainly used for the research and application of deep learning. It provides powerful GPU acceleration functions, dynamic calculation graphs, and easy-to-use APIs, making the construction and training of neural network models more intuitive and efficient. PyTorch is widely used in computer vision, natural language processing, and other fields) (a deep learning model based on a self-attention mechanism, which aims to solve the problems of low parallel computing efficiency and insufficient long-term dependency capture in traditional RNN and CNN when processing sequence data. The model adopts an Encoder-Decoder (encoder-decoder) architecture, including two parts, an encoder and a decoder, which can efficiently process sequence data and has achieved significant performance improvements in multiple fields such as natural language processing) and a patient screening method based on colorectal cancer keyword detection to process the patient's text data, use a U-net3+ model to process and identify the patient's image data, and combine a unique image recognition algorithm for colorectal cancer to segment possible tumor areas, annotate and predict them. After completing the processing and identification of text data and image data respectively, the text and image data are weighted fused and predicted.

[0025] The following is a description of the specific operation process of each module: Structured medical record data collection module: The structured medical record used in this module abandons the randomness and irregularity of free text narration in traditional paper medical records, and instead uses preset templates and standardized fields to systematically collect and manage patients' medical information. Structured medical records not only cover the patient's basic information, chief complaint, current medical history, past medical history, physical examination, auxiliary examination, diagnosis, treatment plan and doctor's advice, but also store them in the electronic medical record system in a structured form, thus achieving data standardization, normalization and interoperability. At the same time, through the use of standardized fields, the consistency and accuracy of the data are ensured, and errors and omissions caused by human factors are reduced.

[0026] like Figure 2 As shown, the structured medical record used in the present invention is to ensure the universality of various diseases, covering six main contents: patient basic information, chief complaint, physical examination, auxiliary examination, clinical physician diagnosis results, and treatment methods. The keyword entry method is adopted in the chief complaint. After the doctor enters the patient data, the system automatically checks and judges the relevant entered data, and performs data cleaning and data normalization, removes irrelevant characters, and unifies the medical record text format, and stores it locally in the hospital or on the medical data cloud in accordance with the medical record confidentiality requirements.

[0027] Patient screening module based on colorectal cancer keyword detection: The main function of this module is to screen out potential colorectal cancer patients from the huge medical record database of the hospital covering various diseases and sub-health conditions, and to form a tuple of patient-data-disease status with all relevant medical records and historical medical data of each patient. Through the selection and statistics of highly correlated keywords in the relevant colorectal cancer clinical complaints and examination reports, as well as further scoring calculations, preliminary potential patient discovery is achieved, data processing is done for subsequent accurate colorectal cancer patient discovery, and unnecessary image calculations are reduced during model operation.

[0028] like Figure 3 As shown in the process, after completing the structured entry of outpatient and inpatient medical records, this module completes the initial screening of potential colorectal cancer patients through the bag-of-words model and the text medical record keyword scoring detection method. The bag-of-words model is a text representation method widely used in natural language processing and information retrieval. This model ignores the grammar and word order of the text and simply regards the text as a collection of words. Each word appears independently without considering the context. By calculating the TF weight and the word numerical feature vector, the text is converted into a numerical feature vector, so that the machine learning algorithm can process text data. The TF weight calculation formula and the word vector calculation formula are as follows: ; ; in, is the number of times the keyword appears, is the total number of occurrences of all keywords. is the word TF weight, is the word numerical feature vector, is the word direction vector coefficient.

[0029] In the present invention, the bag-of-words model is used to quickly capture various text keywords in medical records and examination reports. After the bag-of-words model completes keyword capture, the text medical record keyword scoring detection method proposed in the present invention classifies and scores the keywords, and assigns different types of keywords corresponding colorectal cancer risk scores. , and then evaluate the potential risk of colorectal cancer and determine the range of patients entering the precise colorectal cancer patient discovery module through the text medical record keyword scoring detection formula. The text medical record keyword scoring detection formula is as follows: ; in, is the potential risk of colorectal cancer, Assign a score to each text medical record keyword, M is the sum of the scores of all text medical record keywords in the database, is the colorectal cancer incidence rate parameter by region and patient occupation. The corresponding table of colorectal cancer text medical record keyword scoring variable values ​​is shown in Table 1 below: Table 1 Sample table for keyword scoring of medical records

[0030] In addition, set the colorectal cancer incidence rate parameters by region and patient occupation The formula is as follows: ; in, is the number of confirmed colorectal cancer medical records in each hospital in the region, The number of suspected medical records reported in the area. is the number of medical records of colorectal cancer diagnosed in the institution where the patient is treated, The number of suspected cases reported by the hospital. The area code of the hospital where the patient is treated. is the regional weight coefficient, is the proportion coefficient of the visiting unit. In principle, the proportion coefficient of the visiting unit is subject to the regional proportion coefficient, that is, ; If the hospital you are visiting is a tertiary hospital or a regional central hospital, This coefficient can be adjusted dynamically according to the grade of the hospital visited and the regional status of the hospital. The higher the grade of the hospital, the higher the centrality. The larger the hospital, the lower the centrality. The smaller. is the occupational disease risk factor, The risk time standard is the 24-month hazard standard. is the working time coefficient and occupational disease coefficient The coefficient distribution is shown in Table 2 below: Table 2 Occupational disease coefficient table

[0031] In addition, the time coefficient of employment The conversion is as follows: Let T be the total time spent on the job, in months: ; When the potential risk of colorectal cancer Greater than the regional colorectal cancer risk line When the patient is identified, the corresponding patient will be included in the next phase of precision colorectal cancer patient discovery operations; The calculation method is as follows: ; in, is the number of confirmed colorectal cancer medical records in each hospital in the region, is the number of suspected medical records reported in the area, P is the permanent population in the area, is the elasticity diagnostic conversion coefficient; Finally, the potential patient discrimination formula based on colorectal cancer keyword detection is obtained: ; when When , then the corresponding medical record owner is determined to be a potential colorectal cancer patient, and the data is entered into the second-level screening.

[0032] Multimodal accurate colorectal cancer patient secondary screening module: The main purpose of this module is to further narrow the scope of potential colorectal cancer patients. By combining multimodal accurate identification, clinical physician diagnosis results and medical expert diagnosis results within a smaller range of potential patients, rapid and accurate screening of colorectal cancer patients can be achieved, while efficiently improving the recognition accuracy of the multimodal colorectal cancer screening model. This can reduce the workload of clinicians while maintaining or even improving the accuracy of colorectal cancer diagnosis.

[0033] like Figure 4 As shown in the process, the multimodal precise colorectal cancer patient secondary screening module first obtains a list of potential colorectal cancer patients through the colorectal cancer keyword detection primary screening module, and then extracts all medical images (such as CT, MRI, endoscopic images, etc.) and biomarker detection reports related to the patient from the database, extracts and separates the image information in the report, and transfers it to the U-net3+ model based on the pytorch framework for precise colorectal cancer image recognition. The text part is handed over to the Transformer module for semantic analysis and processing.

[0034] Accurate colorectal cancer patient identification is achieved by using the U-net3+ model based on the pytorch framework to complete the fusion judgment of relevant detection images and text medical records. In terms of image processing, the relevant colorectal examination image data is first preprocessed through the torchvision visual processing package (TorchVision is a powerful visual processing package. It is part of the PyTorch ecosystem and provides developers and researchers with a wide range of tools and functions for processing various computer vision tasks such as image classification, object detection and segmentation, including commonly used data sets, pre-trained models, data conversion and utility functions.) The colorectal detection image size, color and other parameters are unified, and then the unclear, blurred, dark or bright images are removed to ensure the accuracy of the images for training and detection. Finally, the image size is uniformly adjusted to 256×256 for input into the deep learning model.

[0035] After completing data preprocessing, the colorectal images are enhanced by randomly rotating, shearing, scaling, etc. to simulate colorectal images at different shooting angles and distances to prevent the model from overfitting the colorectal direction or structure.

[0036] After completing image data preprocessing and image enhancement, U-net3+ is used as the backbone network to perform fine segmentation of colorectal images. Its encoder is responsible for downsampling and feature extraction, the decoder is responsible for upsampling and segmentation result generation, and the full-scale jump connection realizes feature fusion between different levels. U-net3+ combines low-level details from feature maps of different scales with high-level semantic information through full-scale jump connections and full-scale deep supervision mechanisms, and ultimately realizes the recognition of colorectal cancer lesions.

[0037] The Transformer module is responsible for analyzing the medical records and test report texts extracted from the database. It first removes irrelevant characters, such as special symbols and punctuation marks, through text cleaning, and standardizes and normalizes the text to remove common words that do not contribute to classification, thereby improving machine learning efficiency. Then, through the powerful self-attention mechanism of the Transformer module, it captures the keywords related to colorectal cancer in all text reports, and widely links the keywords to obtain more accurate global information, ultimately achieving detection and information extraction of various medical texts including medical records and test reports.

[0038] After the U-net3+ module and the Transformer module complete the feature extraction of images and texts respectively, the Transformer attention mechanism is used to weight the text and image features so that the model can learn the relative importance of different modalities. At the same time, the multimodal colorectal cancer patient information is classified by combining mainstream multi-classification functions, and a distribution probability of colorectal cancer risk is obtained. , thereby realizing automated comparison and judgment between system identification and manual diagnosis in the third-level screening.

[0039] In image modality recognition, the present invention uses center loss + softmax loss (the above two functions are algorithm functions for image classification after completing convolution and full connection operations. The function is a prior art. The main innovation of the present invention is to combine the value of the function with the text classification function in the following text to generate a distribution probability for comparison with the doctor's diagnosis The classification strategy combined with the integrated algorithm of the classification method is to increase the gap between classes, reduce the gap within classes, and improve the predicted distribution probability. In text modality recognition, the present invention uses cross-entropy loss to perform multi-classification probability prediction on text content. Finally, the weight coefficient is added Weight the image prediction probability and text prediction probability separately to get the final synthetic distribution probability . Each loss function and The calculation method is as follows: Image classification function based on center loss+softmax loss : ; in, represents the softmax loss function, that is ; Represents the center loss function, ; It is a proportional parameter, which is used to adjust the proportion of the center loss function to the total function. It realizes the function of intra-class aggregation and inter-class separation; c is the number of samples accepted for training, is the current sample training feature, is the predicted value of the current feature in the loop, is the predicted value of the target sample; is the feature extracted by the convolutional neural network for the i-th sample, is the average characteristic of the i-th class of samples, and N represents the number of samples; In the process of semantic analysis, the cross-entropy loss function is used to predict the multi-classification probability of the text content. The specific formula is as follows: Text classification function based on cross-entropy loss : ; in, =- , is the prediction test result of each sample T; is the value of the true sample t in the i-th category, The predicted probability of the text analysis prediction model on the real sample t of the i-th category in the multimodal accurate secondary screening module for colorectal cancer patients; Finally add the weight coefficient and Weight the image prediction probability and text prediction probability separately to get the final synthetic distribution probability , the specific formula is as follows: .

[0040] From the characteristics of the classification function, we can see that the image classification function and text classification functions All of them are probability functions with values ​​in the range of (0, 1), so we only need to adjust the weight coefficient according to the relationship between the quantity of image data and text data. , in the case where the amount of data of each mode is different, it is subject to , which means that the distribution probability The final function obtains a reasonable value.

[0041] Final screening module for intelligent multimodal patient identification results: The purpose of the final screening of this module is mainly to compare whether there is a deviation between the intelligent multimodal detection results and the clinical diagnosis results of doctors, so as to ensure the scientific rigor of medical diagnosis and ensure that patients can receive safe and effective diagnosis and treatment. Figure 5 As shown in Figure 2, after completing the classification based on the multi-function fusion classification strategy based on the combination of center loss + softmax loss and cross-entropy loss, a distribution probability of colorectal cancer risk can be obtained. At the same time, the patient’s clinical diagnosis text is analyzed through the transformer network to obtain the doctor’s diagnosis result (Because clinicians will give the final diagnosis based on personal experience, that is, the patient has colorectal cancer or the patient does not have colorectal cancer, is a logical variable, The clinician believes that the patient has colorectal cancer. The clinician believes that the patient does not have colorectal cancer), when the colorectal cancer risk distribution probability When the diagnosis made by the clinician and the intelligent multimodal accurate colorectal cancer patient discovery system is consistent, the patient can be included in the colorectal cancer patient database for management. or The present invention considers that the diagnosis made by the clinician and the intelligent multimodal precise colorectal cancer patient discovery system is inconsistent, and the various types of medical records of the patient are packaged and submitted to higher-level medical experts for diagnosis, and the decision is made based on the medical expert's diagnosis whether to include the patient in the colorectal cancer patient database for management. If the judgment result shows that the intelligent multimodal patient recognition result is incorrect, the corresponding data will be back-propagated and fed back to the multimodal precise colorectal cancer patient secondary screening module, and the recognition ability of the multimodal precise colorectal cancer patient secondary screening module will be optimized (the model's accuracy in distinguishing different images and text data will be improved). When the potential patient is considered to have no colorectal cancer, he / she will not be included in the colorectal cancer patient database, and the data will be fed back to the colorectal cancer keyword detection and initial screening to optimize this part of the model. After summarizing the above formulas, the distribution probability can be obtained And the diagnosis results The corresponding execution table of the values ​​is shown in Table 3 below: Table 3 Corresponding execution table of distribution probability and diagnosis result values

[0042] Intelligent colorectal cancer patient condition summary visualization module: The purpose of this module is to realize the visualization management of colorectal cancer patient data at the hospital or regional level after the discovery of colorectal cancer patients is completed. The present invention is based on Python Pandas and NumPy data processing libraries to build a visualization colorectal cancer patient discovery management system that includes key information such as patient basic information, colorectal cancer development stage, survival time after diagnosis, and treatment medication, and can realize the real-time addition of newly discovered patients.

[0043] In addition, the colorectal cancer patient management system can also be combined with the patient's basic information through the GIS system to realize the map display of the patient's permanent residence, and analyze the potential colorectal cancer environmental impact of the medical institution's location through the patient's permanent residence information, and calculate the regional distribution density of colorectal cancer patients. , analyze the survival time and treatment courses of patients after diagnosis, and calculate the estimated survival rate of patients in the region by combining the regional colorectal cancer full-cycle survival rate and late-stage diagnosis survival rate .

[0044] Regional distribution density of colorectal cancer patients The calculation formula is as follows: ; in, is the area of ​​the selected region, Total area served for patient discovery and management systems, is the number of colorectal cancer patients in the selected area, The total number of colorectal cancer patients in the patient discovery and management system service area. The calculation method of can also be obtained by using known patient distribution data to calculate the scattered point envelope surface without artificially selecting a fixed area. area.

[0045] Estimated survival time of patients in different regions The calculation formula is as follows: ; in, For selected patients, To calculate the survival time of current patients with early colorectal cancer, To count the number of patients with early colorectal cancer in the system, To calculate the survival time of patients with advanced colorectal cancer. Statistics of patients with moderate to late stage colorectal cancer in the system. To estimate the survival time, the two parameters are adapted according to the number of treatment courses and diagnosis of the patient obtained by the system, and obey As the patient's condition worsens or time passes, this parameter group gradually follows .

[0046] Based on the technical solution of the present invention, the following case scenario is used to illustrate the implementation process of the present invention in practical application. The specific application implementation plan is as follows: Case 1: If Figure 6 As shown, after the designated hospital or medical unit in the target area completes the outpatient treatment and inpatient examination work for the day, the AI ​​multimodal colorectal cancer tertiary screening patient discovery system will extract all medical record data for the day and perform multimodal analysis on it. Taking patient A as an example, after the system extracts the textual diagnosis data of patient A during the outpatient or hospitalization process from the medical record data of the day, it will first perform an initial screening of colorectal cancer keyword detection on the text of his outpatient or inpatient medical record. After the initial screening by the system, keywords such as occult blood in stool, colonoscopy polyps, abdominal pain, fever, poor sleep at night, loss of appetite, and less exercise time are detected from the medical record of patient A. After keyword scoring and comparison, the system screens out the effective keywords for colorectal cancer, including occult blood in stool, colonoscopy polyps, abdominal pain, and loss of appetite, and ignores the invalid keywords, such as fever and poor sleep at night. By calculating the keyword scores, and at the same time using the patient's basic information, it is found that the patient is engaged in the Internet industry and has the risk of long-term sitting. The occupational disease coefficient can be queried to obtain the disease coefficient. =1.12. At the same time, by checking the doctor's inquiry text, it can be learned that he has been engaged in this profession for about 5 years, that is, 60 months. The working time coefficient t=2.5 is calculated. The current disease coefficient of the hospital and the region can be retrieved to finally calculate the potential colorectal cancer risk probability of patient A. =3.72, which is greater than the regional colorectal cancer risk line =1.56, patient A will be included in the range of potential colorectal cancer patients, and the basic information of patient A will be submitted to the multimodal precise colorectal cancer patient secondary screening module.

[0047] After reaching the multimodal accurate colorectal cancer patient secondary screening module, all medical examination data related to Patient A and potentially related to the diagnosis of colorectal cancer, as well as various outpatient and inpatient medical records, will be retrieved from the database of all medical units in the target hospital or destination area, and image recognition and semantic analysis will be performed on the above data through image processing and natural language text processing. After completing the processing of the image and text respectively, the system will integrate all textual diagnostic data, test reports, and related CT and MR medical images of Patient A in the hospital or hospital alliance, and derive the distribution probability of colorectal cancer risk through the classification function. = =0.4*0.9+0.6*0.6=0.72, and compared with the patient's clinician's diagnosis =1 for comparison. At this point, the system finds the distribution probability of colorectal cancer risk derived by the system Diagnosis results with clinicians The value range is not consistent, according to the distribution probability And the diagnosis results The corresponding execution table shows that at this time =0.72<0.8, =1, which is the situation described by situation number 4 in Table 3, , it is impossible to directly determine whether the patient has colorectal cancer. In order to determine the patient's actual condition, the system automatically submits the patient data to a more senior medical expert for manual review. After completing the manual review, it is determined that Patient A has colorectal cancer. The system automatically enters the patient data into the colorectal cancer patient discovery and management system, which is convenient for subsequent hospitals and disease control departments to manage and follow up. In the patient management link, the system will record the patient's name, hospital area, medical record number, admission and discharge time, epidemiological questionnaire information, and disease diagnosis (colon cancer / colon adenocarcinoma / rectal cancer, etc.) in detail, effectively improving the hospital's information management level for colorectal patients. In the follow-up link, the system can record the specific circumstances of the follow-up through structured text, including the number of follow-up visits, whether the patient has recently gone to the hospital for a reexamination, whether the patient has recurred or metastasized, and the follow-up doctor's textual follow-up summary record.

[0048] In addition, if the clinician is found to have made a diagnostic error in the previous third-level review, the system can also obtain the name of the clinician who made the wrong diagnosis through text analysis, and provide feedback to the clinician on the problems in the clinical work, and suggest that the clinician correct the relevant medical records or add relevant medical record data. After completing the above work, the colorectal cancer condition of patient A was automatically screened, and the system process of the third-level screening patient discovery for colorectal cancer of patient A was completed.

[0049] Case 2: Taking patient B as an example, when the system extracts patient B's data from the medical record data of the day, it will first perform a colorectal cancer keyword detection and screening on the text of his outpatient or inpatient medical record. When the system detects keywords such as constipation and abdominal distension from patient B's medical record, and compares the keyword scores based on the patient's regional risk factor and occupational risk factor, it still finds that patient B has a potential colorectal cancer risk probability. =2.71 greater than the regional colorectal cancer risk line =1.56, patient B will be included in the scope of potential colorectal cancer patients, and the basic information of patient B will be submitted to the multimodal precise colorectal cancer patient secondary screening module.

[0050] Similar to case 1, the system will automatically extract various information about patient B and make comparisons and predictions to obtain the distribution probability of colorectal cancer risk. =0.33 and the clinical diagnosis result However, after the system judges that the data ranges of the two are consistent and the diagnosis result is that patient B does not have colorectal cancer, the system will no longer submit the information to the more senior medical experts, but directly report the results to the colorectal cancer keyword detection and screening module, and adjust the keyword scoring value. After the scoring value is adjusted, the colorectal cancer keyword detection and screening process is performed on patient B again. If the initial screening result shows that the risk probability of colorectal cancer is Still higher than the regional colorectal cancer risk line , then continue the above cycle and adjust the keyword scoring until the initial screening result is correct. =1.51 lower than the regional colorectal cancer risk line , the initial screening result is correct and no further correction is required, and the process ends.

[0051] Case 3: Taking patient C as an example, the screening results of the colorectal cancer keyword detection primary screening module and the multimodal accurate colorectal cancer patient secondary screening module are the same as those of patient A, and the distribution probability of colorectal cancer risk obtained by the system is Diagnosis results with clinicians The numerical range is not consistent, but the feedback from the senior medical expert is consistent with that of the clinician, but inconsistent with the system's predicted probability. After the system receives feedback from the senior medical expert, it first enters the patient C's information into the colorectal cancer patient discovery and management system, and then adjusts the proportional coefficient of the multimodal accurate colorectal cancer patient secondary screening module. According to the training results, the image probability and text probability can be combined to form a parameter Adjust to 0.45 and 0.55; and at the same time consider further optimizing the loss function in the back propagation of the deep learning system to reduce the number of feature losses. The requirements for recognition training are consistent with the requirements for the colorectal cancer keyword detection screening module in Case 2. After the training is completed, the process ends.

[0052] The system of the present invention combines multimodal data such as clinical patient complaint data after structured processing, medical images (such as CT, MRI, endoscopic images, etc.), biomarker detection, and hospitalization monitoring data. It adopts the Pytorch framework and a multimodal intelligent colorectal cancer screening fusion model built based on Transformer. Through multimodal recognition methods such as text semantic analysis, image segmentation and recognition, and data weighted fusion, it realizes the three-level screening and classification of colorectal cancer patients, aiming to improve the accuracy and efficiency of early screening, diagnosis, and risk assessment of colorectal cancer in different clinical scenarios.

[0053] The above description is only a detailed description of the preferred embodiments and principles of the present invention. For ordinary technicians in this field, according to the ideas provided by the present invention, there will be changes in the specific implementation methods, and these changes should also be regarded as the protection scope of the present invention.

Claims

1. A colorectal cancer patient discovery system based on AI multimodal technology, characterized by: include: Structured medical record data collection module, used to systematically collect and manage patients' medical information using structured medical records; The patient screening module based on colorectal cancer keyword detection is used to screen potential colorectal cancer patients from the medical record database, and to form a tuple of patient-data-disease status with all relevant medical records and historical medical data of each patient to analyze potential colorectal cancer patients; The multimodal accurate secondary screening module for colorectal cancer patients is used to further extract all the corresponding medical data of potential colorectal cancer patients, and analyze the corresponding data through the AI ​​multimodal system to obtain the predicted disease results; The final screening module of intelligent multimodal patient identification results is used to compare the predicted condition results with the diagnosis results given by the clinician, and make a judgment based on the relative deviation value between the clinician's diagnosis results and the predicted condition results to obtain the final screening results; The intelligent colorectal cancer patient condition summary visualization module is used to display the disease risk grading of colorectal cancer patients, the residential distribution of colorectal cancer patients, and the follow-up records.

2. The colorectal cancer patient discovery system based on AI multimodal technology according to claim 1, characterized in that: In the structured medical record data collection module, the content of the structured medical record includes the patient's basic information, chief complaint, physical examination, auxiliary examination, clinical physician's diagnosis results and treatment methods; among them, the keyword entry method is adopted in the chief complaint. After the doctor enters the patient data, the system automatically checks and judges the entered data, and performs data cleaning and data normalization, removes irrelevant characters, and unifies the medical record text format, and stores it locally in the hospital or on the medical data cloud in accordance with the medical record confidentiality requirements.

3. The colorectal cancer patient discovery system based on AI multimodal technology according to claim 1, characterized in that: The patient initial screening module based on colorectal cancer keyword detection specifically includes the following processes: S21, complete the initial screening of potential colorectal cancer patients through the bag-of-words model and the text medical record keyword scoring detection method; convert the text into a numerical feature vector by calculating the TF weight and the word numerical feature vector; the TF weight calculation formula and the word vector calculation formula are as follows: ； ； in, is the number of times the keyword appears, is the total number of occurrences of all keywords. is the word TF weight, is the word numerical feature vector, is the word direction vector coefficient; S22, using the bag-of-words model to quickly capture various text keywords in medical records and examination reports; after the bag-of-words model completes keyword capture, the keywords are classified and segmented, and different types of keywords are assigned corresponding colorectal cancer risk scores , and then evaluate the potential risk of colorectal cancer through the text medical record keyword scoring detection formula, and determine the range of patients entering the multimodal precise colorectal cancer patient secondary screening module; the text medical record keyword scoring detection formula is as follows: ； in, is the potential risk of colorectal cancer, Assign a score to each text medical record keyword, M is the sum of the scores of all text medical record keywords in the database, are the parameters for the incidence rate of colorectal cancer by region and patient occupation.

4. The colorectal cancer patient discovery system based on AI multimodal technology according to claim 3, characterized in that: The patient initial screening module based on colorectal cancer keyword detection specifically includes the following processes: S23, set the colorectal cancer incidence rate parameters by region and patient occupation The formula is as follows: ； in, is the number of confirmed colorectal cancer medical records in each hospital in the region, The number of suspected medical records reported in the area. is the number of medical records of colorectal cancer diagnosed in the institution where the patient is treated, The number of suspected cases reported by the hospital. The area code of the hospital where the patient is treated. is the regional weight coefficient, is the proportion coefficient of the visiting unit. In principle, the proportion coefficient of the visiting unit is subject to the regional proportion coefficient, that is, ; is the occupational disease risk factor, The risk time standard is the 24-month hazard standard. The time factor for working in the profession; S24, working time coefficient The conversion is as follows: Let T be the total time spent on the job, in months: ； When the potential risk of colorectal cancer Greater than the regional colorectal cancer risk line When the patient is identified, the corresponding patient will be included in the next phase of precision colorectal cancer patient discovery operations; The calculation method is as follows: ； in, is the number of confirmed colorectal cancer medical records in each hospital in the region, is the number of suspected medical records reported in the area, P is the permanent population in the area, is the elasticity diagnostic conversion coefficient; S25, finally the potential patient discrimination formula based on colorectal cancer keyword detection is obtained: ； when When , then the corresponding medical record owner is determined to be a potential colorectal cancer patient, and the data is entered into the second-level screening.

5. The colorectal cancer patient discovery system based on AI multimodal technology according to claim 1, characterized in that: The multimodal precise colorectal cancer patient secondary screening module obtains a list of potential colorectal cancer patients through a patient primary screening module based on colorectal cancer keyword detection, and then extracts all medical images and biomarker test reports related to the corresponding patients from the database, extracts and separates the image information in the biomarker test report, and transfers it to the U-net3+ model based on the pytorch framework for precise colorectal cancer image recognition, and the text part is handed over to the Transformer module for semantic analysis and processing.

6. The colorectal cancer patient discovery system based on AI multimodal technology according to claim 5, characterized in that: In the multimodal accurate colorectal cancer patient secondary screening module, the accurate colorectal cancer image recognition specifically includes the following steps: S31, performing image preprocessing on the colorectal examination image data through the torchvision visual processing package; the image preprocessing includes unifying the size and color parameters of the colorectal examination image, removing unclear, blurred, dark or bright images; and finally adjusting the image size to 256×256 for input into the deep learning model; After image preprocessing is completed, the colorectal image is enhanced to simulate colorectal images at different shooting angles and shooting distances; the image enhancement includes random rotation, shearing and scaling; After completing image data preprocessing and image enhancement, the U-net3+ model is used to perform fine segmentation of colorectal images. The encoder in the U-net3+ model is responsible for downsampling and feature extraction, the decoder is responsible for upsampling and segmentation result generation, and the full-scale skip connection realizes feature fusion between different levels. The semantic analysis process specifically includes the following steps: S32, Transformer module is responsible for analyzing the medical record text and test report text extracted from the database; Remove irrelevant characters, including special symbols and punctuation marks, through text cleaning; standardize and normalize the cleaned text to remove common words that do not contribute to classification; Through the self-attention mechanism of the Transformer module, we capture keywords related to colorectal cancer in all text reports, and widely link the keywords to obtain accurate global information, ultimately achieving detection and information extraction of various medical texts including medical records and test reports.

7. The colorectal cancer patient discovery system based on AI multimodal technology according to claim 6, characterized in that: For the accurate image recognition process of colorectal cancer, a classification strategy combining center loss + softmax loss function is used. The specific formula is as follows: Image classification function based on center loss+softmax loss : ； in, represents the softmax loss function, that is ; Represents the center loss function, ; It is a proportional parameter, which is used to adjust the proportion of the center loss function to the total function. It realizes the function of intra-class aggregation and inter-class separation; c is the number of samples accepted for training, is the current sample training feature, is the predicted value of the current feature in the loop, is the predicted value of the target sample; is the feature extracted by the convolutional neural network for the i-th sample, is the average characteristic of the i-th class of samples, and N represents the number of samples; In the process of semantic analysis, the cross-entropy loss function is used to predict the multi-classification probability of the text content. The specific formula is as follows: Text classification function based on cross-entropy loss : ； in, =- , is the prediction test result of each sample T; is the value of the true sample t in the i-th category, is the predicted probability of the true sample t of the i-th category; Finally add the weight coefficient and Weight the image prediction probability and text prediction probability separately to get the final synthetic distribution probability , the specific formula is as follows: 。 8. The colorectal cancer patient discovery system based on AI multimodal technology according to claim 7, characterized in that: The intelligent multimodal patient identification result final screening module specifically includes the following processes: S41, analyze the patient’s clinical diagnosis text through the Transformer network to obtain the doctor’s diagnosis result , is a logical variable, The clinician believes that the patient has colorectal cancer. The clinician believes that the patient does not have colorectal cancer; When the probability of colorectal cancer risk When the diagnosis made by the clinician and the system is consistent, the corresponding patient will be included in the colorectal cancer patient database for management. or When , it is considered that the diagnoses made by the clinician and the system are inconsistent, and the various medical records of the corresponding patient are packaged and submitted to higher-level medical experts for diagnosis, and the decision on whether to include the patient in the colorectal cancer patient database for management is made according to the diagnosis of the medical experts; if the judgment result shows that the system recognition result is incorrect, the corresponding data is back-propagated and fed back to the multimodal precise colorectal cancer patient secondary screening module, and the recognition ability of the multimodal precise colorectal cancer patient secondary screening module is optimized; when , it is considered that the potential patient does not have colorectal cancer and will not be included in the colorectal cancer patient database. The corresponding data will be fed back to the patient initial screening module based on colorectal cancer keyword detection, and the module will be optimized.

9. The colorectal cancer patient discovery system based on AI multimodal technology according to claim 1, characterized in that: The intelligent colorectal cancer patient condition summary visualization module specifically includes the following processes: S51, through the combination of GIS system and basic patient information, the patient's permanent residence map is displayed, and the potential colorectal cancer environmental impact of the medical institution's location is analyzed through the patient's permanent residence information, and the distribution density of colorectal cancer patients in the area is calculated. , analyze the survival time and treatment courses of patients after diagnosis, and calculate the estimated survival rate of patients in the region by combining the regional colorectal cancer full-cycle survival rate and late-stage diagnosis survival rate ; Regional distribution density of colorectal cancer patients The calculation formula is as follows: ； in, is the area of ​​the selected region, Total area served for patient discovery and management systems, is the number of colorectal cancer patients in the selected area, The total number of colorectal cancer patients in the patient discovery and management system service area; Estimated survival time of patients in the region The calculation formula is as follows: ； in, For selected patients, To calculate the survival time of current patients with early colorectal cancer, To count the number of patients with early colorectal cancer in the system, To calculate the survival time of patients with advanced colorectal cancer. Count the number of patients with advanced colorectal cancer in the system; Coordinate parameters for estimated survival time, parameters Adapt according to the number of treatment courses and diagnosis of the patient obtained by the system, and obey ; As the patient's condition worsens or time passes, parameters Gradually obey .

Images