Cervical cancer risk diagnosis system

Abstract

The invention relates to the technical field of clinical diagnosis, in particular to a cervical cancer risk diagnosis system. The image processing module is used for extracting image features based on a preprocessed cervical cell pathological image; the text processing module is used for extracting text features based on the preprocessed medical record book data; the feature alignment module is used for aligning the image features and the text features to obtain the aligned image features and text features; the feature fusion module is used for fusing the aligned image features and text features by using a mutual attention mechanism to obtain fused features; and the prediction module is used for predicting the cervical cancer onset risk of the target patient by using a preset tumor risk prediction model based on the fusion features to obtain a dichotomy prediction result. Therefore, through the cervical cancer risk diagnosis system, the problem of low screening accuracy caused by manual limitation or difficulty in multi-modal data fusion in the existing diagnosis technology is solved, and the accuracy of early screening of cervical cancer is improved.

Description

Technical Field

[0001] The present application relates to the field of clinical diagnosis technology, and in particular to a cervical cancer risk diagnosis system. Background Art

[0002] Cervical cytology screening is a method of cervical cancer screening. It can collect cells from the surface of the cervix and use Thinprep Cytology Test (TCT) technology to diagnose cervical cancer by observing abnormalities in the morphology of cervical cells.

[0003] In the related art, it is proposed to use computer-assisted screening of cervical cell pathology images based on TCT cell pathology images.

[0004] However, this method only performs disease screening based on single modality data, resulting in inaccurate disease screening results, which urgently needs to be improved. Summary of the Invention

[0005] The present application provides a cervical cancer risk diagnosis system to solve the problem of low screening accuracy caused by artificial limitations or difficulties in multimodal data fusion in existing diagnostic technologies, and to improve the accuracy of early cervical cancer screening.

[0006] To achieve the above objectives, the first embodiment of the present application provides a cervical cancer risk diagnosis system, comprising:

[0007] An image processing module is used to obtain a cervical cell pathology image of a target patient, preprocess the cervical cell pathology image, and extract image features based on the preprocessed cervical cell pathology image;

[0008] a text processing module, configured to pre-process the medical records of the target patient using a large language model, and extract text features based on the pre-processed medical records;

[0009] A feature alignment module, configured to align the image features with the text features to obtain aligned image features and aligned text features;

[0010] A feature fusion module, configured to fuse the aligned image features and the aligned text features using a mutual attention mechanism to obtain a fused feature;

[0011] A prediction module is used to predict the risk of cervical cancer in the target patient based on the fusion features using a preset tumor risk prediction model to obtain a binary prediction result, wherein the preset tumor risk prediction model is trained by historical cytopathology image data and historical medical case record data.

[0012] According to one embodiment of the present application, the image processing module includes:

[0013] a first preprocessing unit, configured to segment a target cell region from the cervical cell pathology image and cut the target cell region to obtain an image block of a preset size, wherein the preprocessed cervical cell pathology image is an image block of the preset size;

[0014] The first extraction unit is configured to perform feature extraction on the image block of the preset size to obtain the image features.

[0015] According to one embodiment of the present application, the first pre-processing unit includes:

[0016] a downsampling subunit, configured to perform downsampling processing on the cervical cell pathology image to obtain an image meeting a preset resolution condition;

[0017] A first processing subunit is configured to convert the RGB (Red Green Blue) format of the image meeting the preset resolution condition into the HSV (Hue Saturation Value) format, extract the S channel and perform median filtering to obtain a first image;

[0018] a second processing subunit, configured to determine an optimal threshold value by using a preset algorithm, and perform a binarization threshold segmentation process on the first image based on the optimal threshold value to obtain a binary mask image;

[0019] a third processing subunit, configured to remove burrs and holes in the binary mask image using a morphological strategy to obtain a second image;

[0020] a fitting subunit, configured to determine cell boundaries based on the second image, select boundaries that meet preset conditions from the cell boundaries, determine a circumscribed circle based on the boundaries that meet the preset conditions, and fit the cell shape using the circumscribed circle to obtain the target cell region;

[0021] The cutting subunit is used to cut the target cell area using a non-overlapping cutting strategy to obtain the image block of the preset size.

[0022] According to one embodiment of the present application, the first extraction unit includes:

[0023] The feature extraction subunit is used to use a preset deep convolutional neural network to perform feature extraction on the image block of the preset size to obtain the image features.

[0024] According to one embodiment of the present application, the text processing module includes:

[0025] a second preprocessing unit, configured to input the medical record data of the target patient into the large language model to obtain the preprocessed medical record data, wherein the preprocessed medical record data is structured text including multiple categories;

[0026] The second extraction unit is used to perform feature extraction based on the preprocessed medical record data using a preset neural network model to obtain the text features.

[0027] According to one embodiment of the present application, the second extraction unit includes:

[0028] An extraction subunit is configured to input the text corresponding to each category into a preset neural network model based on the preprocessed medical record data to obtain a plurality of subtext features, wherein each subtext feature corresponds to each category one by one;

[0029] The superposition subunit is used to add the category ID of each sub-text feature to obtain the text feature.

[0030] According to one embodiment of the present application, the multiple categories include menstrual history, gynecological conditions, medication history, HPV (Human Papillomavirus) conditions and current medical history.

[0031] According to one embodiment of the present application, the feature fusion module includes:

[0032] a mapping unit for mapping the aligned text features into queries and the aligned image features into keys and values ​​using different linear mapping functions;

[0033] The fusion unit is used to fuse the mapped results using the mutual attention mechanism to obtain the fused features.

[0034] According to one embodiment of the present application, the prediction module includes:

[0035] a labeling unit, configured to add a preset classification label to the fusion feature to obtain a target input sequence;

[0036] The prediction unit is used to input the target input sequence into the preset tumor risk prediction model to obtain the binary classification prediction result.

[0037] According to one embodiment of the present application, the prediction unit includes:

[0038] A first prediction subunit is configured to input the target input sequence into the Transformer network in the preset tumor risk prediction model to obtain a feature vector corresponding to the preset classification label;

[0039] The second prediction subunit is used to input the feature vector into the fully connected layer binary classifier in the preset tumor risk prediction model to obtain the binary classification prediction result.

[0040] According to the cervical cancer risk diagnosis system proposed in the embodiment of the present application, the image processing module extracts image features based on the preprocessed cervical cell pathology image, the text processing module extracts text features based on the preprocessed medical record data, the feature alignment module aligns the image features and text features to obtain aligned image features and text features, the feature fusion module fuses the aligned image features and text features using the mutual attention mechanism to obtain fused features, and the prediction module predicts the target patient's cervical cancer risk based on the fused features using a preset tumor risk prediction model to obtain a binary classification prediction result. Therefore, the cervical cancer risk diagnosis system solves the problem of low screening accuracy caused by artificial limitations or difficulties in multimodal data fusion in existing diagnostic technologies, thereby improving the accuracy of early cervical cancer screening.

[0041] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0043] Figure 1 This is a block diagram of a cervical cancer risk diagnosis system provided according to an embodiment of the present application;

[0044] Figure 2 This is a schematic diagram of the implementation process of a cervical cancer risk diagnosis system according to one embodiment of the present application. DETAILED DESCRIPTION

[0045] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0046] The following describes a cervical cancer risk diagnosis system proposed according to an embodiment of the present application with reference to the accompanying drawings.

[0047] Before introducing the cervical cancer risk diagnosis system proposed in the embodiment of the present application, the relevant technical background is first introduced.

[0048] Traditional cervical cytology screening is often performed manually by doctors. However, manual reading is time-consuming and labor-intensive, with subjective standards and limited doctor expertise. These factors limit the efficiency and accuracy of traditional manual screening. Therefore, with technological advancements, computer-assisted screening of cervical cell pathology images has begun to be used. Through intelligent screening and diagnosis, the efficiency and accuracy of cervical cytology screening by pathologists have been greatly improved.

[0049] Because cell morphology can change due to factors such as age, gynecological examination results, clinical medication use, and menstrual patterns, it's impossible to accurately determine whether a patient is ill based solely on image data from a single modality. Medical records contain a wealth of this information, which can serve as supplementary information for cytopathology images. Therefore, the fusion of these two modalities can help improve the accuracy of early cervical cancer screening and assist doctors in making more efficient and accurate diagnoses.

[0050] However, most current intelligent screening methods rely solely on TCT cytopathology images, rarely incorporating medical records for auxiliary diagnosis. Building multimodal diagnostic models presents a series of challenges and difficulties: First, there are significant differences between the two modalities, cytopathology images and medical records, and the difference in information magnitude is equally significant. This disparity makes it extremely difficult to effectively and efficiently fuse information from the two modalities. Second, medical records often contain a significant amount of noise, which can arise from irregular recording, incomplete information, or errors. Furthermore, cytopathology images are inherently sparse, meaning that the areas within the image that can provide diagnostic information are very limited. Furthermore, typically, only case-level labels are used as supervisory information, without more detailed annotations. This makes it particularly difficult to extract the key features crucial for diagnosis from the massive amount of data.

[0051] It is based on the above problems that the embodiment of the present application proposes a cervical cancer risk diagnosis system, which only requires weak supervision labels at the case level to effectively combine cytopathology images and medical records, thereby accurately diagnosing cervical cancer. Specifically, the cervical cytopathology images are preprocessed through an automatic data processing process to extract image features at the example level; the medical record text is cleaned and text features are extracted using a large language model to achieve alignment at the example level of image and text data; finally, the mutual attention mechanism is used to fuse the features of the two modalities, and the intelligent risk prediction of cervical cancer is achieved through a deep neural network (i.e., a preset tumor risk prediction model). It solves the problem of low screening accuracy due to artificial limitations or difficulties in multimodal data fusion in existing diagnostic technologies, and improves the accuracy of early screening for cervical cancer.

[0052] Next, the cervical cancer risk diagnosis system is explained in detail.

[0053] Figure 1 It is a block diagram of a cervical cancer risk diagnosis system according to one embodiment of the present application.

[0054] For example, Figure 1 As shown, the cervical cancer risk diagnosis system 10 includes: an image processing module 100 , a text processing module 200 , a feature alignment module 300 , a feature fusion module 400 and a prediction module 500 .

[0055] Among them, the image processing module 100 is used to obtain the cervical cell pathology image of the target patient, preprocess the cervical cell pathology image, and extract image features based on the preprocessed cervical cell pathology image; the text processing module 200 is used to use a large language model to preprocess the medical record data of the target patient, and extract text features based on the preprocessed medical record; the feature alignment module 300 is used to align the image features and the text features to obtain aligned image features and aligned text features; the feature fusion module 400 is used to use the mutual attention mechanism to fuse the aligned image features and the aligned text features to obtain fused features; the prediction module 500 is used to predict the cervical cancer risk of the target patient based on the fused features using a preset tumor risk prediction model to obtain a binary classification prediction result, wherein the preset tumor risk prediction model is trained by historical cell pathology image data and historical medical record data.

[0056] It should be noted that when the cervical cancer risk diagnosis system 10 of the embodiment of the present application predicts the risk of disease, the entire prediction process is divided into three main parts: cytopathology image preprocessing and feature extraction, medical record text report preprocessing and feature extraction, and multimodal feature fusion and risk prediction.

[0057] Cervical cytopathology images refer to images of cervical cell samples observed and photographed under a microscope. These samples are typically obtained through a cervical scraping or biopsy. Cervical cytopathology images are used to diagnose cervical cancer and its precursors and are an important tool for cervical cancer screening and diagnosis. Medical records are important documents that record a patient's personal health information and medical history. They include personal information, medical history, treatment progress, medication use, diagnosis results, and physician recommendations.

[0058] Specifically, the image processing module 100 is primarily used to acquire cervical cytopathology images of target patients. After successfully acquiring these images, it performs preprocessing on the cervical cytopathology images. Once preprocessing is complete, the image processing module 100 applies advanced algorithms to extract key image features based on the preprocessed cervical cytopathology images. These features may include cell shape, size, texture, and nuclear morphology, which are extremely important for subsequent pathological analysis and diagnosis. The text processing module 200 utilizes advanced large language models to preprocess the target patient's medical records. Based on this preprocessing, it further extracts diagnostically valuable text features to facilitate subsequent analysis and processing. The feature alignment module 300 is primarily responsible for accurately aligning image features with text features. This process yields aligned image features and corresponding text features, laying a solid foundation for subsequent processing steps. Subsequently, the feature fusion module 400 utilizes a mutual attention mechanism to further process these aligned features. Through this mechanism, the feature fusion module 400 can intelligently identify and emphasize the correlation between image features and text features, thereby effectively fusing them and ultimately generating a comprehensive fusion feature. This fusion feature not only contains the information of the original image and text, but also the correlation between the two, providing richer and deeper data support for subsequent analysis and understanding. Subsequently, the prediction module 500 uses a preset tumor risk prediction model based on the fused features to assess the target patient's risk of cervical cancer. The model can ultimately output a binary diagnosis of whether the target patient has cervical cancer. It is worth noting that the preset tumor risk prediction model was obtained in the early stage through a complex machine learning training process using a large amount of historical cytopathology image data and rich historical medical records data, thereby ensuring the accuracy and reliability of the prediction.

[0059] For ease of understanding, the image processing module 100 , the text processing module 200 , the feature alignment module 300 , the feature fusion module 400 and the prediction module 500 are respectively described in detail below.

[0060] Optionally, in some embodiments, the image processing module 100 includes: a first preprocessing unit and a first extraction unit, wherein the first preprocessing unit is used to segment the target cell area from the cervical cell pathology image and cut the target cell area to obtain an image block of a preset size, wherein the preprocessed cervical cell pathology image is an image block of a preset size; the first extraction unit is used to perform feature extraction on the image block of the preset size to obtain image features.

[0061] Specifically, cervical cell pathology image preprocessing refers to the segmentation and cutting of TCT cervical cell slices. In TCT slice images, usually only a part of the circular area in the middle contains cells. Therefore, the first task is to accurately segment this cell area from the entire slice to obtain the target cell area. The target cell area can be cut to obtain an image block of a preset size (i.e., the preprocessed cervical cell pathology image). Finally, the image block of the preset size is deeply analyzed and processed to extract key image features.

[0062] Optionally, in some embodiments, the first preprocessing unit includes: a downsampling subunit, a first processing subunit, a second processing subunit, a third processing subunit, a fitting subunit and a cutting subunit, wherein the downsampling subunit is used to perform downsampling processing on the cervical cell pathology image to obtain an image that meets a preset resolution condition; the first processing subunit is used to convert the RGB format of the image that meets the preset resolution condition into an HSV format, and extract the S channel for median filtering to obtain a first image; the second processing subunit is used to determine an optimal threshold using a preset algorithm, and perform binarization threshold segmentation processing on the first image based on the optimal threshold to obtain a binary mask image; the third processing subunit is used to remove burrs and holes in the binary mask image using a morphological strategy to obtain a second image; the fitting subunit is used to determine cell boundaries based on the second image, and select boundaries that meet preset conditions from the cell boundaries, determine a circumscribed circle based on the boundaries that meet the preset conditions, and use the circumscribed circle to fit the cell shape to obtain a target cell area; the cutting subunit is used to cut the target cell area using a non-overlapping cutting strategy to obtain an image block of a preset size.

[0063] Specifically, if Figure 2As shown, for the segmentation part, the embodiment of the present application can first downsample the cervical cell pathology image to obtain a low-resolution thumbnail (meeting the preset resolution conditions, not specifically limited here). The length and width of the thumbnail can be approximately one thousand to two thousand. Next, background segmentation is performed. The image format that meets the preset resolution conditions obtained in the previous step is converted from RGB to HSV, and the S channel is focused on. The median filter technique can be applied to this channel to smooth the image and reduce noise, thereby obtaining a first image. Then, the first image can be binarized. That is, an optimal threshold is determined using a preset algorithm (such as the OTSU method). The optimal threshold is used for binary segmentation to obtain a clear (foreground) binary mask image. This mask image will only contain foreground pixels, while background pixels will be set to another fixed value, thereby completing the binary threshold segmentation of the image. Then, some traditional morphological strategies, such as opening and closing operations, are used to remove burrs and holes in the binary mask image to obtain a second image. Among them, the opening operation is mainly used to remove small noise points and small protrusions in the image, while the closing operation helps to fill small holes and cracks in the image. Both methods are based on basic morphological operations. They use specific structural elements to scan the image to achieve the purpose of smoothing the image, eliminating noise and filling holes. Given that the cell area in the TCT image usually appears in a circular shape, the maximum boundary of the cell area can be determined based on the second image through precise image processing technology. Subsequently, in order to more accurately represent the cell area, the concept of a circumscribed circle can be used to fit the entire area, thereby obtaining a more accurate and complete target cell area.

[0064] After extracting the target cell region, a non-overlapping cutting strategy can be adopted, starting from the upper left corner of the cytopathology image of the target cell region, and cutting without overlapping into image blocks of 256×256 (i.e., the preset size) pixels, thereby avoiding data redundancy or duplication in subsequent processing.

[0065] Optionally, in some embodiments, the first extraction unit includes: a feature extraction subunit, configured to perform feature extraction on an image block of a preset size using a preset deep convolutional neural network to obtain image features.

[0066] Specifically, after obtaining the image blocks of a preset size, a preset deep convolutional neural network (such as Resnet50 (ResidualNetwork 50, a deep convolutional neural network model)) can be used to extract the features of these image blocks. In the process of feature extraction, Resnet50 can capture the key information in the image, such as edges, textures, and shapes. These features can then be packaged into feature vectors, that is, the image features H are obtained. TCT , which is expressed as follows:

[0067]

[0068] Where d is the feature dimension extracted from each image block, and J is the number of image blocks cut from each slice.

[0069] Optionally, in some embodiments, the text processing module 200 includes: a second preprocessing unit and a second extraction unit, wherein the second preprocessing unit is used to input the medical record data of the target patient into the large language model to obtain preprocessed medical record data, wherein the preprocessed medical record data is a structured text including multiple categories; the second extraction unit is used to perform feature extraction based on the preprocessed medical record data using a preset neural network model to obtain text features.

[0070] Optionally, in some embodiments, the multiple categories include menstrual history, gynecological conditions, medication history, HPV conditions, and history of present illness.

[0071] Specifically, if Figure 2 As shown in the figure, the preprocessing of medical record data refers to inputting the target patient's medical record text report into a large language model. After the data is input, clear instructions can be issued to the large language model, which can guide the model on how to process and understand the text data. According to the instructions received, the large language model can conduct in-depth analysis and understanding of the medical record text report. Finally, the model will integrate all the information and output a classified and structured text. This structured output makes the information clearer and easier to understand.

[0072] Among them, the structure of the instruction issued to the large language model can be {prefix+text+suffix}, where text is the text part in the medical record text data; prefix is ​​the instruction text before the text, and here the embodiment of this application uses it as "You are performing a medical text report summary task. Please obtain information from the text and classify and summarize it according to the provided keywords. The text to be extracted is as follows"; suffix is ​​the instruction text after the text, and here the embodiment of this application uses it as "Output is structured text, only retaining the parts related to gynecological diseases and immune diseases. The output structure is {menstrual history}: {text1}, {gynecological conditions}: {text2}, {medication history}: {text3}, {HPV conditions}: {text4}, {current medical history}: {text5}, only output structured results, and do not output other redundant text." The final output of structured text including multiple categories can be divided into "menstrual history", "gynecological conditions", "medication history", "HPV conditions", and "current medical history", and each category corresponds to a piece of text.

[0073] After obtaining structured text including multiple categories (i.e., preprocessed medical record data), a preset neural network model (such as BERT (Bidirectional Encoder Representations from Transformers, a natural language processing pre-training model)) can be used to extract features from the preprocessed medical record data, thereby obtaining the feature representation results of the text part, i.e., text features.

[0074] Optionally, in some embodiments, the second extraction unit includes: an extraction subunit and an overlay subunit, wherein the extraction subunit is used to input the text corresponding to each category into a preset neural network model based on the preprocessed medical record data to obtain multiple sub-text features, wherein each sub-text feature corresponds one-to-one to each category; the overlay subunit is used to add the category ID of each sub-text feature to obtain a text feature.

[0075] Specifically, based on the preprocessed medical record data, the text corresponding to each category can be input into BERT for feature extraction, thereby obtaining multiple sub-text features H text , expressed as:

[0076]

[0077] Where K is the number of text categories, which is 5 here, and d is the feature dimension extracted from the text corresponding to each category. Each sub-text feature category is represented by a different ID, for example, "menstrual history" has an ID of 0, "gynecological conditions" has an ID of 1, and so on. The ID number of each category is embedded, that is, converted into a dense vector representation, that is, In this way, the category ID is converted into a numerical form that can be used for subsequent calculations. Next, the sub-text features extracted from the text can be added to the embedding vector of the corresponding category ID to obtain the text feature H report , which is expressed as follows:

[0078]

[0079] Optionally, in some embodiments, the feature fusion module 400 includes: a mapping unit and a fusion unit, wherein the mapping unit is used to map the aligned text features into queries and the aligned image features into keys and values ​​using different linear mapping functions; the fusion unit is used to fuse the mapped results using a mutual attention mechanism to obtain fused features.

[0080] Specifically, if Figure 2 As shown, the embodiment of the present application can use three linear mapping functions W q 、W k 、W v The aligned text features H report Mapping to query The aligned image feature map is the key Sum value Then, the mutual attention mechanism is used to fuse the mapped results to obtain the fusion feature H fusion , expressed as:

[0081]

[0082] Q=W q ·H report ;

[0083] K=W k ·H TCT ;

[0084] V=W v ·H TCT .

[0085] Optionally, in some embodiments, the prediction module 500 includes: a labeling unit and a prediction unit, wherein the labeling unit is used to add a preset classification label to the fusion feature to obtain a target input sequence; the prediction unit is used to input the target input sequence into a preset tumor risk prediction model to obtain a binary classification prediction result.

[0086] Specifically, after obtaining the fusion features, the fusion features and the preset classification token (cls token) can be used to form a target input sequence {[cls]; H fusion}, and input the target input sequence into the preset tumor risk prediction model for final cervical cancer prediction. The model can finally output a binary classification prediction result of whether the target patient has cervical cancer.

[0087] Optionally, in some embodiments, the prediction unit includes: a first prediction subunit and a second prediction subunit, wherein the first prediction subunit is used to input the target input sequence into the Transformer network in the preset tumor risk prediction model to obtain a feature vector corresponding to a preset classification label; the second prediction subunit is used to input the feature vector into the fully connected layer binary classifier in the preset tumor risk prediction model to obtain a binary classification prediction result.

[0088] Specifically, in the process of prediction based on the target input sequence, the target input sequence is first input into the Transformer network in the preset tumor risk prediction model. The Transformer network can output a feature vector corresponding to the preset classification label. Subsequently, the feature vector can be input as the final feature representation into the fully connected layer binary classifier in the preset tumor risk prediction model, thereby obtaining a binary classification result of predicting whether the target patient has cervical cancer.

[0089] In summary, the cervical cancer risk diagnosis system proposed in the embodiment of the present application can realize automatic pre-processing of the patient's TCT cervical cell pathology images and medical text reports, and extract effective features; it can also realize efficient fusion of TCT cervical cell pathology section images and medical case texts to improve the diagnostic effect; in addition, on a multimodal data set, the cervical cancer risk diagnosis system of the embodiment of the present application can achieve a joint improvement in accuracy, F1-score and AUC (Area Under the Curve, area under the ROC curve). The results show that the accuracy can be improved by 7.3%, the F1-score can be improved by 5.8%, and the AUC can be improved by 6.4%.

[0090] According to the cervical cancer risk diagnosis system proposed in the embodiment of the present application, the image processing module extracts image features based on the preprocessed cervical cell pathology image, the text processing module extracts text features based on the preprocessed medical record data, the feature alignment module aligns the image features and text features to obtain aligned image features and text features, the feature fusion module fuses the aligned image features and text features using the mutual attention mechanism to obtain fused features, and the prediction module predicts the target patient's cervical cancer risk based on the fused features using a preset tumor risk prediction model to obtain a binary classification prediction result. Therefore, the cervical cancer risk diagnosis system solves the problem of low screening accuracy caused by artificial limitations or difficulties in multimodal data fusion in existing diagnostic technologies, thereby improving the accuracy of early cervical cancer screening.

[0091] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0092] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0093] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A cervical cancer risk diagnosis system, characterized in that: include: An image processing module is used to obtain a cervical cell pathology image of a target patient, preprocess the cervical cell pathology image, and extract image features based on the preprocessed cervical cell pathology image; a text processing module, configured to pre-process the medical records of the target patient using a large language model, and extract text features based on the pre-processed medical records; A feature alignment module, configured to align the image features with the text features to obtain aligned image features and aligned text features; A feature fusion module, configured to fuse the aligned image features and the aligned text features using a mutual attention mechanism to obtain a fused feature; A prediction module is used to predict the risk of cervical cancer in the target patient based on the fusion features using a preset tumor risk prediction model to obtain a binary prediction result, wherein the preset tumor risk prediction model is trained by historical cytopathology image data and historical medical case record data.

2. The system according to claim 1, wherein: The image processing module includes: a first preprocessing unit, configured to segment a target cell region from the cervical cell pathology image and cut the target cell region to obtain an image block of a preset size, wherein the preprocessed cervical cell pathology image is an image block of the preset size; The first extraction unit is configured to perform feature extraction on the image block of the preset size to obtain the image features.

3. The system according to claim 2, characterized in that The first pre-processing unit includes: a downsampling subunit, configured to perform downsampling processing on the cervical cell pathology image to obtain an image meeting a preset resolution condition; A first processing subunit is configured to convert the RGB format of the image meeting the preset resolution condition into the HSV format, extract the S channel and perform median filtering to obtain a first image; a second processing subunit, configured to determine an optimal threshold value by using a preset algorithm, and perform a binarization threshold segmentation process on the first image based on the optimal threshold value to obtain a binary mask image; a third processing subunit, configured to remove burrs and holes in the binary mask image using a morphological strategy to obtain a second image; a fitting subunit, configured to determine cell boundaries based on the second image, select boundaries that meet preset conditions from the cell boundaries, determine a circumscribed circle based on the boundaries that meet the preset conditions, and fit the cell shape using the circumscribed circle to obtain the target cell region; The cutting subunit is used to cut the target cell area using a non-overlapping cutting strategy to obtain the image block of the preset size.

4. The system according to claim 3, characterized in that The first extraction unit includes: The feature extraction subunit is used to use a preset deep convolutional neural network to perform feature extraction on the image block of the preset size to obtain the image features.

5. The system according to claim 1, wherein: The text processing module includes: a second preprocessing unit, configured to input the medical record data of the target patient into the large language model to obtain the preprocessed medical record data, wherein the preprocessed medical record data is structured text including multiple categories; The second extraction unit is used to perform feature extraction based on the preprocessed medical record data using a preset neural network model to obtain the text features.

6. The system according to claim 5, characterized in that The second extraction unit includes: An extraction subunit is configured to input the text corresponding to each category into a preset neural network model based on the preprocessed medical record data to obtain a plurality of subtext features, wherein each subtext feature corresponds to each category one by one; The superposition subunit is used to add the category ID of each sub-text feature to obtain the text feature.

7. The system according to claim 6, characterized in that The multiple categories include menstrual history, gynecological conditions, medication history, HPV status, and history of present illness.

8. The system according to claim 1, wherein: The feature fusion module includes: a mapping unit for mapping the aligned text features into queries and the aligned image features into keys and values ​​using different linear mapping functions; The fusion unit is used to fuse the mapped results using the mutual attention mechanism to obtain the fused features.

9. The system according to claim 1, wherein: The prediction module includes: a labeling unit, configured to add a preset classification label to the fusion feature to obtain a target input sequence; The prediction unit is used to input the target input sequence into the preset tumor risk prediction model to obtain the binary classification prediction result.

10. The system according to claim 9, characterized in that The prediction unit includes: A first prediction subunit is configured to input the target input sequence into the Transformer network in the preset tumor risk prediction model to obtain a feature vector corresponding to the preset classification label; The second prediction subunit is used to input the feature vector into the fully connected layer binary classifier in the preset tumor risk prediction model to obtain the binary classification prediction result.

Images