Lymph node metastasis staging prediction method and apparatus, computer device, and storage medium

Abstract

The present application relates to a lymph node metastasis staging prediction method and apparatus, a computer device, and a storage medium. The method comprises: collecting a CT image and a digital pathological image of a patient; performing depth feature extraction on the CT image and the digital pathological image by means of a ResNet50 network and a ViT network, respectively, to obtain a first feature vector of the CT image and a second feature vector of the digital pathological image; combining the first feature vector and the second feature vector to obtain a fused feature vector; and inputting the fused feature vector into a fully connected neural network, and outputting a lymph node metastasis staging prediction result of the patient by means of the fully connected neural network. The present application uses a depth feature fusion strategy for a CT image and a digital pathological image to construct a lymph node metastasis staging prediction model, such that the generalization of the model can be improved while the prediction precision of the lymph node metastasis staging prediction model is improved.

Description

Lymph node metastasis staging prediction method, device, computer equipment and storage medium Technical Field

[0001] The present application belongs to the field of medical image processing technology, and in particular relates to a method, apparatus, computer equipment, and storage medium for predicting lymph node metastasis staging. Background Art

[0002] Currently, lymph node metastasis is common in gastric cancer patients. Therefore, when formulating a surgical plan for a patient, it is necessary to examine the patient's lymph node metastasis stage. Different lymph node metastasis stages correspond to different ranges of lymph node dissection. According to the gastric cancer diagnosis and treatment guidelines, the number of lymph node metastases is generally used as an important basis for staging: patients with no lymph node metastases are staged as N0, patients with 1-2 lymph node metastases are staged as N1, patients with 3-6 lymph node metastases are staged as N2, patients with 7-15 lymph node metastases are staged as N3a, and when the number of lymph node metastases exceeds 15, the patient is staged as N3b. Although CT images can relatively accurately determine the presence of lymph node metastasis in clinical practice, it is often difficult to determine the number of lymph node metastases, making the lymph node metastasis staging of gastric cancer patients difficult to diagnose.

[0003] In the existing technology, research on the prediction of lymph node metastasis in gastric cancer patients mainly uses CT (Computed Tomography). The images are used to establish a lymph node metastasis prediction model through radiomics methods. However, since radiomics methods mainly use manually designed feature extraction operators to extract feature information from CT images, and then use machine learning methods to establish a prediction model, this manual feature extraction method leads to limited model performance.

[0004] While some researchers have proposed auxiliary prediction methods that combine imaging and pathology images, most focus on decision-level fusion. This involves generating predictions based on both imaging and pathology images, followed by a comprehensive analysis of these predictions. The typical approach is to assign different weights to each prediction. The performance of these combined imaging and pathology auxiliary diagnosis models is significantly influenced by the weight coefficients, making them highly dependent on design experience. Furthermore, because decision-level fusion strategies model the two modalities separately and simply combine the results, the model's generalization ability is relatively poor. When external data is introduced, the model's performance typically degrades significantly.

[0005] Summary of the Invention

[0006] The present application provides a lymph node metastasis staging prediction method, apparatus, computer device and storage medium, aiming to solve at least one of the above-mentioned technical problems in the prior art to a certain extent.

[0007] In order to solve the above problems, this application provides the following technical solutions:

[0008] A method for predicting lymph node metastasis staging, comprising:

[0009] Collect patients' CT images and digital pathology images;

[0010] Using a ResNet50 network and a ViT network to perform deep feature extraction on the CT image and the digital pathology image, respectively, to obtain a first feature vector of the CT image and a second feature vector of the digital pathology image;

[0011] Merging the first eigenvector and the second eigenvector to obtain a fused eigenvector;

[0012] The fused feature vector is input into a fully connected neural network, and the fully connected neural network outputs the patient's lymph node metastasis staging prediction result.

[0013] The technical solution adopted in the embodiment of the present application further includes: after collecting the patient's CT image and digital pathology image, it also includes:

[0014] Delineating the tumor lesion area on the CT image and the digital pathology image respectively, and marking the tumor lesion area;

[0015] Based on the delineation result of the tumor lesion area, respectively extracting the tumor lesion area of ​​the CT image and the digital pathology image to obtain a set number of first tumor images and second tumor images;

[0016] The first tumor image and the second tumor image are compressed respectively to obtain a first tensor of the CT image and a second tensor of the digital pathology image.

[0017] The technical solution adopted in the embodiment of the present application further includes: extracting the tumor lesion area from the CT image and the digital pathology image respectively includes:

[0018] Obtaining a slice of the CT image marked with a tumor lesion area, and completely enclosing the tumor lesion area in the slice into a square frame of a set size, where the side length of the square frame is equal to the maximum length of the tumor lesion area on the X or Y axis;

[0019] Extracting a set number of slices at equal intervals, and compressing each slice into a first tumor image of a set size;

[0020] All first tumor images are stitched into a first tensor of set size and saved in .npy format.

[0021] The technical solution adopted in the embodiment of the present application further includes: extracting the tumor lesion area from the CT image and the digital pathology image respectively further includes:

[0022] The tumor lesion area of ​​the digital pathology image is magnified, and a sliding window of a set size is designed. The sliding window is sequentially slid from left to right and from top to bottom within the magnified tumor lesion area. With each sliding step, pixel information covered by the sliding window is extracted as a patch image and saved;

[0023] Compress each patch image into a tumor image of a set size, and randomly select a set number of tumor images from all compressed tumor images as the second tumor images;

[0024] The second tumor image is converted into a second tensor of a set size through matrix stitching and saved in .npy format.

[0025] The technical solution adopted in the embodiment of the present application also includes: using the ResNet50 network and the ViT network to extract deep features of the CT image and the digital pathology image respectively is specifically as follows:

[0026] The first tensor and the second tensor are respectively input into a pre-trained ResNet50 network and a ViT network, and a first eigenvector of the CT image and a second eigenvector of the digital pathology image are respectively output through the ResNet50 network and the ViT network.

[0027] The technical solution adopted in the embodiment of the present application also includes: the network weights of the ResNet50 network are obtained by pre-training a data set containing 1,000 CT images, and the output result is a first eigenvector of 1×2048 dimensions; the network weights of the ViT network are obtained by pre-training a data set of 10,000 patch images, and the output result of the ViT network is a second eigenvector of 1×768 dimensions.

[0028] The technical solution adopted by the embodiment of the present application further includes: after inputting the fused feature vector into a fully connected neural network and outputting the patient's lymph node metastasis staging prediction result through the fully connected neural network, the following steps are further included:

[0029] The weight coefficients of the fully connected layers in the ResNet50 network, the ViT network, and the fully connected neural network are iteratively trained to obtain a lymph node metastasis staging prediction model with optimal network parameters.

[0030] Another technical solution adopted in the embodiment of the present application is: a lymph node metastasis staging prediction device, comprising:

[0031] Image collection module: used to collect patients' CT images and digital pathology images;

[0032] Feature extraction module: used to perform deep feature extraction on the CT image and the digital pathology image using a ResNet50 network and a ViT network, respectively, to obtain a first feature vector of the CT image and a second feature vector of the digital pathology image;

[0033] Feature fusion module: used for merging the first feature vector and the second feature vector to obtain a fused feature vector;

[0034] Prediction module: used to input the fused feature vector into a fully connected neural network, and output the patient's lymph node metastasis staging prediction result through the fully connected neural network.

[0035] Another technical solution adopted by the embodiment of the present application is: a computer device, the computer device comprising a processor and a memory coupled to the processor, wherein:

[0036] The memory stores program instructions for implementing the lymph node metastasis staging prediction method;

[0037] The processor is configured to execute the program instructions stored in the memory to control a lymph node metastasis staging prediction method.

[0038] Another technical solution adopted by the embodiment of the present application is: a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute the lymph node metastasis staging prediction method.

[0039] Compared with the prior art, the beneficial effects of the embodiments of the present application are as follows: the lymph node metastasis staging prediction method, apparatus, computer equipment and storage medium of the embodiments of the present application propose a deep feature fusion strategy of CT images and digital pathology images to construct a lymph node metastasis staging prediction model, which not only can utilize the macroscopic phenotypic information of CT images and the microscopic cell morphology information of digital pathology images, but also give full play to the correlation between the two image data to construct a nonlinear mapping relationship model, thereby improving the prediction accuracy of the lymph node metastasis staging prediction model while also improving the generalization of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] FIG1 is a flow chart of a method for predicting lymph node metastasis staging according to a first embodiment of the present application;

[0041] FIG2 is a flow chart of a method for predicting lymph node metastasis staging according to a second embodiment of the present application;

[0042] FIG3 is a schematic diagram of the structure of a lymph node metastasis staging prediction device according to an embodiment of the present application;

[0043] FIG4 is a schematic diagram of the structure of a computer device according to an embodiment of the present application;

[0044] FIG5 is a schematic diagram of the structure of a storage medium according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0046] The terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of such features. In the description of this application, "multiple" means at least two, for example, two, three, etc., unless otherwise specifically defined. All directional indications in the embodiments of this application (such as up, down, left, right, front, back...) are only used to explain the relative positional relationship, movement, etc. between the components under a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications also change accordingly. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or computer device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products, or computer devices.

[0047] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0048] Specifically, please refer to FIG1 , which is a flow chart of the lymph node metastasis staging prediction method of the first embodiment of the present application. The lymph node metastasis staging prediction method of the first embodiment of the present application includes the following steps:

[0049] S100: Collect CT images and digital pathology images of patients;

[0050] S110: Using a ResNet50 network and a ViT network to perform deep feature extraction on the CT image and the digital pathology image, respectively, to obtain a first feature vector of the CT image and a second feature vector of the digital pathology image;

[0051] S120: Merge the first eigenvector and the second eigenvector to obtain a fused eigenvector;

[0052] S130: Inputting the fused feature vector into a fully connected neural network, and outputting the patient's lymph node metastasis staging prediction result through the fully connected neural network.

[0053] Please refer to FIG2 , which is a flow chart of a method for predicting lymph node metastasis stages according to a second embodiment of the present application. The method for predicting lymph node metastasis stages according to the second embodiment of the present application comprises the following steps:

[0054] S200: Collecting CT images and digital pathology images of patients;

[0055] In this step, after collecting the patient's CT images and digital pathology images, the images need to be saved in a predetermined format. Specifically, the CT images are uniformly saved as .nii format files, and the digital pathology images (Whole Slide Image, abbreviated as WSI) are uniformly saved as standard svs format files.

[0056] S210: outlining the tumor lesion region (region of interest, referred to as ROI) of the CT image and the digital pathology image respectively, and marking the tumor lesion region;

[0057] In this step, because some slices in the CT image contain tumor lesions while others do not, a radiologist needs to manually read each slice and delineate the tumor lesion area using ITK-SNAP. When delineating the ROI in digital pathology images, the pathologist needs to carefully delineate the tumor cell clusters and save the delineated label file as an .xml file.

[0058] S220: Based on the tumor lesion region delineation result, extract the tumor lesion regions from the CT image and the digital pathology image respectively to obtain a set number of first tumor images and second tumor images, and compress the first tumor images and the second tumor images respectively to obtain a first tensor of the CT image and a second tensor of the digital pathology image;

[0059] In this step, the specific method for extracting and compressing the tumor lesion area of ​​the CT image is as follows: To ensure that only tumor tissue in the CT image is used, a slice with the tumor lesion area marked in the CT image is obtained, and the tumor lesion area in the slice is completely enclosed in a square frame. Then, 5 slices are extracted at equal intervals, and the 5 slices are compressed into a first tumor image of 224×224 pixels. The 5 first tumor image matrices are then spliced ​​into a 5×224×224 tensor and saved in .npy format. The size of the square frame can be adjusted according to the size of the tumor lesion area, and the side length of the square frame is equal to the maximum length of the tumor lesion area on the X or Y axis. The number of slices extracted and the pixel size of the first tumor image can be set according to the actual application scenario.

[0060] Furthermore, the slice extraction method is as follows:

[0061] 1) Determine that the slice marked with ROI in the CT image has N layers;

[0062] 2) Set the number of slice intervals to be extracted k = (N+1) / 5, where k is an integer;

[0063] 3) Extract 5 slices at the set interval k, and number the layers of each slice as follows: (N+1) / 2-2k, (N+1) / 2-k, (N+1) / 2, (N+1) / 2+k, (N+1) / 2+2k, where (N+1) / 2 is an integer.

[0064] 4) Compress the tumor lesion areas within the square boxes of the five slices into first tumor images of 224×224 pixels respectively, merge the five 224×224 first tumor image matrices into a first tensor of 5×224×224, and save it in .npy format.

[0065] In an embodiment of the present application, the method for cutting and compressing the tumor lesion area of ​​the digital pathology image is as follows: the outlined tumor lesion area is magnified 400 times, and a sliding window with a sliding window size of 1024×1024 is designed. The sliding window is slid in the magnified tumor lesion area in order from left to right and from top to bottom, with a sliding step of 1024. The pixel information covered by the sliding window is extracted as a patch image and saved for each sliding step, thereby obtaining several patch images. Each patch image is then compressed into a tumor image of 224×224 pixels. The compressed tumor image can be represented as 3×224×224. Five tumor images are randomly selected from all the compressed tumor images as second tumor images. The five second tumor images are converted into a second tensor of 15×224×224 by matrix splicing and saved in .npy format. Among them, parameters such as the magnification of the tumor lesion area, the sliding window size, the compressed size of the patch image, and the number of extracted second tumor images can be set according to the actual application scenario.

[0066] S230: Inputting the first tensor and the second tensor into a pre-trained ResNet50 network and a ViT network for deep feature extraction, and outputting a first feature vector of the CT image and a second feature vector of the digital pathology image through the ResNet50 network and the ViT network respectively;

[0067] In this step, the network weights of the ResNet50 network are obtained by pre-training on a dataset containing 1000 CT images. In this embodiment of the present application, the standard ResNet50 network is modified by removing the last fully connected layer of the standard ResNet50 network, and the output result is a first eigenvector of 1×2048 dimensions. Similarly, the network weights of the ViT network are obtained by pre-training on a dataset of 10,000 patch images. In this embodiment of the present application, the standard ViT network is modified by deleting the last MLP layer, so the output result of the ViT network is a second eigenvector of 1×768 dimensions.

[0068] S240: Merge the first eigenvector and the second eigenvector to obtain a fused eigenvector;

[0069] In this step, after extracting the first eigenvector of the CT image and the second eigenvector of the digital pathology image respectively, the first eigenvector and the second eigenvector are fused by vector splicing to obtain a 1×2816-dimensional fused feature vector.

[0070] S250: inputting the fused feature vector into a fully connected neural network, and outputting the patient's lymph node metastasis staging prediction result through the fully connected neural network;

[0071] In this step, after feature fusion is completed, the fused feature vector is input into a fully connected neural network. The input layer of the fully connected neural network includes 2816 neurons, the output layer includes 5 neurons, and there is a hidden layer between the input and output layers. The number of neurons in the hidden layer is 1408. The final output is the patient's lymph node metastasis staging prediction result (N0, N1, ..., N3b).

[0072] S260: Iteratively train the weight coefficients of the fully connected layers in the ResNet50 network, ViT network, and fully connected neural network to obtain a lymph node metastasis staging prediction model with optimal network parameters;

[0073] In this step, the weight coefficients of the fully connected layers in the ResNet50 network, ViT network, and fully connected neural network were iteratively trained. The training parameters were set to a learning rate of 0.001 and an adaptive gradient optimization algorithm was used in the optimizer until the loss function stabilized. That is, when the variance of the loss value in the last 20 rounds of iterative training was less than 0.05, the iterative training process was terminated to obtain a lymph node metastasis staging prediction model with optimal network parameters.

[0074] Based on the above, the lymph node metastasis staging prediction method of the second embodiment of the present application proposes a deep feature fusion strategy of CT images and digital pathology images to construct a lymph node metastasis staging prediction model. It can not only utilize the macroscopic phenotypic information of CT images and the microscopic cell morphology information of digital pathology images, but also give full play to the correlation between the two image data to construct a nonlinear mapping relationship model, thereby improving the prediction accuracy of the lymph node metastasis staging prediction model while improving the generalization of the model.

[0075] Please refer to FIG3 , which is a schematic diagram of the structure of the lymph node metastasis staging prediction device according to an embodiment of the present application. The lymph node metastasis staging prediction device 40 according to the embodiment of the present application comprises:

[0076] Image collection module 41: used to collect CT images and digital pathology images of patients;

[0077] Feature extraction module 42: configured to perform deep feature extraction on the CT image and the digital pathology image using a ResNet50 network and a ViT network, respectively, to obtain a first feature vector of the CT image and a second feature vector of the digital pathology image;

[0078] Feature fusion module 43: used for merging the first feature vector and the second feature vector to obtain a fused feature vector;

[0079] Prediction module 44: used to input the fused feature vector into a fully connected neural network, and output the patient's lymph node metastasis staging prediction result through the fully connected neural network.

[0080] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0081] The device provided in the embodiment of the present application can be applied in the aforementioned method embodiment. For details, please refer to the description of the aforementioned method embodiment, which will not be repeated here.

[0082] Please refer to FIG4 , which is a schematic diagram of the computer device structure of an embodiment of the present application. The computer device 50 includes:

[0083] A memory 51 storing executable program instructions;

[0084] a processor 52 connected to the memory 51;

[0085] The processor 52 is used to call the executable program instructions stored in the memory 51 and perform the following steps: collect the patient's CT image and digital pathology image; use the ResNet50 network and the ViT network to perform deep feature extraction on the CT image and the digital pathology image respectively to obtain a first eigenvector of the CT image and a second eigenvector of the digital pathology image; merge the first eigenvector and the second eigenvector to obtain a fused eigenvector; input the fused eigenvector into a fully connected neural network, and output the patient's lymph node metastasis staging prediction result through the fully connected neural network.

[0086] The processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip having signal processing capabilities. The processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor or any conventional processor.

[0087] Please refer to Figure 5, which is a schematic diagram of the structure of the storage medium of an embodiment of the present application. The storage medium of the embodiment of the present application stores program instructions 61 that can implement the following steps: collecting CT images and digital pathology images of patients; using the ResNet50 network and the ViT network to perform deep feature extraction on the CT images and digital pathology images respectively to obtain a first feature vector of the CT image and a second feature vector of the digital pathology image; merging the first feature vector and the second feature vector to obtain a fused feature vector; inputting the fused feature vector into a fully connected neural network, and outputting the patient's lymph node metastasis staging prediction result through the fully connected neural network. Among them, the program instructions 61 can be stored in the above-mentioned storage medium in the form of a software product, including several instructions for causing a computer device (which can be a personal computer, server, or network computer device, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program instructions, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a terminal computer device such as a computer, a server, a mobile phone, or a tablet. Among them, the server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), as well as big data and artificial intelligence platforms.

[0088] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0089] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. The above is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for predicting lymph node metastasis staging, characterized in that: include: Collect CT images and digital pathology images of patients; Using a ResNet50 network and a ViT network to perform deep feature extraction on the CT image and the digital pathology image, respectively, to obtain a first feature vector of the CT image and a second feature vector of the digital pathology image; Merging the first feature vector and the second feature vector to obtain a fused feature vector; The fused feature vector is input into a fully connected neural network, and the lymph node metastasis staging prediction result of the patient is output through the fully connected neural network.

2. The method for predicting lymph node metastasis staging according to claim 1, characterized in that: After collecting the patient's CT image and digital pathology image, the method further includes: Delineating the tumor lesion areas of the CT image and the digital pathology image respectively, and marking the tumor lesion areas; Based on the delineation result of the tumor lesion area, respectively extracting the tumor lesion area of ​​the CT image and the digital pathology image to obtain a set number of first tumor images and second tumor images; The first tumor image and the second tumor image are compressed respectively to obtain a first tensor of the CT image and a second tensor of the digital pathology image.

3. The method for predicting lymph node metastasis staging according to claim 2, characterized in that: The extracting of the tumor lesion area of ​​the CT image and the digital pathology image respectively comprises: Obtaining a slice with a tumor lesion region marked in the CT image, and completely including the tumor lesion region in the slice into a square frame of a set size, wherein the side length of the square frame is equal to the maximum length of the tumor lesion region on the X or Y axis; Extracting a set number of slices at equal intervals, and compressing each slice into a first tumor image of a set size; All first tumor images are concatenated into a first tensor of set size and saved in .npy format.

4. The method for predicting lymph node metastasis staging according to claim 2, characterized in that: The extracting of the tumor lesion area from the CT image and the digital pathology image respectively further comprises: The tumor lesion area of ​​the digital pathology image is enlarged, and a sliding window of a set size is designed, and the sliding window is sequentially slid from left to right and from top to bottom in the enlarged tumor lesion area, and pixel information covered by the sliding window is extracted as a patch image and saved at each sliding step; Compressing each patch image into a tumor image of a set size, and randomly selecting a set number of tumor images from all compressed tumor images as second tumor images; The second tumor image is converted into a second tensor of a set size by matrix stitching and saved in .npy format.

5. The method for predicting lymph node metastasis staging according to claim 3 or 4, characterized in that: The use of the ResNet50 network and the ViT network to extract deep features from the CT image and the digital pathology image is specifically as follows: The first tensor and the second tensor are respectively input into a pre-trained ResNet50 network and a ViT network, and a first feature vector of a CT image and a second feature vector of a digital pathology image are respectively output through the ResNet50 network and the ViT network.

6. The method for predicting lymph node metastasis staging according to claim 5, characterized in that: The network weights of the ResNet50 network are obtained by pre-training a data set containing 1000 CT images, and the output result is a first eigenvector of 1×2048 dimensions; the network weights of the ViT network are obtained by pre-training a data set of 10,000 patch images, and the output result of the ViT network is a second eigenvector of 1×768 dimensions.

7. The method for predicting lymph node metastasis staging according to claim 6, characterized in that: After inputting the fused feature vector into a fully connected neural network and outputting the patient's lymph node metastasis staging prediction result through the fully connected neural network, the method further includes: The weight coefficients of the fully connected layers in the ResNet50 network, the ViT network, and the fully connected neural network are iteratively trained to obtain a lymph node metastasis staging prediction model with optimal network parameters.

8. A lymph node metastasis staging prediction device, characterized in that: include: Image collection module: used to collect CT images and digital pathology images of patients; Feature extraction module: used to perform deep feature extraction on the CT image and the digital pathology image using a ResNet50 network and a ViT network, respectively, to obtain a first feature vector of the CT image and a second feature vector of the digital pathology image; Feature fusion module: used for merging the first feature vector and the second feature vector to obtain a fused feature vector; Prediction module: used for inputting the fused feature vector into a fully connected neural network, and outputting the patient's lymph node metastasis staging prediction result through the fully connected neural network.

9. A computer device, characterized in that: The computer device includes a processor and a memory coupled to the processor, wherein: The memory stores program instructions for implementing the lymph node metastasis staging prediction method according to any one of claims 1 to 7; The processor is used to execute the program instructions stored in the memory to control a lymph node metastasis staging prediction method.

10. A storage medium, characterized in that: Program instructions executable by a processor are stored, and the program instructions are used to execute the lymph node metastasis staging prediction method according to any one of claims 1 to 7.

Images