Disease diagnosis system, device and storage medium based on ileocecal ulcer

Abstract

The present application discloses a disease diagnosis system, device and storage medium based on ileocecal ulcer, including: a preprocessing module, a disease diagnosis module and an output module, wherein the disease diagnosis module is used to perform the following operations: using a first encoder to extract features from a first CT feature map to generate a first CT feature vector, using a second encoder to extract features from a second CT feature map to generate a second CT feature vector; using a first fully connected layer to perform feature fusion on the first CT feature vector and the second CT feature vector to generate a first fused feature vector; using a third encoder to perform feature extraction on an endoscopic feature map to generate an endoscopic feature vector; and performing feature fusion and classification on the first fused feature vector, the endoscopic feature vector and the test index feature vector corresponding to the test index data, and determining the diagnosis information corresponding to the patient.

Description

Technical Field

[0001] The present application relates to the field of medical automated diagnosis technology, and in particular to a disease diagnosis system, device and storage medium based on ileocecal ulcer. Background Art

[0002] Ileocecal ulcer is a common manifestation of digestive system diseases. And because many diseases such as Crohn's disease, intestinal tuberculosis and intestinal Behcet's disease can manifest as ileocecal ulcer, it is possible to diagnose the ileocecal ulcer to identify the disease suffered by the patient. However, diagnosing patients based on ileocecal ulcer has always been a major problem in the international medical field.

[0003] Although there are specific examples of using deep learning models to assist in the identification of Crohn's disease, intestinal tuberculosis, and intestinal Behcet's disease, the above-mentioned prior art is only based on the patient's colonoscopy image data and uses deep learning models to complete the classification task. That is, the existing use of deep learning models to assist in the identification of the above three types of digestive system diseases only uses information from a single modality (i.e., only based on the patient's colonoscopy image data). For image data of atypical lesions, only based on information from a single modality, the accuracy of the final diagnosis is often reduced (i.e., it is impossible to accurately identify which disease is Crohn's disease, intestinal tuberculosis, or intestinal Behcet's disease), thereby affecting the doctor's diagnosis and treatment of the patient's disease.

[0004] With regard to the technical problem that the existing method of using deep learning models to assist in the identification of digestive system diseases in the above-mentioned prior art only utilizes information from a single modality, which may lead to a reduction in the accuracy of the final diagnosis, no effective solution has been proposed so far. Summary of the invention

[0005] The embodiments of the present disclosure provide a disease diagnosis system, device and storage medium based on ileocecal ulcer, so as to at least solve the technical problem that the existing method of using deep learning models to assist in the identification of digestive system diseases in the prior art only utilizes information from a single modality, which may lead to a reduction in the accuracy of the final diagnosis.

[0006] According to one aspect of an embodiment of the present disclosure, a disease diagnosis system based on ileocecal ulcer is provided, comprising: a preprocessing module, a disease diagnosis module and an output module, wherein the preprocessing module is connected to the disease diagnosis module and is used to collect first disease diagnosis data corresponding to the patient, preprocess the first disease diagnosis data, and send second disease diagnosis data generated after the preprocessing to the disease diagnosis module, wherein the second disease diagnosis data comprises a first CT feature map, a second CT feature map, an endoscopic feature map and a test index feature vector; the output module is used to output diagnosis information determined by the disease diagnosis module, wherein the diagnosis information is used to indicate that the patient suffers from Crohn's disease, intestinal tuberculosis or intestinal Behcet's disease, and wherein the disease diagnosis module is configured to perform The following operations: using the first encoder to extract features from the first CT feature map to generate a first CT feature vector, using the second encoder to extract features from the second CT feature map to generate a second CT feature vector, wherein the body position of the patient corresponding to the first CT feature map is different from the body position of the patient corresponding to the second CT feature map; using the first fully connected layer to fuse the first CT feature vector and the second CT feature vector to generate a first fused feature vector; using the third encoder to extract features from the endoscope feature map to generate an endoscope feature vector; and performing feature fusion and classification on the first fused feature vector, the endoscope feature vector, and the test index feature vector corresponding to the test index data, and determining the diagnostic information corresponding to the patient.

[0007] According to another aspect of the embodiment of the present disclosure, a storage medium is also provided, the storage medium includes a stored program, wherein when the program is running, the processor executes the following steps: using a first encoder to extract features from a first CT feature map to generate a first CT feature vector, using a second encoder to extract features from a second CT feature map to generate a second CT feature vector, wherein the body position of the patient corresponding to the first CT feature map is different from the body position of the patient corresponding to the second CT feature map; using a first fully connected layer to perform feature fusion on the first CT feature vector and the second CT feature vector to generate a first fused feature vector; using a third encoder to perform feature extraction on an endoscopic feature map to generate an endoscopic feature vector; and performing feature fusion and classification on the first fused feature vector, the endoscopic feature vector and the test index feature vector, and determining the diagnostic information corresponding to the patient.

[0008] According to another aspect of the embodiments of the present disclosure, a disease diagnosis device based on ileocecal ulcer is also provided, including: a CT feature vector generation module, used to use a first encoder to extract features from a first CT feature map to generate a first CT feature vector, and use a second encoder to extract features from a second CT feature map to generate a second CT feature vector, wherein the body position of the patient corresponding to the first CT feature map is different from the body position of the patient corresponding to the second CT feature map; a fusion feature vector generation module, used to use a first fully connected layer to perform feature fusion on the first CT feature vector and the second CT feature vector, thereby generating a first fused feature vector; an endoscopic feature vector generation module, used to use a third encoder to perform feature extraction on the endoscopic feature map, thereby generating an endoscopic feature vector; and a diagnosis information determination module, used to perform feature fusion and classification on the first fusion feature vector, the endoscopic feature vector and the test index feature vector corresponding to the test index data, and determine the diagnosis information corresponding to the patient.

[0009] According to another aspect of the embodiments of the present disclosure, a disease diagnosis device based on ileocecal ulcer is also provided, including: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: using a first encoder to extract features from a first CT feature map to generate a first CT feature vector, using a second encoder to extract features from a second CT feature map to generate a second CT feature vector, wherein the body position of the patient corresponding to the first CT feature map is different from the body position of the patient corresponding to the second CT feature map; using a first fully connected layer to perform feature fusion on the first CT feature vector and the second CT feature vector to generate a first fused feature vector; using a third encoder to perform feature extraction on an endoscopic feature map to generate an endoscopic feature vector; and performing feature fusion and classification on the first fused feature vector, the endoscopic feature vector, and the test index feature vector corresponding to the test index data, and determining the diagnostic information corresponding to the patient.

[0010] The present application provides a disease diagnosis system based on ileocecal ulcer. The system includes a preprocessing module, a disease diagnosis module and an output module. The disease diagnosis module can also perform the following operations: first, a first encoder is used to extract features from a first CT feature map to generate a first CT feature vector, and a second encoder is used to extract features from a second CT feature map to generate a second CT feature vector. Then, a first fully connected layer is used to perform feature fusion on the first CT feature vector and the second CT feature vector to generate a first fused feature vector. Thereafter, a third encoder is used to extract features from an endoscopic feature map to generate an endoscopic feature vector. Finally, the first fused feature vector, the endoscopic feature vector and the test index feature vector are feature fused and classified to determine the diagnostic information corresponding to the patient.

[0011] Since the disease diagnosis module in the present application is based on the CT feature map corresponding to the patient, the endoscopic feature map corresponding to the patient, and the test index feature vector corresponding to the patient, and ultimately determines the diagnostic information corresponding to the patient, compared to the existing method of only using a single modality of data to determine the diagnostic information corresponding to the patient, the present application uses multimodal data as the standard for judging the patient's disease, so that the diagnostic information ultimately determined is more accurate.

[0012] This solves the technical problem that the existing method of using deep learning models to assist in the identification of digestive system diseases in the prior art only uses information from a single modality, which may lead to a reduction in the accuracy of the final diagnosis.

[0013] In addition, since the present application first uses the first encoder to extract features from the first CT feature map to generate a first CT feature vector, and uses the second encoder to extract features from the second CT feature map to generate a second CT feature vector, the first fused feature vector used to finally determine the diagnostic information corresponding to the patient contains both information corresponding to the first CT feature map and information corresponding to the second CT feature map. And since the body position of the patient corresponding to the first CT feature map is different from the body position of the patient corresponding to the second CT feature map, compared to using only a single body position CT feature map to determine the diagnostic information corresponding to the patient, the first fused feature vector in the present application contains more information, so that the diagnostic information finally determined is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0015] Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to Embodiment 1 of the present application;

[0016] Figure 2 is a schematic diagram of a disease diagnosis system based on ileocecal ulcer according to Example 1 of the present application;

[0017] Figure 3 This is a flow chart of the disease diagnosis method based on ileocecal ulcer according to Example 1 of the present application;

[0018] Figure 4 is a schematic diagram of a disease diagnosis module according to Example 1 of the present application;

[0019] Figure 5A is a schematic diagram of the first CT characteristic image according to Example 1 of the present application;

[0020] Figure 5B is a schematic diagram of the second CT characteristic map according to Example 1 of the present application;

[0021] Fig. 6A is a schematic diagram of endoscopic image data according to Example 1 of the present application;

[0022] Figure 6B is a schematic diagram of an endoscope characteristic map according to Example 1 of the present application;

[0023] Figure 7 This is an overall flow chart of the disease diagnosis method based on ileocecal ulcer according to Example 1 of the present application;

[0024] Figure 8 is a schematic diagram of a disease diagnosis device based on ileocecal ulcer according to Example 2 of the present application;

[0025] Fig. 9 This is a schematic diagram of a disease diagnosis device based on ileocecal ulcer according to Example 3 of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0028] Example 1

[0029] According to this embodiment, a method embodiment of disease diagnosis based on ileocecal ulcer is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0030] The method embodiment provided in this embodiment can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computing device for implementing a disease diagnosis method based on ileocecal ulcer. Figure 1 As shown, the computing device may include one or more processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, the transmission device, and the input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0031] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computing device. As involved in the embodiments of the present disclosure, the data processing circuitry acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0032] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the disease diagnosis method based on ileocecal ulcer in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the disease diagnosis method based on ileocecal ulcer of the above-mentioned application is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0033] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computing device. In one example, the transmission device includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0034] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of the computing device.

[0035] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing devices described above.

[0036] Figure 2 Schematic diagram of a disease diagnosis system based on ileocecal ulcer according to this embodiment. Figure 2As shown, the system includes: a preprocessing module, a disease diagnosis module and an output module. The preprocessing module is connected to the disease diagnosis module, and is used to preprocess the first disease diagnosis data, and send the second disease diagnosis data generated after the preprocessing to the disease diagnosis module. The disease diagnosis module is used to determine the diagnosis information corresponding to the patient based on the second disease diagnosis data. The diagnosis information corresponding to the patient is used to indicate that the patient suffers from Crohn's disease, intestinal tuberculosis or intestinal Behcet's disease. The output module is connected to the disease diagnosis module, and is used to output the diagnosis information.

[0037] It should be noted that the preprocessing module, disease diagnosis module and output module in the system can all be applied to the hardware structure described above.

[0038] Under the above operating environment, according to the first aspect of this embodiment, a disease diagnosis method based on ileocecal ulcer is provided. The method comprises: Figure 2 The disease diagnosis system shown in is implemented. Figure 3 A schematic diagram showing the process of the method is shown in FIG. Figure 3 As shown, the method includes:

[0039] S302: extracting features from the first CT feature map using a first encoder to generate a first CT feature vector, and extracting features from the second CT feature map using a second encoder to generate a second CT feature vector, wherein the body position of the patient corresponding to the first CT feature map is different from the body position of the patient corresponding to the second CT feature map;

[0040] S304: using the first fully connected layer to perform feature fusion on the first CT feature vector and the second CT feature vector, thereby generating a first fused feature vector;

[0041] S306: extracting features from the endoscope feature map using a third encoder to generate an endoscope feature vector; and

[0042] S308: Perform feature fusion and classification on the first fusion feature vector, the endoscope feature vector, and the test index feature vector, and determine the diagnosis information corresponding to the patient.

[0043] Specifically, Figure 4 Schematic diagram of the disease diagnosis module according to Example 1 of the present application. Figure 4As shown, first, the disease diagnosis module receives the first CT feature map and the second CT feature map sent by the preprocessing module. The body position of the patient corresponding to the first CT feature map is different from the body position of the patient corresponding to the second CT feature map. Furthermore, in order to display the entire organ, multiple continuous layer images are required. By using the image reconstruction program on the CT device, the layer images of the coronal plane and the sagittal plane can also be reconstructed, and the relationship between the organ and the lesion can be viewed from multiple angles. Therefore, this embodiment provides two CT images corresponding to different body positions of the patient. Figure 5A It is a schematic diagram of the first CT feature map described in Example 1 of the present application. Figure 5A The first CT characteristic map shown may be, for example, a CT characteristic map of a cross section of the patient. Figure 5B It is a schematic diagram of the second CT feature map described in Example 1 of the present application. Figure 5B The second CT characteristic map shown may be, for example, a CT characteristic map of a coronal plane of the patient.

[0044] Then, the disease diagnosis module uses the first encoder to extract features from the first CT feature map to generate a first CT feature vector, and uses the second encoder to extract features from the second CT feature map to generate a second CT feature vector (S302). Figure 4 As shown, after the disease diagnosis module receives the first CT feature map, it inputs the first CT feature map into the first encoder, and uses the residual network of the first encoder to extract features from the first CT feature map, thereby generating a first CT feature vector. At the same time, after the disease diagnosis module receives the second CT feature map, it inputs the second CT feature map into the second encoder, and uses the residual network of the second encoder to extract features from the second CT feature map, thereby generating a second CT feature vector. The first CT feature vector and the second CT feature vector contain information corresponding to different patient positions.

[0045] Afterwards, the disease diagnosis module uses the first fully connected layer to perform feature fusion on the first CT feature vector and the second CT feature vector, thereby generating a first fused feature vector (S304). Figure 4As shown, when the first encoder generates the first CT feature vector based on the first CT feature map, and the second encoder generates the second CT feature vector based on the second CT feature map, the disease diagnosis module uses the first fully connected layer to fuse the first CT feature vector and the second CT feature vector and realize modal alignment, thereby obtaining a complete CT feature vector (i.e., the first fused feature vector). Among them, since the first fused feature vector contains both the information of the patient's body position corresponding to the first CT feature map and the information of the patient's body position corresponding to the second CT feature map, the information contained in the first fused feature vector is relatively rich. Therefore, compared with the existing CT feature map that only determines the patient's diagnostic information based on a single body position corresponding to the patient, the diagnostic information generated based on the first fused feature vector in this application is more accurate.

[0046] At the same time, the disease diagnosis module receives the endoscopic feature map sent by the preprocessing module, and uses the third encoder to extract features from the endoscopic feature map, thereby generating an endoscopic feature vector (S306). Figure 4 As shown, after receiving the endoscopic feature map, the disease diagnosis module inputs the endoscopic feature map into the third encoder, and uses the residual network of the third encoder to extract features from the endoscopic feature map, thereby generating an endoscopic feature vector.

[0047] Finally, the disease diagnosis module performs feature fusion and classification on the first fused feature vector, the endoscopic feature vector and the test index feature vector, and determines the diagnostic information corresponding to the patient (S308). Specifically, when the disease diagnosis module determines the first fused feature vector and the endoscopic feature vector, the third fully connected layer is used to perform feature fusion on the first fused feature vector, the endoscopic feature vector and the test index feature vector corresponding to the test index data to generate a second feature fusion vector. After that, the disease diagnosis uses the softmax classifier to classify the second feature fusion vector, thereby generating probability information corresponding to Crohn's disease, probability information corresponding to intestinal tuberculosis and probability information corresponding to intestinal Behcet's disease. Finally, the disease diagnosis module determines the patient's diagnostic information based on the probability information corresponding to Crohn's disease, the probability information corresponding to intestinal tuberculosis and the probability information corresponding to intestinal Behcet's disease. That is, the patient suffers from Crohn's disease, intestinal tuberculosis or intestinal Behcet's disease. The above content will be described in detail later, so it will not be repeated here.

[0048] As described in the background technology, although there are specific examples of using deep learning models to assist in the identification of Crohn's disease, intestinal tuberculosis and intestinal Behcet's disease, the above-mentioned prior art is only based on the patient's colonoscopy image data and uses deep learning models to complete the classification task. That is, the existing use of deep learning models to assist in the identification of the above three types of digestive system diseases only uses information from a single modality (i.e., only based on the patient's colonoscopy image data). For image data of atypical lesions, only based on information from a single modality, the accuracy of the final diagnosis is often reduced (i.e., it is impossible to accurately identify which disease is Crohn's disease, intestinal tuberculosis and intestinal Behcet's disease), thereby affecting the doctor's diagnosis and treatment of the patient's disease.

[0049] In view of this, the present application provides a disease diagnosis system based on ileocecal ulcer. And because the disease diagnosis module in the present application is based on the CT feature map corresponding to the patient, the endoscopic feature map corresponding to the patient, and the test index feature vector corresponding to the patient, and finally determines the diagnosis information corresponding to the patient, compared with the existing method of determining the diagnosis information corresponding to the patient using only a single modality of data, the present application uses multimodal data as the standard for judging the disease suffered by the patient, so that the diagnosis information finally determined is more accurate.

[0050] This solves the technical problem that the existing method of using deep learning models to assist in the identification of digestive system diseases in the prior art only uses information from a single modality, which may lead to a reduction in the accuracy of the final diagnosis.

[0051] In addition, since the present application first uses the first encoder to extract features from the first CT feature map to generate a first CT feature vector, and uses the second encoder to extract features from the second CT feature map to generate a second CT feature vector, the first fused feature vector used to finally determine the diagnostic information corresponding to the patient contains both information corresponding to the first CT feature map and information corresponding to the second CT feature map. And since the body position of the patient corresponding to the first CT feature map is different from the body position of the patient corresponding to the second CT feature map, compared to using only a single body position CT feature map to determine the diagnostic information corresponding to the patient, the first fused feature vector in the present application contains more information, so that the diagnostic information finally determined is more accurate.

[0052] Optionally, a second fully connected layer is arranged between the first encoder and the first fully connected layer and between the second encoder and the first fully connected layer, and the second fully connected layer is configured to perform the following operations: unify the vector dimension of the first CT feature vector to correspond to the input dimension of the first fully connected layer; and unify the vector dimension of the second CT feature vector to correspond to the input dimension of the first fully connected layer.

[0053] Specifically, refer to Figure 4 As shown, a second fully connected layer is provided between the first encoder and the first fully connected layer of the disease diagnosis module, and between the second encoder and the first fully connected layer. When the disease diagnosis module inputs the first CT feature vector to the second fully connected layer, the second fully connected layer can unify the vector dimension of the first CT feature vector to correspond to the input dimension of the first fully connected layer. When the disease diagnosis module inputs the second CT feature vector to the second fully connected layer, the second fully connected layer can unify the vector dimension of the second CT feature vector to correspond to the input dimension of the first fully connected layer.

[0054] Therefore, when the second fully connected layer unifies the vector dimension of the first CT feature vector and the vector dimension of the second CT feature vector to correspond to the input dimension of the first fully connected layer, the first CT feature vector and the second CT feature vector are sent to the first fully connected layer, so as to facilitate the first fully connected layer to perform feature fusion processing on the first CT feature vector and the second CT feature vector.

[0055] Optionally, the operation of performing feature fusion and classification on the first fused feature vector, the endoscopic feature vector, and the test index feature vector, and determining the diagnostic information corresponding to the patient, includes: using the third fully connected layer to perform feature fusion on the first fused feature vector, the endoscopic feature vector, and the test index feature vector, and generate a second fused feature vector; using a softmax classifier to classify the second fused feature vector, thereby determining the probability information corresponding to Crohn's disease, intestinal tuberculosis, and intestinal Behcet's disease; and based on the probability information, determining the diagnostic information corresponding to the patient. Further optionally, a fourth fully connected layer is provided between the third fully connected layer and the first fully connected layer and between the third fully connected layer and the third encoder, and the fourth fully connected layer is configured to perform the following operations: unify the vector dimension of the first fused feature vector to correspond to the input dimension of the third fully connected layer; unify the vector dimension of the endoscopic feature vector to correspond to the input dimension of the third fully connected layer; and unify the vector dimension of the test index feature vector to correspond to the input dimension of the third fully connected layer.

[0056] Specifically, refer to Figure 4As shown, a fourth fully connected layer is provided between the third fully connected layer and the first fully connected layer, and between the third fully connected layer and the third encoder. When the first fully connected layer inputs the first fused feature vector to the fourth fully connected layer, the fourth fully connected layer can unify the vector dimension of the first fused feature vector to correspond to the input dimension of the third fully connected layer. When the third encoder inputs the endoscopic feature vector to the fourth fully connected layer, the fourth fully connected layer can unify the vector dimension of the endoscopic feature vector to correspond to the input dimension of the third fully connected layer. When the preprocessing module inputs the inspection index feature vector to the fourth fully connected layer, the fourth fully connected layer can unify the vector dimension of the inspection index feature vector to correspond to the input dimension of the third fully connected layer.

[0057] Thus, when the fourth fully connected layer unifies the vector dimension of the first fused feature vector, the vector dimension of the endoscopic feature vector, and the vector dimension of the inspection index feature vector to correspond to the input dimension of the third fully connected layer, the first fused feature vector, the endoscopic feature vector, and the inspection index feature vector are sent to the third fully connected layer, so as to facilitate the third fully connected layer to perform feature fusion processing on the first fused feature vector, the endoscopic feature vector, and the inspection index feature vector.

[0058] Furthermore, the third fully connected layer performs feature fusion on the received first fused feature vector, the endoscope feature vector, and the inspection index feature vector, so as to generate a second fused feature vector.

[0059] Afterwards, the third fully connected layer sends the generated second fused feature vector to the softmax classifier, and uses the softmax classifier to classify the second fused feature vector, thereby determining the probability information corresponding to Crohn's disease, the probability information corresponding to intestinal tuberculosis, and the probability information corresponding to intestinal Behcet's disease. For example, the probability information corresponding to Crohn's disease is 50%, the probability information corresponding to intestinal tuberculosis is 30%, and the probability information corresponding to intestinal Behcet's disease is 20%.

[0060] Finally, the disease diagnosis module determines the diagnosis information corresponding to the patient based on the probability information corresponding to Crohn's disease, the probability information corresponding to intestinal tuberculosis, and the probability information corresponding to intestinal Behcet's disease determined by the softmax classifier. For example, when the probability information corresponding to Crohn's disease is 50%, the probability information corresponding to intestinal tuberculosis is 30%, and the probability information corresponding to intestinal Behcet's disease is 20%, the disease suffered by the patient is Crohn's disease.

[0061] Therefore, the disease diagnosis module can accurately and efficiently determine which disease the patient suffers from, Crohn's disease, intestinal tuberculosis or Behcet's disease, based on the above-mentioned first fusion feature vector, endoscopic feature vector and test index feature vector, and the three aspects of data information corresponding to the patient, thereby facilitating subsequent doctors to provide targeted treatment to the patient.

[0062] Optionally, the preprocessing module collects first disease diagnosis data corresponding to the patient, preprocesses the first disease diagnosis data, and sends the second disease diagnosis data generated after the preprocessing to the disease diagnosis module, including: the preprocessing module collects endoscopic image data, and preprocesses the endoscopic image data to generate an endoscopic feature map; and the preprocessing module collects test index data, and preprocesses the test index data to generate a test index feature vector.

[0063] Specifically, refer to Figure 2 as well as Figure 4 As shown, first, the preprocessing module collects endoscopic image data corresponding to the patient, and preprocesses the endoscopic image data to generate an endoscopic feature map. Fig. 6A It is a schematic diagram of the endoscopic image data described in Example 1 of the present application. Figure 6B is a schematic diagram of an endoscope characteristic map according to Example 1 of the present application. Fig. 6A and Figure 6B As shown, in the preprocessing module, Fig. 6A After preprocessing the endoscopic image data shown in Figure 6B The above contents will be described in detail later, so they will not be repeated here.

[0064] Then, the preprocessing module collects the test index data corresponding to the patient, and preprocesses the test index data to generate a test index feature vector. Among them, the test index data can include 65 test indicators such as the patient's gender, the patient's age, and whether the patient has a fever. Thus, the disease diagnosis module can determine the disease suffered by the patient from multiple angles based on multiple test index data. The above content will be described later, so it will not be repeated here.

[0065] Optionally, the preprocessing module collects endoscopic image data and preprocesses the endoscopic image data to generate an endoscopic feature map, including: the preprocessing module uses image binarization to generate a grayscale image corresponding to the endoscopic image data; and the preprocessing module determines the foreground area of ​​the endoscopic image data and selects the foreground area as the endoscopic feature map.

[0066] Specifically, refer to Figure 2 , Figure 4 , Fig. 6A as well as Figure 6B As shown, when receiving the endoscopic image data, the preprocessing module performs image binarization processing on the endoscopic image data, thereby generating a grayscale image corresponding to the endoscopic image data. For example, the preprocessing module sets the grayscale value of the pixel point on the endoscopic image data to 0 or 255, thereby generating a grayscale image corresponding to the endoscopic image data.

[0067] The preprocessing module then determines the foreground area corresponding to the grayscale image of the endoscopic image data and selects the foreground area as the endoscopic feature map.

[0068] refer to Fig. 6A As shown, since the right half of the endoscopic image data (i.e., the original endoscopic image) is an image of the intestines related to diagnosis, and the left side is text information. When the endoscopic image data is input into the disease diagnosis module, the text information on the left side of the endoscopic image data will be regarded as noise, thereby affecting the final diagnosis result. Therefore, in this application, only the right half of the intestinal images related to diagnosis are retained as the input of the disease diagnosis module, and the left half that may affect the final diagnosis result of the disease diagnosis module is removed.

[0069] Thus, the preprocessing module determines the foreground region corresponding to the grayscale image of the endoscopic image data (ie, Fig. 6A The outer rectangular area of ​​the largest area in the right half shown in FIG. 1 is used as an image to assist in diagnosing the patient's disease, and finally generates the following Figure 6B Endoscopic features shown.

[0070] Thus, by utilizing a preprocessing module to binarize the endoscopic image data (i.e., the original endoscopic image data) and selecting the foreground area of ​​the endoscopic image data as an endoscopic feature map for input into a disease diagnosis module, the technical effect of removing noise and ensuring the output of accurate diagnostic information is achieved.

[0071] Optionally, the preprocessing module collects test indicator data and preprocesses the test indicator data to generate a test indicator feature vector, including: the preprocessing module collects test indicator data and divides the test indicator data into classification indicator data and continuous indicator data; the preprocessing module performs numerical binarization on the classification indicator data; the preprocessing module performs mean binarization on the continuous indicator data; and the preprocessing module generates a test indicator feature vector based on the classification indicator data after numerical binarization and the continuous indicator data after mean normalization.

[0072] Specifically, refer to Figure 2 and Figure 4As shown, first, the preprocessing module collects the test index data and divides the test index data into classification index data and continuous index data. Among them, the classification index data is continuous, and two adjacent values ​​can be infinitely divided. That is, it can take an infinite number of values, such as age. Classification index data is data that reflects the type of things obtained by classifying or grouping according to a certain attribute of the phenomenon, such as dividing the population into male and female according to gender.

[0073] The preprocessing module then assigns the classification indicator data using 0 / 1. For example, if the classification indicator data is gender, the preprocessing module uses 1 to represent males and 0 to represent females. For another example, if the classification indicator data is whether or not a person has a fever, the preprocessing module uses 1 to represent a person with a fever and 0 to represent a person without a fever.

[0074] Furthermore, the preprocessing module processes the continuous indicator data using a standardized processing method with a mean value of 0.5, thereby ensuring that the data range of the continuous indicator data is consistent.

[0075] Finally, the preprocessing module generates a test indicator feature vector based on the classification indicator data after numerical binarization and the continuous indicator data after mean normalization. For example, the classification indicator data and the continuous indicator data can be converted into the test indicator feature vector by a pre-set vector mapping method.

[0076] Thus, by utilizing the preprocessing module to perform numerical binarization processing on the classified index data and performing mean normalization processing on the continuous index data, a technical effect is achieved that can ensure the accuracy of the diagnostic information determined by the disease diagnosis module.

[0077] For the steps of disease diagnosis model training corresponding to the disease diagnosis module: This application uses an NVIDIA GeForce RTX 4090 graphics card with a video memory capacity of 24GB for deep learning model training, and selects the cross entropy loss function of the prediction category to supervise the optimization direction of the disease diagnosis model. Use SDG for stochastic gradient optimization. Use a step-by-step learning rate strategy. Among them, the initial learning rate is 1e-3, and the learning rate is decayed by a ratio of 0.1 when training to 70% to 90%. This application collected 1978 samples as training data and performed 100 iterations with a batch size of 16. At the same time, in order to avoid the occurrence of overfitting, the images are randomly flipped horizontally, vertically flipped, and rotated during the training process to increase the diversity of the data set. Finally, after the network converges, the model parameters with the best performance are taken as the final result.

[0078] Therefore, according to the first aspect of this embodiment, the technical effect of improving the accuracy of diagnostic information is achieved.

[0079] Figure 7 This is an overall flow chart of the disease diagnosis method based on ileocecal ulcer according to Example 1 of the present application. Figure 7 As shown, the method includes:

[0080] S711: The preprocessing module collects endoscopic image data and preprocesses the endoscopic image data to generate an endoscopic feature map;

[0081] S712: The preprocessing module collects the inspection index data and preprocesses the inspection index data to generate an inspection index feature vector;

[0082] S720: The preprocessing module sends the endoscope feature map, the first CT feature map, the second CT feature map, and the test index feature vector to the disease diagnosis module;

[0083] S730: The disease diagnosis module uses the first encoder to extract features from the first CT feature map to generate a first CT feature vector, and uses the second encoder to extract features from the second CT feature map to generate a second CT feature vector;

[0084] S740: The disease diagnosis module uses the second fully connected layer to unify the dimension of the first CT feature vector to correspond to the input dimension of the first fully connected layer, and uses the second fully connected layer to unify the dimension of the second CT feature vector to correspond to the input dimension of the first fully connected layer;

[0085] S750: The disease diagnosis module uses the first fully connected layer to perform feature fusion on the first CT feature vector and the second CT feature vector, thereby generating a first fused feature vector;

[0086] S760: The disease diagnosis module uses the third encoder to extract features from the endoscopic feature map, thereby generating an endoscopic feature vector;

[0087] S770: The disease diagnosis module uses the fourth fully connected layer to unify the dimension of the first fusion feature vector to correspond to the input dimension of the third fully connected layer, uses the fourth fully connected layer to unify the dimension of the endoscope feature vector to correspond to the input latitude of the third fully connected layer, and uses the fourth fully connected layer to unify the dimension of the test index feature vector to correspond to the input dimension of the third fully connected layer;

[0088] S780: The disease diagnosis module uses the third fully connected layer to perform feature fusion on the first fusion feature vector, the endoscopy feature vector, and the test index feature vector, and uses a softmax classifier to classify the second feature fusion vector, thereby determining the diagnosis information corresponding to the patient;

[0089] S790: The output module outputs the diagnosis information determined by the disease diagnosis module.

[0090] In addition, reference Figure 1 As shown, according to the second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0091] Therefore, according to this embodiment, the technical effect of improving the accuracy of diagnostic information is achieved.

[0092] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0093] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0094] Example 2

[0095] Figure 8 The device 800 for diagnosing diseases based on ileocecal ulcer according to this embodiment is shown, and the device 800 corresponds to the method according to embodiment 1. Figure 8As shown, the device 800 includes: a CT feature vector generation module 810, which is used to use a first encoder to extract features from a first CT feature map to generate a first CT feature vector, and use a second encoder to extract features from a second CT feature map to generate a second CT feature vector, wherein the body position of the patient corresponding to the first CT feature map is different from the body position of the patient corresponding to the second CT feature map; a first fusion module 820, which is used to use a first fully connected layer to perform feature fusion on the first CT feature vector and the second CT feature vector to generate a first fused feature vector; an endoscope feature vector generation module 830, which is used to use a third encoder to extract features from an endoscope feature map to generate an endoscope feature vector; and a diagnosis information determination module 840, which is used to perform feature fusion and classification on the first fused feature vector, the endoscope feature vector, and the test index feature vector corresponding to the test index data, and determine the diagnosis information corresponding to the patient.

[0096] Optionally, a second fully connected layer is arranged between the first encoder and the first fully connected layer and between the second encoder and the first fully connected layer, and the second fully connected layer is configured to perform the following operations: a first dimension unification module, used to unify the dimension of the first CT feature vector to correspond to the input dimension of the first fully connected layer; and a second dimension unification module, used to unify the dimension of the second CT feature vector to correspond to the input dimension of the first fully connected layer.

[0097] Optionally, the diagnosis information determination module 840 includes: a second fusion module, used to use the third fully connected layer to perform feature fusion on the first fused feature vector, the endoscopic feature vector and the test index feature vector, and generate a second fused feature vector; a probability information determination module, used to classify the second fused feature vector using a softmax classifier, so as to determine the probability information corresponding to Crohn's disease, intestinal tuberculosis and intestinal Behcet's disease; and a diagnosis information determination submodule, used to determine the diagnosis information corresponding to the patient based on the probability information.

[0098] Optionally, a fourth fully-connected layer is arranged between the third fully-connected layer and the first fully-connected layer and between the third fully-connected layer and the third encoder, and the fourth fully-connected layer is configured to perform the following operations: a third dimension unification module, used to unify the dimension of the first fused feature vector to correspond to the input dimension of the third fully-connected layer; a fourth dimension unification module, used to unify the dimension of the endoscopic feature vector to correspond to the input dimension of the third fully-connected layer; and a fifth dimension unification module, used to unify the dimension of the inspection index feature vector to correspond to the input dimension of the third fully-connected layer.

[0099] Optionally, the preprocessing module includes: a first preprocessing submodule, which is used for the preprocessing module to collect endoscopic image data and preprocess the endoscopic image data to generate an endoscopic feature map; and a second preprocessing submodule, which is used for the preprocessing module to collect inspection index data and preprocess the inspection index data to generate an inspection index feature vector.

[0100] Optionally, the first preprocessing submodule includes: a grayscale image generation module, which is used for the preprocessing module to generate a grayscale image corresponding to the endoscopic image data by using image binarization; and an endoscopic feature map generation module, which is used for the preprocessing module to determine the foreground area of ​​the endoscopic image data and select the foreground area as the endoscopic feature map.

[0101] Optionally, the second preprocessing submodule includes: an indicator data classification module, which is used for the preprocessing module to collect test indicator data and divide the test indicator data into classification indicator data and continuous indicator data; a numerical binarization module, which is used for the preprocessing module to perform numerical binarization on the classification indicator data; a mean standardization module, which is used for the preprocessing module to perform mean standardization on the continuous indicator data; and a test indicator feature vector generation module, which is used for the preprocessing module to generate a test indicator feature vector based on the classification indicator data after numerical binarization and the continuous indicator data after mean standardization.

[0102] Therefore, according to this embodiment, the technical effect of improving the accuracy of diagnostic information is achieved.

[0103] Example 3

[0104] Fig. 9 The device 900 for diagnosing diseases based on ileocecal ulcer according to this embodiment is shown, and the device 900 corresponds to the method according to embodiment 1. Fig. 9 As shown, the device 900 includes: a processor 910; and a memory 920, which is connected to the processor 910 and is used to provide the processor 910 with instructions for processing the following processing steps: using a first encoder to extract features from a first CT feature map to generate a first CT feature vector, using a second encoder to extract features from a second CT feature map to generate a second CT feature vector, wherein the body position of the patient corresponding to the first CT feature map is different from the body position of the patient corresponding to the second CT feature map; using a first fully connected layer to perform feature fusion on the first CT feature vector and the second CT feature vector to generate a first fused feature vector; using a third encoder to perform feature extraction on an endoscopic feature map to generate an endoscopic feature vector; and performing feature fusion and classification on the first fused feature vector, the endoscopic feature vector, and the test index feature vector corresponding to the test index data, and determining the diagnostic information corresponding to the patient.

[0105] Optionally, a second fully connected layer is arranged between the first encoder and the first fully connected layer and between the second encoder and the first fully connected layer, and the second fully connected layer is configured to perform the following operations: unify the dimension of the first CT feature vector to correspond to the input dimension of the first fully connected layer; and unify the dimension of the second CT feature vector to correspond to the input dimension of the first fully connected layer.

[0106] Optionally, the first fused feature vector, the endoscopic feature vector, and the test index feature vector corresponding to the test index data are feature fused and classified, and the operation of determining the diagnostic information corresponding to the patient includes: using the third fully connected layer to perform feature fusion on the first fused feature vector, the endoscopic feature vector, and the test index feature vector, and generate a second fused feature vector; using a softmax classifier to classify the second fused feature vector to determine the probability information corresponding to Crohn's disease, intestinal tuberculosis and intestinal Behcet's disease; and determining the diagnostic information corresponding to the patient based on the probability information.

[0107] Optionally, a fourth fully-connected layer is arranged between the third fully-connected layer and the first fully-connected layer and between the third fully-connected layer and the third encoder, and the fourth fully-connected layer is configured to perform the following operations: unify the dimension of the first fused feature vector to correspond to the input dimension of the third fully-connected layer; unify the dimension of the endoscopic feature vector to correspond to the input dimension of the third fully-connected layer; and unify the dimension of the inspection index feature vector to correspond to the input dimension of the third fully-connected layer.

[0108] Optionally, the preprocessing module collects first disease diagnosis data corresponding to the patient, preprocesses the first disease diagnosis data, and sends the second disease diagnosis data generated after the preprocessing to the disease diagnosis module, including: the preprocessing module collects endoscopic image data, and preprocesses the endoscopic image data to generate an endoscopic feature map; and the preprocessing module collects test index data, and preprocesses the test index data to generate a test index feature vector.

[0109] Optionally, the preprocessing module collects endoscopic image data and preprocesses the endoscopic image data to generate an endoscopic feature map, including: the preprocessing module uses image binarization to generate a grayscale image corresponding to the endoscopic image data; and the preprocessing module determines the foreground area of ​​the endoscopic image data and selects the foreground area as the endoscopic feature map.

[0110] Optionally, the preprocessing module collects test indicator data and preprocesses the test indicator data to generate a test indicator feature vector, including: the preprocessing module collects test indicator data and divides the test indicator data into classification indicator data and continuous indicator data; the preprocessing module performs numerical binarization on the classification indicator data; the preprocessing module performs mean normalization on the continuous indicator data; and the preprocessing module generates a test indicator feature vector based on the classification indicator data after numerical binarization and the continuous indicator data after mean normalization.

[0111] Therefore, according to this embodiment, the technical effect of improving the accuracy of diagnostic information is achieved.

[0112] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0113] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0115] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0117] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0118] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A disease diagnosis system based on ileocecal ulcer, characterized in that: include: Preprocessing module, disease diagnosis module and output module, among which The preprocessing module is connected to the disease diagnosis module, and is used to preprocess the first disease diagnosis data, and send the second disease diagnosis data generated after the preprocessing to the disease diagnosis module, wherein the second disease diagnosis data includes the first CT feature map, the second CT feature map, the endoscope feature map and the test index feature vector; The output module is used to output the diagnosis information determined by the disease diagnosis module, wherein the diagnosis information is used to indicate that the patient suffers from Crohn's disease, intestinal tuberculosis or intestinal Behcet's disease, and wherein The disease diagnosis module is configured to perform the following operations: Using a first encoder to extract features from the first CT feature map to generate a first CT feature vector, and using a second encoder to extract features from the second CT feature map to generate a second CT feature vector, wherein the body position of the patient corresponding to the first CT feature map is different from the body position of the patient corresponding to the second CT feature map; Using a first fully connected layer to perform feature fusion on the first CT feature vector and the second CT feature vector, thereby generating a first fused feature vector; Using a third encoder to extract features from the endoscope feature map, thereby generating an endoscope feature vector; as well as The first fusion feature vector, the endoscope feature vector and the test index feature vector are subjected to feature fusion and classification, and diagnostic information corresponding to the patient is determined, wherein The operation of performing feature fusion and classification on the first fusion feature vector, the endoscope feature vector, and the inspection index feature vector, and determining the diagnosis information corresponding to the patient includes: Using a third fully connected layer to perform feature fusion on the first fused feature vector, the endoscope feature vector, and the inspection index feature vector, and generate a second fused feature vector; classifying the second fused feature vector using a softmax classifier, thereby determining probability information corresponding to the Crohn's disease, the intestinal tuberculosis, and the intestinal Behcet's disease; and Based on the probability information, diagnosis information corresponding to the patient is determined, wherein A fourth fully connected layer is provided between the third fully connected layer and the first fully connected layer and between the third fully connected layer and the third encoder, and the fourth fully connected layer is configured to perform the following operations: Unifying the vector dimension of the first fused feature vector to correspond to the input dimension of the third fully connected layer; Unifying the vector dimension of the endoscopic feature vector to correspond to the input dimension of the third fully connected layer; and The vector dimension of the inspection index feature vector is unified to correspond to the input dimension of the third fully connected layer.

2. The system according to claim 1, characterized in that A second fully connected layer is provided between the first encoder and the first fully connected layer and between the second encoder and the first fully connected layer, and the second fully connected layer is configured to perform the following operations: Unifying the vector dimension of the first CT feature vector to correspond to the input dimension of the first fully connected layer; and The vector dimension of the second CT feature vector is unified to correspond to the input dimension of the first fully connected layer.

3. The system according to claim 1, characterized in that The operation of the preprocessing module collecting first disease diagnosis data corresponding to the patient, preprocessing the first disease diagnosis data, and sending second disease diagnosis data generated after the preprocessing to the disease diagnosis module includes: The preprocessing module collects endoscopic image data and preprocesses the endoscopic image data to generate the endoscopic characteristic map; and The preprocessing module collects inspection index data and preprocesses the inspection index data to generate the inspection index feature vector.

4. The system according to claim 3, characterized in that The operation of the preprocessing module collecting endoscopic image data and preprocessing the endoscopic image data to generate the endoscopic characteristic map includes: The preprocessing module generates a grayscale image corresponding to the endoscopic image data by using image binarization; and The preprocessing module determines a foreground area of ​​the endoscopic image data and selects the foreground area as the endoscopic feature map.

5. The system according to claim 3, characterized in that The operation of the preprocessing module collecting inspection index data and preprocessing the inspection index data to generate the inspection index feature vector includes: The preprocessing module collects the inspection index data and divides the inspection index data into classified index data and continuous index data; The preprocessing module performs numerical binarization processing on the classification index data; The preprocessing module performs mean normalization processing on the continuous indicator data; and The preprocessing module generates the inspection index feature vector based on the classification index data after numerical binarization and the continuous index data after mean normalization.

6. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the processor performs the following steps: Using a first encoder to extract features from a first CT feature map to generate a first CT feature vector, and using a second encoder to extract features from a second CT feature map to generate a second CT feature vector, wherein the body position of a patient corresponding to the first CT feature map is different from the body position of a patient corresponding to the second CT feature map; Using a first fully connected layer to perform feature fusion on the first CT feature vector and the second CT feature vector, thereby generating a first fused feature vector; Using a third encoder to extract features from the endoscopic feature map, thereby generating an endoscopic feature vector; and The first fusion feature vector, the endoscope feature vector and the test index feature vector are subjected to feature fusion and classification, and the diagnosis information corresponding to the patient is determined, wherein The operation of performing feature fusion and classification on the first fusion feature vector, the endoscope feature vector, and the inspection index feature vector, and determining the diagnosis information corresponding to the patient includes: Using a third fully connected layer to perform feature fusion on the first fused feature vector, the endoscope feature vector, and the inspection index feature vector, and generate a second fused feature vector; Using a softmax classifier to classify the second fused feature vector, thereby determining probability information corresponding to Crohn's disease, intestinal tuberculosis, and intestinal Behcet's disease; and Based on the probability information, diagnosis information corresponding to the patient is determined, wherein A fourth fully connected layer is provided between the third fully connected layer and the first fully connected layer and between the third fully connected layer and the third encoder, and the fourth fully connected layer is configured to perform the following operations: Unifying the vector dimension of the first fused feature vector to correspond to the input dimension of the third fully connected layer; Unifying the vector dimension of the endoscopic feature vector to correspond to the input dimension of the third fully connected layer; and The vector dimension of the inspection index feature vector is unified to correspond to the input dimension of the third fully connected layer.

7. A disease diagnosis device based on ileocecal ulcer, characterized in that: include: a CT feature vector generating module, configured to extract features from a first CT feature map using a first encoder to generate a first CT feature vector, and to extract features from a second CT feature map using a second encoder to generate a second CT feature vector, wherein the body position of a patient corresponding to the first CT feature map is different from the body position of a patient corresponding to the second CT feature map; A first fusion module, configured to perform feature fusion on the first CT feature vector and the second CT feature vector using a first fully connected layer, thereby generating a first fused feature vector; An endoscopic feature vector generating module, used for extracting features from the endoscopic feature map using a third encoder, thereby generating an endoscopic feature vector; as well as A diagnosis information determination module is used to perform feature fusion and classification on the first fusion feature vector, the endoscope feature vector and the test index feature vector corresponding to the test index data, and determine the diagnosis information corresponding to the patient, wherein The diagnosis information determination module includes: a second fusion module, which is used to perform feature fusion on the first fused feature vector, the endoscope feature vector and the inspection index feature vector using a third fully connected layer, and generate a second fused feature vector; a probability information determination module, configured to classify the second fused feature vector using a softmax classifier, thereby determining probability information corresponding to Crohn's disease, intestinal tuberculosis, and intestinal Behcet's disease; and A diagnosis information determination submodule is used to determine the diagnosis information corresponding to the patient based on the probability information, wherein A fourth fully connected layer is provided between the third fully connected layer and the first fully connected layer and between the third fully connected layer and the third encoder, and the fourth fully connected layer is configured to perform the following operations: A third dimension unification module, used to unify the dimension of the first fused feature vector to correspond to the input dimension of the third fully connected layer; a fourth dimension unification module, configured to unify the dimension of the endoscopic feature vector to correspond to the input dimension of the third fully connected layer; and The fifth dimension unification module is used to unify the dimension of the inspection index feature vector to correspond to the input dimension of the third fully connected layer.

8. A disease diagnosis device based on ileocecal ulcer, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Using a first encoder to extract features from a first CT feature map to generate a first CT feature vector, and using a second encoder to extract features from a second CT feature map to generate a second CT feature vector, wherein the body position of a patient corresponding to the first CT feature map is different from the body position of a patient corresponding to the second CT feature map; Using a first fully connected layer to perform feature fusion on the first CT feature vector and the second CT feature vector, thereby generating a first fused feature vector; Using a third encoder to extract features from the endoscopic feature map, thereby generating an endoscopic feature vector; as well as The first fusion feature vector, the endoscope feature vector and the test index feature vector corresponding to the test index data are subjected to feature fusion and classification, and the diagnosis information corresponding to the patient is determined, wherein The operation of performing feature fusion and classification on the first fusion feature vector, the endoscope feature vector, and the inspection index feature vector, and determining the diagnosis information corresponding to the patient includes: Using a third fully connected layer to perform feature fusion on the first fused feature vector, the endoscope feature vector, and the inspection index feature vector, and generate a second fused feature vector; Using a softmax classifier to classify the second fused feature vector, thereby determining probability information corresponding to Crohn's disease, intestinal tuberculosis, and intestinal Behcet's disease; and Based on the probability information, diagnosis information corresponding to the patient is determined, wherein A fourth fully connected layer is provided between the third fully connected layer and the first fully connected layer and between the third fully connected layer and the third encoder, and the fourth fully connected layer is configured to perform the following operations: Unifying the vector dimension of the first fused feature vector to correspond to the input dimension of the third fully connected layer; Unifying the vector dimension of the endoscopic feature vector to correspond to the input dimension of the third fully connected layer; and The vector dimension of the inspection index feature vector is unified to correspond to the input dimension of the third fully connected layer.

Images