Knowledge data and artificial intelligence driven multi-disease identification system for ophthalmology

Abstract

The present invention provides a target classification system. The target classification system can specifically be a multi-disease identification system driven by knowledge data and artificial intelligence. The system can be used to obtain medical images and non-image medical information; use the medical expert knowledge base to match information data, and obtain the weight results of ophthalmic diseases by the medical knowledge reasoning algorithm; use the organ tissue segmentation algorithm and disease lesion recognition algorithm in medical image diagnosis to obtain the weight results of ophthalmic diseases; combine the weight results obtained by knowledge reasoning and medical image recognition, and obtain the final disease diagnosis results by weighted calculation. The present invention refers to the medical clinical diagnosis ideas, uses big data to structure medical data, and uses the artificial intelligence model to detect disease lesions to achieve automatic diagnosis of diseases. The system improves the existing medical treatment methods, realizes the artificial intelligence preliminary diagnosis screening of diseases, effectively alleviates the current shortage of medical resources, and has broad application prospects.

Description

Technical Field

[0001] The present invention relates to the field of ophthalmology digitization and artificial intelligence technology, and in particular to a knowledge data and artificial intelligence driven ophthalmology multi-disease recognition system. Background Art

[0002] The epidemic has posed a serious threat to people's life safety and physical health. The epidemic has prompted the widespread application of artificial intelligence in various scenarios, such as auxiliary examination and pathological diagnosis, dissemination of epidemic prevention and control knowledge, locking the movement trajectory of epidemic personnel, and predicting and tracing the development trend of the epidemic. In such an environment, building a health ecosystem based on big data and providing high-quality, efficient and economical new intelligent medical services will become an effective means to solve the contradiction between medical supply and demand in my country and promote the development of medicine.

[0003] Artificial intelligence medical care is an important driving force for realizing the "Healthy China" strategy and has great development prospects. At present, artificial intelligence medical imaging is the most widely used in the medical field, and nearly 70% of clinical diagnoses require the help of medical imaging. In addition, artificial intelligence medical care has also maintained a rapid development trend in the fields of medical imaging examinations such as ultrasound and CT, electronic medical records, surgical robots, precision medicine, and medical payment. While improving the level of diagnosis and treatment, it also saves working time for medical workers. However, the current application of artificial intelligence technology in the medical field still needs to rely on medical staff and medical equipment. The labor force of medical workers has not been greatly liberated, and patients cannot independently screen for diseases. Therefore, using mobile devices such as mobile phones or wearable devices to obtain patients' medical information, and using artificial intelligence algorithms to achieve classified diagnosis of patients' diseases, so as to reduce the dependence of artificial intelligence on medical personnel and medical examination equipment, and enable patients to complete disease diagnosis independently, may become an important direction for the development of medical artificial intelligence in the future.

[0004] Humans obtain more than 80% of external information through their eyes, and ophthalmic diseases have a huge impact on families and society. Eyes are located on the surface of the body. Due to the specificity of transparent eyeball tissue, patients can easily obtain eye imaging data using mobile devices such as mobile phones, and common ophthalmic diseases can be diagnosed with the help of artificial intelligence algorithms.

[0005] Therefore, it is necessary to extract useful information from the huge amount of diverse and complex ophthalmic-related data, use machine learning, deep learning and multi-source data fusion algorithms combined with ophthalmic knowledge, realize intelligent diagnosis of ophthalmic diseases at the individual and family level, assist doctors in clinical diagnosis, alleviate the shortage of medical resources in remote grassroots areas, and truly benefit patients. Summary of the invention

[0006] Ophthalmic diagnosis mainly relies on data from multiple information sources such as clinical interviews and ophthalmic images. In order to characterize the associated semantic information of such multi-source heterogeneous data, the present invention provides a multi-source correlation fusion feature learning method.

[0007] In some aspects, the present disclosure provides a method for classifying a target, comprising:

[0008] Acquiring first information of a target, wherein the first information includes a medical image;

[0009] Acquiring second information of the target, wherein the second information includes non-image medical information;

[0010] Providing the first information to a first neural network including a first neural network model to determine a first vector of the target;

[0011] providing the second information to a second neural network including a second neural network model to determine a second vector of the target;

[0012] Performing data fusion processing on the first eigenvector and the second eigenvector of the target to determine a fused eigenvector of the target;

[0013] According to the fused feature vectors of the targets, the targets are classified using a machine learning model.

[0014] In some embodiments, the medical image comprises an image of a biological organ.

[0015] In some embodiments, the non-image medical information includes biological health-related information.

[0016] In some embodiments, the biological organ image includes one or more of the following: a skin image, an oral image, a bone image, and an eye image.

[0017] In some embodiments, the biological health-related information includes one or more of the following: skin health-related information; oral health-related information; bone health-related information; eye health-related information;

[0018] In some embodiments, the first vector includes one or more of the following: a first skin disease vector, a first oral disease vector, a first bone disease vector, a first ophthalmic disease vector;

[0019] In some embodiments, the second vector includes one or more of the following: a second skin disease vector, a second oral disease vector, a second bone disease vector, a second ophthalmic disease vector.

[0020] In some embodiments, in step a), the medical image is collected by an imaging device, including but not limited to a slit lamp, a fundus camera, an optical coherence tomography scanner, a magnetic resonance imaging scanner, a computer tomography machine, and a smart device (including a mobile phone, a tablet computer, a small computer, and a large or small instrument with a camera system).

[0021] In some embodiments, medical images include but are not limited to slit lamp images, mobile phone images, and wearable device images. The knowledge data and artificial intelligence-driven multi-ophthalmic disease recognition system of the present invention can be integrated on these existing imaging devices, or can be independently set or set on a terminal such as a server.

[0022] In some embodiments, in step b), the non-image medical information includes information related to eye health.

[0023] In some embodiments, the eye health related information includes one or more of the following: necessary eye conditions, possible eye conditions, negative eye conditions, eye medical history and target age, as well as other information required to improve the level of eye disease diagnosis and treatment.

[0024] In some embodiments, in step b), the non-image medical information source includes text, voice, questionnaire, and scale.

[0025] In some embodiments, in step a), the medical image includes one or more images of pathological areas of ocular tissue.

[0026] In some embodiments, the ocular tissue pathological area includes one or more of the following: upper eyelid area, lower eyelid area, caruncle area, conjunctival area, corneal area.

[0027] In some embodiments, the eye tissue case area image is obtained by the following method: the eye image is segmented by an image segmentation model (eg, an image semantic segmentation model) to obtain one or more eye tissue pathology area images.

[0028] In some embodiments, the image segmentation model is trained by the following method:

[0029] S1. Obtain a medical image (such as eye image) dataset;

[0030] S2, segmenting the image data into pathological regions to obtain annotated images;

[0031] S3. A semantic image segmentation model is constructed based on the U Net structure with attention mechanism, and the image segmentation model is trained using labeled images to obtain an image segmentation model.

[0032] In some embodiments, the image segmentation model includes: using a threshold to segment a medical image (such as an eye image), and then using a maximum connected domain algorithm to determine a set of pixels in the same connected region to obtain the final eye tissue region location.

[0033] In some embodiments, in step c), the first neural network includes a FasterR-CNN neural network based on ResNet-101.

[0034] In some embodiments, in step d), the second neural network comprises a GAN neural network.

[0035] In some embodiments, in step d), the second neural network performs weight calculations on multiple non-image medical information (eg, necessary symptoms, possible symptoms, negative symptoms, medical history, and age) to determine a second vector of the target.

[0036] In some embodiments, the second feature determination unit includes a knowledge reasoning module (such as an ophthalmic knowledge reasoning module), which first obtains a set of disease candidates based on necessary symptoms and age, and then increases the disease candidate set or increases the weight of diseases related to the medical history in combination with the medical history, and at the same time reduces the disease candidate set in combination with negative symptoms, and finally obtains the diagnosis result by sorting and calculating the weights according to the number of symptoms, that is, the second vector of the target.

[0037] In some embodiments, step e) comprises:

[0038] The first eigenvector and the second eigenvector are projected onto the manifold feature, and a metric constraint is introduced to minimize the distance between features of similar targets in the manifold feature space. The specific optimization objectives are as follows:

[0039] L=L p +λL C

[0040] Among them, L p It is the loss function of the classification result using the projected features, L C is the distance to the feature mean of similar targets.

[0041] In some embodiments, in step f), the classification result includes one or more eye-related symptoms.

[0042] In some embodiments, the eye-related symptoms are selected from one or more of the following:

[0043] Corneal diseases (such as keratitis, corneal opacity, other corneal lesions, corneal senile ring);

[0044] Conjunctival diseases (such as conjunctivitis, pterygium, palpebral fissure, conjunctival hemorrhage, and other conjunctival lesions);

[0045] Lens diseases (eg, cataracts; uveal diseases including uveitis);

[0046] Postoperative status (e.g., postoperative status after ocular surface surgery, postoperative status after anti-glaucoma surgery);

[0047] Eyelid disease (eg, eyelid disease, trichiasis);

[0048] Ocular surface diseases (such as ocular surface tumors, dry eyes, meibomian gland dysfunction);

[0049] Others (such as hyphema);

[0050] In good health.

[0051] In some embodiments, the target comprises an organism, for example comprises a mammal, for example comprises a human.

[0052] In some embodiments, the target classification method further includes providing treatment recommendations based on the classification results.

[0053] In some aspects, the present disclosure also provides a target classification system, comprising:

[0054] A first acquisition module, used to acquire first information of a target, wherein the first information includes a medical image;

[0055] A second acquisition module, used to acquire second information of the target, wherein the second information includes non-image medical information;

[0056] A first feature determination unit is used to provide first information to a first neural network including a first neural network model to determine a first vector of the target;

[0057] A second feature determination unit, configured to provide the second information to a second neural network including a second neural network model to determine a second vector of the target;

[0058] A fusion unit, used for performing data fusion processing on the first feature vector and the second feature vector of the target to determine a fused feature vector of the target;

[0059] A classification unit is used to classify each target by using a machine learning model according to the fused feature vectors of each target.

[0060] In some embodiments, the medical image comprises an image of a biological organ;

[0061] In some embodiments, the non-image medical information includes biological health-related information.

[0062] In some embodiments, the biological organ image includes one or more of the following: a skin image, an oral image, a bone image, an eye image;

[0063] In some embodiments, the biological health-related information includes one or more of the following: skin health-related information; oral health-related information; bone health-related information; eye health-related information;

[0064] In some embodiments, the first vector includes one or more of the following: a first skin disease vector, a first oral disease vector, a first bone disease vector, a first ophthalmic disease vector;

[0065] In some embodiments, the second vector includes one or more of the following: a second skin disease vector, a second oral disease vector, a second bone disease vector, a second ophthalmic disease vector.

[0066] In some embodiments, the first acquisition module includes an imaging device;

[0067] Optionally, imaging devices include but are not limited to slit lamps, fundus cameras, optical coherence tomography scanners, magnetic resonance imaging scanners, computer tomography machines, smart devices (including mobile phones, tablet computers, small computers, and large or small instruments with camera function systems).

[0068] In some embodiments, the non-image medical information includes information related to eye health;

[0069] Preferably, the eye health-related information includes one or more of the following: necessary eye disease, possible eye disease, denied eye disease, eye medical history and target age, as well as other information required to improve the level of eye disease diagnosis and treatment.

[0070] In some embodiments, the non-image medical information includes: text, voice, questionnaire, and scale.

[0071] In some embodiments, the medical image includes one or more images of pathological regions of ocular tissue;

[0072] Preferably, the ocular tissue pathological area includes one or more of the following: upper eyelid area, lower eyelid area, lacrimal caruncle area, conjunctival area, corneal area.

[0073] In some embodiments, the target classification system further includes an image segmentation module, which segments a medical image (eg, an eye image) to obtain one or more eye tissue pathology region images.

[0074] In some embodiments, the target classification system further comprises

[0075] A training data acquisition module, used to acquire a medical image (such as eye image) data set;

[0076] The training data labeling module is used to segment the pathological area of ​​the image data and obtain the labeled image;

[0077] The training module is used to build an image style model based on the U Net structure using the attention mechanism, and train the image segmentation model using the labeled images to obtain the image segmentation module.

[0078] In some embodiments, in the image segmentation module: a threshold is used to segment a medical image (such as an eye image), and then a maximum connected domain algorithm is used to determine a set of pixels in the same connected area to obtain multiple pathological tissue areas (such as eye pathological tissue areas).

[0079] In some embodiments, the first neural network comprises a Faster R-CNN neural network based on ResNet-101.

[0080] In some embodiments, the second neural network comprises a GAN neural network.

[0081] In some embodiments, the first feature determination unit includes an ophthalmic image recognition model, which is used to input image data into an image segmentation module to obtain an eye tissue segmentation map (the image data can come from a mobile phone picture or a slit lamp picture), and transfer weights through transfer learning to increase the generalization ability of the segmentation model, extract the eye pathological tissue area in the region extraction part, and then input the eye pathological tissue area into the first neural network to obtain the lesion category and location, that is, the first vector of the target.

[0082] In some embodiments, the second feature determination unit includes a knowledge reasoning module (such as an ophthalmic knowledge reasoning module), which first obtains a set of disease candidates based on necessary symptoms and age, and then increases the disease candidate set or increases the weight of diseases related to the medical history in combination with the medical history, and at the same time reduces the disease candidate set in combination with negative symptoms, and finally obtains the diagnosis result by sorting and calculating the weights according to the number of symptoms, that is, the second vector of the target.

[0083] In some embodiments, the second feature determination unit is used to perform weight calculation on multiple non-image medical information (eg, necessary symptoms, possible symptoms, negative symptoms, medical history, and age) to determine the second vector of the target.

[0084] In some embodiments, the fusion unit is used to project the first feature vector and the second feature vector to the flow pattern feature, and introduce a metric constraint so that the features of similar targets are close in distance in the flow pattern feature space. The specific optimization objectives are as follows:

[0085] L=L p +λL C

[0086] Among them, Lp is the loss function of the classification result using the projected features, and LC is the distance from the feature mean of the same type of target.

[0087] In some embodiments, the classification results include one or more eye conditions (eg, eye diseases).

[0088] In some embodiments, the ocular condition is selected from one or more of the following:

[0089] Corneal diseases (such as keratitis, corneal opacity, other corneal lesions, corneal senile ring);

[0090] Conjunctival diseases (such as conjunctivitis, pterygium, palpebral fissure, conjunctival hemorrhage, and other conjunctival lesions);

[0091] Lens diseases (eg, cataracts; uveal diseases including uveitis);

[0092] Postoperative status (e.g., postoperative status after ocular surface surgery, postoperative status after anti-glaucoma surgery);

[0093] Eyelid disease (eg, eyelid disease, trichiasis);

[0094] Ocular surface diseases (such as ocular surface tumors, dry eyes, meibomian gland dysfunction);

[0095] Others (eg, hyphema).

[0096] In some embodiments, the target comprises an organism, for example comprises a mammal, for example comprises a human.

[0097] In some embodiments, the target classification system further comprises

[0098] The treatment suggestion providing module is used to provide treatment suggestions based on the classification results.

[0099] In some aspects, the present disclosure also provides a target classification device, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the target classification method described in any one of the above claims based on instructions stored in the memory device.

[0100] In some aspects, the present disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the target classification methods described above.

[0101] In some embodiments herein, "medical information" as used herein refers to information related to the physiological state of the target (eg, health state, disease state). The medical information has a direct or indirect reference role in diagnosing whether the target has a disease.

[0102] In some embodiments, "medical image" as used herein refers to an image produced by a medical imaging modality. In some embodiments, "medical image" as used herein may include, but is not limited to, ultrasound images, magnetic resonance (MR) images, X-ray images, computed tomography (CT) images, positron emission tomography (PET) images, and model images obtained by modeling an object.

[0103] In some embodiments, "imaging equipment" as used herein broadly includes any imaging modality, as understood in the art of the present disclosure and as contemplated hereinafter, for performing diagnostic imaging of an anatomical region or anatomical organ. Examples of medical imaging machines include, but are not limited to, X-ray machines, ultrasound machines, computed tomography (CT) machines, magnetic resonance imaging (MRI) machines, positron emission tomography (PET) machines, single photon emission computed tomography (SPECT) machines, and diffuse optical tomography (DOT) machines.

[0104] In some embodiments, the "medical images" described herein may include images captured using various types of image capture modes. For example, medical images may include (but are not limited to): radiation therapy (RT) images, X-ray (XR) images, digital radiography (DX) X-ray images, X-ray angiography (XA) images, panoramic X-ray (PX) images, computed tomography (CT) images, mammography (MG) images (including tomosynthesis devices), magnetic resonance imaging (MRI) images, ultrasound (US) images, color flow Doppler (CD) images, position emission tomography (PET) images, single photon emission computed tomography (SPECT) images, nuclear medicine (NM) images, etc. The medical images may also include a synthesized version of a local medical image such as a synthetic X-ray (SXR) image, a modified or enhanced version of a local medical image, an enhanced version of a local medical image, etc., the synthesized version being generated using one or more image processing techniques. The medical imaging processing model disclosed herein may also be configured to process 3D images.

[0105] As used herein, a "target classification method" or "target classification system" is customized to perform image processing / analysis tasks on one or more medical images. For example, medical imaging processing / analysis tasks may include (but are not limited to): disease / disorder classification, disease region segmentation, organ segmentation, disease quantification, disease / disorder grading, risk prediction, temporal analysis, abnormality detection, anatomical feature characterization, medical image reconstruction, etc. A "target classification method" or "target classification system" may be a "medical image inference algorithm", "medical image inference model", "medical image processing algorithm", "medical image processing model", "medical image analysis model", etc. The output result of a "target classification method" or "target classification system" may be an artifact generated by an algorithm executed using one or more medical images as input. The output may be in different formats, such as: Digital Imaging and Communications in Medicine (DICOM) Structured Report (SR), DICOM Assisted Capture, DICOM Parameter Mapping, Image, Text and / or JavaScript Object Annotation (JSON).

[0106] In some embodiments, "disease" as used herein generally refers to any abnormal or pathological condition affecting a subject.

[0107] In some embodiments, "disease diagnosis" as used herein generally refers to diagnosing or screening for a disease, stratifying the risk of disease occurrence, monitoring the progression or remission of a disease, formulating a treatment plan for a disease, or any combination thereof.

[0108] In some embodiments, "ophthalmic" as used herein means relating to or associated with the eyes and vision.

[0109] In some embodiments, an "ophthalmic disease" or "ophthalmic disorder" as used herein refers to a disease or disorder involving the anatomy and / or function of the visual system.

[0110] In some embodiments, a “vector” as used herein may broadly include a feature vector extracted from a reconstructed medical image by an image reconstruction preprocessor, the feature vector representing a classification / prediction feature of the reconstructed medical image.

[0111] In some embodiments, "algorithm" and "model" as used herein may be used interchangeably herein, unless the context requires a special distinction between, in some embodiments, as used herein.

[0112] In some embodiments, as used herein, "second neural network" refers to an AI / ML algorithm configured to perform image processing or analysis tasks on an image. The image processing or analysis tasks may vary. In various embodiments, image processing tasks may include (but are not limited to): image restoration tasks (e.g., for improving image quality), image synthesis tasks (e.g., for constructing a three-dimensional image based on multiple two-dimensional images), image enhancement tasks (e.g., for improving images by using filters or adding information that will help with visualization) and image compression tasks (e.g., for reducing the size of images to enhance the required transmission time and storage). Some example image analysis tasks may include, but are not limited to, classification tasks (e.g., diagnosis), segmentation tasks (e.g., organ segmentation, region of interest segmentation, etc.), object recognition tasks, motion detection tasks, video tracking tasks, optical flow tasks, etc. The image inference model described here may include a 2D image processing model as well as a 3D image processing model. The image processing model may use various types of AI / ML algorithms, including (but not limited to): deep learning models, neural network models, deep neural network models (DNN), convolutional neural network models (CNN), generated adversarial neural network models (GANS), etc. However, the concepts of algorithms or models described here are not limited to machine learning or deep learning models.

[0113] Beneficial Effects

[0114] The technical solution disclosed in the present invention may have one or more of the following advantages:

[0115] 1. The receipt collection mechanism provided by some solutions of the present invention does not rely on medical resources. The user's medical information can be obtained using devices such as computers and mobile phones. The system can automatically standardize the information, reducing the dependence of smart medical care on medical personnel and medical examination equipment.

[0116] 2. The knowledge reasoning module (such as the ophthalmic knowledge reasoning module) proposed in some schemes of the present invention includes an ophthalmic standardization mechanism, an ophthalmic expert knowledge base, and an ophthalmic knowledge reasoning algorithm. The ophthalmic knowledge reasoning algorithm calculates weights for necessary symptoms, possible symptoms, negative symptoms, medical history, and age.

[0117] 3. The eye tissue segmentation algorithm proposed in some schemes of the present invention proposes a U-Net with attention mechanism to guide the algorithm to learn more distinguishing features of eye tissue to separate various eye tissue parts, and at the same time use transfer learning to increase the generalization ability of the segmentation algorithm recognition.

[0118] 3. The ophthalmic lesion detection algorithm proposed in some schemes of the present invention collects ophthalmic image data taken by different devices, uses an eye tissue segmentation algorithm to locate the eye case area, and uses transfer learning to increase the generalization ability of the lesion detection algorithm.

[0119] 4. Image and data fusion analysis proposed in some solutions of the present invention. A multi-source data fusion analysis algorithm is used, based on multi-core learning data fusion, combining complementary information from multiple data sources such as images, symptoms, and medical history, to improve the accuracy of ophthalmic disease diagnosis.

[0120] 5. The diagnostic system provided by some schemes of the present invention is combined with ophthalmic treatment guidelines to reduce medical costs, provide reference for grassroots hospitals with scarce medical resources, and provide ophthalmic disease screening for individuals and families, so as to detect ophthalmic diseases in time and protect eye health. BRIEF DESCRIPTION OF THE DRAWINGS

[0121] Figure 1 It is a schematic diagram of the structure of a multi-disease identification system for ophthalmology driven by knowledge data and artificial intelligence in some embodiments.

[0122] Figure 2 Schematic diagram of the flow of ophthalmic knowledge reasoning algorithm in some embodiments.

[0123] Figure 3 is a flowchart of an ophthalmic image recognition model in some embodiments.

[0124] Figure 4 2 is a diagram of segmentation results of an eye tissue structure segmentation model in some embodiments.

[0125] Figure 5 These are the change curves of loss, loss_clc, and loss_bbox of the ophthalmic lesion classification algorithm in some embodiments.

[0126] Figure 6 A schematic diagram illustrating an object classification apparatus according to some embodiments.

[0127] Figure 7 Schematic diagrams showing target classification devices according to some further embodiments.

[0128] Figure 8 A schematic diagram illustrating a target classification method according to some embodiments.

[0129] Fig. 9 A schematic diagram illustrating a target classification system of some embodiments. DETAILED DESCRIPTION

[0130] The embodiments of the present invention will be described in detail below in conjunction with the examples, but those skilled in the art will appreciate that the following examples are only used to illustrate the present invention and should not be considered to limit the scope of the present invention. If no specific conditions are specified in the examples, they are carried out according to normal conditions or the conditions recommended by the manufacturer. If the manufacturer is not specified for the reagents or instruments used, they are all conventional products that can be obtained commercially.

[0131] Figure 1The target classification system (knowledge data and artificial intelligence driven ophthalmic multi-disease identification system) of one embodiment of the present invention is shown. The system is implemented by big data processing and artificial intelligence imaging, and includes:

[0132] S1. Data collection module: Use the ophthalmic examination function to collect examination information such as visual acuity and astigmatism, and collect basic information, medical history information and symptom information through the medical interview data collection mechanism.

[0133] S2, knowledge reasoning module: standardize the data, use knowledge reasoning methods to obtain expert knowledge diagnosis of ophthalmic diseases, and build an ophthalmic expert knowledge base platform. Ophthalmic knowledge reasoning mainly consists of knowledge reasoning about necessary symptoms, possible symptoms, negative symptoms, age and acute and severe eye diseases. The ophthalmic knowledge reasoning algorithm process is as follows: Figure 2 shown.

[0134] S3, image recognition module; ophthalmic image recognition includes eye tissue structure segmentation algorithm and ophthalmic lesion detection algorithm. First, based on the relevant characteristics of eye tissue, a U-Net eye tissue segmentation model related to eye tissue location is proposed. In addition, we established an image dataset that has been accurately labeled by multiple ophthalmologists. On this basis, we combined the detection algorithm in deep learning with measure learning, integrated the experience of medical diagnosis, and constructed a convolutional neural network model. The combination of eye tissue segmentation algorithm and ophthalmic lesion detection algorithm ultimately achieves accurate identification of ophthalmic diseases. The flow chart of the ophthalmic image recognition model is shown below. Figure 3 shown.

[0135] S4, fusion analysis module (image and data); combines the data information of the intelligent consultation module and the image information of the intelligent diagnosis module for weighting, and stores the patient data such as symptoms, medical history, images, age, etc. into the knowledge base in a structured manner to discover the correlation between multi-source data, and finally obtain the weighted diagnosis results of ophthalmic diseases.

[0136] S5, treatment guide module: Based on ophthalmology knowledge, an ophthalmology treatment guide is constructed. The guide includes subspecialty recommendations, disease profile, treatment principles and treatment methods. Based on the intelligent diagnosis results, subspecialty consultation and corresponding treatment plans are automatically recommended.

[0137] In some embodiments, the target classification system is a knowledge data and artificial intelligence driven ophthalmic multi-disease identification system, which includes: a data acquisition module; an ophthalmic knowledge data reasoning module; an ophthalmic image diagnosis module, including an eye tissue structure segmentation algorithm and an ophthalmic lesion detection algorithm; an image and data fusion analysis module; and an ophthalmic treatment guide module. The ophthalmic diagnosis system and method based on big data and artificial intelligence of the present invention is based on big data and artificial intelligence technology, and can realize intelligent diagnosis of ophthalmic diseases, refer to the clinical diagnosis ideas of ophthalmology, use big data to structure medical data, and use artificial intelligence models to detect ophthalmic disease lesions, thereby realizing automatic diagnosis of ophthalmic consultation data and ophthalmic images, improving existing ophthalmic treatment methods, realizing preliminary diagnosis and screening of ophthalmic diseases at the individual and family end, alleviating the current shortage of medical resources, and is expected to realize data and artificial intelligence assisted medical diagnosis.

[0138] In some embodiments, the target classification system is a knowledge data and artificial intelligence driven ophthalmic multi-disease identification system, including: obtaining basic ophthalmic information, medical history information, examination information, symptom information and imaging information; using an ophthalmic expert knowledge base to match information data, and obtaining ophthalmic disease weight results by an ophthalmic knowledge reasoning algorithm; using an eye tissue segmentation algorithm and an eye disease lesion recognition algorithm in ophthalmic imaging diagnosis to obtain ophthalmic disease weight results; combining the weight results obtained by knowledge reasoning and ophthalmic image recognition, and combining data from multiple information sources such as medical history, symptoms, examinations, and images, and performing weighted calculations to obtain the final ophthalmic disease diagnosis results.

[0139] In some embodiments, the target classification method (system) is a knowledge data and artificial intelligence driven ophthalmic multi-disease recognition system. The system is implemented by big data processing and artificial intelligence imaging.

[0140] In some embodiments, the non-image medical information includes examination information such as visual acuity and astigmatism collected using an ophthalmic examination function, basic target information, medical history information, and symptom information collected through a medical interview data collection mechanism.

[0141] In some embodiments, the non-image medical information includes questionnaire information.

[0142] In some embodiments, the questionnaire includes one or more of the following questions:

[0143] 1. Is there any visual impairment?

[0144] □None□Decline slowly□Decline suddenly□Poor eyesight since childhood

[0145] 2. What you see close up is blurry, but what you see far away is clearer?

[0146] □Yes □No

[0147] 3. Do you see flashes of light before your eyes?

[0148] □Yes □No

[0149] 4. Is there a loss of vision? (An area in front of the eyes becomes dark)

[0150] □Yes □No

[0151] 5. When you look at an object, is the color of the object different from usual?

[0152] □Yes □No

[0153] 6. See if things are deformed?

[0154] □Yes □No

[0155] 7. Do you experience any eye pain?

[0156] □No □Eyelid pain □Eyeball pain

[0157] 8. Is it invisible at night?

[0158] □Yes □No

[0159] 9. Poor vision during the day, but better vision at night (day blindness)?

[0160] □Yes □No

[0161] 10. Are there any floaters in front of your eyes?

[0162] □Yes □No

[0163] 11. Do you see double images?

[0164] □Yes □No

[0165] 12. Are you afraid of light (always feel it is glaring)?

[0166] □Yes □No

[0167] 13. Is there any dryness?

[0168] □Yes □No

[0169] 14. Itchy eyes?

[0170] □Yes □No

[0171] 15. Are you crying or weeping?

[0172] □Yes □No

[0173] 16. Do you get easily tired when looking at things?

[0174] □Yes □No

[0175] 17. Jealous?

[0176] □No □A lot of blood □Flake-like red

[0177] 18. Do your eyelids droop? (Upper eyelids cover most of your eyes)

[0178] □Yes □No

[0179] 19. Do you feel like there is sand in your eyes (foreign body sensation)?

[0180] □Yes □No

[0181] 20. Is there an increase in eye discharge (e.g. eye mucus)?

[0182] □Yes □No

[0183] 21. Difficulty opening your eyes?

[0184] □Yes □No

[0185] 22. Blinking all the time?

[0186] □Yes □No

[0187] 23. Is there something growing on the eye?

[0188] □No □Eyelid □Eyeball (eyeball)

[0189] 24. Is there any eye injury?

[0190] □Yes □No

[0191] In some embodiments, the second feature determination module includes a knowledge reasoning module (such as an ophthalmic knowledge reasoning module), which performs standardization processing on the second information and uses a knowledge reasoning method to obtain expert knowledge diagnosis of ophthalmic diseases. In some embodiments, the second feature determination module includes an ophthalmic expert knowledge base. Ophthalmic knowledge reasoning mainly includes relevant knowledge reasoning of necessary symptoms, possible symptoms, negative symptoms, age, and acute and severe eye diseases.

[0192] In some embodiments, the ophthalmic knowledge reasoning algorithm performed by the ophthalmic knowledge reasoning module is as follows: Figure 2 The process is as follows: the knowledge reasoning algorithm first obtains the disease candidate set based on the necessary symptoms and age, then increases the disease candidate set or increases the weight of diseases related to the medical history based on the medical history, and reduces the disease candidate set based on the negative symptoms, and finally obtains the diagnosis result by sorting the weights according to the number of symptoms.

[0193] In some embodiments, the first feature determination unit includes an ophthalmic imaging diagnosis module.

[0194] In some embodiments, the first feature determination unit includes an eye tissue structure segmentation module and an ophthalmic lesion detection module. The eye tissue segmentation algorithm and the ophthalmic lesion detection algorithm are combined to ultimately achieve accurate identification of ophthalmic diseases.

[0195] In some embodiments, the flow chart of the ophthalmic image recognition model is as follows Figure 3 The process is as follows: the ophthalmic image recognition model first inputs the image data into the image segmentation module to obtain the eye tissue segmentation map (the image data can come from mobile phone pictures or slit lamp pictures), and transfers the weights through transfer learning to increase the generalization ability of the segmentation model, extracts the eye local map (eye pathological tissue area) in the region extraction part, and then inputs the local map (eye pathological tissue area) into the second neural network to obtain the lesion category and location, and then fuses the eye tissue information and lesion information in the fusion module to finally obtain the ophthalmic image diagnosis result.

[0196] In some embodiments, the eye tissue structure segmentation module proposes a U-Net eye tissue segmentation model related to the eye tissue part according to the eye tissue related characteristics.

[0197] In some embodiments, the ophthalmic lesion detection module includes an image dataset annotated by experts (ophthalmologists), on this basis, the detection algorithm in deep learning is combined with measurement learning, and the experience of medical diagnosis is integrated to construct an ophthalmic lesion detection module.

[0198] In some embodiments, the fusion unit includes an image (medical image) and data (non-image medical information) fusion analysis module. This module combines the image information of the intelligent diagnosis module acquired by the first acquisition module and the data information of the intelligent consultation module acquired by the second acquisition module for weighting. In some embodiments, the patient data such as symptoms, medical history, images, and age are structured and stored in the knowledge base to discover the association between multi-source data, and finally obtain the weighted ophthalmic disease diagnosis results.

[0199] In some embodiments, the treatment suggestion providing module includes an ophthalmology treatment guide module. This module constructs an ophthalmology treatment guide based on ophthalmology knowledge. The guide includes subspecialty recommendations, disease profile, treatment principles and treatment methods. Based on the intelligent diagnosis results, the subspecialty and corresponding treatment plan are automatically recommended.

[0200] In some embodiments, the target classification system of the present invention (i.e., the knowledge data and artificial intelligence driven ophthalmic multi-disease identification system) includes a first acquisition module, a second acquisition module, a first feature determination unit, a second feature determination unit, a fusion unit, and a classification unit. When the actual system is applied, it can be merged or split to achieve the system function.

[0201] In some implementations, the first acquisition module and the second acquisition module independently collect data in the following ways: obtaining ophthalmic consultation knowledge data in the form of questionnaires in actual application scenarios, and obtaining ophthalmic image data by taking photos or videos; collecting and structuring ophthalmic consultation information based on doctor and instrument diagnosis information. Among them, the consultation data collection adopts the form of questionnaire filling to structure the patient's condition. According to the doctor and instrument diagnosis information, the diseased areas and lesions in the sample image are annotated by annotation tools.

[0202] In some embodiments, the second feature determination unit includes a knowledge reasoning module (e.g., an ophthalmology knowledge reasoning module). The module creates a standardized ophthalmology standard database based on Hadoop's distributed big data acquisition technology, NoSQL technology, and Apache Hadoop open source software framework. The module includes a knowledge extraction algorithm that allows the computer to extract knowledge-related rules from massive medical literature.

[0203] In some implementation schemes, the knowledge reasoning module (eg, ophthalmic knowledge reasoning module) uses professional ophthalmic prior knowledge to vectorize the relevant rules and probabilistically represent the knowledge relationships.

[0204] In some implementation schemes, a knowledge reasoning module (such as an ophthalmic knowledge reasoning module) performs corresponding knowledge reasoning based on a large amount of patient clinical data to construct a knowledge reasoning algorithm for all ophthalmic disease knowledge (including 16 major categories and 71 minor categories).

[0205] In some embodiments, the knowledge reasoning module (such as the ophthalmic knowledge reasoning module) also includes an acute and severe ophthalmic disease early warning module.

[0206] In some embodiments, the knowledge reasoning module (such as the ophthalmic knowledge reasoning module) has an accuracy rate of 82.4% in ophthalmic disease reasoning and recognition based on joint testing of the algorithm by multiple ophthalmic centers.

[0207] In some embodiments, the image segmentation module is used to implement an eye tissue structure segmentation algorithm.

[0208] In some embodiments, the image segmentation module includes a standardization processing module, which performs standardization processing on the collected ophthalmic image data.

[0209] In some embodiments, the image segmentation module includes a labeling module, which labels / segments the case area of ​​the image data according to an expert (professional ophthalmologist) using ophthalmic prior knowledge.

[0210] Specifically, the pathological areas in the sample images were annotated using the Labelme annotation tool, and the annotated labels included five categories: palpebra superior, palpebra inferior, carunculae, lacrimalis, conjunctiva, and cornea.

[0211] In some embodiments, the image segmentation module includes a preprocessing module that performs one or more of the following thermal processing on the annotated image: resizing, flipping, sharpening, embossing, non-linear spatial transformation data enhancement operations using albumentations.

[0212] In some embodiments, the image segmentation module is trained through a training module, which uses a u-net network with an attention mechanism to train a segmentation model for the eye tissue structure, performs hyperparameter adjustment, and finally obtains an eye tissue structure segmentation module.

[0213] In some embodiments, the accuracy of the image segmentation module (eye tissue segmentation module) is shown in Table 1, and the eye tissue segmentation results are shown in Table 1. Figure 4 shown.

[0214] Figure 4 A1 shows an eye photo taken by a slit lamp camera, and B1 shows the segmentation result. The segmented eye tissue pathological areas include: lacrimal caruncle area 1, corneal area 2, conjunctival area 3, upper eyelid area 4, and lower eyelid area 5.

[0215] Figure 4 A2 shows an eye photo taken by a mobile phone camera, and B2 shows the segmentation result. The segmented eye tissue pathological areas include: lacrimal caruncle area 1, corneal area 2, conjunctival area 3, upper eyelid area 4, and lower eyelid area 5.

[0216] Table 1

[0217]

[0218] In some embodiments, the image segmentation module is based on a deep learning ophthalmic lesion detection algorithm; after segmenting the collected ophthalmic image data, the ocular tissue pathology segmentation positioning (ocular tissue pathology area image) is obtained.

[0219] In some implementation schemes, lesions are annotated in each case region in the pathological localization area based on the prior knowledge of ophthalmology experts, and the annotated data is normalized.

[0220] In some embodiments, the second feature determination unit includes an ophthalmic lesion detection module, which uses the FasterRCNN-Resnet101 target detection network for model training and integrates medical diagnosis experience for algorithm construction.

[0221] In some embodiments, the ophthalmic lesion detection module includes a YOLOv3 framework, a FasterRCNN-Resnet50 framework, or a FasterRCNN-Resnet101 framework.

[0222]

[0223] In some embodiments, the ophthalmic lesion detection module includes the FasterRCNN-Resnet101 framework. This module has the best effect, with an average accuracy of 0.967. The average accuracy of the three network frameworks is shown in Table 2. The FasterRCNN-resnet101 network framework training loss, loss_clc, and loss_bbox changes are shown in Figure 5 shown.

[0224] In some embodiments, eye disorders include one or more of the following: corneal diseases include keratitis, corneal opacities, other corneal lesions, and corneal senile rings; conjunctival diseases include conjunctivitis, pterygium, palpebral fissures, conjunctival hemorrhage, and other conjunctival lesions; lens diseases include cataracts; uveal diseases include uveitis; postoperative conditions include postoperative status of ocular surface surgery and anti-glaucoma surgery; eyelid diseases include eyelid diseases and trichiasis; ocular surface diseases include ocular surface tumors, dry eyes, and meibomian gland dysfunction; others include hyphema.

[0225] In some embodiments, the fusion unit includes an image and data fusion analysis module. The data of different information sources of each patient are projected onto the flow pattern feature, and a metric constraint is introduced to minimize the distance between the features of similar patients in the flow pattern feature space. The specific optimization objectives are as follows:

[0226]

[0227] Among them, Lp is the loss function of disease diagnosis using projected features, and LC is the distance from the feature mean of similar patients. This model fully considers the feature relevance of each information source, and at the same time makes the distance between the features of similar patients in the low-dimensional feature space close, making the final diagnosis result more relevant to the multi-source diagnosis information of the patient.

[0228] In some embodiments, the target classification system includes a validation module. The validation module includes a set of specialized ophthalmic disease validation sets screened by ophthalmologists. The method accuracy is tested on the validation data set using the knowledge data and artificial intelligence driven ophthalmic multi-disease recognition method. The recognition results of the method are analyzed as follows: Figure 7 shown.

[0229] In some embodiments, the target classification system includes a clinical diagnosis and treatment recommendation improvement module, which is used to collect data, search similar cases, match the ophthalmologist knowledge base through clinical treatment rules, and output a treatment plan.

[0230] Figure 6 A schematic diagram illustrating an object classification apparatus according to some embodiments.

[0231] As shown in the figure, the target classification device 7 of this embodiment includes: a memory 71 and a processor 72 coupled to the memory 71, and the processor 72 is configured to execute the target classification method in any embodiment of the present disclosure based on the instructions stored in the memory 71.

[0232] The memory 71 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, a database, and other programs.

[0233] Figure 7 Some schematic diagrams of target classification devices are shown.

[0234] As shown in the figure, the target classification device 8 of this embodiment includes: a memory 810 and a processor 820 coupled to the memory 810, and the processor 820 is configured to execute the method for target classification in any one of the aforementioned embodiments based on instructions stored in the memory 810.

[0235] The memory 810 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs.

[0236] The target classification device 8 may also include an input / output interface 830, a network interface 840, a storage interface 850, etc. These interfaces 830, 840, 850, the memory 810, and the processor 820 may be connected, for example, via a bus 860. The input / output interface 830 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, and a touch screen. The network interface 840 provides a connection interface for various networked devices. The storage interface 850 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0237] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0238] So far, the method for object classification, the object classification device and the computer-readable storage medium according to the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

[0239] The method and system of the present disclosure may be implemented in many ways. For example, the method and system of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0240] Figure 8 A target classification method according to an embodiment is shown, comprising:

[0241] Step 101: Acquire first information of a target, where the first information includes a medical image;

[0242] Step 102: Acquire second information of the target, where the second information includes non-image medical information;

[0243] Step 103: providing the first information to a first neural network including a first neural network model to determine a first vector of the target;

[0244] Step 104: providing the second information to a second neural network including a second neural network model to determine a second vector of the target;

[0245] Step 105: performing data fusion processing on the first feature vector and the second feature vector of the target to determine a fused feature vector of the target;

[0246] Step 106: Classify the targets using a machine learning model based on the fused feature vectors of the targets.

[0247] Fig. 9 A target classification system according to an embodiment is shown, comprising:

[0248] A first acquisition module 201 is used to acquire first information of a target, wherein the first information includes a medical image;

[0249] A second acquisition module 202, configured to acquire second information of the target, wherein the second information includes non-image medical information;

[0250] A first feature determination unit 301 is used to provide first information to a first neural network including a first neural network model to determine a first vector of the target;

[0251] A second feature determination unit 302 is used to provide the second information to a second neural network including a second neural network model to determine a second vector of the target;

[0252] A fusion unit 401 is used to perform data fusion processing on the first feature vector and the second feature vector of the target to determine a fused feature vector of the target;

[0253] The classification unit 501 is used to classify each target according to the fused feature vector of each target using a machine learning model.

[0254] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A knowledge data and artificial intelligence driven multi-disease recognition system for ophthalmology, including: A first acquisition module, used to acquire first information of a target, wherein the first information includes a medical image; A second acquisition module, used to acquire second information of the target, wherein the second information includes non-image medical information; A first feature determination unit is used to provide first information to a first neural network including a first neural network model to determine a first vector of the target; A second feature determination unit, configured to provide the second information to a second neural network including a second neural network model to determine a second vector of the target; A fusion unit, used for performing data fusion processing on the first vector and the second vector of the target to determine a fusion feature vector of the target; A classification unit, used to classify each target by using a machine learning model according to the fused feature vector of each target; A training data acquisition module, used to acquire a medical image dataset, wherein the medical image dataset is an eye image; The training data labeling module is used to segment the pathological area of ​​the image data and obtain the labeled image; A training module, for building an image segmentation model based on a U-Net structure using an attention mechanism, and training the image segmentation model using an annotated image to obtain an image segmentation module, wherein a threshold is used to segment the medical image, and then a maximum connected domain algorithm is used to determine a pixel set in the same connected region to obtain multiple ocular pathological tissue regions; The first feature determination unit includes an ophthalmic image recognition model, which is used to input image data into an image segmentation module to obtain an eye tissue segmentation map, and increase the generalization ability of the segmentation model by transferring weights through transfer learning, extract eye pathological tissue regions in the region extraction part, and then input the eye pathological tissue regions into the first neural network to obtain the lesion category and location, that is, the first vector of the target; The second feature determination unit is used to perform weight calculation on multiple non-image medical information to determine the second vector of the target, The fusion unit is used to project the first vector and the second vector to the manifold feature, and introduce a metric constraint so that the features of similar targets have a small distance in the manifold feature space. The specific optimization objectives are as follows: Among them, L p It is the loss function of the classification result using the projected features, L C is the distance to the feature mean of similar targets.

2. The ophthalmic multi-disease identification system according to claim 1, wherein: The medical image includes an image of a biological organ; The non-image medical information includes biological health-related information.

3. The ophthalmic multi-disease identification system according to claim 2, characterized in that Any of the following: The biological organ image includes one or more of the following: skin image, oral image, bone image, eye image; The biological health-related information includes one or more of the following: skin health-related information; oral health-related information; bone health-related information; eye health-related information; The first vector includes one or more of the following: a first skin disease vector, a first oral disease vector, a first bone disease vector, a first ophthalmic disease vector; The second vector includes one or more of the following: a second skin disease vector, a second oral disease vector, a second bone disease vector, and a second ophthalmic disease vector.

4. According to the ophthalmic multi-disease identification system according to claim 1, the first acquisition module includes an imaging device.

5. The ophthalmic multi-disease identification system according to claim 4, The imaging equipment includes a slit lamp, a fundus camera, an optical coherence tomography scanner, a magnetic resonance imaging scanner, a computerized tomography machine, and an intelligent device.

6. According to the ophthalmic multi-disease identification system of claim 5, the smart device includes a mobile phone, a tablet computer, a small computer and a large or small instrument with a camera function system.

7. According to the ophthalmic multi-disease identification system of claim 1, the non-image medical information includes information related to eye health.

8. The ophthalmic multi-disease identification system according to claim 7, The eye health related information includes one or more of the following: must have eye conditions, may have eye conditions, deny eye conditions, eye medical history and target age.

9. The ophthalmic multi-disease identification system according to claim 1, wherein the non-image medical information comprises: Text, voice, questionnaires, and scales.

10. The ophthalmic multi-disease identification system according to claim 1, wherein the medical image comprises one or more images of pathological areas of ocular tissue.

11. The ophthalmic multi-disease identification system according to claim 10, The ocular tissue pathological area includes one or more of the following: upper eyelid area, lower eyelid area, lacrimal caruncle area, conjunctival area, and corneal area.

12. According to the ophthalmic multi-disease recognition system according to claim 1, the first neural network comprises a Faster R-CNN neural network based on ResNet-101.

13. According to the ophthalmic multi-disease identification system according to claim 1, the second neural network comprises a GAN neural network.

14. The ophthalmic multi-disease identification system according to claim 1, wherein the image data is from a mobile phone picture or a slit lamp picture.

15. The ophthalmic multi-disease identification system according to claim 1, wherein the plurality of non-image medical information includes necessary symptoms, possible symptoms, negative symptoms, medical history and age.

16. The ophthalmic multi-disease identification system according to claim 1, The second feature determination unit includes an ophthalmic knowledge reasoning module, which first obtains a set of disease candidates based on necessary symptoms and age, and then increases the disease candidate set or increases the weight of diseases related to the medical history in combination with the medical history, and at the same time reduces the disease candidate set in combination with negative symptoms, and finally calculates the weights by sorting the number of symptoms to obtain the diagnosis result, that is, the second vector of the target.

17. The ophthalmic multi-disease identification system according to claim 1, wherein the classification result comprises one or more eye diseases.

18. The ophthalmic multiple disease identification system according to claim 17, The eye condition is selected from one or more of the following: Corneal disease; Conjunctivitis; Lens disease; Uveal disease; Postoperative status; Eyelid disease; Ocular surface diseases; Hyphema.

19. According to the ophthalmic multi-disease identification system of claim 18, the corneal diseases include keratitis, corneal opacity, and corneal senile ring; the conjunctival diseases include conjunctivitis, pterygium, palpebral fissure, and conjunctival hemorrhage; the lens diseases include cataracts; uveal diseases include uveitis; postoperative status includes postoperative status of ocular surface surgery and postoperative status of anti-glaucoma surgery; eyelid diseases include eyelid diseases and trichiasis; ocular surface diseases include ocular surface tumors, dry eyes, and meibomian gland dysfunction.

20. The ophthalmic multi-disease identification system according to claim 1, wherein the target comprises a living being.

21. The ophthalmic multi-disease identification system according to claim 1, wherein the target comprises a mammal.

22. The ophthalmic multi-disease identification system according to claim 1, wherein the target comprises a person.

23. The ophthalmic multiple disease identification system according to claim 1, further comprising The treatment suggestion providing module is used to provide treatment suggestions based on the classification results.

24. A target classification method, comprising: Acquiring first information of a target, wherein the first information includes a medical image; Acquiring second information of the target, wherein the second information includes non-image medical information; Providing the first information to a first neural network including a first neural network model to determine a first vector of the target; providing the second information to a second neural network including a second neural network model to determine a second vector of the target; Performing data fusion processing on the first vector and the second vector of the target to determine a fusion feature vector of the target; According to the fused feature vectors of the targets, the targets are classified using a machine learning model, The target classification method is performed using the ophthalmic multi-disease recognition system described in any one of claims 1 to 23.

25. A target classification device, comprising: Memory; and a processor coupled to the memory, the processor being configured to execute the target classification method of claim 24 based on instructions stored in the memory device.

26. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the target classification method according to claim 24 is implemented.

Images