Image recognition method and system for cataract

Abstract

The invention relates to an image recognition method and system for cataract. The method comprises the following steps: acquiring an eye examination medical image of a patient; inputting the eye examination medical image into a cataract image recognition model to obtain a cataract diagnosis result; wherein the cataract image recognition model is obtained by training a deep learning model based on a plurality of sample eye examination medical images and corresponding annotation data; moreover, the training process of the deep learning model comprises the steps: extracting a crystalline lens region image in the sample eye examination medical image, extracting a plurality of different types of image features from the crystalline lens region image, and fusing and outputting cataract type information based on the plurality of different types of image features. And updating the deep learning model based on the difference between the cataract type information and the type label in the corresponding annotation data, and ending the training until the loss value of the loss function of the deep learning model meets the convergence condition.

Description

Technical Field

[0001] The disclosed embodiments relate to the field of AI-assisted medical technology, and in particular, to an image recognition method and system for cataracts. Background Art

[0002] Cataract is a disease in which the lens of the eye becomes cloudy due to various reasons, causing visual impairment. Usually, doctors need to arrange relevant examinations to diagnose and confirm cataracts, and at the same time, they must rule out misdiagnosis of other eye diseases, which leads to low diagnostic efficiency and requires professional doctors to judge, and the accuracy needs to be improved. In recent years, artificial intelligence (AI) has gradually been applied to the medical field to promote medical development by improving efficiency and accuracy. In related technologies, the diagnosis of cataracts is mainly based on AI technology for eye image recognition, which can quickly identify disease features in images and determine whether the patient has cataracts based on the extracted features. However, the application of AI in cataract diagnosis and treatment is still in its infancy. It is currently limited to the preliminary diagnosis and screening of cataracts. For the detailed diagnosis of different types of cataracts such as moderate and severe, effective solutions are still needed. Summary of the invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide an image recognition method and system for cataracts.

[0004] In a first aspect, an embodiment of the present disclosure provides an image recognition method for cataracts, comprising: Obtain medical images of the patient's eye examination; Inputting the eye examination medical image into a cataract image recognition model to obtain a cataract diagnosis result; wherein the cataract image recognition model is obtained by training a deep learning model based on multiple sample eye examination medical images and corresponding annotated data; And wherein, the training process of the deep learning model includes: extracting the lens area image in the sample eye examination medical image, extracting multiple different types of image features from the lens area image, outputting cataract type information based on the fusion of the multiple different types of image features, and updating the deep learning model based on the difference between the cataract type information and the type label in the corresponding annotated data, until the loss value of the loss function of the deep learning model meets the convergence condition, and the training is terminated.

[0005] In one embodiment, the multiple different types of image features include at least intensity features, color features, and texture features of the lens area image; the training process of the deep learning model specifically includes: the deep learning model determines the fused image features based on the intensity features, color features, and texture features, and outputs cataract type information based on the fused image features.

[0006] In one embodiment, the deep learning model determines the fused image features based on the image intensity features, color features, and texture features, including: Obtain a preset first weight value, a second weight value, and a third weight value; wherein the first weight value is greater than the second weight value, and the second weight value is greater than the third weight value; and obtain a fused image feature by weighted summation based on the first weight value, the second weight value, and the third weight value and the corresponding texture features, color features, and intensity features.

[0007] In one embodiment, the method further comprises: Cropping the eye examination medical image to obtain an image of the lens area of ​​the eye of a preset size; wherein the preset size is related to the type of the deep learning model; The lens area image of the eye is input into the cataract image recognition model to obtain a cataract diagnosis result.

[0008] In one embodiment, the eye examination medical images and sample eye examination medical images comprise at least slit lamp microscope images.

[0009] In one embodiment, the multiple different types of image features also include the image position of the lens area image, and the image position represents the relative position of the lens in the lens area image and the eye in the sample eye examination medical image; the training process of the deep learning model specifically also includes: the deep learning model outputs cataract type information based on the fused image features and the image position.

[0010] In one embodiment, the cataract diagnosis result includes at least a cataract type parameter, which identifies a numerical range corresponding to the degree of lens opacity, and different cataract type parameters identify different types of cataracts.

[0011] In a second aspect, an embodiment of the present disclosure provides an image recognition system for cataracts, comprising: An acquisition module, used for acquiring a medical image of an eye examination of a patient; A recognition module, used for inputting the eye examination medical image into a cataract image recognition model to obtain a cataract diagnosis result; wherein the cataract image recognition model is obtained by training a deep learning model based on multiple sample eye examination medical images and corresponding annotated data; And wherein, the training process of the deep learning model includes: extracting the lens area image in the sample eye examination medical image, extracting multiple different types of image features from the lens area image, outputting cataract type information based on the fusion of the multiple different types of image features, and updating the deep learning model based on the difference between the cataract type information and the type label in the corresponding annotated data, until the loss value of the loss function of the deep learning model meets the convergence condition, and the training is terminated.

[0012] In a third aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image recognition method for cataracts described in any of the above embodiments.

[0013] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, including: Processor; and Memory for storing computer programs; The processor is configured to execute the image recognition method for cataracts described in any of the above embodiments by executing the computer program.

[0014] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages: The image recognition method and system for cataracts provided by the embodiments of the present disclosure obtain a patient's eye examination medical image; input the eye examination medical image into a cataract image recognition model to obtain a cataract diagnosis result; wherein the cataract image recognition model is obtained by training a deep learning model based on multiple sample eye examination medical images and corresponding annotation data; and the training process of the deep learning model includes: extracting a lens area image in the sample eye examination medical image, extracting multiple different types of image features from the lens area image, outputting cataract type information based on a fusion of the multiple different types of image features, and updating the deep learning model based on the difference between the cataract type information and the type label in the corresponding annotation data, until the loss value of the loss function of the deep learning model meets the convergence condition and the training is terminated. In this way, during the model training process, the cataract type information is output based on the fusion of multiple different types of image features of the acquired lens area image to update the model, avoiding the interference of other areas in the sample eye examination medical image on the recognition. Only the lens area image most relevant to cataract diagnosis is recognized and processed, and multiple different types of image features are extracted instead of a single feature such as texture for fusion recognition processing. This can deeply explore the most relevant deep features in the learning image, improve the recognition accuracy of the trained cataract image recognition model, and accurately identify different types of cataracts. AI-assisted diagnosis can help improve clinical work, expand the scope of cataract screening, and alleviate the pressure of insufficient medical resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Figure 1 This is a flow chart of an image recognition method for cataracts according to an embodiment of the present disclosure; Figure 2 A schematic diagram of the architecture of a deep learning model according to an embodiment of the present disclosure; Figure 3 This is a flow chart of an image recognition method for cataracts according to another embodiment of the present disclosure; Figure 4 is a schematic diagram of an image recognition system for cataracts according to an embodiment of the present disclosure; Figure 5Schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0019] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0020] It should be understood that, in the following, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0021] Figure 1 The flowchart of the image recognition method for cataracts according to the embodiment of the present disclosure is shown in FIG. 1 . The image recognition method for cataracts can be executed by a computing device such as a computer or a mobile terminal, and specifically may include the following steps: Step S101: Obtain a medical image of the patient's eye examination.

[0022] Exemplarily, in one embodiment, the eye examination medical image may at least include but is not limited to a slit lamp microscope image. A slit lamp microscope is an important instrument that is indispensable for ophthalmic examination. A slit lamp microscope image is an eye image of a patient when an eye examination is performed using a slit lamp microscope. The slit lamp microscope image can be obtained through an image acquisition device such as a camera, but is not limited thereto.

[0023] Step S102: input the eye examination medical image into a cataract image recognition model to obtain a cataract diagnosis result; wherein the cataract image recognition model is obtained by training a deep learning model based on multiple sample eye examination medical images and corresponding annotated data.

[0024] Among them, the training process of the deep learning model includes: extracting the lens area image in the sample eye examination medical image, extracting multiple different types of image features from the lens area image, outputting cataract type information based on the fusion of the multiple different types of image features, and updating the deep learning model based on the difference between the cataract type information and the type label in the corresponding annotated data, until the loss value of the loss function of the deep learning model meets the convergence condition and the training is terminated.

[0025] Exemplarily, the sample eye examination medical images may include at least slit lamp microscope images, for example, slit lamp microscope images of multiple historical patients are obtained as sample eye examination medical images. At the same time, the sample eye examination medical images may be annotated, such as obtaining annotation data based on historical examination results, such as type labels indicating specific types of cataracts, such as label 0 indicating type 1, label 1 indicating type 2, label 2 indicating type 3, and so on. Figure 2 As shown, the deep learning model can be but is not limited to a convolutional neural network model, which can include a feature extraction network 201 and a convolutional neural network 202. The feature extraction network 201 can extract the lens area image in the sample eye examination medical image, for example, obtain the lens area image by image segmentation, and then extract multiple different types of image features such as at least 2 different types of image features from the lens area image. The convolutional neural network 202 outputs the predicted cataract type based on the fusion of multiple different types of image features, and updates the deep learning model based on the difference between the cataract type and the type label in the corresponding annotation data, that is, the feature extraction network 201 and the convolutional neural network 202 are updated simultaneously, until the loss function of the deep learning model, such as the loss value of the loss function of the convolutional neural network 202, meets the convergence condition, such as less than the preset value, and the training is terminated. After the training is completed, the cataract image recognition model can be obtained, and then the eye examination medical image in step S101 is input into the cataract image recognition model to obtain the cataract diagnosis result.

[0026] In one embodiment, the cataract diagnosis result may include at least a cataract type parameter, which identifies a numerical range corresponding to the degree of lens opacity, and different cataract type parameters identify different types of cataracts. Exemplarily, cataract types can be divided into immature stage, expansion stage, mature stage, and overmature stage based on the degree of lens opacity, which can be represented by different type parameters. In this embodiment, each cataract type parameter can identify a numerical range corresponding to the degree of lens opacity, and different numerical ranges correspond to different types of cataracts. The corresponding type of cataract can be determined by the numerical range in which the cataract type parameter output by the model falls. In this way, different types of cataracts can be subdivided more accurately, thereby improving the recognition accuracy of the cataract image recognition model.

[0027] In the above scheme of the disclosed embodiment, during the model training process, the cataract type information is output based on the fusion of multiple different types of image features of the accurately acquired lens area image to update the model, thereby avoiding the interference of other areas in the sample eye examination medical image on the recognition, and only the lens area image most relevant to cataract diagnosis is recognized and processed, and multiple different types of image features are extracted instead of a single feature such as texture for fusion recognition processing. In this way, the most relevant deep features in the learning image can be deeply mined to improve the recognition accuracy of the trained cataract image recognition model, and different types of cataracts can be accurately identified. AI-assisted diagnosis can help improve clinical work, expand the scope of cataract screening, and alleviate the pressure of insufficient medical resources.

[0028] On the basis of the above embodiments, in one embodiment, the feature extraction network 201 may include an image segmentation subnetwork and a feature extraction subnetwork. The image segmentation subnetwork performs image segmentation on the sample eye examination medical image to obtain a lens area image, and then the feature extraction subnetwork extracts multiple different types of image features from the lens area image. After that, the convolutional neural network 202 outputs cataract type information based on the fusion of multiple different types of image features. Among them, the feature extraction subnetwork may include multiple image encoders, each image encoder corresponding to extracting one type of image feature in the lens area image. In this way, multiple different types of image features can be accurately segmented and accurately extracted, thereby making the most relevant deep features in the image for deep mining and learning more accurate for feature fusion recognition processing, and ultimately further improving the recognition accuracy of the trained cataract image recognition model.

[0029] Based on any of the above embodiments, in one embodiment, the multiple different types of image features may at least include but are not limited to intensity features of the lens region image, i.e., image intensity (Image Intensity), color features (RGB features), and texture features. Accordingly, the training process of the deep learning model may specifically include: the deep learning model determines fused image features based on the intensity features, color features, and texture features, and outputs cataract type information based on the fused image features.

[0030] Exemplarily, after the feature extraction network 201 extracts multiple different types of image features such as image intensity X1, color feature X2, and texture feature X3, the convolutional neural network 202 determines the fused image feature X4 based on the image intensity X1, color feature X2, and texture feature X3, such as multiplication, splicing, etc., predicts and outputs the cataract type information based on the fused image feature X4, and updates the deep learning model based on the difference between the cataract type and the type label in the corresponding annotation data, that is, simultaneously updates the feature extraction network 201 and the convolutional neural network 202. In this embodiment, the inventors have found that for the final model output result, that is, the cataract recognition result, the above-mentioned different types of image features, namely image intensity, color features, and texture features, have the most relevant and important impact on the recognition result. Therefore, the recognition accuracy of the cataract image recognition model obtained by the fusion training update based on the above-mentioned specific different types of image features is further improved.

[0031] In one embodiment, in conjunction with reference Figure 3 As shown, the deep learning model such as the convolutional neural network 202 determines the fused image features based on the image intensity features, color features, and texture features, which may specifically include the following steps: Step S301: obtaining a preset first weight value, a second weight value and a third weight value; wherein the first weight value is greater than the second weight value, and the second weight value is greater than the third weight value.

[0032] Since the above-mentioned different types of image features, namely image intensity, color features, and texture features, have the most relevant and important influence on the recognition result of the model, the first weight value, the second weight value, and the third weight value corresponding to the texture feature, the color feature, and the intensity feature can be specifically and accurately determined. The first weight value is greater than the second weight value, that is, the effect weight of the texture feature is greater than the effect weight of the color feature, and the second weight value is greater than the third weight value, that is, the effect weight of the color feature is greater than the effect weight of the intensity feature.

[0033] Exemplarily, during the model training process, the image features of the input model and the corresponding output cataract type information are processed by the model interpretation tool to obtain the percentage value of the effect of the image features of the input model on the output result, i.e., the cataract type information, as the weight value of the corresponding type of image feature. The model interpretation tool can be Shapash, which is a machine learning model interpretation tool suitable for classification and regression problems. The content of the tool can be specifically understood by referring to the prior art and will not be repeated here.

[0034] Step S302: Obtain fused image features by weighted summing up the first weight value, the second weight value, the third weight value and the corresponding texture features, color features and intensity features.

[0035] For example, after obtaining the image intensity X1, the color feature X2, the texture feature X3 and their respective weight values, the weighted sum can be used to obtain the fused image feature X4: X4=X1*wb1+X2*wb2+X3*wb3; wherein wb1 is the third weight value, wb2 is the second weight value, and wb3 is the first weight value.

[0036] The model interpretation tool can be used to accurately obtain the weight values ​​of different types of image features in advance, and then the fused image features can be accurately obtained based on this, so that the most relevant deep features in the image learned through deep mining based on fused feature recognition processing are more accurate, and ultimately the recognition accuracy of the trained cataract image recognition model is further improved.

[0037] Based on any one of the above embodiments, in one embodiment, the method may further include: cropping the eye examination medical image to obtain an eye lens area image of a preset size; wherein the preset size is related to the type of the deep learning model; inputting the eye lens area image into the cataract image recognition model to obtain a cataract diagnosis result.

[0038] Exemplarily, the eye examination medical image is cropped to obtain a lens area image of a preset size, that is, the lens area image of a suitable size related to the type of deep learning model is cropped, and then the cataract diagnosis result is obtained based on the lens area image of the suitable size and using the cataract image recognition model. Since different types of models correspond to different image sizes, selecting the lens area image of a suitable size that is cropped for the convolutional neural network model of this embodiment for recognition can improve the recognition accuracy of the cataract image recognition model.

[0039] In one embodiment, the multiple different types of image features also include the image position of the lens area image, and the image position represents the relative position of the lens in the lens area image and the eye in the sample eye examination medical image; the training process of the deep learning model specifically also includes: the deep learning model outputs cataract type information based on the fused image features and the image position.

[0040] It is understandable that the clinical diagnosis of cataracts depends not only on the degree of lens opacity observed under a slit lamp biological microscope, but also on information such as the location of the lens opacity area. Therefore, in this embodiment, in order to further improve the recognition accuracy of the cataract image recognition model, the multiple different types of image features extracted by the feature extraction network 201 can also be set to include the image position of the lens area image during the model training process, that is, the information of the relative position of the lens in the lens area image and the eye in the sample eye examination medical image is added, and the deep learning model such as the convolutional neural network 202 outputs the cataract type information based on the above-mentioned fused image features and the image position, and updates the deep learning model based on the difference between the cataract type and the type label in the corresponding annotation data, that is, the feature extraction network 201 and the convolutional neural network 202 are updated simultaneously until convergence is completed.

[0041] Among them, the feature extraction subnetwork can also be used to extract the image position of the lens area image, for example, another image encoder is set to extract the image position, which directly processes the sample eye examination medical image to extract the image position of the lens area image, rather than processing the segmented lens area image. In this way, the fused image features and image positions can be accurately obtained, thereby making the most relevant deep features in the image learned by deep mining based on the fused features and image position recognition processing more accurate, and ultimately further improving the recognition accuracy of the trained cataract image recognition model.

[0042] It should be noted that, although the steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc. In addition, it is also easy to understand that these steps may be, for example, executed synchronously or asynchronously in multiple modules / processes / threads.

[0043] like Figure 4 As shown, the image recognition system for cataracts provided by the embodiment of the present disclosure includes: An acquisition module 401 is used to acquire a medical image of an eye examination of a patient; The recognition module 402 is used to input the eye examination medical image into a cataract image recognition model to obtain a cataract diagnosis result; wherein the cataract image recognition model is obtained by training a deep learning model based on multiple sample eye examination medical images and corresponding annotated data; and wherein the training process of the deep learning model includes: extracting the lens area image in the sample eye examination medical image, extracting multiple different types of image features from the lens area image, outputting cataract type information based on the fusion of the multiple different types of image features, and updating the deep learning model based on the difference between the cataract type information and the type label in the corresponding annotated data, until the loss value of the loss function of the deep learning model meets the convergence condition and the training is terminated.

[0044] The deep learning model may be, but is not limited to, a convolutional neural network model, which may include a feature extraction network 201 and a convolutional neural network 202. The feature extraction network 201 may extract a lens region image from a sample eye examination medical image, such as obtaining a lens region image by image segmentation, and then extract multiple different types of image features, such as at least two different types of image features, from the lens region image. The convolutional neural network 202 outputs a predicted cataract type based on a fusion of multiple different types of image features, and updates the deep learning model based on the difference between the cataract type and the type label in the corresponding annotated data, i.e., simultaneously updates the feature extraction network 201 and the convolutional neural network 202, until the loss function of the deep learning model, such as the loss value of the loss function of the convolutional neural network 202, meets the convergence condition, such as being less than a preset value, and the training is terminated.

[0045] In one embodiment, the multiple different types of image features include at least intensity features, color features, and texture features of the lens area image; the training process of the deep learning model specifically includes: the deep learning model determines the fused image features based on the intensity features, color features, and texture features, and outputs cataract type information based on the fused image features.

[0046] In one embodiment, the feature extraction network 201 may include an image segmentation subnetwork and a feature extraction subnetwork. The image segmentation subnetwork performs image segmentation on the sample eye examination medical image to obtain a lens region image. Then, the feature extraction subnetwork extracts multiple different types of image features from the lens region image. After that, the convolutional neural network 202 outputs cataract type information based on the fusion of multiple different types of image features. The feature extraction subnetwork may include multiple image encoders, each of which extracts one type of image feature from the lens region image.

[0047] In one embodiment, the deep learning model determines the fused image features based on the image intensity features, color features, and texture features, including: obtaining a preset first weight value, a second weight value, and a third weight value; wherein the first weight value is greater than the second weight value, and the second weight value is greater than the third weight value; and obtaining the fused image features by weighted summation based on the first weight value, the second weight value, and the third weight value and the corresponding texture features, color features, and intensity features.

[0048] In one embodiment, the image features of the input model and the corresponding output cataract type information are processed by a model interpretation tool to obtain the percentage value of the degree of influence of the image features of the input model on the output result, i.e., the cataract type information, as the weight value of the corresponding type of image feature.

[0049] In one embodiment, the system may also include a cropping module, which is used to: crop the eye examination medical image to obtain an eye lens area image of a preset size; wherein the preset size is related to the type of the deep learning model; the recognition module may also be used to input the eye lens area image into the cataract image recognition model to obtain a cataract diagnosis result.

[0050] In one embodiment, the eye examination medical images and sample eye examination medical images comprise at least slit lamp microscope images.

[0051] In one embodiment, the multiple different types of image features also include the image position of the lens area image, and the image position represents the relative position of the lens in the lens area image and the eye in the sample eye examination medical image; the training process of the deep learning model specifically also includes: the deep learning model outputs cataract type information based on the fused image features and the image position.

[0052] In one embodiment, the feature extraction subnetwork can also be used to extract the image position of the lens area image, for example, another image encoder is set to extract the image position, which directly processes the sample eye examination medical image to extract the image position of the lens area image.

[0053] In one embodiment, the cataract diagnosis result includes at least a cataract type parameter, which identifies a numerical range corresponding to the degree of lens opacity, and different cataract type parameters identify different types of cataracts.

[0054] Regarding the system in the above embodiment, the specific manner in which each module performs operations and the corresponding technical effects brought about have been described in detail in the embodiment of the method, and will not be elaborated here.

[0055] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of a module or unit described above can be further divided into multiple modules or units for concretization. The components displayed as modules or units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying creative work.

[0056] The embodiments of the present disclosure further provide a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the image recognition method for cataracts described in any one of the above embodiments is implemented.

[0057] Exemplarily, the readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0058] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0059] The present disclosure also provides an electronic device, including a processor and a memory, wherein the memory is used to store a computer program, wherein the processor is configured to execute the image recognition method for cataracts in any of the above embodiments by executing the computer program.

[0060] Refer to the following Figure 5The electronic device 600 according to this embodiment of the present invention is described. Figure 5 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0061] like Figure 5 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0062] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method embodiment section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 The steps of the method shown in .

[0063] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0064] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0065] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0066] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. In addition, the electronic device 600 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 through a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0067] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the method steps of the above-mentioned embodiments according to the implementation methods of the present disclosure.

[0068] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0069] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for image recognition of cataracts, characterized in that: include: Obtain medical images of the patient's eye examination; Inputting the eye examination medical image into a cataract image recognition model to obtain a cataract diagnosis result; wherein the cataract image recognition model is obtained by training a deep learning model based on multiple sample eye examination medical images and corresponding annotated data; And wherein, the training process of the deep learning model includes: extracting the lens area image in the sample eye examination medical image, extracting multiple different types of image features from the lens area image, outputting cataract type information based on the fusion of the multiple different types of image features, and updating the deep learning model based on the difference between the cataract type information and the type label in the corresponding annotated data, until the loss value of the loss function of the deep learning model meets the convergence condition, and the training is terminated.

2. The method according to claim 1, characterized in that The plurality of different types of image features at least include intensity features, color features, and texture features of the lens region image; The training process of the deep learning model specifically includes: the deep learning model determines the fused image features based on the intensity features, color features, and texture features, and outputs cataract type information based on the fused image features.

3. The method according to claim 2, characterized in that The deep learning model determines the fused image features based on the image intensity features, color features, and texture features, including: Obtain a preset first weight value, a second weight value, and a third weight value; wherein the first weight value is greater than the second weight value, and the second weight value is greater than the third weight value; and obtain a fused image feature by weighted summation based on the first weight value, the second weight value, and the third weight value and the corresponding texture features, color features, and intensity features.

4. The method according to any one of claims 1 to 3, characterized in that The method further includes: Cropping the eye examination medical image to obtain an image of the lens area of ​​the eye of a preset size; wherein the preset size is related to the type of the deep learning model; The lens area image of the eye is input into the cataract image recognition model to obtain a cataract diagnosis result.

5. The method according to any one of claims 1 to 3, characterized in that: The eye examination medical images and the sample eye examination medical images include at least slit lamp microscope images.

6. The method according to claim 2 or 3, characterized in that: The multiple different types of image features also include the image position of the lens area image, and the image position represents the relative position of the lens in the lens area image and the eye in the sample eye examination medical image; the training process of the deep learning model specifically also includes: the deep learning model outputs cataract type information based on the fused image features and the image position.

7. The method according to claim 6, characterized in that The cataract diagnosis result at least includes a cataract type parameter, which identifies a numerical range corresponding to the degree of lens opacity, and different cataract type parameters identify different types of cataracts.

8. An image recognition system for cataracts, characterized in that: include: An acquisition module, used for acquiring a medical image of an eye examination of a patient; A recognition module, used for inputting the eye examination medical image into a cataract image recognition model to obtain a cataract diagnosis result; wherein the cataract image recognition model is obtained by training a deep learning model based on multiple sample eye examination medical images and corresponding annotated data; And wherein, the training process of the deep learning model includes: extracting the lens area image in the sample eye examination medical image, extracting multiple different types of image features from the lens area image, outputting cataract type information based on the fusion of the multiple different types of image features, and updating the deep learning model based on the difference between the cataract type information and the type label in the corresponding annotated data, until the loss value of the loss function of the deep learning model meets the convergence condition, and the training is terminated.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image recognition method for cataracts according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; as well as Memory for storing computer programs; Wherein, the processor is configured to execute the image recognition method for cataracts according to any one of claims 1 to 7 by executing the computer program.

Images