Retinal crack detection model and training method, retinal crack detection method, program product, medium and equipment

Abstract

The invention provides a retinal crack detection model, a training method, a retinal crack detection method, a program product, a medium and equipment. In the training of the retinal crack detection model, the crack detection model is enabled to be more accurate by adopting the model training which is more focused on image difference (also more relevant information of a target); in addition, due to the emphasis on feature differences, the model structure is simpler, and the training process is more efficient.

Description

Technical Field

[0001] The present invention relates to the field of retinal break detection, and in particular to a retinal break detection model and training method, a retinal break detection method, a program product, a medium and a device. Background Art

[0002] Artificial intelligence (AI) is committed to using computer technology to simulate human intelligent behavior with minimal human intervention. Given its unique advantages in multimodal complex data processing and feature learning, AI has been widely studied and applied in medical diagnosis and treatment. In particular, in ophthalmology, where diagnosis is mainly based on images and digital forms, the potential of AI is unquestionable. Therefore, research on AI for retinal disease classification, screening and diagnosis has been conducted.

[0003] In daily ophthalmic diagnosis, fundus photography is one of the commonly used examination methods in ophthalmology, and it can also help ophthalmologists obtain a large amount of information. Fundus photography examines the morphological changes of the entire retina. The principle is to use a special camera to record the scene seen under the fundusscope. Therefore, fundus photography can observe the morphology of the retina, optic disc, macular area, retinal blood vessels, and whether there are changes such as bleeding, exudation, hemangioma, retinal degeneration area, retinal holes, neovascularization, atrophic spots, pigment disorders, etc. on the retina. Among them, retinal hole detection is an area of ​​concern. At present, some researchers have used AI for retinal hole detection, but the detection accuracy obtained needs to be improved. The reason may be that it is difficult to detect retinal holes in the early stage, especially when the holes are small. However, the development of retinal holes is extremely harmful, and in severe cases it can cause retinal detachment and the risk of blindness. Therefore, a more accurate retinal hole detection method is needed. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a retinal break detection model and training method, a retinal break detection method, a program product, a medium and a device, which mainly solve the problem of low accuracy of the existing AI retinal break detection method.

[0005] The objective of the present invention is achieved through the following solutions:

[0006] In a first aspect, according to an embodiment of the present invention, a training method for a retinal hole detection model is provided, wherein the retinal hole detection model detects whether a subject has a retinal hole based on a fundus image, and the training method comprises: S1, constructing the retinal hole detection model, wherein the retinal hole detection model comprises a first feature extraction network, a second feature extraction network and a subtractor network, wherein the first feature extraction network and the second feature extraction network are used to extract features of the fundus image, and the input of the subtractor network is the two extracted feature maps; S2, creating a training data set, wherein the training data set comprises fundus images of multiple subjects and a reference fundus image, wherein the reference fundus image corresponds to a fundus image of a retinal hole without a hole, wherein each fundus image also comprises a label indicating whether there is a hole; S3, training the retinal hole detection model based on the created training data set, wherein a preset loss function is used to calculate the loss during the training process, and the model parameters are updated.

[0007] According to an embodiment of the present invention, the retinal hole detection model also includes an attention network, and the attention model is used to weight the feature map output by the subtractor network.

[0008] According to an embodiment of the present invention, weighting the feature map output by the subtractor network is to divide the output feature map into regions, and then the attention network assigns weights to each region.

[0009] According to an embodiment of the present invention, the loss is a cross entropy loss, a logarithmic loss or a mean square error loss.

[0010] According to an embodiment of the present invention, the cross entropy loss is:

[0011]

[0012] Among them, y represents the label hole type, Indicates the predicted hiatus type.

[0013] In a second aspect, according to another embodiment of the present invention, a retinal hole detection model is provided, which detects whether there is a retinal hole in a subject based on a fundus image, and the model is configured to be trained using the method described in the first aspect, and includes: a first feature extraction network and a second feature extraction network, which are used to extract features of the fundus image; a subtractor network, which is used to perform a subtraction operation on the two extracted feature maps.

[0014] In a third aspect, according to another embodiment of the present invention, a retinal hole detection method is provided, which detects whether there is a retinal hole in a subject based on a fundus image, and the method comprises: P1, acquiring a fundus image of the subject and a reference fundus image; and P2, using a retinal hole detection model as described in the second aspect to perform hole detection on the fundus image.

[0015] In a fourth aspect, according to another embodiment of the present invention, a computer program product is provided, wherein the computer program product includes a computer program code, and when the computer program code runs on an electronic device, the electronic device executes the method as described in the first aspect or the third aspect.

[0016] In the fifth aspect, according to another embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program can be executed by a processor to implement the steps of the method described in the first aspect or the third aspect.

[0017] In the sixth aspect, according to another embodiment of the present invention, an electronic device is provided, comprising: one or more processors; and a memory, wherein the memory is used to store executable instructions; the one or more processors are configured to implement the steps of the method described in the first aspect or the third aspect by executing the executable instructions.

[0018] Compared with the prior art, the present invention has the following beneficial effects: by adopting a model training that focuses more on image differences (more relevant information about the target), the crack detection model is more accurate; in addition, thanks to the focus on feature differences, the model structure is simpler and the training process is more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The embodiments of the present invention are further described below with reference to the accompanying drawings:

[0020] Figure 1 is the target fundus image of the embodiment of the present invention.

[0021] Figure 2 Flow chart of a method for training a retinal break detection model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0023] As described in the background art, the existing retinal hole detection methods are not accurate enough. The applicant found that the problem may be that the fundus image contains too much information. Figure 1As shown in Figure 1, the fundus image reflects information about the retina, optic disc, macula, retinal blood vessels, etc. Therefore, generally, information related to retinal tears (if any) only occupies a relatively small part of the image, unless the severity of retinal detachment has been reached (which is difficult to ignore). Therefore, in model training based on conventional image classification, even if the complexity of the model is continuously increased, it is often difficult to learn these subtle differences.

[0024] In response to the above problems, Figure 2 As shown, according to an embodiment of the present invention, a training method for a retinal hole detection model is provided, wherein the retinal hole detection model detects whether there is a hole in the retina of a subject based on a fundus image, and the training method comprises: S1, constructing the retinal hole detection model, wherein the retinal hole detection model comprises a first feature extraction network, a second feature extraction network and a subtractor network, wherein the first feature extraction network and the second feature extraction network are used to extract features of the fundus image, and the input of the subtractor network is the two extracted feature maps; S2, creating a training data set, wherein the training data set comprises fundus images of multiple subjects and a reference fundus image, wherein the reference fundus image corresponds to a fundus image without a retinal hole, wherein each fundus image also comprises a label indicating whether there is a hole; S3, training the retinal hole detection model based on the created training data set, wherein a preset loss function is used to calculate the loss during the training process, and the model parameters are updated. By adopting a model training that focuses more on image differences (target-related information), the hole detection model is made more accurate; in addition, thanks to the focus on feature differences, the model structure is simpler and the training process is more efficient.

[0025] In detail, the first feature extraction network is used for feature extraction of the fundus image of the subject, the second feature extraction network is used for feature extraction of the reference fundus image of the subject, and the subtractor network is used to obtain the difference between the feature maps of the two. The obtained feature map is then passed through the projection network and the classification network to obtain the prediction result. Among them, the first feature extraction network and the second feature extraction network share network parameters.

[0026] In order to improve the performance and generalization ability of the model, according to an embodiment of the present invention, the retinal hole detection model also includes an attention network, and the attention model is used to weight the feature map output by the subtractor network. In detail, according to an embodiment of the present invention, weighting the feature map output by the subtractor network is to divide the output feature map into regions, and then the attention network assigns weights to each region.

[0027] According to an embodiment of the present invention, the loss is a cross entropy loss, a logarithmic loss or a mean square error loss. In detail, when the cross entropy loss is adopted, the cross entropy loss is:

[0028]

[0029] Among them, y represents the label hole type, Indicates the predicted hiatus type.

[0030] According to another embodiment of the present invention, a retinal hole detection model is provided, which detects whether a retinal hole exists in a subject based on a fundus image. The model is configured to be trained using the method described above, and includes: a first feature extraction network and a second feature extraction network, which are used to extract features of the fundus image; and a subtractor network, which is used to perform a subtraction operation on the two extracted feature maps.

[0031] According to another embodiment of the present invention, a retinal hole detection method is provided, which detects whether there is a retinal hole in a subject based on a fundus image, and the method comprises: P1, acquiring a fundus image of the subject and a reference fundus image; and P2, using a retinal hole detection model as described above to perform hole detection on the fundus image.

[0032] According to yet another embodiment of the present invention, a computer program product is provided. The computer program product includes a computer program code. When the computer program code is executed on an electronic device, the electronic device executes the method as described above.

[0033] According to another embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. The computer program can be executed by a processor to implement the steps of the method described above.

[0034] According to another embodiment of the present invention, there is provided an electronic device, comprising: one or more processors; and a memory, wherein the memory is used to store executable instructions; the one or more processors are configured to implement the steps of the method described above by executing the executable instructions.

[0035] Through the retinal break detection method provided by the present invention, the examinee can better monitor the changes in his / her fundus condition and can be informed in time when changes occur, so that the examinee can better understand his / her fundus condition and can intervene in time when necessary to avoid the development of a serious situation.

[0036] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0037] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0038] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A training method for a retinal hole detection model, wherein the retinal hole detection model detects whether a subject has a retinal hole based on a fundus image, characterized in that: The training method comprises: S1. Constructing the retinal hole detection model, the retinal hole detection model includes a first feature extraction network, a second feature extraction network and a subtractor network, wherein the first feature extraction network and the second feature extraction network are used to extract features of the fundus image, and the input of the subtractor network is the two extracted feature maps; S2. Create a training data set, wherein the training data set includes fundus images of multiple subjects and reference fundus images, wherein the reference fundus images correspond to fundus images without retinal holes, wherein each fundus image also includes a label indicating whether there is a hole; S3. Training the retinal break detection model based on the created training data set, using a preset loss function to calculate the loss during the training process, and updating the model parameters.

2. A retinal break detection model training method as claimed in claim 1, characterized in that: The retinal hole detection model also includes an attention network, which is used to weight the feature map output by the subtractor network.

3. A training method for a retinal break detection model as claimed in claim 2, characterized in that: The feature map output by the subtractor network is weighted to divide the output feature map into regions, and the attention network then assigns weights to each region.

4. The training method of a retinal break detection model according to claim 1, characterized in that: The loss is cross entropy loss, logarithmic loss or mean square error loss.

5. A method for training a retinal break detection model as claimed in claim 4, characterized in that: The cross entropy loss is: Among them, y represents the label hole type, Indicates the predicted hiatus type.

6. A retinal hole detection model, which detects whether there is a retinal hole in the subject based on fundus images, characterized in that: The model is configured to be trained using the method according to any one of claims 1 to 5, and comprises: The first feature extraction network and the second feature extraction network are used for extracting features of the fundus image; The subtractor network is used to perform a subtraction operation on the two extracted feature maps.

7. A retinal hole detection method, based on fundus images, to detect whether a subject has a retinal hole, characterized in that: The method comprises: P1. Obtain the fundus image of the subject and the reference fundus image; and P2. Use a retinal hole detection model as described in claim 6 to perform hole detection on fundus images.

8. A computer program product, characterized in that The computer program product comprises a computer program code, and when the computer program code is run on an electronic device, the electronic device executes the method as claimed in any one of claims 1-5 and 7.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method in any one of claims 1-5 and 7.

10. An electronic device, characterized in that: include: one or more processors; as well as A memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1-5 and 7 by executing the executable instructions.

Images