Lesion type identification method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a lesion type identification method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring at least one fundus image to be recognized; based on a pre-constructed optic disk recognition model, determining whether the to-be-recognized fundus image has the proliferative diabetic retinopathy; wherein the optic disk identification model is a model obtained by replacing an original convolution module in an original convolutional neural network model with a tree convolution module. According to the technical scheme of the embodiment of the invention, in the optic disk recognition model, the blood vessel features can be accurately recognized through the tree convolution module, so that the effect of effectively and accurately recognizing whether the value-added diabetic retinopathy exists in the fundus image to be recognized is achieved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of medical technology, and in particular to a lesion type identification method, device, electronic device and storage medium. Background Art

[0002] Diabetic retinopathy (DR) is a fundus disease caused by diabetes. Diabetic retinopathy is mainly divided into early non-proliferative diabetic retinopathy (NPDR) and proliferative diabetic retinopathy (PDR). DR screening is crucial for the eye health of diabetic patients.

[0003] In the related art, a convolutional neural network for identifying fundus images is usually pre-built to complete the identification of fundus lesion types. However, in the process of implementing the present invention, it is found that the related art has at least the following technical problems: Since the fundus blood vessels are slender, the blood vessels only occupy a very small part of the entire image, and the composition of pixels is extremely limited. In addition, the structure of blood vessels is complex and changeable, and the morphological changes in different areas depend on factors such as the number of branches and the position of the bifurcation. Therefore, ordinary convolution in the convolutional neural network cannot extract the characteristics of the blood vessels well, thereby affecting the accuracy of the recognition results. Summary of the invention

[0004] The embodiments of the present invention provide a lesion type identification method, device, electronic device and storage medium to achieve the purpose of effectively and accurately identifying whether proliferative diabetic retinopathy exists in a fundus image to be identified.

[0005] According to one aspect of the present invention, a method for identifying a lesion type is provided, comprising:

[0006] Acquire at least one fundus image to be identified;

[0007] Based on a pre-built optic disc recognition model, determining whether there is proliferative diabetic retinopathy in the fundus image to be recognized;

[0008] The optic disc recognition model is a model obtained by replacing the original convolution module in the original convolution neural network model with a tree convolution module.

[0009] According to another aspect of the present invention, a lesion type identification device is provided, the device comprising:

[0010] An image acquisition module, used to acquire at least one fundus image to be identified;

[0011] A lesion type determination module, used to determine whether proliferative diabetic retinopathy exists in the fundus image to be identified based on a pre-built optic disc recognition model;

[0012] The optic disc recognition model is a model obtained by replacing the original convolution module in the original convolution neural network model with a tree convolution module.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the lesion type identification method described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the lesion type identification method described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention obtains at least one fundus image to be identified; determines whether there is proliferative diabetic retinopathy in the fundus image to be identified through a pre-constructed optic disc recognition model; wherein the optic disc recognition model is a model obtained by replacing the original convolution module in the original convolution neural network model with a tree convolution module. Since the structure of fundus blood vessels is complex and tree-like, the tree convolution module in the optic disc recognition model can accurately identify the characteristics of blood vessels, thereby achieving the effect of effectively and accurately identifying whether there is proliferative diabetic retinopathy in the fundus image to be identified.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a lesion type identification method provided according to an embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of the structure of a lesion type identification device provided according to an embodiment of the present invention;

[0023] Figure 3 It is a schematic diagram of the structure of an electronic device for implementing the lesion type identification method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "etc." and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0027] Figure 1 This is a flow chart of a lesion type identification method provided according to an embodiment of the present invention. This embodiment can be applied to identifying whether there is proliferative diabetic retinopathy in a fundus image. The method can be performed by a lesion type identification device, which can be implemented in the form of hardware and / or software.

[0028] like Figure 1 As shown, the method of this embodiment may specifically include:

[0029] S110, obtaining at least one fundus image to be identified.

[0030] In a specific implementation, at least one fundus image can be acquired by a fundus camera device as the fundus image to be identified. In addition, an image preprocessing operation can be performed on the fundus image to be identified so as to improve the recognition accuracy of the fundus image to be identified. Specifically, the image preprocessing operation performed on the fundus image to be identified may include at least one of brightness adjustment, cropping, enlarging, rotating and reducing.

[0031] It should be noted that an optic disc recognition model can be pre-built, and the optic disc recognition model can be a neural network model. The optic disc recognition model is used to identify whether proliferative diabetic retinopathy exists in the fundus image to be identified.

[0032] Optionally, in order to facilitate the pre-constructed optic disc recognition model to process the fundus image to be identified and obtain an effective recognition result, before determining whether proliferative diabetic retinopathy exists in the fundus image to be identified based on the pre-constructed optic disc recognition model, it also includes: adjusting the image size of the fundus image to be identified to a preset size.

[0033] The preset size may be 640 pixels × 640 pixels. It should be noted that those skilled in the art may determine the preset size according to the input requirements of the pre-built video disc recognition model, and this embodiment does not specifically limit this.

[0034] Specifically, for each fundus image to be identified that is acquired, if the size of the fundus image to be identified is larger than a preset size, the fundus image to be identified is reduced so that the size of the reduced fundus image to be identified is equal to the preset size; if the size of the fundus image to be identified is smaller than the preset size, the fundus image to be identified is enlarged so that the size of the enlarged fundus image to be identified is equal to the preset size.

[0035] In this embodiment, by adjusting the size of each fundus image to be identified to a preset size, the optic disc recognition model can uniformly process the fundus images to be identified, thereby improving the convenience of determining the optic disc of the fundus image to be identified.

[0036] S120: Based on the pre-built optic disc recognition model, determine whether there is proliferative diabetic retinopathy in the fundus image to be recognized.

[0037] The optic disc recognition model is a model obtained by replacing the original convolution module in the original convolution neural network model with a tree convolution module. The original convolution neural network model is a convolution neural network model used to identify whether there is proliferative diabetic retinopathy in the fundus image to be identified.

[0038] It should be noted that the tree convolution module can be a two-dimensional convolution. Taking the horizontal direction as an example, the specific position of each grid in the convolution kernel of the tree convolution is expressed as (x i±c ,y i±c ), where x i±c The horizontal coordinate of the grid at a distance of ±c from the central grid; y i±c Indicates the ordinate of the grid that is ±c away from the central grid in the horizontal direction; c represents the distance from the central grid.

[0039] In order to make the convolution kernel more flexible to focus on the complex geometric features of the target, a deformation offset is introduced. The deformation offset must be taken into account for each grid position in the convolution kernel. The horizontal axis position of each grid in the convolution kernel of the tree convolution module can be expressed as The vertical position of each grid in the convolution kernel of the tree convolution module can be expressed as Among them, x i+c represents the horizontal coordinate of the grid with a distance c from the central grid; y i Indicates the ordinate of the central grid in the horizontal direction; Δy i+c Indicates the deformation offset in the horizontal direction, compared with the grid with a distance c from the center grid and the grid with a distance c-1. j Indicates the horizontal coordinate of the central grid in the vertical direction; Δx j+c Indicates the deformation offset in the vertical direction, compared with the grid with a distance of c from the center grid and the grid with a distance of c-1; j+c represents the ordinate of the grid whose vertical distance from the central grid is c. For example, c={0, 1, 2, 3, 4}.

[0040] In this embodiment, an optic disc recognition model for identifying the optic disc area in the fundus image to be identified can be pre-constructed. Specifically, a plurality of sample fundus images can be acquired through a fundus camera device to form a training set, and the optic disc area in each sample fundus image can be marked. Exemplarily, the optic disc area is marked with a rectangular frame, and the coordinate positions of the upper left point and the lower right point of the rectangular frame are recorded. The pre-constructed replacement convolutional neural network model is trained with sample fundus images with the optic disc area marked to obtain an optic disc recognition model. Among them, the replacement convolutional neural network model is a model obtained by replacing the original convolution module in the original convolutional neural network model with a tree convolution module. Exemplarily, the original convolutional neural network model may include YoloV5.

[0041] In a specific implementation, the fundus image to be identified can be input into the optic disc recognition model, and based on the output result of the optic disc recognition model, it is determined whether there is proliferative diabetic retinopathy in the fundus image to be identified, wherein the output result can be the position information of the optic disc to be identified.

[0042] The technical solution of the embodiment of the present invention obtains at least one fundus image to be identified; determines whether there is proliferative diabetic retinopathy in the fundus image to be identified through a pre-constructed optic disc recognition model; wherein the optic disc recognition model is a model obtained by replacing the original convolution module in the original convolution neural network model with a tree convolution module. Since the structure of fundus blood vessels is complex and tree-like, the tree convolution module in the optic disc recognition model can accurately identify the characteristics of blood vessels, thereby achieving the effect of effectively and accurately identifying whether there is proliferative diabetic retinopathy in the fundus image to be identified.

[0043] In this embodiment, based on a pre-constructed optic disc recognition model, determining whether proliferative diabetic retinopathy exists in the fundus image to be identified includes: inputting the fundus image to be identified into the optic disc recognition model to obtain the position information of the optic disc to be identified in the fundus image to be identified and the optic disc confidence; determining the optic disc shape probability of the fundus image to be identified based on the position information; and determining whether proliferative diabetic retinopathy exists in the fundus image to be identified based on the optic disc confidence and the optic disc shape probability.

[0044] The optic disc shape probability is used to reflect the similarity between the first optic disc shape of the optic disc to be identified and the second optic disc shape corresponding to proliferative diabetic retinopathy.

[0045] In a specific implementation, the fundus image to be identified is input into the optic disc recognition model to obtain the position information and optic disc confidence of the optic disc to be identified in the fundus image to be identified. The position information includes the vertex position of the optic disc area corresponding to the optic disc to be identified. For example, if the optic disc area is a rectangular area, the position information is the first coordinate position of the upper left point and the second coordinate position of the lower right point of the rectangular area. The optic disc confidence reflects the probability that the area corresponding to the determined position information is the optic disc.

[0046] In this embodiment, it can be determined whether there is proliferative diabetic retinopathy in the fundus image to be identified based on the position information and the optic disc confidence. It should be noted that the higher the optic disc confidence, the greater the probability that there is proliferative diabetic retinopathy in the fundus image to be identified; conversely, the smaller the probability that there is proliferative diabetic retinopathy in the fundus image to be identified.

[0047] Specifically, the optic disc area is a rectangular area, and the first shape feature of the first optic disc shape can be determined through the position information, and the similarity between the first shape feature and the second shape feature is determined based on the second shape feature of the second optic disc shape acquired in advance, so as to determine the similarity as the optic disc shape probability. The higher the optic disc shape probability, the more likely the optic disc to be identified is an optic disc with hypertrophic diabetic retinopathy; the lower the optic disc shape probability, the lower the possibility that the optic disc to be identified is an optic disc with hypertrophic diabetic retinopathy.

[0048] Optionally, based on the optic disc confidence and the optic disc shape probability, the method for determining whether proliferative diabetic retinopathy exists in the fundus image to be identified may be: if the optic disc confidence is greater than a preset execution threshold, and the optic disc shape probability is greater than a preset probability threshold, it can be determined that proliferative diabetic retinopathy exists in the fundus image to be identified; or, if the sum of the optic disc confidence and the optic disc shape probability is greater than a first preset sum value, it is determined that proliferative diabetic retinopathy exists in the fundus image to be identified.

[0049] In this embodiment, the optic disc shape probability is determined based on the position information output by the optic disc recognition model, and whether the fundus image to be identified has proliferative diabetic retinopathy is determined based on the optic disc shape probability and the optic disc confidence. From the similarity between the first optic disc shape and the second optic disc shape, and the confidence of the identified optic disc to be identified, it is determined whether the fundus image to be identified has proliferative diabetic retinopathy, which is conducive to improving the accuracy of the determination result.

[0050] Optionally, the optic disc recognition model includes a tree convolution module, a pooling module, a feature fusion module and a detection module; in a specific implementation, the fundus image to be recognized is input into the optic disc recognition model, and the position information of the optic disc to be recognized in the fundus image to be recognized and the optic disc confidence are obtained by: inputting the fundus image to be recognized into the tree convolution module to obtain a feature map corresponding to the fundus image to be recognized; inputting the feature map into the pooling module to obtain a feature vector corresponding to the feature map; inputting the feature vector into the feature fusion module to obtain a feature fusion map corresponding to the feature vector; inputting the feature fusion map into the detection module to obtain the position information of the optic disc to be recognized in the fundus image to be recognized and the optic disc confidence.

[0051] Among them, the pooling module includes SPP-Net. After inputting at least one feature map obtained into the pooling module, the feature map can be pooled, and the operation result is reduced in dimension through the fully connected layer to obtain a feature vector of fixed size. The feature fusion module includes a feature pyramid network (FPN) to extract features in the feature vector to obtain a feature fusion image. By inputting the feature fusion image into the detection module, the position information and the confidence of the optic disc are obtained. In this embodiment, the quality of the target frame of the optic disc recognition model can be accurately evaluated by the CIoU loss function, and the performance of the model to be recognized in complex scenes can be improved.

[0052] This embodiment discloses a process of obtaining position information and optic disc confidence after the fundus image to be identified is input into the optic disc identification model, and effectively provides a method for determining the position information and optic disc confidence through the optic disc identification model.

[0053] In this embodiment, based on the position information, the probability of the optic disc shape of the fundus image to be identified is determined, including: based on the position information, determining the length value and width value corresponding to the optic disc to be identified, determining a first ratio between the square of the width value and the square of the length value, determining the inverse tangent value of the first ratio, and determining the product of the inverse tangent value and a preset value as the optic disc shape probability.

[0054] The first optic disc shape may be a rectangle, and the first shape feature may be a length value and a width value of the first optic disc shape. Based on the determined length value and width value, the optic disc shape probability of the fundus image to be identified is determined.

[0055] In this embodiment, the method of determining the length value and width value corresponding to the video disk to be identified includes: if the position information can be the length value and width value of the rectangular area, the length value and width value of the video disk to be identified are determined by directly reading the position information. Alternatively, if the position information is the first coordinate position of the upper left point and the second coordinate position of the lower right point of the rectangular area, the position difference between the abscissa of the first coordinate position and the second coordinate position is determined as the length value of the video disk to be identified; and the position difference between the ordinate of the first coordinate position and the second coordinate position is determined as the width value of the video disk to be identified.

[0056] Specifically, the formula for determining the probability of the optic disc shape can be:

[0057]

[0058] Among them, ρ2 represents the probability of the optic disc shape, It can be a preset value, v is pi, arctan represents the inverse tangent function, h represents the length value, and w represents the width value.

[0059] This embodiment provides a method for determining the probability of a disc shape, which can effectively and quickly output the probability of a disc shape.

[0060] In a specific implementation, after acquiring at least one fundus image to be identified, it also includes: inputting the fundus image to be identified into a pre-constructed vascular clarity classification model to obtain the vascular clarity probability of the fundus image to be identified; based on the optic disc confidence and the optic disc shape probability, determining whether there is proliferative diabetic retinopathy in the fundus image to be identified, including: determining whether there is proliferative diabetic retinopathy in the fundus image to be identified based on the vascular clarity probability, the optic disc shape probability and the optic disc confidence.

[0061] Among them, the blood vessel clarity classification model includes a deep residual network model.

[0062] In order to more accurately determine whether there is proliferative diabetic retinopathy in the fundus image to be identified, the fundus image to be identified can be input into the vascular clarity classification model to determine the vascular clarity of the fundus image to be identified. The higher the probability of vascular clarity, the higher the vascular clarity of the fundus image to be identified; conversely, the lower the vascular clarity of the fundus image to be identified. The accuracy of the optic disc shape probability and the optic disc confidence can be verified through the vascular clarity probability. Specifically, the greater the probability of vascular clarity, the higher the accuracy of the optic disc shape probability and the optic disc confidence can be verified; conversely, the lower the accuracy of the optic disc shape probability and the optic disc confidence can be verified.

[0063] In this embodiment, based on the blood vessel clear probability, the optic disc shape probability and the optic disc confidence, the method of determining whether there is proliferative diabetic retinopathy in the fundus image to be identified can be: determining the sum of the blood vessel clear probability, the optic disc shape probability and the optic disc confidence, and if the sum is greater than the second preset sum, it can be determined that there is proliferative diabetic retinopathy in the fundus image to be identified; if the sum is less than or equal to the second preset sum, it is determined that there is no proliferative diabetic retinopathy in the fundus image to be identified. Alternatively, the average value of the blood vessel clear probability, the optic disc shape probability and the optic disc confidence can be determined, and if the average value is greater than the preset average value, it can be determined that there is proliferative diabetic retinopathy in the fundus image to be identified; if the average value is less than or equal to the preset average value, it is determined that there is no proliferative diabetic retinopathy in the fundus image to be identified.

[0064] This embodiment combines the probability of blood vessel clarity when determining whether there is proliferative diabetic retinopathy in the fundus image to be identified, thereby combining multiple factors to further more accurately determine whether there is proliferative diabetic retinopathy in the fundus image to be identified.

[0065] In a specific implementation, based on the probability of clear blood vessels, the probability of optic disc shape and the confidence of optic disc, it is determined whether there is proliferative diabetic retinopathy in the fundus image to be identified, including: by inputting the probability of clear blood vessels, the probability of optic disc shape and the confidence of optic disc into a pre-constructed gradient boosting decision tree model, so as to perform weighted summation on the probability of clear blood vessels, the probability of optic disc shape and the confidence of optic disc according to pre-trained weights, and determining the sum as the probability value of the presence of proliferative diabetic retinopathy in the fundus image to be identified; when the probability value is greater than a preset probability threshold, it is determined that there is proliferative diabetic retinopathy in the fundus image to be identified.

[0066] In order to further improve the accuracy of the lesion type recognition results, the weights of the blood vessel clarity probability, optic disc shape probability and optic disc execution degree can be determined based on the pre-built gradient boosting decision tree model. The higher the weight, the more important it is. Specifically, the gradient boosting decision tree model can be used to perform a weighted summation of the blood vessel clarity probability, optic disc shape probability and optic disc confidence according to the pre-trained weights, and the sum value can be determined as the probability value of the presence of proliferative diabetic retinopathy in the fundus image to be identified. The probability value determination formula is as follows:

[0067] P=ω1ρ1+ω2ρ2+ω3ρ3

[0068] Among them, P represents the probability value, ρ1 represents the optic disc confidence, ρ2 represents the optic disc shape probability, and ρ3 represents the blood vessel clear probability; ω1, ω2 and ω3 represent the weights of the optic disc confidence, optic disc shape probability and blood vessel clear probability respectively.

[0069] Specifically, a preset probability threshold may be pre-set, and illustratively, the value range of the preset probability threshold may be 0.5 to 1. If the probability value is greater than the preset probability threshold, it is determined that proliferative diabetic retinopathy exists in the fundus image to be identified. If the probability value is less than or equal to the preset probability threshold, it is determined that proliferative diabetic retinopathy does not exist in the fundus image to be identified.

[0070] This embodiment sets corresponding weights for the blood vessel clarity probability, optic disc shape probability and optic disc confidence, thereby increasing the influence of important factors on the determined probability value, which is conducive to determining the probability value more reasonably and effectively.

[0071] Figure 2 1 is a schematic diagram of the structure of a lesion type identification device provided according to an embodiment of the present invention, and the device is used to execute the lesion type identification method provided in any of the above embodiments. The device and the lesion type identification method of the above embodiments belong to the same inventive concept, and the details not described in detail in the embodiment of the lesion type identification device can refer to the embodiment of the above lesion type identification method. Figure 2 As shown, the device comprises:

[0072] An image acquisition module 10, used to acquire at least one fundus image to be identified;

[0073] The lesion type determination module 11 is used to determine whether there is proliferative diabetic retinopathy in the fundus image to be identified based on a pre-built optic disc recognition model;

[0074] Among them, the optic disc recognition model is a model obtained by replacing the original convolution module in the original convolutional neural network model with a tree convolution module.

[0075] Based on any optional technical solution in the embodiment of the present invention, optionally, the lesion type determination module 11 includes:

[0076] The first image input submodule is used to input the fundus image to be identified into the optic disc recognition model to obtain the position information and optic disc confidence of the optic disc to be identified in the fundus image to be identified;

[0077] A probability determination submodule is used to determine the probability of the optic disc shape of the fundus image to be identified based on the position information; wherein the optic disc shape probability is used to reflect the similarity between the first optic disc shape of the optic disc to be identified and the second optic disc shape corresponding to the hyperplastic diabetic retinopathy;

[0078] The lesion type determination submodule is used to determine whether there is proliferative diabetic retinopathy in the fundus image to be identified based on the optic disc confidence and the optic disc shape probability.

[0079] On the basis of any optional technical solution in the embodiment of the present invention, optionally, the optic disc recognition model includes a tree convolution module, a pooling module, a feature fusion module and a detection module;

[0080] The first image input submodule comprises:

[0081] An image input unit, used for inputting the fundus image to be identified into the tree convolution module to obtain a feature map corresponding to the fundus image to be identified;

[0082] A feature map input unit, used to input the feature map into the pooling module to obtain a feature vector corresponding to the feature map;

[0083] A feature vector input unit, used to input the feature vector into a feature fusion module to obtain a feature fusion graph corresponding to the feature vector;

[0084] The fusion image input unit is used to input the feature fusion image into the detection module to obtain the position information and confidence of the optic disc to be identified in the fundus image to be identified.

[0085] Based on any optional technical solution in the embodiments of the present invention, optionally, the probability determination submodule includes:

[0086] The probability determination unit is used to determine the length value and width value corresponding to the disc to be identified based on the position information, determine a first ratio between the square of the width value and the square of the length value, determine an arc tangent value of the first ratio, and determine the product of the arc tangent value and a preset value as the disc shape probability.

[0087] Based on any optional technical solution in the embodiments of the present invention, optionally, the method further includes:

[0088] A second image input submodule is used for, after acquiring at least one fundus image to be identified, inputting the fundus image to be identified into a pre-built blood vessel clarity classification model to obtain a blood vessel clarity probability of the fundus image to be identified; wherein the blood vessel clarity classification model includes a deep residual network model;

[0089] The lesion type determination submodule includes:

[0090] The lesion type determination unit is used to determine whether there is proliferative diabetic retinopathy in the fundus image to be identified based on the blood vessel clarity probability, the optic disc shape probability and the optic disc confidence.

[0091] Based on any optional technical solution in the embodiments of the present invention, optionally, the lesion type determination unit includes:

[0092] a probability value determination subunit, for inputting the blood vessel clarity probability, the optic disc shape probability and the optic disc confidence into a pre-built gradient boosting decision tree model, so as to perform weighted summation on the blood vessel clarity probability, the optic disc shape probability and the optic disc confidence according to pre-trained weights, and determining the sum value as the probability value of the presence of proliferative diabetic retinopathy in the fundus image to be identified;

[0093] The lesion type determination subunit is used to determine whether proliferative diabetic retinopathy exists in the fundus image to be identified when the probability value is greater than a preset probability threshold.

[0094] Based on any optional technical solution in the embodiments of the present invention, optionally, the method further includes:

[0095] The size adjustment module is used to adjust the image size of the fundus image to be identified to a preset size before determining whether there is proliferative diabetic retinopathy in the fundus image to be identified based on a pre-built optic disc recognition model.

[0096] The technical solution of the embodiment of the present invention obtains at least one fundus image to be identified; determines whether there is proliferative diabetic retinopathy in the fundus image to be identified through a pre-constructed optic disc recognition model; wherein the optic disc recognition model is a model obtained by replacing the original convolution module in the original convolution neural network model with a tree convolution module. Since the structure of fundus blood vessels is complex and tree-like, the tree convolution module in the optic disc recognition model can accurately identify the characteristics of blood vessels, thereby achieving the effect of effectively and accurately identifying whether there is proliferative diabetic retinopathy in the fundus image to be identified.

[0097] It is worth noting that in the embodiment of the above-mentioned lesion type identification device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0098] Figure 3 : is a schematic diagram of the structure of an electronic device that implements the lesion type identification method of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0099] like Figure 3 As shown, the electronic device 20 includes at least one processor 21, and a memory connected to the at least one processor 21, such as a read-only memory (ROM) 22, a random access memory (RAM) 23, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 21 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 22 or the computer program loaded from the storage unit 28 to the random access memory (RAM) 23. In RAM23, various programs and data required for the operation of the electronic device 20 can also be stored. The processor 21, ROM22 and RAM23 are connected to each other through a bus 24. An input / output (I / O) interface 25 is also connected to the bus 24.

[0100] A number of components in the electronic device 20 are connected to the I / O interface 25, including: an input unit 26, such as a keyboard, a mouse, etc.; an output unit 27, such as various types of displays, speakers, etc.; a storage unit 28, such as a disk, an optical disk, etc.; and a communication unit 29, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 29 allows the electronic device 20 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0101] The processor 21 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 21 performs the various methods and processes described above, such as a lesion type recognition method.

[0102] In some embodiments, the lesion type identification method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 28. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 20 via the ROM 22 and / or the communication unit 29. When the computer program is loaded into the RAM 23 and executed by the processor 21, one or more steps of the lesion type identification method described above may be performed. Alternatively, in other embodiments, the processor 21 may be configured to perform the lesion type identification method in any other appropriate manner (e.g., by means of firmware).

[0103] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0105] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0107] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0108] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0109] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0110] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying lesion types, characterized in that: include: Acquire at least one fundus image to be identified; Based on a pre-built optic disc recognition model, determining whether there is proliferative diabetic retinopathy in the fundus image to be recognized; The optic disc recognition model is a model obtained by replacing the original convolution module in the original convolution neural network model with a tree convolution module.

2. The lesion type identification method according to claim 1, characterized in that: Determining whether there is proliferative diabetic retinopathy in the fundus image to be identified based on a pre-built optic disc recognition model includes: Inputting the fundus image to be identified into the optic disc identification model to obtain the position information and optic disc confidence of the optic disc to be identified in the fundus image to be identified; Based on the position information, determining the optic disc shape probability of the fundus image to be identified; wherein the optic disc shape probability is used to reflect the similarity between the first optic disc shape of the optic disc to be identified and the second optic disc shape corresponding to proliferative diabetic retinopathy; Based on the optic disc confidence and the optic disc shape probability, it is determined whether proliferative diabetic retinopathy exists in the fundus image to be identified.

3. The lesion type identification method according to claim 2, characterized in that: The optic disc recognition model includes the tree convolution module, the pooling module, the feature fusion module and the detection module; The step of inputting the fundus image to be identified into the optic disc identification model to obtain the position information and optic disc confidence of the optic disc to be identified in the fundus image to be identified includes: Inputting the fundus image to be identified into the tree convolution module to obtain a feature map corresponding to the fundus image to be identified; Inputting the feature map into the pooling module to obtain a feature vector corresponding to the feature map; Inputting the feature vector into the feature fusion module to obtain a feature fusion graph corresponding to the feature vector; The feature fusion image is input into the detection module to obtain the position information and confidence level of the optic disc to be identified in the fundus image to be identified.

4. The lesion type identification method according to claim 2, characterized in that: The determining, based on the position information, the probability of the optic disc shape of the fundus image to be identified comprises: Based on the position information, a length value and a width value corresponding to the optic disc to be identified are determined, a first ratio between the square of the width value and the square of the length value is determined, an inverse tangent value of the first ratio is determined, and a product of the inverse tangent value and a preset value is determined as the optic disc shape probability.

5. The method for identifying the lesion type according to claim 2, characterized in that: After obtaining at least one fundus image to be identified, the method further includes: Inputting the fundus image to be identified into a pre-built blood vessel clarity classification model to obtain the blood vessel clarity probability of the fundus image to be identified; wherein the blood vessel clarity classification model includes a deep residual network model; The determining whether there is proliferative diabetic retinopathy in the fundus image to be identified based on the optic disc confidence and the optic disc shape probability comprises: Based on the blood vessel clarity probability, the optic disc shape probability and the optic disc confidence, it is determined whether proliferative diabetic retinopathy exists in the fundus image to be identified.

6. The method for identifying the lesion type according to claim 5, characterized in that: The determining whether there is proliferative diabetic retinopathy in the fundus image to be identified based on the blood vessel clarity probability, the optic disc shape probability and the optic disc confidence includes: By inputting the blood vessel clarity probability, the optic disc shape probability and the optic disc confidence into a pre-constructed gradient boosting decision tree model, the blood vessel clarity probability, the optic disc shape probability and the optic disc confidence are weighted and summed according to pre-trained weights, and the sum is determined as the probability value of the presence of proliferative diabetic retinopathy in the fundus image to be identified; When the probability value is greater than a preset probability threshold, it is determined that proliferative diabetic retinopathy exists in the fundus image to be identified.

7. The lesion type identification method according to claim 1, characterized in that: Before determining whether proliferative diabetic retinopathy exists in the fundus image to be identified based on the pre-built optic disc recognition model, the method further includes: The image size of the fundus image to be identified is adjusted to a preset size.

8. A lesion type identification device, characterized in that: include: An image acquisition module, used for acquiring at least one fundus image to be identified; A lesion type determination module, used to determine whether proliferative diabetic retinopathy exists in the fundus image to be identified based on a pre-built optic disc recognition model; The optic disc recognition model is a model obtained by replacing the original convolution module in the original convolution neural network model with a tree convolution module.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the lesion type identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the lesion type identification method according to any one of claims 1 to 7 when executed.

Images