A method, device, equipment and storage medium for retinal image scoring

Abstract

The present invention relates to the technical field of image processing, and specifically relates to a method, device, equipment and storage medium for scoring retinal images. The present invention first divides a retinal image into a plurality of image blocks, applies a trained deep learning model to each image block to obtain a classification result for each image block, then screens out target image blocks from the plurality of image blocks according to the classification results, and obtains a scoring result of the retinal image according to the target image blocks. From the above analysis, it can be seen that the evaluation result of the present invention includes the classification result of each target image block and each target image block, thereby improving the accuracy of the scoring result.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method, device, equipment and storage medium for scoring retinal images. Background Art

[0002] A retinal image of a human body is collected and scored. Different scores reflect different degrees of retinal lesions corresponding to the retinal image. The prior art only scores the retinal image based on the presence or absence of lesions, thereby reducing the accuracy of the scoring result.

[0003] In summary, the prior art reduces the accuracy of the scoring result.

[0004] Therefore, the prior art still needs to be improved and enhanced. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a method, device, equipment and storage medium for scoring retinal images, which solves the problem that the prior art reduces the accuracy of the scoring result.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for scoring a retinal image, which includes:

[0008] Obtain a retinal image, divide the retinal image into a plurality of image blocks, and apply a trained deep learning model to each of the image blocks to obtain a classification result for each of the image blocks;

[0009] According to the classification result, screen out target image blocks from the plurality of image blocks;

[0010] According to the target image blocks, obtain a scoring result of the retinal image.

[0011] In one implementation, the step of dividing the retinal image into a plurality of image blocks and applying a trained deep learning model to each of the image blocks to obtain a classification result for each of the image blocks includes:

[0012] Set a maximum inscribed square on the retinal image;

[0013] Divide the maximum inscribed square into a plurality of equal-sized image blocks.

[0014] Apply each of the trained deep learning models to each of the image blocks respectively to obtain output results of each of the trained deep learning models, and each of the trained deep learning models corresponds to a type of lesion;

[0015] Based on each of the said output results, obtain the classification result of each of the said image blocks.

[0016] In one implementation, the step of screening out target image blocks from a number of the said image blocks according to the classification result includes:

[0017] Screen out the lesion classification result from the classification result;

[0018] Take the image block corresponding to the lesion classification result as the target image block.

[0019] In one implementation, the step of obtaining the scoring result of the retinal image according to the target image block includes:

[0020] Determine the main score corresponding to the lesion classification result of the target image block;

[0021] Count the total number of pixel points of the target image block;

[0022] According to the lesion classification result, determine the weight coefficient of the total number of pixel points;

[0023] Obtain a sub-score according to the total number of pixel points and the weight coefficient;

[0024] Obtain the scoring result of the retinal image according to the main score and the sub-score.

[0025] In one implementation, the step of obtaining a sub-score according to the total number of pixel points and the weight coefficient includes:

[0026] Determine the maximum number of pixel points and the minimum number of pixel points corresponding to the lesion classification result;

[0027] According to the maximum number of pixel points and the minimum number of pixel points, perform normalization processing on the total number of pixel points to obtain the total number of pixel points after normalization;

[0028] Multiply the total number of pixel points after normalization by the weight coefficient to obtain a sub-score.

[0029] In one implementation, the trained deep learning model is a deep learning model after training. The deep learning model includes two parallel deep learning sub-models, and each deep learning sub-model includes an input layer, a convolutional layer, a fully connected layer, and an output layer connected in sequence.

[0030] In one implementation, the training method of the trained deep learning model includes:

[0031] Obtain a number of retinal sample images marked with classification information, and divide each of the retinal sample images into a number of sub-image sample blocks;

[0032] Screen out target sample image blocks that match the classification information from a number of the sub-image sample blocks;

[0033] Obtain a number of retinal reference images, and divide each of the retinal reference images into a number of sub-image reference blocks;

[0034] Input the target sample image blocks of each of the retinal sample images into one of the deep learning sub-models to obtain a sub-training result output by one of the deep learning sub-models;

[0035] Input the sub-image reference blocks of each of the retinal reference images into another one of the deep learning sub-models to obtain a sub-reference result output by another one of the deep learning sub-models;

[0036] Obtain an objective function based on the sub-training result corresponding to each of the target sample image blocks and the sub-reference result corresponding to each of the sub-image reference blocks;

[0037] Train the two deep learning sub-models according to the objective function, and use one of the trained deep learning sub-models as the trained deep learning model.

[0038] In a second aspect, an embodiment of the present invention further provides a retinal image scoring device, where the device includes the following components:

[0039] A classification module, configured to obtain a retinal image and apply the trained deep learning model to the retinal image to obtain a main classification result of the retinal image;

[0040] An image division module, configured to divide the retinal image into a number of image blocks, and screen out target image blocks that match the main classification result from the number of image blocks;

[0041] A scoring module, configured to obtain a scoring result of the retinal image based on the main classification result and the target image blocks.

[0042] In a third aspect, an embodiment of the present invention further provides a terminal device, where the terminal device includes a memory, a processor, and a retinal image scoring program stored in the memory and executable on the processor. When the processor executes the retinal image scoring program, the steps of the above-mentioned retinal image scoring method are implemented.

[0043] Fourthly, an embodiment of the present invention further provides a computer-readable storage medium, on which a retinal image scoring program is stored. When the retinal image scoring program is executed by a processor, the steps of the above-mentioned retinal image scoring method are implemented.

[0044] Beneficial effects: The present invention first divides a retinal image into a plurality of image blocks, applies a trained deep learning model to each image block to obtain a classification result for each image block, then screens out target image blocks from the plurality of image blocks according to the classification results, and obtains a scoring result of the retinal image according to the target image blocks. From the above analysis, it can be seen that the evaluation result of the present invention includes the classification result of each target image block and each target image block, thereby improving the accuracy of the scoring result. Description of the Drawings

[0045] Figure 1 is the overall flowchart of the present invention;

[0046] Figure 2 is the structural diagram of the retinal image scoring device provided by the present invention;

[0047] Figure 3 is the internal structure principle block diagram of the terminal device provided by the embodiment of the present invention. Detailed Embodiments

[0048] The following combines embodiments and the accompanying drawings of the specification to clearly and completely describe the technical solutions in the present invention. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present invention.

[0049] It has been found through research that by collecting a retinal image of a human body and scoring the retinal image, different scores reflect different degrees of retinal lesions corresponding to the retinal image. The prior art only scores the retinal image based on the presence or absence of lesions, thereby reducing the accuracy of the scoring result.

[0050] To solve the above technical problems, the present invention provides a retinal image scoring method, device, equipment and storage medium, which solves the problem that the prior art reduces the accuracy of the scoring result. Specifically in implementation, first divide the retinal image into a plurality of image blocks, apply a trained deep learning model to each image block to obtain a classification result for each image block; then screen out target image blocks from the plurality of image blocks according to the classification results; finally, obtain a scoring result of the retinal image according to the target image blocks.

[0051] The retinal image scoring method of this embodiment can be applied to a terminal device, and the terminal device can be a terminal product with image processing functions, such as a computer, etc. In this embodiment, asFigure 1 As shown, the method for scoring a retinal image specifically includes the following steps:

[0052] S100, obtain a retinal image, divide the retinal image into a plurality of image blocks, and apply a trained deep learning model to each of the image blocks to obtain a classification result for each of the image blocks.

[0053] S200, based on the classification result, screen out target image blocks from the plurality of image blocks.

[0054] S300, based on the target image blocks, obtain a scoring result for the retinal image.

[0055] In one embodiment, the deep learning model in step S100 is composed of two parallel deep learning sub-models A and B, there is no information interaction between the two deep learning sub-models, and each deep learning sub-model includes an input layer, a convolutional layer, a fully connected layer, and an output layer connected in sequence. Among them, the input layer contains 36 neurons, the convolutional layer is 3 series-connected convolutional layers, the fully connected layer is 3 series-connected fully connected layers, and the activation function values of each neuron on the deep learning sub-model are 0, 0.5, and 1 respectively.

[0056] The retina includes eight types of lesions, namely dot hemorrhage, microaneurysm, yellowish-white hard exudate, white cotton-like soft exudate combined with hemorrhage spots, retinal neovascular membrane, retinal neovascular membrane combined with retinal vitreous hemorrhage, neovascularization combined with retinal fibrosis, neovascularization combined with retinal hyperplasia, and proliferative retinal traction causing retinal detachment. For each type of lesion, a deep learning model is trained, that is, eight deep learning models are trained. Training each deep learning model includes the following specific steps S01 to S07:

[0057] S01, obtain a plurality of retinal sample images marked with classification information, and divide each of the retinal sample images into a plurality of sub-image sample blocks.

[0058] If the original retinal image comes from a human body with retinal dot hemorrhage, then the classification information of this original retinal image is dot hemorrhage. Perform position registration on all such original retinal images. After registration, perform size normalization on all original retinal images in the manner of the largest inscribed square of equal size. That is, perform position registration on all original retinal images with the same classification information, and then intercept all such original retinal images in the manner of the same-sized inscribed square. The image covered by the largest inscribed square is used as the retinal sample image.

[0059] Each of the above retinal sample images is divided into a number of sub-image sample blocks, and each sub-image sample block is a non-overlapping image block of equal size of 6 pixels x 6 pixels.

[0060] S02, screen out target sample image blocks that match the classification information from a number of the sub-image sample blocks.

[0061] For example, if a retinal sample image is divided into eight sub-image sample blocks, and two of the sub-image sample blocks have no lesions, then the remaining six sub-image sample blocks are the target sample image blocks. Only using images with lesions to train the deep learning model has the following beneficial effects:

[0062] For the image blocks in the area without lesions, it is not necessary to calculate the network output results, nor is it necessary to calculate the difference values between them and the corresponding image blocks in the comparison data subset. Therefore, not using the image blocks in the area without lesions can save training time without affecting the final training accuracy.

[0063] S03, obtain a number of retinal reference images, and divide each of the retinal reference images into a number of sub-image reference blocks.

[0064] The retinal reference images are normal images, that is, images without any lesion types. The number of sub-image reference blocks is equal to the number of target sample image blocks.

[0065] S04, input the target sample image blocks of each of the retinal sample images into one of the deep learning sub-models to obtain a sub-training result output by one of the deep learning sub-models.

[0066] S05, input the sub-image reference blocks of each of the retinal reference images into another deep learning sub-model to obtain a sub-reference result output by another deep learning sub-model.

[0067] The deep learning model of this embodiment is composed of two deep learning sub-models A and B, and the two deep learning sub-models are executed in parallel, that is, there is no interaction between the two deep learning sub-models.

[0068] S06, obtain the objective function f based on the sub-training results corresponding to each of the target sample image blocks and the sub-reference results corresponding to each of the sub-image reference blocks.

[0069]

[0070] Wherein, a is the total number of retinal sample images included in each type of classification information, that is, the total number of retinal sample images included in the typical features of each type of lesion, b is the total number of retinal reference images, and J is the number of sub-image reference blocks or the number of target sample image blocks. p ij is the sub-training result corresponding to the j-th target sample image block of the i-th retinal sample image, q mj is the sub-reference result corresponding to the j-th sub-image reference block of the m-th retinal reference image, p i+k,j is the sub-training result corresponding to the j-th target sample image block of the (i + k)-th retinal sample image, q m+n,j is the sub-reference result corresponding to the j-th sub-image reference block of the (m + n)-th retinal reference image, || || 2 represents the two-norm.

[0071] The first term in f represents the difference between the network output result corresponding to the data subset of the typical features of this type of lesion and the network output result corresponding to the comparison data subset, the second term represents the difference between the network output results corresponding to the data subset of the typical features of this type of lesion, and the third term represents the difference between the network output results corresponding to the comparison data subset.

[0072] S07. According to the objective function, train the two deep learning sub-models, and use one of the trained deep learning sub-models as the trained deep learning model.

[0073] Train the two deep learning sub-models simultaneously. After training is completed, only use one of the deep learning sub-models as the trained deep learning model. That is, during later application, only input the image blocks to be classified into one of the trained deep learning sub-models to obtain an output result. Input the reference image blocks marked as non-lesioned into the other trained deep learning sub-model to obtain another output result. Determine whether there is a lesion and the type of lesion based on the difference between the two output results.

[0074] At the starting step of network training, set each weight coefficient of the network to a random number between -1 and 1. The network training process aims to maximize the value of the objective function f, that is, adjust the network weight coefficients to maximize the value of the objective function f.

[0075] The meaning of the objective function f is as follows:

[0076] The difference between the network output results of any retinal image in the data subset of the typical features of the lesion and each retinal image in the comparison data subset, and maximize the sum of these difference values and minimize the variance of the output values of the two neurons in the output layer of the comparison detection model as the training objective function.

[0077] By training the deep learning model using the images corresponding to different classification information with the above method, there will be a corresponding trained deep learning model for each classification information, that is, there will be a trained deep learning model for each typical feature of the lesion, and there will be eight trained deep learning models for eight typical features of the lesion.

[0078] In one embodiment, step S100 includes the following specific steps S101 to S104:

[0079] S101, set the maximum inscribed square on the retinal image.

[0080] S102, divide the maximum inscribed square into a number of image patches.

[0081] The size of each image patch is 6 pixels x 6 pixels, and there is no overlap between the image patches.

[0082] S103, apply each trained deep learning model to each of the image patches respectively to obtain the output results of each of the trained deep learning models, and each of the trained deep learning models corresponds to a lesion type.

[0083] S104, obtain the classification result of each image patch according to each of the output results.

[0084] In this embodiment, there are eight trained deep learning models, which are respectively used to identify eight lesion types. Each image patch is input into the above eight trained deep learning models respectively, and each of the eight trained deep learning models outputs a result, that is, each image patch has eight output results. Calculate the difference between the output result of each trained deep learning model and the benchmark result corresponding to each trained deep learning model. If the difference is greater than the threshold corresponding to the trained deep learning model, then the classification result of the image patch is the lesion type corresponding to the trained deep learning model.

[0085] Use P i′j′ to represent the output result of the i'-th image patch input into the j'-th trained deep learning model. If the lesion type corresponding to the j'-th trained deep learning model is dot hemorrhage, and the threshold corresponding to the j'-th trained deep learning model is Y j′ , input the image benchmark patch without lesion into the trained deep learning model j', and the output of the trained deep learning model j' is the benchmark result X j′ . If the difference between P i′j′ and the benchmark result X j′ is greater than Y j′ , then the classification result of the i'-th image patch is dot hemorrhage.

[0086] In this embodiment, if the difference between the output result of a trained deep learning model and its reference result is not greater than the threshold, the classification result of the image patch is a non-lesion result.

[0087] In one embodiment, the specific process of step S200 is as follows: select the lesion classification results from the classification results; use the image patches corresponding to the lesion classification results as target image patches.

[0088] That is, a retinal image is divided into several image patches. Some image patches have lesion areas, and some do not. The image patches with lesion areas are used as target image patches. This is because the image patches without lesion areas are meaningless for the final assessment of the lesion degree, and removing these image patches can also reduce the computational amount.

[0089] In one embodiment, step S300 includes the following specific steps S301 to S307:

[0090] S301, determine the main score S corresponding to the lesion classification result of the target image patch i″ .

[0091] S i″ = NUM i″ * s i″

[0092] NUM i″ is the total number of target image patches belonging to the i''-th lesion classification result, and s i″ is the scoring score corresponding to the i''-th lesion classification result.

[0093] The relationship between the lesion classification result and the main score is shown in Table 1. That is, when the lesion classification result is dot hemorrhage, the scoring score of this lesion classification result is 5; when the lesion classification result is microaneurysm, the scoring score of this lesion classification result is 7, and so on.

[0094] S302, count the total number of pixel points h of the target image patches belonging to the same lesion classification result i″ .

[0095] h i″ = NUM i″ · 36

[0096] h i″ is the total number of pixel points belonging to the i''-th lesion classification result. For example, if the number of target image patches belonging to the lesion classification result of dot hemorrhage is three, and since the number of pixel points of each target image patch in this embodiment is 36, then the total number of pixel points corresponding to the lesion classification result of dot hemorrhage is 108.

[0097] S303. Determine the total number h of the pixel points according to the lesion classification result. i″ The weight coefficient w i″ .

[0098] As shown in Table 1, when the target image block corresponding to the total number of pixel points belongs to the lesion classification result of punctate hemorrhage, the weight coefficient of the total number of pixel points is 1.

[0099] Table 1

[0100]

[0101] S304. Determine the maximum number and minimum number of pixel points corresponding to the lesion classification result.

[0102] For an entire image, the maximum number x and minimum number y of pixel points corresponding to each lesion classification result i″ are fixed, that is, x and y are obtained based on statistics.

[0103] S305. Normalize the total number h of the pixel points according to the maximum number and minimum number of pixel points to obtain the normalized total number z of the pixel points i″ . i″ .

[0104]

[0105] z i″ The value of z is between 0 and 1. The statistics of x and y need to be performed on a batch of retinal images with typical lesion features (i.e., lesion classification results).

[0106] S306. Multiply the normalized total number z of the pixel points i″ by the weight coefficient w i″ to obtain a sub-score.

[0107] S307. Obtain the scoring result R of the retinal image according to the main score and the sub-score.

[0108]

[0109] In summary, the scoring method of the present invention is applicable to the discrimination of diabetic retinopathy. The present invention can finely evaluate the degree of the lesion. For example, for different patients with the same type of diabetic retinopathy, the degree of the lesion is different. For example, the number of the same type of lesion areas such as bleeding points means a certain difference in their conditions. The method of the present invention can identify the above differences to better evaluate the degree of the lesion.

[0110] This embodiment also provides a retinal image scoring device, as Figure 2 shown. The device includes the following components:

[0111] A classification module 01, configured to obtain a retinal image, divide the retinal image into a plurality of image blocks, and apply a trained deep learning model to each of the image blocks to obtain a classification result for each of the image blocks;

[0112] An image screening module 02, configured to screen out target image blocks from the plurality of image blocks according to the classification result;

[0113] A scoring module 03, configured to obtain a scoring result of the retinal image according to the target image blocks.

[0114] Based on the above embodiment, the present invention further provides a terminal device, and its principle block diagram can be as Figure 3 shown. The terminal device includes a processor, a memory, a network interface, and a display screen connected through a system bus. Among them, the processor of the terminal device is used to provide computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a retinal image scoring method. The display screen of the terminal device can be a liquid crystal display screen or an electronic ink display screen.

[0115] Those skilled in the art can understand that Figure 3 the principle block diagram shown in

[0116] merely shows the block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the terminal device to which the solution of the present invention is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0117] Obtain a retinal image, divide the retinal image into a plurality of image blocks, and apply a trained deep learning model to each of the image blocks to obtain a classification result for each of the image blocks;

[0118] Screen out target image blocks from the plurality of image blocks according to the classification result;

[0119] Based on the target image block, obtain the scoring result of the retinal image.

[0120] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for scoring retinal images, characterized in that, including: obtaining a retinal image, dividing the retinal image into a plurality of image patches, applying a trained deep learning model to each of the image patches to obtain a classification result for each of the image patches, there are eight trained deep learning models, which are respectively used to identify eight types of lesions, and the eight types of lesions are respectively dot hemorrhage, microaneurysm, yellowish-white hard exudate, white cotton-like soft exudate combined with hemorrhage spots, retinal neovascular membrane, retinal neovascular membrane combined with retinal vitreous hemorrhage, neovascularization combined with retinal fibrosis, neovascularization combined with retinal hyperplasia, and proliferative retinal traction causing retinal detachment, the trained deep learning model is a deep learning model after training, the deep learning model includes two parallel deep learning sub-models, and each of the deep learning sub-models includes an input layer, a convolutional layer, a fully connected layer, and an output layer connected in sequence; screening out target image patches from the plurality of image patches according to the classification result; obtaining a scoring result of the retinal image according to the target image patches; the screening out target image patches from the plurality of image patches according to the classification result includes: screening out lesion classification results from the classification result; using the image patches corresponding to the lesion classification results as target image patches; the obtaining a scoring result of the retinal image according to the target image patches includes: Determine the main score S corresponding to the lesion classification result of the target image patch i″ : S i″ = NUM i″ * s i″ NUM i″ is the total number of target image patches belonging to the i''-th lesion classification result, s i″ is the scoring score corresponding to the i''-th lesion classification result; counting the total number of pixel points of the target image patches belonging to the same lesion classification result; determining a weight coefficient of the total number of pixel points according to the lesion classification result; obtaining a sub-score according to the total number of pixel points and the weight coefficient; Based on the main score and the sub-score, obtain the scoring result R of the retinal image: w i″ is the weight coefficient, z i″ is the total number of pixel points after normalization; the dividing the retinal image into a plurality of image patches and applying a trained deep learning model to each of the image patches to obtain a classification result for each of the image patches includes: setting a maximum inscribed square on the retinal image; equally dividing the maximum inscribed square into a plurality of image patches; applying each of the trained deep learning models to each of the image patches respectively to obtain output results of each of the trained deep learning models, and each of the trained deep learning models corresponds to one type of lesion; obtaining a classification result for each of the image patches according to the respective output results; the training method of the trained deep learning model includes: obtaining a plurality of retinal sample images marked with classification information, and dividing each of the retinal sample images into a plurality of image sample patches; screening out target sample image patches matching the classification information from the plurality of image sample patches; obtaining a plurality of retinal reference images, and dividing each of the retinal reference images into a plurality of image reference patches; inputting the target sample image patches of each of the retinal sample images into one of the deep learning sub-models to obtain a sub-training result output by one of the deep learning sub-models; inputting the image reference patches of each of the retinal reference images into the other deep learning sub-model to obtain a sub-reference result output by the other deep learning sub-model; Based on the sub-training results corresponding to each of the target sample image patches and the sub-reference results corresponding to each of the image reference patches, an objective function is obtained; Based on the objective function, two of the deep learning sub-models are trained, and one of the trained deep learning sub-models is used as the trained deep learning model.

2. The retinal image scoring method according to claim 1, wherein The obtaining of the sub-score based on the total number of pixel points and the weight coefficient includes: Determining the maximum number of pixel points and the minimum number of pixel points corresponding to the lesion classification result; Normalizing the total number of pixel points based on the maximum number of pixel points and the minimum number of pixel points to obtain the normalized total number of pixel points; Multiplying the normalized total number of pixel points by the weight coefficient to obtain the sub-score.

3. A retinal image scoring device, characterized in that, The device includes the following components: A classification module, configured to obtain a retinal image, divide the retinal image into a plurality of image patches, apply the trained deep learning model to each of the image patches to obtain the classification result of each of the image patches. There are eight trained deep learning models, which are respectively used to identify eight types of lesions. The eight types of lesions are dot hemorrhage, microaneurysm, yellowish-white hard exudate, white cotton-like soft exudate combined with hemorrhage spot, retinal neovascular membrane, retinal neovascular membrane combined with retinal vitreous hemorrhage, neovascularization combined with retinal fibrosis, neovascularization combined with retinal proliferation, and proliferative retinal traction causing retinal detachment. The trained deep learning model is the deep learning model after training. The deep learning model includes two parallel deep learning sub-models, and each of the deep learning sub-models includes an input layer, a convolutional layer, a fully connected layer, and an output layer connected in sequence; An image screening module, configured to screen out target image patches from a plurality of the image patches according to the classification result; A scoring module, configured to obtain the scoring result of the retinal image according to the target image patches; The screening out of target image patches from a plurality of the image patches according to the classification result includes: Screening out the lesion classification result from the classification result; Taking the image patch corresponding to the lesion classification result as the target image patch; The obtaining of the scoring result of the retinal image according to the target image patches includes: Determine the main score S corresponding to the lesion classification result of the target image patch i″ : S i″ = NUM i″ * s i″ NUM i″ is the total number of target image patches belonging to the i''-th lesion classification result, s i″ is the scoring score corresponding to the i''-th lesion classification result; Counting the total number of pixel points of the target image patches belonging to the same lesion classification result; Determining the weight coefficient of the total number of pixel points according to the lesion classification result; Obtaining the sub-score according to the total number of pixel points and the weight coefficient; Based on the main score and the sub-score, obtain the scoring result R of the retinal image: w i″ is the weight coefficient, and z i″ is the total number of pixel points after normalization; The dividing of the retinal image into a plurality of image patches and applying the trained deep learning model to each of the image patches to obtain the classification result of each of the image patches includes: Setting a maximum inscribed square on the retinal image; Equally dividing the maximum inscribed square into a plurality of image patches; Applying each of the trained deep learning models to each of the image patches respectively to obtain the output results of each of the trained deep learning models, and each of the trained deep learning models corresponds to a type of lesion; Based on each of the said output results, obtain the classification result of each of the said image blocks; The training method of the said trained deep learning model includes: Obtain a number of retinal sample images marked with classification information, and divide each of the said retinal sample images into a number of image sample blocks; Select target sample image blocks matching the said classification information from a number of the said image sample blocks; Obtain a number of retinal reference images, and divide each of the said retinal reference images into a number of image reference blocks; Input the target sample image blocks of each of the said retinal sample images into one of the said deep learning sub-models, and obtain a sub-training result output by one of the said deep learning sub-models; Input the image reference blocks of each of the said retinal reference images into another one of the said deep learning sub-models, and obtain a sub-reference result output by another one of the said deep learning sub-models; Based on the sub-training result corresponding to each of the said target sample image blocks and the sub-reference result corresponding to each of the said image reference blocks, obtain an objective function; Based on the objective function, train the two said deep learning sub-models, and use one of the trained said deep learning sub-models as the trained deep learning model.

4. A terminal device, characterized in that, The said terminal device includes a memory, a processor, and a retinal image scoring program stored in the memory and operable on the processor. When the processor executes the retinal image scoring program, the steps of the retinal image scoring method according to any one of claims 1-2 are implemented.

5. A computer-readable storage medium, characterized in that, A retinal image scoring program is stored on the said computer-readable storage medium. When the retinal image scoring program is executed by a processor, the steps of the retinal image scoring method according to any one of claims 1-2 are implemented.

Images