Method, device, medium and equipment for determining fundus structural features

Abstract

The present invention provides a method, device, medium and equipment for determining fundus structure features, the method comprising: acquiring a fundus image; inputting the fundus image into a pre-trained target frame detection model, generating a first target detection frame and a second target detection frame on the fundus image; determining a first target detection point and a second target detection point according to the first target detection frame, determining an optic disc center point according to the first target detection point and the second target detection point, and determining a third target detection point and a fourth target detection point according to the second target detection frame; determining a fundus vascular arch fitting curve according to the optic disc center point, the third target detection point and the fourth target detection point, thereby determining the vascular arch. By accurately determining the fundus vascular arch and its fitting curve, it is not only helpful to help identify the eye, but also helpful to understand fundus abnormalities and detailed abnormality analysis.

Description

Technical Field

[0001] The present invention relates to the field of fundus image feature acquisition, and in particular to a method, device, medium and equipment for determining fundus structure features. Background Art

[0002] In fundus images, vascular arches are one of the basic features of the fundus. Detecting and identifying them is not only helpful for distinguishing eyes (left and right eyes), but also helps to understand fundus abnormalities, and also provides more valuable information for fundus disease analysis. Segmenting (i.e. identifying blood vessels from the background) and classifying (i.e. classifying blood vessels) blood vessels in fundus images are of great significance in clinical diagnosis and treatment. Taking fundus blood vessels as an example, the fundus is the only area of ​​the human body where blood vessels can be directly observed without intervention. Many systemic diseases and cardiovascular and cerebrovascular diseases will affect the morphology of fundus blood vessels and have different effects on the formation of arteries and veins. For example, clinical studies have confirmed that a decrease in the fundus arteriovenous width ratio will increase the risk of stroke; the narrowing of fundus arteries is related to the development of hypertension and diabetes.

[0003] In the process of realizing the present invention, the inventors found that there are at least the following problems in the prior art: the prior art lacks a method for extracting fundus vascular arches. Summary of the invention

[0004] In view of this, an object of embodiments of the present invention is to provide a method, device, medium and equipment for determining fundus structure characteristics, which can quickly and accurately determine the characteristics of the vascular arch and obtain its fitting curve.

[0005] To achieve the above object, in a first aspect, a method for determining fundus structure features is provided, comprising:

[0006] Acquire fundus images;

[0007] Inputting the fundus image and the fundus image into a pre-trained target frame detection model, generating a first target detection frame and a second target detection frame on the fundus image;

[0008] Determine a first target detection point and a second target detection point according to the first target detection frame, determine a optic disc center point according to the first target detection point and the second target detection point, and determine a third target detection point and a fourth target detection point according to the second target detection frame;

[0009] A fundus vascular arch fitting curve is determined according to the optic disc center point, the third target detection point and the fourth target detection point, and the fundus vascular arch is determined by the fundus vascular arch fitting curve.

[0010] In some possible implementations, the method further includes: acquiring a fundus training image; annotating the fundus image to obtain a training sample; and inputting the training sample into a deep learning target frame detection network for training to obtain the target frame detection model.

[0011] In some possible implementations, the labeling of the fundus training image to obtain a training sample specifically includes:

[0012] Determine a circumscribed rectangular frame of an optic disc in the fundus training image, obtain a first sample detection frame, determine a first vertex and a second vertex of a diagonal line of the first sample detection frame, obtain a first sample detection point and a second sample detection point, and determine a center point of the optic disc according to the first sample detection point and the second sample detection point;

[0013] Determine the lower half and the upper half of the main blood vessel in the fundus image, take the optic disc center point as the center and draw a circle with an arbitrary radius length as the radius, intersect with the lower half of the main blood vessel to obtain a third sample detection point, draw a vertical line through the third sample detection point to intersect with the upper half of the main blood vessel to obtain a first auxiliary intersection point, draw a horizontal line through the first auxiliary intersection point to intersect with the vertical line through the optic disc center point to obtain a fourth sample detection point, and determine a second sample detection frame according to the third sample detection point, the first auxiliary intersection point and the fourth sample detection point;

[0014] The training sample is acquired according to the first sample detection frame and the second sample detection frame.

[0015] In some possible implementations, inputting the training samples into a deep learning target box detection network for training to obtain a target box detection model specifically includes:

[0016] Step S1: input the training sample into a deep learning target box detection network for convolution to obtain feature maps of different scales;

[0017] Step S2: on the feature maps of different scales, taking the center points of different cells as the center, generating a plurality of target pre-selection boxes with different aspect ratios;

[0018] Step S3: determining a first target detection frame and a second target detection frame according to a preset intersection-and-union ratio for the target pre-selection frame, determining a first target detection point and a second target detection point according to the first target detection frame, and calculating coordinate error values ​​of the first target detection point and the second target detection point relative to the first sample detection point and the second sample detection point, determining coordinates of a third target detection point and a fourth target detection point according to the second target detection frame, and calculating coordinate error values ​​of the third target detection point and the fourth target detection point relative to the third sample detection point and the fourth sample detection point;

[0019] The above steps S1 to S3 are continuously iterated until the error value is reduced to a preset error value, and the target frame detection model is obtained.

[0020] In some possible implementations, inputting the fundus image into a pre-trained target frame detection model to generate a first target detection frame and a second target detection frame on the fundus image specifically includes:

[0021] Inputting the fundus image into the target frame detection model for convolution to obtain feature maps of different scales;

[0022] On each of the scale feature maps, taking the center points of different cells as the center, generating a plurality of target pre-selection boxes with different aspect ratios;

[0023] The target pre-selected boxes on the feature maps of different scales are combined, and the first target detection box and the second target detection box are respectively selected using a non-maximum suppression method.

[0024] In some possible implementations, obtaining the vascular arch fitting curve according to the optic disc center point, the third target detection point, and the fourth target detection point specifically includes:

[0025] Connect the third target detection point, the optic disc center point and the fourth target detection point in sequence,

[0026] The third target detection point is used as a starting endpoint and the fourth target detection point is used as a terminating endpoint, and a fundus vascular arch fitting curve is determined by Bessel function fitting.

[0027] In some possible implementations, the method further includes: determining an opening angle of a fundus vascular arch according to the optic disc center point, the third target detection point, and the fourth target detection point.

[0028] In a second aspect, the present invention provides a device for determining fundus structural features, comprising:

[0029] An acquisition module, for acquiring fundus images;

[0030] A target frame detection module, used for inputting the fundus image into a pre-trained target frame detection model, and generating a first target detection frame and a second target detection frame on the fundus image;

[0031] A first determination module is used to determine a first target detection point and a second target detection point according to the first target detection frame, determine a optic disc center point according to the first target detection point and the second target detection point, and determine a third target detection point and a fourth target detection point according to the second target detection frame;

[0032] The second determination module is used to determine a fundus vascular arch fitting curve according to the optic disc center point, the third target detection point and the fourth target detection point, and determine the fundus vascular arch through the fundus vascular arch fitting curve.

[0033] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned methods for determining fundus structure features.

[0034] In a fourth aspect, the present invention provides a device for determining fundus structure features, comprising:

[0035] one or more processors;

[0036] A storage device for storing one or more programs;

[0037] When the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above methods for determining fundus structure features.

[0038] The above technical solution has the following beneficial effects:

[0039] The embodiment of the present invention generates a first target detection frame and a second target detection frame on the fundus image by inputting the fundus image into a pre-trained target frame detection model; determines the optic disc center point according to the first target detection frame, and determines the first target detection point and the second target detection point according to the second target detection frame; obtains a fundus vascular arch fitting curve according to the optic disc center point, the first target detection point and the second target detection point, and determines the fundus vascular arch through the fundus vascular arch fitting curve. By performing detection through the pre-trained target frame detection model, the vascular arch can be accurately determined. By accurately determining the vascular arch, the vascular arch, which is a basic feature of the fundus, can be detected, which is not only helpful in helping to identify the eye, but also helpful in understanding fundus abnormalities, and is the basis for subsequent fundus disease analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 is a flow chart of a method for determining fundus structure features according to an embodiment of the present invention;

[0042] Figure 2A is a schematic diagram of obtaining a training sample as an example in an embodiment of the present invention;

[0043] Figure 2B is a schematic diagram of calculating an intersection-over-union ratio according to an embodiment of the present invention;

[0044] Figure 2C is a schematic diagram of obtaining a target frame detection model as an example in an embodiment of the present invention;

[0045] Figure 2D is a schematic diagram of target pre-selection boxes of different aspect ratios generated on feature maps of different scales according to an embodiment of the present invention;

[0046] Figure 3A is a schematic diagram of a fundus vascular arch and its fitting curve according to an embodiment of the present invention;

[0047] Figure 3B is a schematic diagram of measuring the opening angle of a vascular arch as an example in an embodiment of the present invention;

[0048] Figure 4A is a functional block diagram of a device for determining fundus structure features according to an embodiment of the present invention;

[0049] Figure 4B is a structural block diagram of a model determination module in an embodiment of the present invention;

[0050] Figure 5 is a functional block diagram of a computer-readable storage medium according to an embodiment of the present invention;

[0051] Figure 6 It is a functional block diagram of a device for determining fundus structure features according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The vascular arch is the area formed by the fundus retinal aorta. Because of its near-opic arc shape, it is called the vascular arch. It can be divided into the fundus retinal arterial arch, the fundus retinal venous arch and the vascular arch (that is, the arch fitted according to the direction of the arteries and veins), which includes the arterial arch and the venous arch.

[0054] Embodiment 1

[0055] Figure 1 FIG. 1 is a flow chart of a method for determining fundus structure features according to an embodiment of the present invention. Figure 1As shown, the method comprises the following steps:

[0056] S110: Acquire fundus image.

[0057] Specifically, in this step, the fundus image can be preprocessed first, and then the fundus blood vessels can be extracted from the preprocessed fundus image. Preprocessing of the fundus image can specifically include denoising, normalization and enhancement. Denoising is mainly to remove the noise of the fundus image during the imaging process and reduce its interference on the vascular characteristics. Normalization is mainly to achieve the unification of differentiated exposure, color and brightness of the fundus image, so that the grayscale value range of different images is unified, thereby improving the algorithm's generalization ability for massive images and making the technology product possible. Enhancement is mainly to widen the difference between the features of interest and the background features, so that the features are more prominent, which is convenient for threshold segmentation and feature extraction.

[0058] S120: Input the fundus image into a pre-trained target frame detection model, and generate a first target detection frame and a second target detection frame on the fundus image.

[0059] In some embodiments, the training process of the target box detection model in step S120 specifically includes:

[0060] S121: Acquire fundus training images;

[0061] S122: labeling the fundus training image to obtain training samples;

[0062] In some embodiments, annotating the fundus training image to obtain the training sample specifically includes:

[0063] Figure 2A FIG. 1 is a schematic diagram of obtaining training samples as an example in an embodiment of the present invention. Figure 2A As shown:

[0064] Taking the left eye as an example (the right eye is a mirror image of the left eye), determine the circumscribed rectangular frame of the optic disc of the fundus training image, obtain the first sample detection frame, determine the first vertex E and the second vertex F of the diagonal of the first detection frame, use the first vertex E as the first sample detection point, and the second vertex F as the second sample detection point, and determine the optic disc center point D according to the first vertex E and the second vertex F, that is, determine the optic disc center point D according to the first sample detection point and the second sample detection point;

[0065] Determine the lower half G1 and the upper half G2 of the main blood vessels in the fundus image, take the optic disc center point D as the center and draw a circle with an arbitrary radius length as the radius (preferably, the diameter length is 1pd to 3pd), intersect with the lower half G1 of the main blood vessel to obtain a third sample detection point B, draw a vertical line L1 through the third sample detection point B to intersect with the upper half G2 of the main blood vessel to obtain a first auxiliary intersection point C, draw a horizontal line L2 through the first auxiliary intersection point C to intersect with the vertical line L3 through the optic disc center point D to obtain a fourth sample detection point A, determine the second sample detection frame according to the third sample detection point B, the first auxiliary intersection point C and the fourth sample detection point A; and obtain training samples according to the first sample detection frame and the second sample detection frame.

[0066] Step S123: input the training sample into the deep learning target frame detection network for training to obtain a target frame detection model;

[0067] In some embodiments, in step S123, the training sample is input into the deep learning target frame detection network for training to obtain the target frame detection model, which specifically includes:

[0068] Step S1: Input the training sample into the deep learning target box detection network for convolution to obtain feature maps of different scales;

[0069] Step S2: On feature maps of different scales, multiple target pre-selection boxes with different aspect ratios are generated with the center points of different cells as the center;

[0070] Step S3: Step S3: Determine the first target detection frame and the second target detection frame according to the preset intersection-and-union ratio of the target pre-selection frame, determine the first target detection point and the second target detection point according to the first target detection frame, and calculate the coordinate error values ​​of the first target detection point and the second target detection point relative to the first sample detection point and the second sample detection point, determine the coordinates of the third target detection point and the fourth target detection point according to the second target detection frame, and calculate the coordinate error values ​​of the third target detection point and the fourth target detection point relative to the third sample detection point and the fourth sample detection point;

[0071] The above steps S1 to S3 are continuously iterated until the error value is reduced to a preset error value, and the target frame detection model is obtained.

[0072] Among them, the intersection over union ratio, full name Intersection over Union (IoU for short), is the ratio of the intersection and union of the "predicted bounding box" and the "real bounding box". Figure 2B It is a schematic diagram of calculating the intersection-over-union ratio according to an embodiment of the present invention.

[0073] Figure 2C FIG. 1 is a schematic diagram of a target frame detection model obtained as an example in an embodiment of the present invention. Figure 2C As shown:

[0074] The training samples and the annotated coordinate points are input into the deep learning target box detection network for convolution to obtain feature maps of different scales, for example, Figure 2C Features in Figure 1 , feature map 2, feature map 3; on feature maps of different scales, multiple target pre-selection boxes with different aspect ratios are generated with the center points of different cells as the center. Figure 2D is a schematic diagram of target pre-selection boxes with different aspect ratios generated on feature maps of different scales according to an embodiment of the present invention, such as Figure 2D As shown in the figure, each scale feature map is divided into multiple cells, for example, an 8x8 feature map is divided into 64 cells, and a 4x4 feature map is divided into 16 cells. Each cell corresponds to a set of features in the original image. On feature maps of different scales, multiple target pre-selected boxes with different aspect ratios are generated with the center points of different cells as the center, and then multiple target detection boxes are determined according to a preset intersection-union ratio; the intersection-union ratio is the ratio of the intersection and union of the "predicted bounding box" and the "real bounding box", that is, the ratio of the "intersection" and "union" of the pre-selected target box and the target detection box, that is, the ratio of the intersection and union of the pre-selected target box and the target detection box;

[0075] The center point of the optic disc is determined according to the first target detection frame, and the coordinate error value of the coordinate of the center point of the optic disc relative to the first sample detection point is calculated. The coordinates of the first detection point and the second target detection point are determined according to the second target detection frame, and the error values ​​of the coordinates of the first target detection point and the second target detection point relative to the coordinates of the first sample detection point and the second sample detection point are calculated, that is, (x1', y1'), (x2', y2'), (x3', y3')..., and the difference between the determined coordinate point and the marked coordinate point (x1, y1), (x2, y2), (x3, y3)... is calculated, that is, |x1'-x1|=△x1, |y1'-y1|=△y1..., and the above steps are continuously iterated until the difference (△x1, △y1...) is reduced to the preset coordinate error.

[0076] In some embodiments, the deep learning target box detection network is a Single Shot MultiBox Detector (SSD deep learning network for short). The SSD deep learning network uses a variety of data enhancement algorithms, including horizontal flipping, cropping, zooming in and out, etc., which can significantly improve the performance of the algorithm, making the method more robust to target detection of different sizes and shapes of inputs.

[0077] S130: Determine a first target detection point and a second target detection point according to the first target detection frame, determine a optic disc center point according to the first target detection point and the second target detection point, and determine a third target detection point and a fourth target detection point according to the second target detection frame.

[0078] In some embodiments, a first target detection point and a second target detection point are determined based on two vertices on a diagonal line of the first target detection frame, a optic disc center point is determined through the first target detection point and the second target detection point, and a third target detection point and a fourth target detection point are determined based on two vertices on a diagonal line of the second target detection frame.

[0079] S140: Determine a fundus vascular arch fitting curve according to the optic disc center point, the third target detection point and the fourth target detection point, and determine the fundus vascular arch through the fundus vascular arch fitting curve.

[0080] In some embodiments, in step S140, a fundus vascular arch fitting curve is determined according to the optic disc center point, the third target detection point, and the fourth target detection point, and the fundus vascular arch is determined by the fundus vascular arch fitting curve, specifically including:

[0081] Figure 3A is a schematic diagram of a fundus vascular arch fitting curve according to an embodiment of the present invention, such as Figure 3A As shown, the third target detection point B, the optic disc center point D and the first auxiliary intersection point C are connected in sequence, with the third target detection point B as the starting endpoint and the first auxiliary intersection point C as the ending endpoint, and the fundus vascular arch fitting curve L is determined by Bessel function fitting, and the fundus vascular arch is determined by the fundus vascular arch fitting curve L.

[0082] By using a pre-trained target box detection model for detection, the vascular arch can be accurately determined. By accurately determining the vascular arch, the vascular arch, which is the basic feature of the fundus, can be detected. This is not only helpful in helping to identify the eye, but also helps to understand fundus abnormalities. It is the basis for subsequent fundus disease analysis.

[0083] In some embodiments, the method further includes: determining the opening angle of the fundus vascular arch according to the optic disc center point, the third target detection point and the fourth target detection point, Figure 3B FIG3 is a schematic diagram of measuring the vascular arch opening angle as an example of an embodiment of the present invention. As shown in FIG3 , the optic disc center point D is connected to the third target detection point B to determine the first line segment L DG , connect the disc center point D with the first auxiliary intersection point C to determine the second line segment L DC , the first line segment L DB With the second line segment L DC The angle between them is the opening angle θ of the vascular arch. By measuring the opening angle of the vascular arch, it is convenient to analyze the changing trend of the fundus blood vessels.

[0084] The method for determining the fundus feature structure provided in the embodiment of the present invention can accurately determine the vascular arch by performing detection through a pre-trained target frame detection model. By accurately determining the vascular arch, the vascular arch, which is a basic feature of the fundus, can be detected, which is not only helpful in helping to identify the eye, but also helpful in understanding fundus abnormalities, and is the basis for subsequent fundus disease analysis.

[0085] Embodiment 2

[0086] Figure 4A is a functional block diagram of a device for determining fundus structure features according to an embodiment of the present invention, such as Figure 4A As shown, based on similar inventive concepts, the device 400 for determining fundus structure features includes:

[0087] The acquisition module 410 is used to acquire fundus images; specifically, the acquisition module is used to perform preprocessing on the fundus images, and then perform fundus blood vessel extraction on the preprocessed fundus images. The preprocessing of the fundus images may specifically include denoising, normalization and enhancement. Denoising is mainly to remove the noise in the fundus images during the imaging process and reduce its interference on the vascular features. Normalization is mainly to achieve the differential exposure, color and brightness of the fundus images, so that the grayscale value range of different images is unified, thereby improving the algorithm's generalization ability for massive images and making the technology product possible. Enhancement is mainly to widen the difference between the features of interest and the background features, so that the features are more prominent, which is convenient for threshold segmentation and feature extraction.

[0088] The target frame detection module 420 inputs the fundus image into a pre-trained target frame detection model, and generates a first target detection frame and a second target detection frame on the fundus image; specifically, in this embodiment, the device 400 for determining fundus structure features further includes:

[0089] Model training module 450, such as Figure 4B As shown, the model training module 450 includes:

[0090] A training image acquisition unit 451 is used to acquire a fundus training image;

[0091] A labeling unit 452 is used to label the fundus training image to obtain a training sample;

[0092] In some embodiments, Figure 2A As shown, the annotation unit is specifically used for:

[0093] Taking the left eye as an example (the right eye is a mirror image of the left eye), determine the circumscribed rectangular frame of the optic disc of the fundus training image, obtain the first sample detection frame, determine the first vertex E and the second vertex F of the diagonal of the first detection frame, use the first vertex E as the first sample detection point, and the second vertex F as the second sample detection point, and determine the optic disc center point D according to the first vertex E and the second vertex F, that is, determine the optic disc center point D according to the first sample detection point and the second sample detection point;

[0094] Determine the lower half G1 and the upper half G2 of the main blood vessels in the fundus image, take the optic disc center point D as the center and draw a circle with an arbitrary radius length as the radius (preferably, the diameter length is 1pd to 3pd), intersect with the lower half G1 of the main blood vessel to obtain a third sample detection point B, draw a vertical line L1 through the third sample detection point B to intersect with the upper half G2 of the main blood vessel to obtain a first auxiliary intersection point C, draw a horizontal line L2 through the first auxiliary intersection point C to intersect with the vertical line L3 through the optic disc center point D to obtain a fourth sample detection point A, determine the second sample detection frame according to the third sample detection point B, the first auxiliary intersection point C and the fourth sample detection point A; and obtain training samples according to the first sample detection frame and the second sample detection frame.

[0095] The model acquisition unit 453 is used to input the training samples into the deep learning target frame detection network for training, and obtain the target frame detection model; specifically, the model acquisition unit 453 inputs the training samples into the deep learning target frame detection network for convolution, obtains feature maps of different scales, generates multiple target pre-selected boxes with different aspect ratios on the feature maps of different scales with the center points of different cells as the center, determines the first target detection box and the second target detection box according to the preset intersection ratio, determines the first target detection point and the second target detection point according to the first target detection box, and calculates the coordinate error value of the first target detection point and the second target detection point relative to the first sample detection point and the second sample detection point, determines the coordinates of the third target detection point and the fourth target detection point according to the second target detection box, and calculates the coordinate error value of the third target detection point and the fourth target detection point relative to the third sample detection point and the fourth sample detection point; iterates here continuously until the error value is reduced to the preset error value, and obtains the target frame detection model. The first determination module 430 determines the first target detection point and the second target detection point according to the first target detection frame, determines the optic disc center point according to the first target detection point and the second target detection point, and determines the third target detection point and the fourth target detection point according to the second target detection frame; specifically, the first determination module 430 determines the first target detection point and the second target detection point according to two vertices on the diagonal line of the first target detection frame, determines the optic disc center point through the first target detection point and the second target detection point, and determines the third target detection point and the fourth target detection point according to two vertices on the diagonal line of the second target detection frame.

[0096] The second determination module 440 determines the fundus vascular arch fitting curve according to the optic disc center point, the third target detection point and the fourth target detection point, and determines the fundus vascular arch through the fundus vascular arch fitting curve. Figure 3A As shown, the second determination module 440 sequentially connects the third target detection point B, the optic disc center point D and the first auxiliary intersection point C, takes the third target detection point B as the starting endpoint and the first auxiliary intersection point C as the ending endpoint, determines the fundus vascular arch fitting curve L by Bessel function fitting, and determines the fundus vascular arch by the fundus vascular arch fitting curve L. The fundus feature structure determination device provided by the embodiment of the present invention can accurately determine the vascular arch by detecting with a pre-trained target frame detection model, and can detect the vascular arch as the basic feature of the fundus by accurately determining the vascular arch, which is not only helpful for helping to identify the eye, but also helpful for understanding fundus abnormalities, and is the basis for subsequent fundus disease analysis.

[0097] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0098] Embodiment 3

[0099] Figure 5 is a computer-readable storage medium 500 provided by an embodiment of the present invention. Figure 5 As shown, the computer readable storage medium 500 stores a computer program 510. When the computer program 510 is executed by a processor, the following is achieved:

[0100] Acquire fundus images;

[0101] Inputting the fundus image into a pre-trained target frame detection model, generating a first target detection frame and a second target detection frame on the fundus image;

[0102] Determine a first target detection point and a second target detection point according to the first target detection frame, determine a optic disc center point according to the first target detection point and the second target detection point, and determine a third target detection point and a fourth target detection point according to the second target detection frame;

[0103] A fundus vascular arch fitting curve is determined according to the optic disc center point, the third target detection point and the fourth target detection point, and the fundus vascular arch is determined through the fundus vascular arch fitting curve.

[0104] A computer-readable storage medium provided by an embodiment of the present invention can accurately determine the vascular arch by performing detection through a pre-trained target frame detection model. By accurately determining the vascular arch, the vascular arch, which is a basic feature of the fundus, can be detected, which is not only helpful in helping to identify the eye, but also helpful in understanding fundus abnormalities, and is the basis for subsequent fundus disease analysis.

[0105] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. Of course, there are other ways of readable storage media, such as quantum memory, graphene memory, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0106] Embodiment 4

[0107] The embodiment of the present invention further provides an electronic device 600, such as Figure 6 As shown, it includes one or more processors 601 , a communication interface 602 , a memory 603 and a communication bus 604 , wherein the processor 601 , the communication interface 602 , and the memory 603 communicate with each other via the communication bus 604 .

[0108] Memory 603, used for storing computer programs;

[0109] The processor 601 is used to execute the program stored in the memory 603 to achieve:

[0110] Acquire fundus images;

[0111] Inputting the fundus image into a pre-trained deep learning-based target frame detection model to generate a first target detection frame and a second target detection frame on the fundus image;

[0112] Determine a first target detection point and a second target detection point according to the first target detection frame, determine a optic disc center point according to the first target detection point and the second target detection point, and determine a third target detection point and a fourth target detection point according to the second target detection frame;

[0113] A fundus vascular arch fitting curve is determined according to the optic disc center point, the third target detection point and the fourth target detection point, and the fundus vascular arch is determined through the fundus vascular arch fitting curve.

[0114] In some embodiments, the processing performed by the processor 601, before acquiring the fundus image, further includes:

[0115] Acquire fundus training images;

[0116] Annotate the fundus training images to obtain training samples;

[0117] The training samples are input into the deep learning target box detection network for training to obtain the target box detection model.

[0118] In some embodiments, in the process performed by the processor 601, the fundus training image is annotated to obtain a training sample, which specifically includes:

[0119] Determine the circumscribed rectangular frame of the optic disc in the fundus training image, obtain a first sample detection frame, determine a first vertex and a second vertex of a diagonal line of the first sample detection frame, obtain a first sample detection point and a second sample detection point, and determine the center point of the optic disc according to the first sample detection point and the second sample detection point;

[0120] Determine the lower half and the upper half of the main blood vessel in the fundus image, take the center point of the optic disc as the center and draw a circle with an arbitrary radius length as the radius, intersect with the lower half of the main blood vessel to obtain a third sample detection point, draw a vertical line through the third sample detection point to intersect with the upper half of the main blood vessel to obtain a first auxiliary intersection point, draw a horizontal line through the first auxiliary intersection point to intersect with the vertical line through the center point of the optic disc to obtain a fourth sample detection point, and determine the second sample detection frame according to the third sample detection point, the first auxiliary intersection point and the fourth sample detection point;

[0121] A training sample is obtained according to the first sample detection frame and the second sample detection frame.

[0122] In some embodiments, in the process performed by the processor 601, the training sample is input into the deep learning target frame detection network for training to obtain the target frame detection model, which specifically includes:

[0123] Step S1: Input the training sample into the deep learning target box detection network for convolution to obtain feature maps of different scales;

[0124] Step S2: On feature maps of different scales, multiple target pre-selection boxes with different aspect ratios are generated with the center points of different cells as the center;

[0125] Step S3: determining the first target detection frame and the second target detection frame according to the preset intersection-and-union ratio of the target pre-selection frame, determining the first target detection point and the second target detection point according to the first target detection frame, and calculating the coordinate error values ​​of the first target detection point and the second target detection point relative to the first sample detection point and the second sample detection point, determining the coordinates of the third target detection point and the fourth target detection point according to the second target detection frame, and calculating the coordinate error values ​​of the third target detection point and the fourth target detection point relative to the third sample detection point and the fourth sample detection point;

[0126] The above steps S1 to S3 are continuously iterated until the error value is reduced to a preset error value, and a target frame detection model is obtained.

[0127] In some embodiments, in the process performed by the processor 601, the fundus image is input into a pre-trained target frame detection model, and a first target detection frame and a second target detection frame are generated on the fundus image, specifically including:

[0128] The fundus image is input into the target frame detection model for convolution to obtain feature maps of different scales;

[0129] On feature maps of different scales, multiple target pre-selection boxes of different shapes are generated with the center points of different cells as the center;

[0130] The target pre-selected boxes on feature maps of different scales are combined, and the first target detection box and the second target detection box are selected respectively using the non-maximum suppression method.

[0131] In some embodiments, in the processing performed by the processor 601, obtaining a vascular arch fitting curve according to the optic disc center point, the third target detection point, and the fourth target detection point, and determining the fundus vascular arch through the fundus vascular arch fitting curve specifically includes:

[0132] Connect the third target detection point, the optic disc center point and the fourth target detection point in sequence.

[0133] The third target detection point is taken as the starting endpoint and the fourth target detection point is taken as the ending endpoint, a fundus vascular arch fitting curve is determined by Bessel function fitting, and the fundus vascular arch is determined by the fundus vascular arch fitting curve.

[0134] In some embodiments, the processing performed by the processor 601 further includes: determining the opening angle of the fundus vascular arch according to the optic disc center point, the third target detection point and the fourth target detection point.

[0135] An electronic device provided by an embodiment of the present invention can accurately determine the vascular arch by performing detection through a pre-trained target frame detection model. By accurately determining the vascular arch, the vascular arch, which is a basic feature of the fundus, can be detected, which is not only helpful in helping to identify the eye, but also helpful in understanding fundus abnormalities, and is the basis for subsequent fundus disease analysis.

[0136] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used for communication between the above electronic device and other devices.

[0137] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0138] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0139] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0140] Although the present application provides method operation steps as described in the embodiments or flow charts, more or fewer operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, in a parallel processor or multi-threaded processing environment, or even in a distributed data processing environment).

[0141] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0142] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0143] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for determining fundus structural features, characterized in that: include: Acquire fundus images; Inputting the fundus image into a pre-trained deep learning-based target frame detection model, and generating a first target detection frame and a second target detection frame on the fundus image; Determine a first target detection point and a second target detection point according to two vertices on a diagonal line of the first target detection frame, determine a optic disc center point according to the first target detection point and the second target detection point, and determine a third target detection point and a fourth target detection point according to two vertices on a diagonal line of the second target detection frame; Determine a fundus vascular arch fitting curve according to the optic disc center point, the third target detection point and the fourth target detection point, and determine the fundus vascular arch by using the fundus vascular arch fitting curve; Wherein, obtaining a vascular arch fitting curve according to the optic disc center point, the third target detection point and the fourth target detection point, and determining the fundus vascular arch through the fundus vascular arch fitting curve specifically includes: Determine a first auxiliary intersection point by using the third target detection point and the fourth target detection point; connecting the third target detection point, the optic disc center point and the first auxiliary intersection point in sequence; Taking the third target detection point as the starting endpoint and the first auxiliary intersection point as the ending endpoint, the fundus vascular arch fitting curve is determined by Bessel function fitting, and the fundus vascular arch is determined by the fundus vascular arch fitting curve.

2. The method according to claim 1, characterized in that Before acquiring fundus images, it also includes: Acquire fundus training images; Annotating the fundus training image to obtain a training sample; The training samples are input into a deep learning target frame detection network for training to obtain the target frame detection model.

3. The method according to claim 2, characterized in that The step of labeling the fundus training image to obtain a training sample specifically includes: Determine a circumscribed rectangular frame of an optic disc in the fundus training image, obtain a first sample detection frame, determine a first vertex and a second vertex of a diagonal line of the first sample detection frame, obtain a first sample detection point and a second sample detection point, and determine a center point of the optic disc according to the first sample detection point and the second sample detection point; Determine the lower half and the upper half of the main blood vessel in the fundus image, take the optic disc center point as the center and draw a circle with an arbitrary radius length as the radius, intersect with the lower half of the main blood vessel to obtain a third sample detection point, draw a vertical line through the third sample detection point to intersect with the upper half of the main blood vessel to obtain a first auxiliary intersection point, draw a horizontal line through the first auxiliary intersection point to intersect with the vertical line through the optic disc center point to obtain a fourth sample detection point, and determine a second sample detection frame according to the third sample detection point, the first auxiliary intersection point and the fourth sample detection point; The training sample is acquired according to the first sample detection frame and the second sample detection frame.

4. The method according to claim 3, characterized in that The step of inputting the training samples into a deep learning target frame detection network for training to obtain a target frame detection model specifically includes: Step S1: input the training sample into a deep learning target box detection network for convolution to obtain feature maps of different scales; Step S2: on the feature maps of different scales, taking the center points of different cells as the center, generating a plurality of target pre-selection boxes with different aspect ratios; Step S3: determining a first target detection frame and a second target detection frame according to a preset intersection-and-union ratio for the target pre-selection frame, determining a first target detection point and a second target detection point according to the first target detection frame, and calculating coordinate error values ​​of the first target detection point and the second target detection point relative to the first sample detection point and the second sample detection point, determining coordinates of a third target detection point and a fourth target detection point according to the second target detection frame, and calculating coordinate error values ​​of the third target detection point and the fourth target detection point relative to the third sample detection point and the fourth sample detection point; The above steps S1 to S3 are continuously iterated until the error value is reduced to a preset error value, thereby obtaining the target frame detection model.

5. The method according to claim 1, characterized in that The step of inputting the fundus image into a pre-trained target frame detection model and generating a first target detection frame and a second target detection frame on the fundus image specifically includes: Inputting the fundus image into the target frame detection model for convolution to obtain feature maps of different scales; On the feature maps of different scales, taking the center points of different cells as the center, generating a plurality of target pre-selection boxes of different shapes; The target pre-selected boxes on the feature maps of different scales are combined, and the first target detection box and the second target detection box are respectively selected using a non-maximum suppression method.

6. The method according to any one of claims 1 to 5, characterized in that Also includes: The opening angle of the fundus vascular arch is determined according to the optic disc center point, the third target detection point and the fourth target detection point.

7. A device for determining fundus structural features, characterized in that: include: An acquisition module, for acquiring fundus images; a target frame detection module, configured to input the fundus image and the fundus image into a pre-trained target frame detection model, and generate a first target detection frame and a second target detection frame on the fundus image; A first determination module is used to determine a first target detection point and a second target detection point according to two vertices on a diagonal line of the first target detection frame, determine a optic disc center point according to the first target detection point and the second target detection point, and determine a third target detection point and a fourth target detection point according to two vertices on a diagonal line of the second target detection frame; A second determination module is used to determine a fundus vascular arch fitting curve according to the optic disc center point, the third target detection point and the fourth target detection point, and determine the fundus vascular arch through the fundus vascular arch fitting curve; Wherein, the second determining module is specifically used for: Determine a first auxiliary intersection point by using the third target detection point and the fourth target detection point; connecting the third target detection point, the optic disc center point and the first auxiliary intersection point in sequence; Taking the third target detection point as the starting endpoint and the first auxiliary intersection point as the ending endpoint, the fundus vascular arch fitting curve is determined by Bessel function fitting, and the fundus vascular arch is determined by the fundus vascular arch fitting curve.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for determining fundus structure features as described in any one of claims 1 to 6 is implemented.

9. A device for determining fundus structural features, characterized in that: It includes: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining fundus structure features as described in any one of claims 1 to 6.

Images