Method, device, medium and equipment for detecting cross compression characteristics of fundus arteries and veins

Abstract

The embodiment of the present invention provides a method, device, medium and equipment for detecting the arteriovenous cross compression feature of the fundus, including: obtaining a fundus image, resizing the fundus image; inputting the resized fundus image into a pre-trained deep learning-based target detection model, generating multiple target detection frames on the fundus image, and obtaining the probability value of each target detection frame containing the arteriovenous cross compression feature; determining the arteriovenous cross compression feature according to the probability value of the arteriovenous cross compression feature. The present invention can quickly, accurately and intuitively detect the arteriovenous cross compression feature of the fundus retina.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device, medium and equipment for detecting cross compression characteristics of fundus arteries and veins. Background Art

[0002] Arteriovenous nicking (AV nicking) refers to the phenomenon that due to increased blood pressure (i.e., hypertension), the vein is compressed by the hardened artery at the intersection of the vein and the artery. In retinal color photographs, AV nicking is manifested as a decrease in the caliber of the veins on both sides of the arteriovenous (AV) intersection. The AV nicking in the fundus of the human eye is not only related to the current blood pressure of the human body, but also to the blood pressure in the past. It is a marker of the persistent and long-term effects of hypertension. Therefore, the identification and detection of AV nicking is very important for the early detection of cardiovascular disease.

[0003] When the existing technology detects arteriovenous crossing compression signs in fundus images, the implementation process requires complete retinal vascular cross-sectional information and vascular structure information. The classification features are more obvious only when the resolution is high, the imaging quality is high, and the vascular segments in the ROI area are clear. However, in the application process, the imaging quality and information of the fundus image are different, there may be lesions (fundus hemorrhage, exudation, etc.) in the ROI area, and the vascular structure is complex (large degree of vascular curvature, severe vascular crossing or parallel overlap, etc.), which will directly affect the extraction of vascular information and the accuracy of classification feature parameters.

[0004] At the same time, due to the influence of shooting equipment and shooting operation, the quality of fundus images will vary greatly. In addition, due to the differences in fundus structure of each person, the robustness of arteriovenous segmentation using existing algorithms (such as the unsupervised Kmeans algorithm) is poor. The inaccuracy of arteriovenous information will directly affect the accuracy of the arteriovenous crossing compression sign detection results. In addition, since the grayscale values ​​of arteries and veins in fundus images are relatively close, the difference in the arteriovenous appearance is low, which will also reduce the accuracy of the arteriovenous segmentation results, thereby affecting the accuracy of the arteriovenous crossing compression sign detection results. Summary of the invention

[0005] In view of this, an object of embodiments of the present invention is to provide a method, device, medium and equipment for detecting fundus arteriovenous cross compression characteristics, so as to quickly and accurately determine the fundus retinal arteriovenous cross compression characteristics.

[0006] To achieve the above-mentioned purpose, in a first aspect, an embodiment of the present invention provides a method for detecting cross compression characteristics of fundus arteries and veins, which specifically includes:

[0007] Acquire a fundus image, and perform size transformation on the fundus image;

[0008] Inputting the resized fundus image into a pre-trained deep learning-based target detection model, generating a plurality of target detection frames and probability values ​​of arteriovenous cross compression features corresponding to the target detection frames on the fundus image;

[0009] According to the probability value of the arteriovenous cross compression feature, it is determined whether the target detection frame contains the arteriovenous cross compression feature.

[0010] In some possible implementations, before acquiring the fundus image, the method further includes:

[0011] acquiring a plurality of fundus training images;

[0012] Annotating the multiple fundus training images to obtain training samples;

[0013] Transforming the size of the training sample to obtain a plurality of training samples of the same size;

[0014] The multiple training samples of the same size are input into a deep learning target detection network for training to obtain the target detection model.

[0015] In some possible implementations, labeling the multiple fundus training images to obtain training samples specifically includes:

[0016] Marking a rectangular frame at a position of the fundus training image where the arteriovenous cross compression feature is present, and obtaining a plurality of sample detection frames;

[0017] Determine a first vertex and a second vertex on any diagonal line of the sample detection frame to obtain a plurality of sample detection points;

[0018] The training samples are obtained according to the multiple sample detection frames and the multiple sample detection points.

[0019] In some possible implementations, inputting the training samples into a deep learning target detection network for training to obtain the target detection model specifically includes:

[0020] Step S1: input the training sample into a deep learning target detection network for convolution to obtain feature maps of different scales;

[0021] Step S2: on the feature maps of different scales, taking the center points of different cells as the center, generating a plurality of target pre-selection boxes of different aspect ratios;

[0022] Step S3: determining a plurality of target detection frames according to a preset intersection-and-union ratio from the plurality of target pre-selected frames, determining a first vertex and a second vertex on a diagonal line of the plurality of target detection frames, obtaining a plurality of target detection points, and calculating coordinate offset values ​​of the plurality of target detection points relative to the plurality of sample detection points;

[0023] The above steps S1 to S3 are continuously iterated until the coordinate offset value decreases to a preset offset value or below the preset offset value, thereby obtaining the target detection model.

[0024] In some possible implementations, the size-transformed fundus image is input into a pre-trained deep learning-based target detection model, and multiple target detection frames and probability values ​​of arteriovenous cross compression features corresponding to the target detection frames are generated on the fundus image, specifically including:

[0025] Inputting the size-transformed fundus image into the target detection model for convolution to obtain a plurality of feature maps of different scales;

[0026] On the feature maps of different scales, taking the center points of different cells as the center, generating a plurality of target pre-selection boxes of different shapes;

[0027] The target pre-selected boxes on the feature maps of different scales are combined, and the multiple target detection boxes are selected respectively using a non-maximum suppression method.

[0028] In some possible implementations, the method further includes: obtaining the morphological changes of the veins, the changing trends of the vein diameter values, and / or the changing trends of the vein curvature values ​​according to a deep learning-based target detection model, and determining the probability value of the target detection frame containing the feature of arteriovenous cross compression.

[0029] In some possible implementations, determining whether the target detection frame includes an arteriovenous cross compression feature according to the probability value of the arteriovenous cross compression specifically includes:

[0030] When the probability value of arteriovenous cross compression on the target detection frame is greater than or equal to a preset probability threshold, it is determined that the target detection frame contains the feature of arteriovenous cross compression, and the intersection position contained in the target detection frame is arteriovenous cross compression.

[0031] In some possible implementations, after detecting the arteriovenous cross compression feature in the target detection frame according to the probability value of the arteriovenous cross compression, the method further includes:

[0032] Combined with the nerve fiber layer on the fundus image and the age information of the patient corresponding to the fundus image, the erroneously extracted arteriovenous cross compression feature and the corresponding target detection frame are deleted.

[0033] In a second aspect, the present invention provides a device for detecting cross compression characteristics of fundus arteries and veins, comprising:

[0034] An acquisition module, used for acquiring a fundus image and performing size transformation on the fundus image;

[0035] A target generation module is used to input the size-transformed fundus image into a pre-trained deep learning-based target detection model, and generate multiple target detection frames and probability values ​​of arteriovenous cross compression corresponding to the target detection frames on the fundus image;

[0036] The feature detection module is used to determine whether the target detection frame contains an arteriovenous cross compression feature according to the probability value of the arteriovenous cross compression.

[0037] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned methods for detecting fundus arteriovenous cross-compression features.

[0038] In a fourth aspect, the present invention provides a device for detecting arteriovenous cross compression characteristics, comprising:

[0039] one or more processors;

[0040] A storage device for storing one or more programs;

[0041] When the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned methods for detecting fundus arteriovenous cross compression features.

[0042] The above technical solution has the following beneficial effects:

[0043] The embodiment of the present invention generates multiple target detection frames and probability values ​​of arteriovenous crossing compression corresponding to the target detection frames on the fundus image by inputting the size-transformed fundus image into a pre-trained deep learning-based target detection model; according to the probability values ​​of arteriovenous crossing compression, it is determined whether the target detection frame contains arteriovenous crossing compression features, thereby quickly, accurately and intuitively determining the fundus retinal vascular crossing compression features. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 is a flow chart of a method for detecting cross compression features of fundus arteries and veins according to an embodiment of the present invention;

[0046] Figure 2A This is a schematic diagram of obtaining training samples according to an embodiment of the present invention;

[0047] Figure 2B is a schematic diagram of calculating an intersection-over-union ratio according to an embodiment of the present invention;

[0048] Figure 2C is a schematic diagram of obtaining a target detection model according to an embodiment of the present invention;

[0049] Figure 2D is a schematic diagram of target pre-selection boxes with different aspect ratios generated on feature maps of different scales according to an embodiment of the present invention;

[0050] Figure 3A is a schematic diagram of generating an arteriovenous cross compression probability value according to an embodiment of the present invention;

[0051] Figure 3B is a schematic diagram of changes in venous morphology in an arteriovenous cross compression phenomenon in an embodiment of the present invention;

[0052] Figure 4 It is a functional block diagram of a device for detecting cross compression of fundus arteries and veins according to an embodiment of the present invention;

[0053] Figure 5 is a functional block diagram of a computer-readable storage medium according to an embodiment of the present invention;

[0054] Figure 6 It is a functional block diagram of a detection device for fundus arteriovenous cross compression according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Embodiment 1

[0057] Figure 1 FIG. 1 is a flow chart of a method for detecting cross compression characteristics of fundus arteries and veins according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0058] S110: Acquire a fundus image and perform size transformation on the fundus image.

[0059] Specifically, in this step, after the fundus image is resized, preprocessing may be performed on the fundus image, which may specifically include the following steps:

[0060] S101: extracting a region of interest (ROI region) from the fundus image, separating the channels of the fundus image, and selecting any channel image, which may be the R channel, the G channel, the B channel, or any one of the H channel, the I channel, and the S channel;

[0061] S102: threshold segmentation is performed on any of the above-separated channels according to one or more of the grayscale mean, deviation, gradient and other characteristics, and holes are filled in the region of interest obtained after the threshold segmentation to ensure that the obtained ROI region image is a continuous region. In this embodiment, holes are filled in the region of interest by dilation. The region of interest is first "enlarged" by dilation, and the background points in contact with the region of interest are merged into the fundus image, so that the boundary of the region of interest is expanded outward to fill certain holes in the region of interest and eliminate small particle noise contained in the region of interest;

[0062] S103: removing isolated small dots, burrs, small bridges and other features through morphological operations (such as opening operations, the so-called opening operations include corrosion and expansion) and morphological feature extraction;

[0063] S104: performing enhancement processing on the image of the region of interest, so as to increase the contrast between the feature of interest and the background feature, thereby making the feature of the region of interest more prominent.

[0064] S120: Input the size-transformed fundus image into a pre-trained deep learning-based target detection model, generate multiple target detection frames on the fundus image, and a probability value of the arteriovenous cross compression feature corresponding to each target detection frame.

[0065] In some possible implementations, the training process of the target box detection model in step S120 specifically includes:

[0066] S121: Acquire multiple fundus training images;

[0067] S122: labeling multiple fundus training images to obtain training samples;

[0068] Figure 2A FIG. 1 is a schematic diagram of obtaining training samples as an example in an embodiment of the present invention. Figure 2A As shown, a rectangular frame is marked at the position of the arteriovenous intersection compression feature on the fundus training image to obtain a sample detection frame K1 and a sample detection frame K2. Here, the size of the sample detection frame K1 and the sample detection frame K2 can not only satisfy the requirement of simultaneously including information such as arterial vessels, venous vessels and intersection coordinates, but also exclude most of the surrounding interference information. Preferably, the size of the rectangular frame area can be determined according to the caliber of the blood vessel at the corresponding intersection position. As an example, the size of the rectangular frame area can vary with the caliber value of the arterial and / or venous blood vessels at the position of the arteriovenous intersection node. For example, when the caliber of the blood vessel at the position of the arteriovenous intersection is larger, the area of ​​the rectangular frame increases accordingly; when the caliber of the blood vessel at the position of the arteriovenous intersection is smaller, the area of ​​the rectangular frame decreases accordingly. Determine the first vertex A and the second vertex B on any diagonal line of the sample detection frame K1, and the first vertex C and the second vertex D on any diagonal line of the sample detection frame K2, obtain sample detection points A, B, C and D, and obtain training samples according to the sample detection frame K1, the sample detection frame K2 and the multiple sample detection points A, B, C and D.

[0069] In some possible embodiments, the sample detection frames may be 1, 2, 3 or more, the specific number of which may be determined according to the abnormal fundus condition, and may correspond to multiple sample detection points.

[0070] S123: transform the size of the multiple training samples to obtain multiple training samples with the same size;

[0071] In this embodiment, by resizing multiple training sample images, the sizes of different fundus images are made to be within a unified range without losing image detail features, which can reduce the specificity of the image during training and thus improve the accuracy of the deep learning network training model.

[0072] Step S124: input the training samples into the deep learning target box detection network for training to obtain the target detection model.

[0073] In some possible implementations, in step S124, the training samples are input into the deep learning target detection network for training to obtain the target detection model, which may specifically include the following sub-steps:

[0074] Step S1: Input the training sample into the deep learning target box detection network for convolution to obtain feature maps of different scales;

[0075] Step S2: On feature maps of different scales, multiple target pre-selection boxes with different aspect ratios are generated with the center points of different cells as the center;

[0076] Step S3: Determine multiple target detection frames according to a preset intersection-and-union ratio from multiple target pre-selected frames, determine the first vertex and the second vertex on the diagonal of the target detection frame, obtain multiple target detection points, and calculate the coordinate offset values ​​of the multiple target detection points relative to the coordinate offset values ​​of the multiple sample detection points;

[0077] The above steps S1 to S3 are continuously iterated until the coordinate offset value is reduced to a preset offset value, and the target frame detection model is obtained.

[0078] Among them, the intersection over union ratio, full name Intersection over Union (IoU for short), is the ratio of the intersection and union of the "predicted bounding box" and the "real bounding box". Figure 2B It is a schematic diagram of calculating the intersection-over-union ratio according to an embodiment of the present invention.

[0079] Figure 2C FIG. 1 is a schematic diagram of obtaining a target detection model as an example in an embodiment of the present invention. Figure 2C As shown:

[0080] Multiple training samples of the same size (sample 1, sample 2, sample 3...) and the marked coordinate points ((x1, y1), (x2, y2), (x3, y3)...) are input into the deep learning target detection network for convolution to obtain features of different scales. Figure 1 , Feature Map 2 and Feature Map 3; Figure 2D is a schematic diagram of target pre-selection boxes with different aspect ratios generated on feature maps of different scales according to an embodiment of the present invention, such as Figure 2D As shown in the figure, each scale feature map is divided into multiple cells, for example, an 8x8 feature map is divided into 64 cells, and a 4x4 feature map is divided into 16 cells. Each cell corresponds to a set of features in the original image. On feature maps of different scales, multiple target pre-selected boxes with different aspect ratios are generated with the center points of different cells as the center, and then multiple target detection boxes are determined according to the preset intersection-union ratio; the intersection-union ratio is the ratio of the "intersection" and "union" of the "predicted bounding box" and the "real bounding box", that is, the ratio of the "intersection" and "union" of the pre-selected target box and the target detection box;

[0081] Determine the first vertex and the second vertex on the diagonal of each target detection frame, obtain multiple target detection points, and calculate the offset value of the coordinates of each target detection point and the coordinates of the corresponding sample detection point, that is, (x1', y1'), (x2', y2'), (x3', y3')..., calculate the difference between the determined coordinate point and the marked coordinate point (x1, y1), (x2, y2), (x3, y3)..., that is, |x1'-x1|=△x1, |y1'-y1|=△y1..., the above steps are continuously iterated until the difference (△x1, △y1...) is reduced to the preset coordinate offset value.

[0082] In some embodiments, the number of feature maps of different scales is determined by the number of convolutional layers in a deep learning-based object detection network, and different convolutional layers output feature maps of different scales.

[0083] In some embodiments, the deep learning target box detection network is a Single Shot MultiBox Detector (SSD deep learning network for short). The SSD deep learning network uses a variety of data enhancement algorithms, including horizontal flipping, cropping, zooming in and out, etc., which can significantly improve the performance of the algorithm, making the method more robust to target detection of different sizes and shapes of inputs.

[0084] Figure 3A Schematic diagram of generating arteriovenous cross compression probability values ​​according to an embodiment of the present invention. Figure 3A As shown, the probability value of arteriovenous cross-compression corresponding to the target detection frame area is determined by detecting the morphological changes of the veins in the arteriovenous cross-compression data. For example, within the target detection frame, the intersection position of the target detection frame area is determined to be the probability value θ of arteriovenous cross-compression by calculating the diameter value of the veins, the curvature value of the veins, and / or the morphological changes of the veins.

[0085] Specifically, Figure 3B is a schematic diagram of the morphological changes of venous blood vessels in an arteriovenous cross compression phenomenon according to an embodiment of the present invention, such as Figure 3BAs shown, a is a normal arteriovenous crossing; b is an arteriovenous crossing compression phenomenon, in which the vein at the intersection is hidden, that is, it shrinks into a pen tip shape on both sides of the artery. At this time, the probability value of arteriovenous crossing compression is judged according to the diameter value of the blood vessel at the pen tip; c is an arteriovenous crossing compression phenomenon, in which the vein is in a state of being crushed. At this time, the probability value of arteriovenous crossing compression is judged according to the degree of vein crushing; d is an arteriovenous crossing compression phenomenon, in which the vein is reduced to a fusiform shape. At this time, the probability value of arteriovenous crossing compression is judged by judging the diameter value of the place where the vein is reduced to a fusiform shape; e is an arteriovenous crossing compression phenomenon, in which the distal end of the vein is dilated and tumor-like. At this time, by measuring the distal end of the vein The size of the expansion is used to determine the probability of arteriovenous cross compression; f is the deviation of the vein in the phenomenon of arteriovenous cross compression. At this time, the probability of arteriovenous cross compression is determined by the degree of curvature of the venous blood vessels; g is the phenomenon of arteriovenous cross compression. When the vein is located below the artery, the vein is compressed in an S shape. At this time, the curvature of the venous blood vessels is used to determine the probability of arteriovenous cross compression; h is the phenomenon of arteriovenous cross compression. When the vein is located above the artery, the bulge of the vein crosses the artery in the shape of an arch bridge. At this time, the probability of arteriovenous cross compression is determined based on the size of the arch bridge or the curvature of the venous blood vessels; where 1 is the vein and 2 is the artery.

[0086] S130: Determine whether the target detection frame contains an arteriovenous cross compression feature according to the probability value of the arteriovenous cross compression.

[0087] In this embodiment, if the probability value of arteriovenous cross compression generated on the target detection frame is greater than or equal to the preset probability threshold, it is determined that the target detection frame contains arteriovenous cross compression features. As an example, when the preset probability threshold is 0.6, it is determined that the target detection frame corresponding to the probability value greater than or equal to 0.6 contains arteriovenous cross compression features.

[0088] In some embodiments, after detecting the arteriovenous cross compression feature in the target detection frame according to the probability value of the arteriovenous cross compression, the method further includes:

[0089] Combined with the nerve fiber layer on the fundus image and the age information of the patient corresponding to the fundus image, the erroneously extracted arteriovenous cross compression features and their corresponding target detection frames are deleted.

[0090] In some embodiments, in combination with the nerve fiber layer on the fundus image, the age information of the corresponding patient, and other features, some of the locations of the generated target detection frames may not be true arteriovenous cross compression, which means that the generated target detection frame does not belong to the arteriovenous cross compression feature, and the target detection frame will be deleted. For example, hypertension is more common in people over 30 years old. When patients under 30 years old are detected with suspected arteriovenous cross compression, it is generally believed that it is due to image quality or nerve fiber layer. The generated target detection frame is deleted, the network detection results are cleared, and the accuracy of the detection is improved.

[0091] The embodiment of the present invention obtains a target detection model based on deep learning, and can quickly, accurately and intuitively identify the arteriovenous crossing compression features of fundus images without relying on the quality of the image. By identifying the arteriovenous crossing compression features, not only can many fundus complications of chronic diseases and cardiovascular and cerebrovascular diseases be discovered, but it can also reflect to a certain extent the structural damage caused by chronic diseases and cardiovascular and cerebrovascular diseases to the body, providing a basic reference for the evaluation and progression assessment of chronic diseases and cardiovascular and cerebrovascular diseases.

[0092] Embodiment 2

[0093] Figure 4 1 is a functional block diagram of a device for detecting cross compression characteristics of fundus arteries and veins according to an embodiment of the present invention. Figure 4 As shown, based on similar inventive concepts, the fundus arteriovenous cross compression feature detection device 400 includes:

[0094] An acquisition module 410 is used to acquire a fundus image and perform size transformation on the fundus image;

[0095] The target generation module 420 is used to input the size-transformed fundus image into a pre-trained deep learning-based target detection model, and generate multiple target detection frames and probability values ​​of arteriovenous cross compression features corresponding to the target detection frames on the fundus image;

[0096] The feature detection module 430 is used to determine whether the target detection frame contains the arteriovenous cross compression feature according to the probability value of the arteriovenous cross compression feature.

[0097] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0098] Embodiment 3

[0099] The embodiment of the present invention further provides a computer-readable storage medium 500, in which a computer program 510 is stored. When the computer program 510 is executed by a processor, the following is achieved:

[0100] Acquire a fundus image, and perform size transformation on the fundus image;

[0101] The resized fundus image is input into a pre-trained deep learning-based target detection model, and multiple target detection frames and probability values ​​of arteriovenous cross compression features corresponding to the target detection frames are generated on the fundus image;

[0102] According to the probability value of the arteriovenous cross compression feature, it is determined whether the target detection frame contains the arteriovenous cross compression feature.

[0103] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. Of course, there are other ways of readable storage media, such as quantum memory, graphene memory, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0104] Embodiment 4

[0105] The embodiment of the present invention further provides an electronic device 600, such as Figure 6 As shown, it includes one or more processors 601 , a communication interface 602 , a memory 603 and a communication bus 604 , wherein the processor 601 , the communication interface 602 , and the memory 603 communicate with each other via the communication bus 604 .

[0106] Memory 603, used for storing computer programs;

[0107] The processor 601 is used to execute the program stored in the memory 603 to achieve:

[0108] Acquire a fundus image, and perform size transformation on the fundus image;

[0109] The resized fundus image is input into a pre-trained deep learning-based target detection model, and multiple target detection frames and probability values ​​of arteriovenous cross compression features corresponding to the target detection frames are generated on the fundus image;

[0110] It is used to determine whether the target detection frame contains the arteriovenous cross compression feature according to the probability value of the arteriovenous cross compression feature.

[0111] In a possible design, the processing performed by the processor 601 further includes, before acquiring the fundus image, the following steps:

[0112] acquiring a plurality of fundus training images;

[0113] Annotate multiple fundus training images to obtain training samples;

[0114] Transforming the sizes of multiple training samples to obtain multiple training samples with the same size;

[0115] The training samples are input into the deep learning target detection network for training to obtain the target detection model.

[0116] In a possible design, the processing performed by the processor 601 further includes labeling multiple fundus training images to obtain training samples, specifically including:

[0117] Marking a rectangular frame at a location of an arteriovenous cross compression feature in a fundus training image to obtain multiple sample detection frames;

[0118] Determine a first vertex and a second vertex on a diagonal line of a sample detection frame, and obtain a plurality of sample detection points;

[0119] Training samples are obtained according to multiple sample detection frames and multiple sample detection points.

[0120] In a possible design, the processing performed by the processor 601 further includes inputting the training sample into the deep learning target detection network for training to obtain the target detection model, specifically including:

[0121] Step S1: Input the training sample into the deep learning target detection network for convolution to obtain feature maps of different scales;

[0122] Step S2: On feature maps of different scales, multiple target pre-selection boxes with different aspect ratios are generated with the center points of different cells as the center;

[0123] Step S3: Determine multiple target detection frames according to a preset intersection-and-union ratio from multiple target pre-selected frames, determine the first vertex and the second vertex on the diagonal of the target detection frame, obtain multiple target detection points, and calculate the coordinate offset values ​​of the multiple target detection points relative to the coordinate offset values ​​of the multiple sample detection points;

[0124] The above steps S1 to S3 are continuously iterated until the coordinate offset value is reduced to a preset offset value or below the preset offset value, thereby obtaining a target detection model.

[0125] In a possible design, the processing performed by the processor 601 further includes inputting the size-transformed fundus image into a pre-trained deep learning-based target detection model to generate multiple target detection frames on the fundus image, specifically including:

[0126] The fundus image is input into the target detection model for convolution to obtain multiple feature maps of different scales;

[0127] On feature maps of different scales, multiple target pre-selection boxes of different shapes are generated with the center points of different cells as the center;

[0128] The target pre-selected boxes on feature maps of different scales are combined, and the non-maximum suppression method is used to select multiple target detection boxes respectively.

[0129] In one possible design, the processing performed by processor 601 further includes:

[0130] The probability value of cross compression corresponding to the target detection frame is determined based on the morphological changes of the veins, the changing trends of the diameter values ​​of the veins, and / or the changing trends of the curvature values ​​of the veins obtained by the deep learning target detection model.

[0131] In a possible design, the processing performed by the processor 601 includes detecting the arteriovenous cross compression feature in the target detection frame according to the probability value of the arteriovenous cross compression, specifically including:

[0132] When the probability value of arteriovenous cross compression generated on the target detection frame is greater than or equal to a preset probability threshold, the intersection position in the target detection frame is determined to be arteriovenous cross compression, and the arteriovenous cross compression feature is detected in the target detection frame.

[0133] In a possible design, in the processing performed by the processor 601, after extracting the arteriovenous cross compression feature according to the probability value of the arteriovenous cross compression, the processing also includes:

[0134] Combined with the nerve fiber layer on the fundus image and the age information of the patient corresponding to the fundus image, the erroneously extracted arteriovenous cross compression features and their corresponding target detection frames are deleted.

[0135] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used for communication between the above electronic device and other devices.

[0136] Bus 604 includes hardware, software or both, for coupling the above-mentioned components to each other.For example, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations.In suitable cases, bus may include one or more buses.Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0137] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0138] The memory 603 may include a large capacity memory for data or instructions. For example, but not limitation, the memory 603 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 603 may include a removable or non-removable (or fixed) medium. In a specific embodiment, the memory 603 is a non-volatile solid-state memory. In a specific embodiment, the memory 603 includes a read-only memory (ROM). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0139] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0140] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0141] Although the present application provides method operation steps as described in the embodiments or flow charts, more or fewer operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, in a parallel processor or multi-threaded processing environment, or even in a distributed data processing environment).

[0142] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0145] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0146] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0147] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for detecting cross compression characteristics of fundus arteries and veins, characterized in that: include: Acquire a fundus image, and perform size transformation on the fundus image; Inputting the resized fundus image into a pre-trained deep learning-based target detection model, generating a plurality of target detection frames and probability values ​​of arteriovenous cross compression features corresponding to the target detection frames on the fundus image; Determine whether the target detection frame contains an arteriovenous crossing compression feature according to a probability value of the arteriovenous crossing compression feature being contained in the target detection frame; delete the erroneously extracted arteriovenous crossing compression feature and the target detection frame corresponding thereto, in combination with the nerve fiber layer on the fundus image and the age information of the patient corresponding to the fundus image; Before acquiring the fundus image, the method further includes: acquiring a plurality of fundus training images; Annotating the multiple fundus training images to obtain training samples; Performing size transformation on the plurality of training samples to obtain a plurality of training samples with the same size; The plurality of training samples of the same size are input into a deep learning target detection network for training to obtain the target detection model; wherein, The step of labeling the plurality of fundus training images to obtain training samples specifically includes: Marking a rectangular frame at the position of the arteriovenous intersection compression feature on the fundus training image to obtain multiple sample detection frames; the size of the rectangular frame area varies with the diameter value of the arterial blood vessel and / or the venous blood vessel at the position of the arteriovenous intersection node; Determine a first vertex and a second vertex on any diagonal line of the sample detection frame to obtain a plurality of sample detection points; Obtain the training sample according to the multiple sample detection frames and the multiple sample detection points; The step of inputting the training samples into a deep learning target detection network for training to obtain the target detection model specifically includes: Step S1: input the training sample into a deep learning target detection network for convolution to obtain feature maps of different scales; Step S2: on the feature maps of different scales, taking the center points of different cells as the center, generating a plurality of target pre-selection boxes of different aspect ratios; Step S3: determining a plurality of target detection frames according to a preset intersection-and-union ratio from the plurality of target pre-selected frames, determining a first vertex and a second vertex on a diagonal line of the plurality of target detection frames, obtaining a plurality of target detection points, and calculating coordinate offset values ​​of the plurality of target detection points relative to the plurality of sample detection points; The above steps S1 to S3 are continuously iterated until the coordinate offset value decreases to a preset offset value or below the preset offset value, thereby obtaining the target detection model.

2. The method according to claim 1, characterized in that The step of inputting the scale-transformed fundus image into a pre-trained deep learning-based target detection model, and generating multiple target detection frames and probability values ​​of arteriovenous cross compression features corresponding to the target detection frames on the fundus image, specifically includes: Inputting the fundus image into the target detection model for convolution to obtain a plurality of feature maps of different scales; On the feature maps of different scales, taking the center points of different cells as the center, generating a plurality of target pre-selection boxes of different shapes; Combining the target pre-selected boxes on the feature maps of different scales, and selecting the multiple target detection boxes respectively using a non-maximum suppression method; According to the deep learning-based target detection model, the morphological changes of the veins, the changing trends of the vein diameter values, and / or the changing trends of the vein curvature values ​​are obtained to determine the probability value of the target detection frame containing the feature of arteriovenous cross compression.

3. The method according to claim 1, characterized in that The determining, according to the probability value of the arteriovenous cross compression feature, whether the target detection frame contains the arteriovenous cross compression feature specifically includes: When the probability value of arteriovenous cross compression on the target detection frame is greater than or equal to a preset probability threshold, it is determined that the target detection frame contains the feature of arteriovenous cross compression, and the intersection position contained in the target detection frame is arteriovenous cross compression.

4. A device for detecting cross compression characteristics of fundus arteries and veins, characterized in that: The device is used to implement a method for detecting fundus arteriovenous cross compression characteristics as described in any one of claims 1 to 3, comprising: An acquisition module, used for acquiring a fundus image and performing size transformation on the fundus image; A target generation module is used to input the size-transformed fundus image into a pre-trained deep learning-based target detection model, and generate multiple target detection frames and probability values ​​of arteriovenous cross compression features corresponding to the target detection frames on the fundus image; The feature detection module is used to determine whether the target detection frame contains an arteriovenous cross compression feature according to the probability value of the arteriovenous cross compression.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a method for detecting fundus arteriovenous cross compression characteristics as described in any one of claims 1 to 3 is implemented.

6. A device for detecting cross compression characteristics of fundus arteries and veins, characterized in that: It includes: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a method for detecting fundus arteriovenous cross compression features as described in any one of claims 1 to 3.

Images