A fundus examination image processing optimization method

Abstract

The present invention discloses a fundus examination image processing optimization method, which relates to the field of image processing technology. The method comprises: obtaining multiple fundus examination images for pixel gradient amplitude iteration identification to obtain an authentication result; when the authentication result is authentication passed, performing splicing and fusion trajectory identification, determining a global splicing and fusion trajectory, splicing and fusing multiple fundus examination images to obtain a target fundus examination image; constructing multiple feature perception subunits of a multi-target feature recognition unit; calling the multi-target feature recognition unit to perform abnormal feature recognition on the target fundus examination image to obtain an abnormal feature recognition result. The present invention solves the technical problem in the prior art that fundus examination images are blurred due to limited field of view and the accuracy of abnormal feature recognition is low, and achieves the technical effect of improving the quality of fundus examination images and improving the accuracy of abnormal feature recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a fundus examination image processing optimization method. Background Art

[0002] During fundus examination, the image-collecting device can usually only capture the fundus area at a certain angle. For example, the field of view of a device can only cover part of the retina, and other parts of the patient's fundus are not fully captured. If there is a tiny lesion somewhere in the fundus, but it happens to be in an uncovered area, this abnormality will not be discovered. The existing technology has technical problems such as fundus examination images are blurred due to the limited field of view, and the accuracy of abnormal feature recognition is low. Summary of the invention

[0003] The present application provides a fundus examination image processing optimization method for solving the technical problems in the prior art that fundus examination images are blurred and have low accuracy in abnormal feature recognition due to limited field of view.

[0004] In view of the above problems, the present application provides a method for optimizing fundus examination image processing.

[0005] In a first aspect of the present application, a method for optimizing fundus examination image processing is provided, the method comprising:

[0006] Acquire multiple fundus examination images for iterative pixel gradient amplitude recognition to obtain an authentication result, wherein the multiple fundus examination images are images obtained by collecting fundus images of the same examination object at different angles, and the multiple fundus examination images have overlapping areas;

[0007] When the authentication result is authentication passed, performing stitching and fusion trajectory identification on the overlapping areas of the multiple fundus examination images, determining a global stitching and fusion trajectory, and stitching and fusion the multiple fundus examination images according to the global stitching and fusion trajectory to obtain a target fundus examination image;

[0008] Performing preliminary abnormal similarity matching on the target fundus examination image, obtaining multiple abnormal area bandwidths, and constructing multiple feature perception subunits of a multi-target feature recognition unit based on the multiple abnormal area bandwidths;

[0009] The multiple feature sensing subunits of the multi-target feature recognition unit are called to perform abnormal feature recognition on the target fundus examination image, and the recognition results are summarized to obtain abnormal feature recognition results.

[0010] In a second aspect, the present application provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is used to execute a fundus examination image processing optimization method provided in the present application.

[0011] In a third aspect, the present application provides a computer-readable storage medium storing a computer program, and the computer program is used to execute a fundus examination image processing optimization method provided by the present application.

[0012] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0013] The present application obtains the authentication result by acquiring multiple fundus examination images for pixel gradient amplitude iterative identification, wherein the multiple fundus examination images are images obtained by collecting fundus images of the same examination object at different angles, and there are overlapping areas between the multiple fundus examination images. Then, when the authentication result is authentication passed, the overlapping areas of the multiple fundus examination images are identified by splicing and fusion trajectories, the global splicing and fusion trajectory is determined, and the multiple fundus examination images are spliced ​​and fused according to the global splicing and fusion trajectory to obtain the target fundus examination image, and then the target fundus examination image is preliminarily matched for abnormal similarity, multiple abnormal area bandwidths are obtained, and multiple feature perception subunits of the multi-target feature recognition unit are constructed based on the multiple abnormal area bandwidths, and the target fundus examination image is identified for abnormal features by calling multiple feature perception subunits of the multi-target feature recognition unit, and the identification results are summarized to obtain the abnormal feature recognition results. The technical effect of improving the reliability of splicing and fusion of multi-angle fundus examination images, improving the quality of fundus examination images, and thus improving the accuracy of abnormal feature recognition is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 A schematic diagram of a flow chart of a fundus examination image processing optimization method provided in an embodiment of the present application;

[0016] Figure 2 A schematic diagram of images before and after splicing and fusion in a fundus examination image processing optimization method provided in an embodiment of the present application;

[0017] Figure 3 A schematic diagram of the structure of an electronic device provided in this application.

[0018] Description of the reference numerals: processor 21 , memory 22 , input device 23 , output device 24 . DETAILED DESCRIPTION

[0019] The present application provides a fundus examination image processing optimization method for solving the technical problems in the prior art that fundus examination images are blurred and have low accuracy in abnormal feature recognition due to limited field of view.

[0020] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] It should be noted that the terms "including" and "having" are intended to cover non-exclusive inclusions. For example, a process, method, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules that are not explicitly listed or inherent to these processes, methods, products or devices.

[0022] Embodiment 1, as Figure 1 As shown, the present application provides a method for optimizing fundus examination image processing, wherein the method comprises:

[0023] S100: Acquire multiple fundus examination images for iterative pixel gradient amplitude recognition to obtain an authentication result, wherein the multiple fundus examination images are images acquired by collecting fundus images of the same examination object at different angles, and the multiple fundus examination images have overlapping areas;

[0024] In a possible embodiment, the multiple fundus examination images are images obtained by collecting fundus images of the same examination object at different angles, and each image provides a different part of the fundus, thereby providing data support for the subsequent provision of comprehensive fundus examination images. Optionally, there are overlapping areas between the multiple fundus examination images, and the overlapping area can range from 15% to 25%. There are overlapping areas between fundus images taken at different angles. This means that part of the content in the multiple images is the same or similar, and these overlapping parts will become the basis for subsequent image stitching and authentication.

[0025] After obtaining the multiple fundus examination images, the quality of each fundus examination image needs to be authenticated. Optionally, the clarity of each fundus examination image is determined by iteratively identifying the pixel gradient amplitude of the multiple fundus examination images. When the authentication result is passed, the multiple fundus examination images can be applied to the subsequent image stitching and fusion process. When the authentication result is not passed, the fundus examination image acquisition needs to be performed again. The technical effect of laying the foundation for subsequent image stitching and fusion and abnormal feature recognition is achieved.

[0026] Further, a plurality of fundus examination images are obtained for iterative pixel gradient amplitude recognition to obtain an authentication result. Step S100 of the embodiment of the present application further includes:

[0027] Randomly extracting a preset number of sample pixel points from the multiple fundus examination images to perform pixel gradient amplitude recognition, and obtaining multiple sample pixel gradient amplitude sets;

[0028] Performing mean processing on the plurality of sample pixel gradient amplitude sets respectively to obtain a plurality of sample pixel gradient amplitude mean values;

[0029] Taking the mean values ​​of the plurality of sample pixel gradient amplitudes as a plurality of iteration center points respectively, performing iterative search in the plurality of sample pixel gradient amplitude sets with a preset step size, determining a plurality of target iteration center points, and taking the plurality of sample pixel gradient amplitudes corresponding to the plurality of target iteration center points as a plurality of authentication pixel gradient amplitudes;

[0030] When the multiple authentication pixel gradient amplitudes are greater than or equal to the preset authentication pixel gradient amplitude, authentication pass is taken as the authentication result.

[0031] Further, the mean values ​​of the plurality of sample pixel gradient amplitudes are respectively used as a plurality of iteration center points, and an iterative search is performed in the plurality of sample pixel gradient amplitude sets with a preset step size to determine a plurality of target iteration center points. In the embodiment of the present application, step S100 further includes:

[0032] Taking the mean values ​​of the gradient amplitudes of the multiple sample pixels as multiple iteration center points respectively, performing iterative search in the multiple sample pixel gradient amplitude sets with a preset step size to obtain multiple stage iteration center points;

[0033] When the iteration density of the multiple stage iteration center points is less than the iteration density of the multiple iteration center points, continue to use the multiple iteration center points as iteration starting points and perform iterative search in the multiple sample pixel gradient amplitude sets;

[0034] When the iteration density of the multiple stage iteration center points is greater than or equal to the iteration density of the multiple iteration center points and the iteration density difference between the multiple stage iteration center points and the multiple iteration center points is less than or equal to the preset density difference, the iteration is stopped and the multiple stage iteration center points obtained in the last iteration are used as the multiple target iteration center points.

[0035] In a possible embodiment, when the fundus examination image is clearer, the pixel gradient amplitude is larger, and when the fundus examination image is blurred, the pixel gradient amplitude is smaller. Therefore, by iteratively identifying the pixel gradient amplitudes of the multiple fundus examination images, the pixel gradient amplitude changes of the global images are determined, and then it is determined whether the image is authenticated.

[0036] Optionally, a preset number of sample pixel points (the number of pixels to be extracted is pre-set by a person skilled in the art according to the area of ​​the fundus examination image) are randomly extracted from the multiple fundus examination images for pixel gradient amplitude recognition to obtain multiple sample pixel gradient amplitude sets. After the pixel points are extracted according to the preset number, multiple sample pixel point sets are obtained. The distance between any two sample pixel points in the multiple sample pixel point sets is respectively identified. When the distance does not meet the preset distance threshold, the sample pixel points are reselected to avoid falling into the clarity recognition of the local area.

[0037] Optionally, after obtaining multiple sample pixel point sets, the Sobel operator is used to identify the gradients of the multiple sample pixel point sets in the horizontal direction and the vertical direction to obtain multiple sample horizontal gradient sets and multiple sample transverse gradient sets. The multiple sample horizontal gradient sets and the multiple sample transverse gradient sets are calculated using a gradient amplitude formula to obtain multiple sample pixel gradient amplitude sets. The multiple sample pixel gradient amplitude sets reflect the clarity of the locations where the multiple sample pixel point sets are located. The gradient amplitude formula is: .in, is the sample pixel gradient amplitude, is the horizontal gradient of a sample pixel, is the vertical gradient of a sample pixel.

[0038] The plurality of sample pixel gradient amplitude sets are respectively subjected to mean processing to obtain a plurality of sample pixel gradient amplitude means, wherein the plurality of sample pixel gradient amplitude means reflect the average image clarity of the plurality of fundus examination images. Further, the plurality of sample pixel gradient amplitude means are respectively used as a plurality of iteration center points, and an iterative search is performed in the plurality of sample pixel gradient amplitude sets with a preset step length to determine a plurality of target iteration center points, and the plurality of sample pixel gradient amplitudes corresponding to the plurality of target iteration center points are used as a plurality of authentication pixel gradient amplitudes. The preset step length is the pixel gradient amplitude size of the interval between the iteration center point and the stage iteration center point when performing iterative analysis in the plurality of fundus examination images, which is preset by a technician in this field.

[0039] Optionally, the mean values ​​of the multiple sample pixel gradient amplitudes are respectively used as multiple iteration center points, and an iterative search is performed in the multiple sample pixel gradient amplitude sets with a preset step size to obtain multiple stage iteration center points. Furthermore, when the iteration density of the multiple stage iteration center points is less than the iteration density of the multiple iteration center points, it indicates that the number of sample pixel gradient amplitudes distributed around the stage iteration center point is less than the number of sample pixel gradient amplitudes distributed around the iteration center point. At this time, the multiple iteration center points are continued to be used as the iteration starting points, and an iterative search is performed in the multiple sample pixel gradient amplitude sets with a preset step size. The iteration density of the multiple stage iteration center points is the ratio of the number of sample pixel gradient amplitudes distributed in an area constructed with the multiple stage iteration center points as the center and the preset iteration step size as the radius to the area of ​​the area, reflecting the density of the sample pixel gradient amplitudes distributed around the multiple stage iteration center points.

[0040] When the iteration density of the multiple stage iteration center points is greater than or equal to the iteration density of the multiple iteration center points and the iteration density difference between the multiple stage iteration center points and the multiple iteration center points is less than or equal to the preset density difference, it indicates that the area where the sample pixel gradient amplitude is densely distributed has been reached. At this time, the iteration is stopped, and the multiple stage iteration center points obtained in the last iteration are used as the multiple target iteration center points. Thus, multiple target iteration center points that can represent the pixel gradient amplitude of multiple fundus examination images are obtained. Furthermore, the multiple sample pixel gradient amplitudes corresponding to the multiple target iteration center points are used as multiple authentication pixel gradient amplitudes.

[0041] In one embodiment, when the plurality of authentication pixel gradient amplitudes are greater than or equal to a preset authentication pixel gradient amplitude, it indicates that the clarity of the fundus examination image has passed the authentication, and the authentication pass is taken as the authentication result, thereby achieving the technical effect of improving data reliability.

[0042] S200: When the authentication result is authentication passed, performing stitching and fusion trajectory identification on the overlapping areas of the multiple fundus examination images, determining a global stitching and fusion trajectory, and stitching and fusion the multiple fundus examination images according to the global stitching and fusion trajectory to obtain a target fundus examination image;

[0043] In a possible embodiment, when the authentication result is authentication passed, positioning is performed according to the acquisition angles of the multiple fundus examination images, and multiple overlapping areas of the multiple fundus examination images are obtained, and the stitching and fusion trajectories of the multiple overlapping areas are identified to determine the global stitching and fusion trajectory, so as to pave the way for stitching and fusing the multiple fundus examination images to obtain the target fundus examination image.

[0044] Further, when the authentication result is that the authentication is passed, the overlapping areas of the multiple fundus examination images are identified by stitching and fusion trajectories, a global stitching and fusion trajectory is determined, and the multiple fundus examination images are stitched and fused according to the global stitching and fusion trajectory to obtain a target fundus examination image. Step S200 of the embodiment of the present application also includes:

[0045] Perform overlapping area identification based on the multiple fundus examination images to obtain multiple overlapping areas, wherein each overlapping area is an overlapping area between two fundus examination images with the largest overlapping area;

[0046] Using a structural similarity perception unit to identify structural deviation factors of the multiple overlapping regions to obtain multiple structural deviation coefficients;

[0047] When the multiple structural deviation coefficients are less than or equal to the preset deviation coefficients, randomly selecting multiple first trajectory points from the multiple overlapping areas and obtaining multiple first trajectory neighborhoods of the multiple first trajectory points, wherein the multiple first trajectory neighborhoods include sixteen neighboring pixel points of the multiple first trajectory points in the multiple overlapping areas;

[0048] Randomly generate splicing and fusion sub-trajectories based on the multiple first trajectory points and the multiple first trajectory neighborhoods respectively, to obtain multiple first splicing and fusion sub-trajectories;

[0049] Using the plurality of first concatenated and fused sub-trajectories, segmenting the plurality of first trajectory neighborhoods to generate a plurality of first sub-trajectory left neighborhoods and a plurality of first sub-trajectory right neighborhoods;

[0050] Calling a pixel gradient identifier to identify left and right side gradient differences of the plurality of first track points, the plurality of first sub-track left neighborhoods, and the plurality of first sub-track right neighborhoods, respectively, to obtain a plurality of first sub-track gradient differences;

[0051] Again, randomly generating splicing and fusion sub-trajectories based on the plurality of first trajectory points and the plurality of first trajectory neighborhoods, respectively, to obtain a plurality of second splicing and fusion sub-trajectories, and obtaining a plurality of second sub-trajectory gradient differences of the plurality of second splicing and fusion sub-trajectories;

[0052] When the number of randomly generated splicing fusion sub-trajectories meets the preset number, the plurality of first sub-trajectory gradient differences, the plurality of second sub-trajectory gradient differences and the plurality of splicing fusion sub-trajectories corresponding to the minimum value in the plurality of sub-trajectory gradient difference sets are used as the plurality of best splicing fusion sub-trajectories of the plurality of first trajectory points;

[0053] Based on the multiple best spliced ​​fusion sub-trajectories of the multiple first trajectory points, global trajectory derivation is performed in the multiple overlapping areas to obtain multiple regional best spliced ​​fusion trajectories of the multiple overlapping areas;

[0054] The optimal stitching and fusion tracks of the multiple regions are used as the global stitching and fusion track.

[0055] Furthermore, the pixel gradient identifier includes a pixel gradient identification formula, and the pixel gradient identification formula is:

[0056] ;

[0057] in, is the gradient difference of the first sub-trajectory, is the pixel value of the i-th neighboring pixel in the left neighborhood of the first sub-trajectory, is the pixel value of the jth neighboring pixel in the neighborhood on the right side of the first sub-trajectory, IQ is the pixel value of the first trajectory point, and l+p=16.

[0058] Further, based on the multiple best stitching and fusion sub-trajectories of the multiple first trajectory points, global trajectory derivation is performed in the multiple overlapping areas to obtain multiple regional best stitching and fusion trajectories of the multiple overlapping areas. In the embodiment of the present application, step S200 further includes:

[0059] Using the front and rear ends of the multiple best spliced ​​fusion sub-trajectories as derivative trajectory points to obtain multiple front-end derivative trajectory points and multiple back-end derivative trajectory points;

[0060] Acquire a plurality of front-end derived trajectory neighborhoods based on the plurality of front-end derived trajectory points, wherein the plurality of front-end derived trajectory neighborhoods include sixteen neighboring pixel points located in front of the plurality of front-end derived trajectory points;

[0061] Acquire a plurality of back-end derived trajectory neighborhoods based on the plurality of back-end derived trajectory points, wherein the plurality of back-end derived trajectory neighborhoods include sixteen neighboring pixel points located behind the plurality of back-end derived trajectory points;

[0062] Based on the multiple front-end derived trajectory points, the multiple back-end derived trajectory points, the multiple front-end derived trajectory neighborhoods and the multiple back-end derived trajectory neighborhoods, global trajectory derivation of the multiple optimal splicing and fusion sub-trajectories is performed to obtain multiple regional optimal splicing and fusion trajectories of the multiple overlapping areas.

[0063] In a possible embodiment, positioning is performed according to the acquisition angles of the multiple fundus examination images, and the multiple fundus examination images are preliminarily spliced ​​according to the positioning results to obtain multiple overlapping areas. The multiple overlapping areas are obtained by performing overlapping area analysis on multiple overlapping fundus examination images and retaining only the two fundus examination images with the largest overlapping areas, that is, each overlapping area is the overlapping area between the two fundus examination images with the largest overlapping area.

[0064] Then, the position deviation of fundus structures (such as arteries and veins) in multiple overlapping areas is identified to determine whether the positions of multiple fundus examination images are adjusted in place. Optionally, the overlapping parts of multiple fundus examination image combinations (each combination includes two fundus examination images) in multiple overlapping areas are input into the structural similarity perception unit for structural factor deviation analysis to obtain the multiple structural deviation coefficients. The structural deviation coefficient reflects the position deviation of the fundus structure in the overlapping area.

[0065] Optionally, a plurality of sample fundus examination overlapping image combinations are obtained, wherein each sample fundus examination overlapping image combination includes a plurality of sample structural deviation coefficients, and the plurality of sample fundus examination image combinations and the plurality of sample structural deviation coefficients are used as training data, and supervised training is performed on a framework constructed based on a convolutional neural network, and each sample fundus examination overlapping image combination is processed by a plurality of convolutional layers and pooling layers to extract local features and global structural information in the image, and each layer of convolution operation extracts features at different levels, including edges and blood vessels, etc., and a structural similarity loss function (such as structural similarity index SSIM, MSE, etc.) is used to calculate the loss of the network to ensure that the trained network can accurately perceive the structural similarity between images, and obtain training output results, and the training output results are verified using a plurality of sample structural deviation coefficients, and the degree of deviation between the training output results and the corresponding sample structural deviation coefficients is calculated to obtain accuracy, and when the accuracy meets the preset accuracy, the trained structural similarity perception unit is obtained.

[0066] The multiple structural deviation coefficients are compared with the preset deviation coefficients (the minimum deviation coefficient for splicing and fusion) pre-set by those skilled in the art. When the multiple structural deviation coefficients are less than or equal to the preset deviation coefficients, multiple pixels are randomly selected from the multiple overlapping areas as multiple first trajectory points, and multiple first trajectory neighborhoods of the multiple first trajectory points are obtained, wherein the multiple first trajectory neighborhoods include the sixteen neighboring pixel points of the multiple first trajectory points in the multiple overlapping areas. In other words, each first trajectory neighborhood includes the sixteen closest pixel points of a first trajectory point in any direction in the corresponding overlapping area.

[0067] Then, multiple neighboring pixel points are selected from the multiple first track neighborhoods and connected to the multiple first track points to obtain multiple first splicing and fusion sub-tracks. Then, the multiple first splicing and fusion sub-tracks are used to split the multiple first track neighborhoods into left and right to obtain multiple first sub-track left neighborhoods and multiple first sub-track right neighborhoods.

[0068] In one embodiment, in order to identify the pixel gradient difference of the left and right neighborhoods after the multiple first spliced ​​fusion sub-trajectories divide the multiple first trajectory neighborhoods into left and right, the pixel gradient identifier is used for identification. The multiple first trajectory points, the multiple first sub-trajectory left neighborhoods, and the multiple first sub-trajectory right neighborhoods are respectively input into the pixel gradient identification formula to calculate the pixel gradient difference of the left and right sides, and obtain the multiple first sub-trajectory gradient differences. Among them, the smaller the first sub-trajectory gradient difference, the better the segmentation effect, and the smoother the transition after the two fundus examination images are spliced ​​and fused.

[0069] The pixel gradient identifier includes a pixel gradient identification formula, and the pixel gradient identification formula is:

[0070] ;

[0071] in, is the gradient difference of the first sub-trajectory, which reflects the gradient difference between the left and right neighbors. is the pixel value of the i-th neighboring pixel in the left neighborhood of the first sub-trajectory, is the pixel value of the jth neighboring pixel in the neighborhood on the right side of the first sub-trajectory, IQ is the pixel value of the first trajectory point, and l+p=16.

[0072] Based on the same principle of obtaining the multiple first splicing and fusion sub-trajectories, the splicing and fusion sub-trajectories are randomly generated based on the multiple first trajectory points and the multiple first trajectory neighborhoods, and multiple second splicing and fusion sub-trajectories are obtained, and multiple second sub-trajectory gradient differences of the multiple second splicing and fusion sub-trajectories are obtained. The multiple second sub-trajectory gradient differences reflect the splicing and fusion transition effect of the multiple second splicing and fusion sub-trajectories.

[0073] After multiple random generation until the preset number of times is met, multiple first sub-trajectory gradient differences, multiple second sub-trajectory gradient differences, and multiple splicing and fusion sub-trajectories corresponding to the minimum value in the multiple sub-trajectory gradient difference sets are used as multiple best splicing and fusion sub-trajectories of multiple first trajectory points. In other words, multiple splicing and fusion sub-trajectories with the best splicing and fusion effects are selected. This achieves the technical effect of paving the way for improving the subsequent global splicing and fusion quality.

[0074] In one embodiment, the multiple best stitching and fusion sub-trajectories of the multiple first trajectory points are used as the starting points for derivation of the global trajectory, and the global trajectory is derived in the multiple overlapping areas, so as to obtain multiple regional best stitching and fusion trajectories of the multiple overlapping areas, and then the multiple regional best stitching and fusion trajectories are used as the global stitching and fusion trajectory when stitching and fusion are performed on multiple fundus examination images. The above method for obtaining the global stitching and fusion trajectory is applied to the stitching and fusion of multiple fundus examination images. For example, the image comparison before and after the stitching and fusion of the fundus blood vessel images is performed. Figure 2 shown.

[0075] In one embodiment, the front and rear ends of the multiple best spliced ​​fusion sub-trajectories are used as derivative trajectory points to obtain multiple front-end derivative trajectory points and multiple back-end derivative trajectory points. That is, trajectory derivation starts from the front and rear directions of the multiple best spliced ​​fusion sub-trajectories at the same time. Then, multiple front-end derivative trajectory neighborhoods are obtained based on the multiple front-end derivative trajectory points, wherein the multiple front-end derivative trajectory neighborhoods include sixteen neighboring pixel points located in front of the multiple front-end derivative trajectory points. At this time, the multiple front-end derivative trajectory neighborhoods are all pixel points located in front of the multiple front-end derivative trajectory points.

[0076] In one embodiment, a plurality of back-end derived trajectory neighborhoods are obtained with the plurality of back-end derived trajectory points as starting points, wherein the plurality of back-end derived trajectory neighborhoods include sixteen adjacent pixel points located behind the plurality of back-end derived trajectory points. That is to say, at this time, the plurality of back-end derived trajectory neighborhoods are all pixel point sets located behind the plurality of back-end derived trajectory points.

[0077] Based on the same obtaining principle as the multiple optimal splicing and fusion sub-trajectories, global trajectory derivation of the multiple optimal splicing and fusion sub-trajectories is performed based on the multiple front-end derived trajectory points, the multiple back-end derived trajectory points, the multiple front-end derived trajectory neighborhoods and the multiple back-end derived trajectory neighborhoods, thereby obtaining multiple regional optimal splicing and fusion trajectories for the multiple overlapping areas.

[0078] S300: performing preliminary abnormal similarity matching on the target fundus examination image, obtaining multiple abnormal region bandwidths, and constructing multiple feature perception subunits of a multi-target feature recognition unit based on the multiple abnormal region bandwidths;

[0079] Furthermore, step S300 of the embodiment of the present application further includes:

[0080] Using the target fundus examination image as an index, performing similar examination image matching in an image database to obtain a plurality of similar fundus examination images, wherein the plurality of similar fundus examination images include a plurality of similar fundus examination abnormal region area sets;

[0081] According to the size of the abnormal area, clustering the multiple similar fundus examination abnormal area sets across images to obtain multiple clustered similar fundus examination abnormal area sets;

[0082] Calculating the mean of each of the plurality of clustered similar fundus examination abnormal area sets to obtain the mean of the plurality of clustered abnormal area areas;

[0083] The area means of the multiple clustered abnormal regions are used as the multiple abnormal region bandwidths, and the receptive fields of the multiple feature perception sub-units are configured according to the multiple abnormal region bandwidths to obtain the multiple feature perception sub-units that have been configured.

[0084] In one possible embodiment, the target fundus examination image needs to be processed to identify abnormal areas that may exist in the image. The abnormal area may be information such as lesions and structural abnormalities, and specific features are usually extracted through image analysis algorithms. However, since the sizes of abnormal areas vary, when performing identification and analysis, the target fundus examination image is first used as an index to perform similar image matching to determine multiple abnormal area bandwidths of a multi-target feature recognition unit for abnormal identification of the target fundus examination image. Furthermore, the multiple abnormal area bandwidths are configured with the receptive fields of multiple feature perception subunits to determine the network parameters of the multiple feature perception subunits, and the multiple feature perception subunits that have been constructed are obtained.

[0085] In one embodiment, the target fundus examination image is used as an index to search for similar fundus examination images in the image database and perform similar examination image matching. In this step, images similar to the target fundus examination image in image features are found through historical data in the image database. Among the multiple similar fundus examination images matched, each similar fundus examination image contains one or more abnormal areas. By extracting the area information of the abnormal area from these similar fundus examination images, a plurality of abnormal area area sets are formed, and the plurality of similar fundus examination abnormal area area sets are obtained. Each abnormal area area set represents the size of the corresponding abnormal area in different similar fundus images.

[0086] In one embodiment, the plurality of similar fundus examination abnormal area sets are aggregated across regions to obtain an aggregated similar fundus examination abnormal area set. Further, a first aggregated similar fundus examination abnormal area is randomly extracted from the aggregated similar fundus examination abnormal area set as a first partitioning node, and the fundus examination abnormal area areas in the aggregated similar fundus examination abnormal area set whose area difference with the first aggregated similar fundus examination abnormal area is within a preset area difference are added to the first clustered similar fundus examination abnormal area set using the first partitioning node.

[0087] Then, the first cluster of similar fundus examination abnormal area set is removed from the summary of similar fundus examination abnormal area set, and the second summary of similar fundus examination abnormal area is randomly extracted from the summary of similar fundus examination abnormal area set after removal as the second division node. Based on the same acquisition principle as the first cluster of similar fundus examination abnormal area set, the second cluster of similar fundus examination abnormal area set is obtained. After multiple divisions, the multiple clusters of similar fundus examination abnormal area sets are obtained.

[0088] In one embodiment, the mean values ​​of the sets of area of ​​the multiple clustered similar fundus examination abnormal regions are calculated respectively to obtain the mean values ​​of the area of ​​the multiple clustered abnormal regions. The mean values ​​of the area of ​​the multiple clustered abnormal regions reflect the average area size of each clustering result. Optionally, the mean values ​​of the area of ​​the multiple clustered abnormal regions are used as multiple abnormal region bandwidths. With the multiple abnormal region bandwidths as indexes, the abnormal region bandwidth-receptive field mapping relationship set by a technician in this field is retrieved to obtain the receptive fields of the multiple feature perception subunits. According to the receptive field matching unit construction parameters of the multiple feature perception subunits, the unit parameters include the convolution kernel size, the pooling layer, and the convolution kernel moving step size, etc. The unit parameters are used to perform the initial configuration of the feature perception subunits to obtain multiple initial feature perception subunits with the initial configuration of the parameters.

[0089] Extract the first initial feature perception subunit from the multiple initial feature perception subunits, and obtain the first sample training data based on the average area of ​​the clustered abnormal region corresponding to the first initial feature perception subunit, wherein the first sample training data includes a first sample fundus examination image set with the first clustered abnormal region area average and a corresponding first sample abnormal feature recognition result set. Use the first sample training data to train the first initial feature perception subunit, learn the mapping relationship between the first sample fundus examination image and the first sample abnormal feature recognition result, until convergence is reached, and obtain the first initial feature perception subunit that has been trained. Based on the same principle, train multiple initial feature perception subunits until convergence is reached, and obtain the multiple feature perception subunits that have been trained. The technical effect of multi-scale abnormal feature recognition of the target fundus examination image is achieved.

[0090] S400: Calling multiple feature perception subunits of the multi-target feature recognition unit to perform abnormal feature recognition on the target fundus examination image, and summarizing the recognition results to obtain abnormal feature recognition results.

[0091] In one possible embodiment, the target fundus examination image is input into the multiple feature sensing subunits of the multi-target feature recognition unit for feature recognition, and multiple sub-abnormal feature recognition results are obtained after recognition, and the multiple sub-abnormal feature recognition results are summarized to obtain the abnormal feature recognition result. The abnormal feature recognition result reflects the abnormality of the target fundus examination image, including the abnormal area identification and the abnormal probability identification. The technical effect of efficiently processing the fundus examination image and improving the recognition reliability is achieved.

[0092] In summary, the embodiments of the present application have at least the following technical effects:

[0093] 1. When processing multiple fundus examination images acquired from different angles, the present application can effectively extract and analyze pixel gradient amplitude information, and through multiple iterative searches, ultimately determine multiple target iteration center points, which are used as the basis for authentication to ensure the clarity of the image, thereby achieving the technical effect of providing reliable basic data support for fundus examination image processing.

[0094] 2. The present application has the technical effect of improving the reliability of stitching and fusion of multiple fundus examination images by identifying the stitching and fusion trajectories of multiple overlapping areas.

[0095] 3. The present application achieves the technical effect of improving the accuracy of abnormal feature recognition of fundus examination images by configuring multiple feature perception sub-units according to the bandwidths of multiple abnormal areas obtained by analyzing the target fundus examination image, and then using multiple feature perception units to perform feature recognition.

[0096] Embodiment 2: Figure 3 The structural schematic diagram of the electronic device provided in the second embodiment of the present application shows a block diagram of an exemplary electronic device suitable for implementing the embodiment of the present invention. Figure 3 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. Figure 3 As shown, the electronic device includes a processor 21, a memory 22, an input device 23 and an output device 24; the number of processors 21 in the electronic device can be one or more. Figure 3 Taking a processor 21 as an example, the processor 21, memory 22, input device 23 and output device 24 in the electronic device can be connected through a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0097] The memory 22 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to a fundus examination image processing optimization method in an embodiment of the present application. The processor 21 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 22, that is, realizing the above-mentioned fundus examination image processing optimization method.

[0098] Any step of the method described above can be stored as a computer instruction or program in an unlimited computer memory, and can be called and recognized by an unlimited computer processor to implement any method in the embodiments of the present application, without any unnecessary restrictions.

[0099] Furthermore, the first or second mentioned above may not only represent an order relationship, but may also represent a specific concept, and / or refer to multiple elements that can be selected individually or in full. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these changes and variations.

[0100] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0102] This specification and drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A fundus examination image processing optimization method, characterized in that: The method comprises: Acquire multiple fundus examination images for iterative pixel gradient amplitude recognition to obtain an authentication result, wherein the multiple fundus examination images are images obtained by collecting fundus images of the same examination object at different angles, and the multiple fundus examination images have overlapping areas; When the authentication result is authentication passed, performing stitching and fusion trajectory identification on the overlapping areas of the multiple fundus examination images, determining a global stitching and fusion trajectory, and stitching and fusion the multiple fundus examination images according to the global stitching and fusion trajectory to obtain a target fundus examination image; Performing preliminary abnormal similarity matching on the target fundus examination image, obtaining multiple abnormal area bandwidths, and constructing multiple feature perception subunits of a multi-target feature recognition unit based on the multiple abnormal area bandwidths; Calling multiple feature sensing subunits of the multi-target feature recognition unit to perform abnormal feature recognition on the target fundus examination image, and summarizing the recognition results to obtain an abnormal feature recognition result; Wherein, when the authentication result is authentication passed, the overlapping areas of the multiple fundus examination images are identified with a stitching and fusion trajectory, a global stitching and fusion trajectory is determined, and the multiple fundus examination images are stitched and fused according to the global stitching and fusion trajectory to obtain a target fundus examination image, including: Perform overlapping area identification based on the multiple fundus examination images to obtain multiple overlapping areas, wherein each overlapping area is an overlapping area between two fundus examination images with the largest overlapping area; Using a structural similarity perception unit to identify structural deviation factors of the multiple overlapping regions to obtain multiple structural deviation coefficients; When the multiple structural deviation coefficients are less than or equal to the preset deviation coefficients, randomly selecting multiple first trajectory points from the multiple overlapping areas and obtaining multiple first trajectory neighborhoods of the multiple first trajectory points, wherein the multiple first trajectory neighborhoods include sixteen neighboring pixel points of the multiple first trajectory points in the multiple overlapping areas; Randomly generate splicing and fusion sub-trajectories based on the multiple first trajectory points and the multiple first trajectory neighborhoods respectively, to obtain multiple first splicing and fusion sub-trajectories; Using the plurality of first concatenated and fused sub-trajectories, segmenting the plurality of first trajectory neighborhoods to generate a plurality of first sub-trajectory left neighborhoods and a plurality of first sub-trajectory right neighborhoods; Calling a pixel gradient identifier to identify left and right side gradient differences of the plurality of first track points, the plurality of first sub-track left neighborhoods, and the plurality of first sub-track right neighborhoods, respectively, to obtain a plurality of first sub-track gradient differences; Again, randomly generating splicing and fusion sub-trajectories based on the plurality of first trajectory points and the plurality of first trajectory neighborhoods, respectively, to obtain a plurality of second splicing and fusion sub-trajectories, and obtaining a plurality of second sub-trajectory gradient differences of the plurality of second splicing and fusion sub-trajectories; When the number of randomly generated splicing fusion sub-trajectories meets the preset number, the plurality of first sub-trajectory gradient differences, the plurality of second sub-trajectory gradient differences and the plurality of splicing fusion sub-trajectories corresponding to the minimum value in the plurality of sub-trajectory gradient difference sets are used as the plurality of best splicing fusion sub-trajectories of the plurality of first trajectory points; Based on the multiple best spliced ​​fusion sub-trajectories of the multiple first trajectory points, global trajectory derivation is performed in the multiple overlapping areas to obtain multiple regional best spliced ​​fusion trajectories of the multiple overlapping areas; The optimal stitching and fusion tracks of the multiple regions are used as the global stitching and fusion track.

2. The method for optimizing fundus examination image processing according to claim 1, characterized in that: Acquire multiple fundus examination images for iterative pixel gradient amplitude recognition to obtain certification results, including: Randomly extracting a preset number of sample pixel points from the multiple fundus examination images to perform pixel gradient amplitude recognition, and obtaining multiple sample pixel gradient amplitude sets; Performing mean processing on the plurality of sample pixel gradient amplitude sets respectively to obtain a plurality of sample pixel gradient amplitude mean values; Taking the mean values ​​of the plurality of sample pixel gradient amplitudes as a plurality of iteration center points respectively, performing iterative search in the plurality of sample pixel gradient amplitude sets with a preset step size, determining a plurality of target iteration center points, and taking the plurality of sample pixel gradient amplitudes corresponding to the plurality of target iteration center points as a plurality of authentication pixel gradient amplitudes; When the multiple authentication pixel gradient amplitudes are greater than or equal to the preset authentication pixel gradient amplitude, authentication pass is taken as the authentication result.

3. A fundus examination image processing optimization method as claimed in claim 2, characterized in that: Taking the mean values ​​of the gradient amplitudes of the multiple sample pixels as multiple iteration center points respectively, performing iterative searches in the multiple sample pixel gradient amplitude sets with a preset step size to determine multiple target iteration center points, including: Taking the mean values ​​of the gradient amplitudes of the multiple sample pixels as multiple iteration center points respectively, performing iterative search in the multiple sample pixel gradient amplitude sets with a preset step size to obtain multiple stage iteration center points; When the iteration density of the multiple stage iteration center points is less than the iteration density of the multiple iteration center points, continue to use the multiple iteration center points as iteration starting points and perform iterative search in the multiple sample pixel gradient amplitude sets; When the iteration density of the multiple stage iteration center points is greater than or equal to the iteration density of the multiple iteration center points and the iteration density difference between the multiple stage iteration center points and the multiple iteration center points is less than or equal to the preset density difference, the iteration is stopped and the multiple stage iteration center points obtained in the last iteration are used as the multiple target iteration center points.

4. The method for optimizing fundus examination image processing according to claim 1, characterized in that: The pixel gradient identifier includes a pixel gradient identification formula, and the pixel gradient identification formula is: ； in, is the gradient difference of the first sub-trajectory, is the pixel value of the i-th neighboring pixel in the left neighborhood of the first sub-trajectory, is the pixel value of the jth neighboring pixel in the neighborhood on the right side of the first sub-trajectory, IQ is the pixel value of the first trajectory point, and l+p=16.

5. The method for optimizing fundus examination image processing according to claim 1, characterized in that: Based on the multiple best spliced ​​fusion sub-trajectories of the multiple first trajectory points, a global trajectory is derived in the multiple overlapping areas to obtain multiple regional best spliced ​​fusion trajectories of the multiple overlapping areas, including: Using the front and rear ends of the multiple best spliced ​​fusion sub-trajectories as derivative trajectory points to obtain multiple front-end derivative trajectory points and multiple back-end derivative trajectory points; Acquire a plurality of front-end derived trajectory neighborhoods based on the plurality of front-end derived trajectory points, wherein the plurality of front-end derived trajectory neighborhoods include sixteen neighboring pixel points located in front of the plurality of front-end derived trajectory points; Acquire a plurality of back-end derived trajectory neighborhoods based on the plurality of back-end derived trajectory points, wherein the plurality of back-end derived trajectory neighborhoods include sixteen neighboring pixel points located behind the plurality of back-end derived trajectory points; Based on the multiple front-end derived trajectory points, the multiple back-end derived trajectory points, the multiple front-end derived trajectory neighborhoods and the multiple back-end derived trajectory neighborhoods, global trajectory derivation of the multiple optimal splicing and fusion sub-trajectories is performed to obtain multiple regional optimal splicing and fusion trajectories of the multiple overlapping areas.

6. The method for optimizing fundus examination image processing according to claim 1, characterized in that: include: Using the target fundus examination image as an index, performing similar examination image matching in an image database to obtain a plurality of similar fundus examination images, wherein the plurality of similar fundus examination images include a plurality of similar fundus examination abnormal region area sets; According to the size of the abnormal area, clustering the multiple similar fundus examination abnormal area sets across images to obtain multiple clustered similar fundus examination abnormal area sets; Calculating the mean of each of the plurality of clustered similar fundus examination abnormal area sets to obtain the mean of the plurality of clustered abnormal area areas; The area means of the multiple clustered abnormal regions are used as the multiple abnormal region bandwidths, and the receptive fields of the multiple feature perception sub-units are configured according to the multiple abnormal region bandwidths to obtain the multiple feature perception sub-units that have been configured.

7. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; Wherein, the processor is used to execute the fundus examination image processing optimization method described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, so that the computer program is used to execute the fundus examination image processing optimization method described in any one of claims 1 to 6.

Images