Macular pigment density calculation method and device, electronic equipment and storage medium

Abstract

The invention relates to a macular pigment density calculation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-processed fundus image and historical fundus images, the historical fundus images being at least one fundus image of a shooting object of the to-be-processed fundus image in a past preset time period; determining an illumination component of the fundus image to be processed and an illumination component of the historical fundus image, and determining an average value of the illumination component of the historical fundus image to obtain a reference illumination component; correcting the illumination component of the eye fundus image to be processed based on the reference illumination component to obtain a corrected image; and according to the corrected image, determining a target macular pigment density corresponding to the eye fundus image to be processed. The problems of unstable quality and inconsistent brightness or chromaticity of the fundus image caused by different shooting conditions can be avoided, so that the corrected image can more truly reflect the current eye lesion condition of the shot object, and the real macular pigment density level of a photographer can be more reliably evaluated.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and particularly relates to a macular pigment density calculation method and device, an electronic device and a storage medium. BACKGROUND

[0002] The macula is a very important area on the retina, and the macular pigment contained therein plays a key role in maintaining normal visual function, such as protecting the retina from light damage, reducing oxidative stress, etc. Determining the density of macular pigment helps to understand the health status of the macula.

[0003] In the prior art, the density of macular pigment can be measured by photographing a fundus image. First, a fundus image is obtained, then the macular region and the foveal region are identified in the fundus image, and then the color values of the two regions are extracted, and the density of macular pigment is calculated according to the color values.

[0004] However, this method is more dependent on the shooting quality of the fundus image. Since the shooting process is affected by factors such as light and the degree of cooperation of the photographer, the fundus image obtained by shooting will show differences in brightness or chroma, and cannot accurately reflect the true macular pigment density level of the photographer, thereby leading to low accuracy and large fluctuations in the calculated macular pigment density. SUMMARY

[0005] The present disclosure provides a macular pigment density calculation system, method, device, electronic device and storage medium to at least solve the problem that the fundus image cannot accurately reflect the true macular pigment density level of the photographer in the related art, thereby leading to low accuracy and large fluctuations in the calculated macular pigment density. The technical solutions of the present disclosure are as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, a macular pigment density calculation method is provided, comprising:

[0007] obtaining a to-be-processed fundus image and a historical fundus image, the historical fundus image being at least one fundus image of a subject of the to-be-processed fundus image in a past preset time period;

[0008] determining the light component of the to-be-processed fundus image and the light component of the historical fundus image, and determining the average value of the light component of the historical fundus image to obtain a reference light component;

[0009] correcting the light component of the to-be-processed fundus image based on the reference light component to obtain a corrected image;

[0010] determining the target macular pigment density corresponding to the to-be-processed fundus image according to the corrected image.

[0011] Optionally, the obtaining the to-be-processed fundus image and the historical fundus image comprises:

[0012] obtaining the to-be-processed fundus image;

[0013] extracting image features of the to-be-processed fundus image;

[0014] performing feature matching based on the image features to obtain the historical fundus image.

[0015] Optionally, the correcting the illumination component of the to-be-processed fundus image based on the reference illumination component to obtain a corrected image comprises:

[0016] converting the illumination component of the to-be-processed fundus image and the reference illumination component into frequency domains respectively to obtain a first frequency spectrum and a second frequency spectrum;

[0017] correcting the frequency distribution of the first frequency spectrum based on the second frequency spectrum to obtain a correction map;

[0018] converting the correction map into a spatial domain to obtain the corrected image.

[0019] Optionally, before the determining the illumination component of the to-be-processed fundus image and the illumination component of the historical fundus image, the method further comprises:

[0020] performing adaptive threshold segmentation on the to-be-processed fundus image to detect whether a shadow area exists in the to-be-processed fundus image;

[0021] if the shadow area exists, performing the steps of determining the illumination component of the to-be-processed fundus image and the illumination component of the historical fundus image.

[0022] Optionally, the determining the target macular pigment density corresponding to the to-be-processed fundus image according to the corrected image comprises:

[0023] performing macular detection on the corrected image to determine an initial macular pigment density;

[0024] obtaining a historical macular pigment density corresponding to the historical fundus image, and performing smoothing processing on the initial macular pigment density and the historical macular pigment density to obtain the target macular pigment density.

[0025] Optionally, the performing smoothing processing on the initial macular pigment density and the historical macular pigment density to obtain the target macular pigment density comprises:

[0026] obtaining a time interval between the historical fundus image and the to-be-processed fundus image, and determining a weight of each historical fundus image based on the time interval, the weight being negatively correlated with the time interval;

[0027] Based on the weight, the initial macular pigment density and the historical macular pigment density are weighted and smoothed to obtain a target macular pigment density.

[0028] Optionally, after the target macular pigment density corresponding to the to-be-processed fundus image is determined according to the corrected image, the method further includes:

[0029] A historical macular pigment density corresponding to a reference fundus image is obtained as a reference macular pigment density, the reference fundus image being a historical fundus image with the smallest time interval from the to-be-processed fundus image;

[0030] Based on the reference macular pigment density, the target macular pigment density is calibrated to obtain a calibrated macular pigment density.

[0031] Optionally, the calibration of the target macular pigment density based on the reference macular pigment density to obtain a calibrated macular pigment density includes:

[0032] Based on a time interval between the reference fundus image and the to-be-processed fundus image, a fluctuation range is determined.

[0033] The reference macular pigment density is added to or subtracted from the fluctuation range respectively to obtain a maximum value and a minimum value.

[0034] The target macular pigment density is calibrated according to the maximum value and the minimum value to obtain a calibrated macular pigment density, the calibrated macular pigment density having a value between the maximum value and the minimum value.

[0035] According to a second aspect of the embodiments of the present disclosure, a macular pigment density calculation device is provided, including:

[0036] An acquisition module is configured to acquire a to-be-processed fundus image and historical fundus images, the historical fundus images being at least one fundus image of a subject of the to-be-processed fundus image in a past preset time period;

[0037] An analysis module is configured to determine an illumination component of the to-be-processed fundus image and an illumination component of the historical fundus images, and determine an average value of the illumination component of the historical fundus images to obtain a reference illumination component;

[0038] A correction module is configured to correct the illumination component of the to-be-processed fundus image based on the reference illumination component to obtain a corrected image;

[0039] A determination module is configured to determine a target macular pigment density corresponding to the to-be-processed fundus image according to the corrected image.

[0040] According to a third aspect of the embodiments of the present disclosure, a macular pigment density calculation electronic device is provided, comprising:

[0041] a processor;

[0042] a memory for storing processor-executable instructions;

[0043] wherein the processor is configured to execute the instructions to implement the macular pigment density calculation method of any one of the embodiments.

[0044] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, when instructions in the computer-readable storage medium are executed by a processor of a macular pigment density calculation electronic device, the macular pigment density calculation electronic device is enabled to perform the macular pigment density calculation method of any one of the embodiments.

[0045] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, comprising computer programs / instructions, which, when executed by a processor, implement the macular pigment density calculation method of any one of the embodiments.

[0046] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects:

[0047] By acquiring the to-be-processed fundus image and the historical fundus image, and using the historical fundus image of the same photographed object to remove and correct the to-be-processed fundus image, the influence of external factors can be significantly reduced or eliminated, the problem of unstable fundus image quality and inconsistent brightness or chroma caused by differences in shooting conditions can be avoided, and the corrected image can more truly and accurately reflect the current eye lesion condition of the photographed object. Therefore, the macular pigment density is calculated based on the corrected image, which can significantly improve the accuracy and stability of the calculation result, thereby more reliably evaluating the real macular pigment density level of the photographer.

[0048] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0049] The accompanying drawings incorporated in the specification and forming a part of it, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure, and do not limit the present disclosure.

[0050] Figure 1 is a flowchart of a macular pigment density calculation method according to an exemplary embodiment.

[0051] Figure 2 is a specific flowchart of a macular pigment density calculation method according to an exemplary embodiment.

[0052] Figure 3 is a block diagram of a macular pigment density calculation device according to an exemplary embodiment.

[0053] Figure 4 is a block diagram of an electronic device for macular pigment density calculation according to an exemplary embodiment.

[0054] Figure 5 is a block diagram of a device for macular pigment density calculation according to an exemplary embodiment. DETAILED DESCRIPTION

[0055] In order for those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings.

[0056] It should be noted that the terms "first", "second", and the like in the specification and claims of the present disclosure and the accompanying drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0057] In the related art, the macular pigment density can be measured by shooting a fundus image. First, a fundus image is obtained, then the macular region and the foveal region are identified in the fundus image, then the color values of the two regions are extracted, and the macular pigment density is calculated according to the color values. However, this method is more dependent on the shooting quality of the fundus image. Since the shooting process can be affected by factors such as lighting and the degree of cooperation of the shooter, the fundus image obtained by shooting can reflect differences in brightness or chroma, and cannot accurately reflect the true macular pigment density level of the shooter, thereby resulting in low accuracy and large fluctuations in the calculated macular pigment density. Based on this, the macular pigment density calculation method in the present application is proposed to solve the above problems.

[0058] Figure 1 is a flowchart of a macular pigment density calculation method according to an exemplary embodiment, as Figure 1 shown, the macular pigment density calculation method comprises:

[0059] In step S11, a fundus image to be processed and a historical fundus image are obtained. The historical fundus image is at least one fundus image of the shooting object of the fundus image to be processed within a predetermined time period in the past.

[0060] Specifically, the to-be-processed fundus image is an image currently in need of macular pigment density calculation. The to-be-processed fundus image can be obtained by photographing the fundus of a subject by using a professional fundus photographing device, such as a fundus camera.

[0061] The historical fundus image refers to at least one fundus image obtained by photographing the fundus of the same subject by using the same or similar fundus photographing device within a pre-set time period in the past, which can be used as an important reference for subsequent processing. The pre-set time period can be flexibly set according to actual needs, for example, one month, three months, half a year, etc. The historical fundus image can be obtained from a pre-set database, or from a cloud server or a network, and the specifics are not limited.

[0062] In step S12, the illumination component of the to-be-processed fundus image and the illumination component of the historical fundus image are determined, and the average value of the illumination component of the historical fundus image is determined to obtain a reference illumination component.

[0063] Since the fundus image is easily disturbed by environmental light, shooting angle and other factors during the photographing process, resulting in shadows, and the illumination component is closely related to the shadows, therefore, the illumination component of the to-be-processed fundus image and the historical fundus image can be separated first. For example, the to-be-processed fundus image and the historical fundus image can be converted to the logarithmic domain respectively, and specific image processing techniques such as Retinex theory and homomorphic filtering algorithm can be used to decompose the images into two parts, i.e., the illumination component and the reflection component, so as to obtain the illumination component corresponding to the to-be-processed fundus image and each historical fundus image.

[0064] Then, after obtaining the illumination components of the historical fundus images, the illumination components can be integrated and analyzed to determine the average value of the illumination components of the historical fundus images to obtain a reference illumination component. The reference illumination component represents the average state of the illumination of the fundus image of the subject within the pre-set time period in the past, and can reflect a relatively stable and ideal illumination distribution. For example, the gray values of the corresponding pixel positions in the illumination component images of each historical fundus image can be summed and divided by the number of historical fundus images, thereby calculating the average gray value of each pixel position, i.e., the reference illumination component.

[0065] In step S13, the illumination component of the to-be-processed fundus image is corrected based on the reference illumination component to obtain a corrected image.

[0066] In this step, the illumination component of the to-be-processed fundus image can be corrected based on the reference illumination component to obtain a corrected image. The correction process can include adjusting the brightness, contrast, chrominance, etc. of the to-be-processed fundus image to eliminate or reduce the illumination difference caused by changes in the shooting conditions, so that the illumination conditions of the to-be-processed fundus image are as close as possible to the standard state represented by the reference illumination component.

[0067] After the illumination correction, a new fundus image, i.e. a corrected image, is obtained. The corrected image is more uniform and stable in illumination conditions, effectively solving the problem of unstable fundus image quality caused by changes in shooting conditions, and providing a more reliable basis for subsequent macular pigment density calculation.

[0068] In step S14, the target macular pigment density corresponding to the to-be-processed fundus image is determined according to the corrected image.

[0069] After obtaining the corrected image, the macular region and the foveal region can be accurately identified by the trained target detection network, and then the pixel values of the two regions are extracted. The corrected image has eliminated the interference factors such as illumination, so that the extracted pixel values can more truly reflect the characteristics of the macular pigment. Further, based on these pixel values, a specific algorithm can be used for calculation to obtain the target macular pigment density which can accurately reflect the real macular pigment density level of the shooting object.

[0070] In an implementation manner, the to-be-processed fundus image and the historical fundus image are obtained in step S11, including:

[0071] The to-be-processed fundus image is obtained.

[0072] The image features of the to-be-processed fundus image are extracted.

[0073] Feature matching is performed based on the image features to obtain the historical fundus image.

[0074] In this implementation manner, first, the to-be-processed fundus image is obtained, for example, by a professional fundus imaging device such as a fundus camera, etc. to shoot the fundus of a patient, thereby obtaining the to-be-processed fundus image which needs to be analyzed and processed at present.

[0075] Then, feature extraction can be performed on the to-be-processed fundus image, so that a historical fundus image that matches the to-be-processed fundus image can be accurately found. For example, the to-be-processed fundus image can be input into a feature extraction network trained by using a deep learning network. The feature extraction network is trained by using a large amount of data, and can learn representative feature information in the to-be-processed fundus image. Through processing of the feature extraction network, a 1024-dimensional feature vector can be extracted from the to-be-processed fundus image. The feature vector contains information such as texture, structure, and color distribution of the to-be-processed fundus image, and can uniquely identify the features of the to-be-processed image.

[0076] Further, feature matching can be performed based on the extracted image features to obtain a historical fundus image. Specifically, the extracted 1024-dimensional feature vector can be searched and matched in a database. The database stores a large number of historical fundus images and corresponding feature vectors. By calculating the similarity between the feature vector of the to-be-processed fundus image and the feature vectors of the historical fundus images in the database, a historical fundus image with high similarity can be found. These matched historical fundus images are the fundus images of the photographed object in the past predetermined time period.

[0077] The image features of the fundus image are unique biological features of each person, and have high individual specificity. Through this feature matching-based manner, a historical fundus image related to the current to-be-processed image can be accurately found from a large number of historical fundus images, recognition errors can be reduced, and the accuracy and reliability of re-identification can be improved. This provides a reliable reference basis for subsequent shadow removal and macular pigment density calculation.

[0078] In an implementation manner, in step S13, the illumination component of the to-be-processed fundus image is corrected based on the reference illumination component to obtain a corrected image, including:

[0079] The illumination component of the to-be-processed fundus image and the reference illumination component are respectively converted to a frequency domain to obtain a first frequency spectrum and a second frequency spectrum;

[0080] The frequency distribution of the first frequency spectrum is corrected based on the second frequency spectrum to obtain a correction map;

[0081] The correction map is converted to a spatial domain to obtain the corrected image.

[0082] Since it can be difficult to accurately separate and correct the illumination change by directly processing the illumination component in the spatial domain, in this implementation manner, first, Fourier transform or other technologies can be used to convert the illumination component of the to-be-processed fundus image and the reference illumination component to the frequency domain. The illumination component of the to-be-processed fundus image is converted into a first frequency spectrum, and the reference illumination component is converted into a second frequency spectrum. The second frequency spectrum of the reference illumination component represents an ideal and uniform illumination frequency distribution state.

[0083] Then, by comparing the first spectrum and the second spectrum, the difference between the light component of the to-be-processed fundus image and the reference light component in the frequency distribution is analyzed in the frequency domain. For example, if the energy of some low-frequency components in the first spectrum is too high, it indicates that the light in the corresponding area is too strong; if some frequency components are missing or the energy is too low, it indicates that the light in the area is insufficient. In view of these differences, by designing a specific frequency domain filter, the components deviating from the reference frequency distribution in the first spectrum are adjusted, such as attenuating the frequency components with too high energy, and enhancing the frequency components with missing or too low energy, so that the frequency distribution of the first spectrum is closer to the second spectrum, and a corrected image is obtained.

[0084] Further, the inverse Fourier transform can be used to convert the corrected image in the frequency domain back to the spatial domain to obtain a corrected image. The corrected light component has eliminated the difference with the reference light component, and presents a more uniform and reasonable light distribution, so that the corrected light component is combined with the reflection component of the original to-be-processed fundus image, and a corrected image with removed shadows and uniform light is obtained. The corrected image can more truly reflect the actual situation of the fundus, effectively avoiding the interference of uneven light on the subsequent macular pigment density calculation, and providing a reliable data basis for accurate assessment of the macular health status.

[0085] In an implementation manner, before the step of determining the light component of the to-be-processed fundus image and the light component of the historical fundus image in step S12, the method further includes:

[0086] performing adaptive threshold segmentation on the to-be-processed fundus image to detect whether there is a shadow area in the to-be-processed fundus image;

[0087] if there is, performing the step of determining the light component of the to-be-processed fundus image and the light component of the historical fundus image.

[0088] Since the fundus image is affected by factors such as light conditions and shooting angles during shooting, local shadow areas may appear, and these shadows will interfere with the accurate calculation of the subsequent macular pigment density. Therefore, before determining the light component of the to-be-processed fundus image and the light component of the historical fundus image, adaptive threshold segmentation can be performed on the to-be-processed fundus image.

[0089] Specifically, for each pixel point in the to-be-processed fundus image, the adaptive threshold segmentation method can calculate a local threshold according to the pixel features in the neighborhood of the pixel point. For example, the mean adaptive threshold method calculates the mean gray value of the pixels in a small neighborhood centered on the current pixel point as the local threshold of the pixel point; the Gaussian adaptive threshold method calculates the weighted gray mean value of the pixels in the neighborhood using Gaussian weighted averaging as the local threshold; and the entropy-based adaptive threshold method determines the local threshold using the information entropy of the image, measures the degree of disorder of the gray distribution in the neighborhood by calculating the entropy value of the image, and then determines a suitable threshold according to the entropy value.

[0090] According to the calculated local threshold, each pixel point in the to-be-processed fundus image can be compared with the corresponding local threshold. If the gray value of the pixel point is less than its local threshold, the pixel point is determined to be a pixel in the shadow region; if the gray value of the pixel point is greater than or equal to its local threshold, the pixel point is determined to be a pixel in the non-shadow region. In this way, by judging and marking each pixel point, the to-be-processed fundus image can be segmented into two parts, the shadow region and the non-shadow region.

[0091] Further, the subsequent operation can be determined according to the detection result. If it is detected that there is a shadow region in the to-be-processed fundus image, the step of determining the illumination component of the to-be-processed fundus image and the illumination component of the historical fundus image is performed; if the detection result shows that there is no shadow region, it indicates that the illumination distribution of the to-be-processed fundus image is relatively uniform, and no shadow removal operation is needed, and the process of determining the macular pigment density according to the to-be-processed fundus image can be directly entered.

[0092] Through this detection mechanism, the to-be-processed fundus image that needs to be processed by shadow removal can be accurately identified, unnecessary waste of computing resources is avoided while ensuring the processing effect, and the overall efficiency and accuracy of macular pigment density calculation are improved.

[0093] In an implementation manner, in step S14, the target macular pigment density corresponding to the to-be-processed fundus image is determined according to the corrected image, including:

[0094] Performing macular detection on the corrected image to determine an initial macular pigment density;

[0095] Obtaining a historical macular pigment density corresponding to the historical fundus image, and performing smoothing processing on the initial macular pigment density and the historical macular pigment density to obtain the target macular pigment density.

[0096] In this implementation, first, the macula of the corrected image can be detected using computer vision and image processing techniques to determine the initial macular pigment density. Specifically, the macular region and foveal region can be accurately located in the corrected image through a trained deep learning model (such as a convolutional neural network) or a traditional image processing algorithm (such as threshold segmentation, region growing, edge detection, etc.). After positioning, the pixel value information of the two regions is extracted, and based on the absorption characteristics of macular pigment to specific wavelengths of light, combined with a specific conversion formula, the initial macular pigment density is calculated.

[0097] Then, the historical macular pigment density corresponding to the historical fundus image can be obtained. The historical macular pigment density corresponding to the historical fundus image can be obtained by macular detection of the historical fundus image, or directly obtained from a certain database or server, without limitation.

[0098] Further, considering that a single measurement may have errors, and the change of macular pigment density over a period of time has a certain continuity, the initial macular pigment density and the historical macular pigment density can be smoothed to obtain the target macular pigment density. The specific smoothing method can include:

[0099] Weighted average, the initial macular pigment density and the historical macular pigment density are weighted and averaged, and the weight can be set according to the time distance, confidence or other factors. For example, the most recent initial MPOD can be given a higher weight, and the historical value can be given a lower weight.

[0100] Moving average / exponential smoothing, if there are multiple historical macular pigment densities, the initial macular pigment density can be considered as the latest data point, and then the moving average containing each data point is calculated, or the exponential smoothing method is used to give the new data point a greater impact.

[0101] Trend fitting and correction, analyze the trend of the historical macular pigment density, and fit the initial macular pigment density into this trend for correction.

[0102] The target macular pigment density obtained after smoothing is the macular pigment density corresponding to the fundus image to be processed, which is calibrated and stabilized by historical data. It combines the direct measurement results of the fundus image to be processed and the historical macular pigment density of the photographed object, so it is usually more stable and reliable than the initial macular pigment density, and can better reflect the true and smoothed macular pigment density state.

[0103] In one implementation, the initial macular pigment density and the historical macular pigment density are smoothed to obtain the target macular pigment density, including:

[0104] an interval between the historical fundus image and the fundus image to be processed is obtained, and a weight of each historical fundus image is determined based on the interval, the weight being negatively correlated with the interval;

[0105] Based on the weight, the initial macular pigment density and the historical macular pigment density are weighted and smoothed to obtain a target macular pigment density.

[0106] That is, first, an interval between the historical fundus image and the fundus image to be processed can be obtained, wherein the interval can be obtained by calculating the difference between the shooting time of the historical fundus image and the shooting time of the fundus image to be processed, or can be directly obtained from a certain database or server, and the specific limitation is not made.

[0107] Then, the weight of each historical fundus image can be determined based on the interval, since the recent fundus image can better reflect the current macular pigment density condition of the shooting object, and the fundus image with longer interval is less valuable for the current situation due to physiological changes, measurement errors and other factors, therefore the weight is designed to be negatively correlated with the interval.

[0108] For example, an exponential decay function can be used to quantify the relationship between the weight and the interval, that is, the longer the shooting time of the historical fundus image is from the shooting time of the current fundus image, the smaller the influence of the historical fundus image on the current result is. The function is expressed as the following formula:

[0109]

[0110] wherein W i attenuation represents the weight of the historical fundus image, reflecting the referenceability of the historical fundus image to the current prediction. e is a natural constant, about 2.71828, and the nature of the exponential function determines that W i attenuation will be exponentially attenuated with the increase of the interval Δt. θ is a constant, which determines the speed of exponential decay, that is, controls the speed of change of the weight of the historical fundus image with time. By adjusting the value of θ, the influence of the historical fundus image on the current result can be flexibly adjusted according to the actual situation. When θ is large, the weight of the historical fundus image attenuates slowly with time, indicating that more attention is paid to the influence of the historical fundus image shot earlier on the current result; on the contrary, when θ is small, the weight of the historical fundus image attenuates quickly, indicating that more attention is paid to the influence of the historical fundus image shot recently on the current result.

[0111] Further, after obtaining the weight corresponding to each historical fundus image, the historical macular pigment density of each historical fundus image can be multiplied by the corresponding weight and summed up by using a weighted average method, and then the initial macular pigment density multiplied by the corresponding weight is added to obtain the target macular pigment density. The corresponding weight of the initial macular pigment density can be that the initial macular pigment density is regarded as new data, the weight is determined according to the time interval calculation method, or a specific weight can also be assigned to it, and the specific weight is not limited.

[0112] For example, the initial macular pigment density and the historical macular pigment density are weighted and smoothed based on the weight to obtain the target macular pigment density, which can be expressed as the following formula:

[0113]

[0114] wherein R s represents the target macular pigment density after weighted smoothing, R i represents the macular pigment density of the i th fundus image (including historical fundus images and fundus images to be processed), represents the macular pigment density of each fundus image R i is multiplied by the corresponding weight and then summed up. represents the sum of all weights, which is used to normalize the sum of the molecules.

[0115] After such weighted smoothing, the current measurement result and the historical trend are comprehensively considered, and different weights are given to the macular pigment densities of different historical periods according to the different time intervals from the current shooting time. The historical macular pigment density with a shorter time interval has a relatively higher weight, and has a greater influence on the final result, thereby effectively reducing the influence of single measurement error, and the target macular pigment density obtained can more accurately reflect the current real macular pigment density of the photographer, and provide more reliable data support for clinical diagnosis and health assessment.

[0116] In one implementation, after determining the target macular pigment density corresponding to the fundus image to be processed according to the corrected image in step S14, the method further comprises:

[0117] obtaining the historical macular pigment density corresponding to the reference fundus image as the reference macular pigment density, the reference fundus image being the historical fundus image with the smallest time interval from the fundus image to be processed;

[0118] calibrating the target macular pigment density based on the reference macular pigment density to obtain a calibrated macular pigment density.

[0119] That is, after determining the target macular pigment density corresponding to the to-be-processed fundus image according to the corrected image, calibration can be further performed based on the reference fundus image. First, from the historical fundus image data, the historical fundus image with the smallest time interval from the to-be-processed fundus image is screened out, which is defined as the reference fundus image, and the historical macular pigment density corresponding to the reference fundus image is obtained as the reference macular pigment density.

[0120] Then, the target macular pigment density can be calibrated based on the reference macular pigment density to obtain the calibrated macular pigment density. Specifically, the target macular pigment density can be compared and analyzed with the reference macular pigment density by establishing a suitable calibration model or rule. For example, a reasonable fluctuation range function based on the time interval is set, and according to the reference macular pigment density and the fluctuation range, the calibrated macular pigment density is limited within the fluctuation range to avoid abnormal fluctuations caused by measurement errors. Or, if the difference between the reference macular pigment density and the target macular pigment density is small and linear, a simple linear adjustment method can be used, that is, the target macular pigment density is corrected according to the proportion or fixed value of the difference. In addition, if the difference is large or nonlinear, more complex nonlinear adjustment methods such as curve fitting, machine learning models, etc. may be needed. The specific implementation is not limited.

[0121] Since the reference fundus image is taken closest to the to-be-processed fundus image, it has more similarity in physiological state and measurement environment, and its corresponding historical macular pigment density can also reflect the approximate value of the current state. Therefore, considering the possible errors in the measurement process and the limited changes of macular pigment density in a short period of time, the reference macular pigment density can be extracted as a key reference for subsequent calibration.

[0122] In an implementation manner, the target macular pigment density is calibrated based on the reference macular pigment density to obtain the calibrated macular pigment density, including:

[0123] Determine the fluctuation range based on the time interval between the reference fundus image and the to-be-processed fundus image;

[0124] Add or subtract the reference macular pigment density from the fluctuation range to obtain the maximum value and the minimum value;

[0125] Calibrate the target macular pigment density according to the maximum value and the minimum value to obtain the calibrated macular pigment density, and the value of the calibrated macular pigment density is between the maximum value and the minimum value.

[0126] That is, the target macular pigment density can be calibrated in combination with the time interval factor. First, the fluctuation range can be determined based on the time interval between the reference fundus image and the fundus image to be processed. For example, the fluctuation range can be determined using a pre-set function according to the time interval between the reference fundus image and the fundus image to be processed, which can be based on a large amount of clinical data and research analysis, for example, by fitting a mathematical model of the time interval and the macular pigment density fluctuation through regression analysis, so that the determination of the fluctuation range is more scientific and accurate.

[0127] Then, the reference macular pigment density can be added or subtracted from the fluctuation range respectively to obtain the maximum value and the minimum value, which together define a reasonable value interval of the target macular pigment density under the current circumstances, which takes into account the general rules and recent state of macular pigment density changes, providing clear boundary conditions for subsequent calibration.

[0128] Further, the target macular pigment density can be calibrated according to the maximum value and the minimum value to obtain a calibrated macular pigment density, the value of which is between the maximum value and the minimum value. Through such a calibration process, unreasonable extreme values of the target macular pigment density caused by measurement errors, individual differences and other factors are effectively avoided, so that the final calibrated macular pigment density more truly and accurately reflects the current macular pigment density of the photographer, providing more reliable data support for medical diagnosis and health assessment.

[0129] For example, the process of this implementation can be implemented using the following formula:

[0130] R f = clip(R s ,Rf t-1 +f(Δ t ),Rf t-1 -f(Δ t ))

[0131] Where R f represents the calibrated macular pigment density, R s is the target macular pigment density, Rf t-1 represents the reference macular pigment density, and f(Δ t ) represents the fluctuation range of the macular pigment density determined according to the time interval Δt between the reference fundus image and the fundus image to be processed, for example, if the time interval is long, the allowed fluctuation range will be relatively large; if the time interval is short, the fluctuation range will be relatively small. The clip function is used to limit R s to a specific range, which is determined by Rf t-1 +f(Δ t ) and Rf t-1-f (Δ t ) is determined, that is, the maximum value and the minimum value.

[0132] Figure 2 is a specific flowchart of a macular pigment density calculation method according to an exemplary embodiment. The embodiment includes the following steps:

[0133] After obtaining the to-be-processed fundus image of the shooting object, the to-be-processed fundus image is taken as an input image and sent to a feature extraction network trained by a deep learning network to extract 1024-dimensional image features of the to-be-processed fundus image and perform query matching of the image features to match the historical fundus image shot by the person.

[0134] Then, the shadow area of the to-be-processed fundus image is detected by an adaptive threshold segmentation method. When the to-be-processed fundus image has a shadow area, the historical fundus images are combined to perform illumination normalization processing on the to-be-processed fundus image. The illumination components of the to-be-processed fundus image and the historical fundus images are determined, and then the average value of the illumination components of the historical fundus images is determined to obtain a reference illumination component. Then, the illumination component of the to-be-processed fundus image is corrected based on the reference illumination component to obtain a corrected image. The initial macular pigment density is calculated by using the corrected image.

[0135] Further, the historical macular pigment density of the shooting object and the time interval of shooting are queried, and the initial macular pigment density calculated at present and the historical macular pigment density of the person are weighted and smoothed to obtain a target macular pigment density. When weighting, the weighting weight is determined based on the time interval.

[0136] To ensure that the difference between the macular pigment density in the result of this examination of the shooting object and the macular pigment density in the result of the last examination is reasonable within a certain time range, the historical macular pigment density of the shooting object at the last time can be taken as a reference macular pigment density, and the time interval of the reference macular pigment density can be taken as a reference factor to calibrate the target macular pigment density to obtain a calibrated macular pigment density.

[0137] As can be seen from the above, the technical scheme provided by the embodiment of the disclosure can obtain the to-be-processed fundus image and the historical fundus image, and use the historical fundus image of the same shooting object to remove and correct the shadow of the to-be-processed fundus image, which can significantly reduce or eliminate the influence of external factors, avoid the problem of unstable fundus image quality, inconsistent brightness or chroma caused by differences in shooting conditions, and make the corrected image more truly and accurately reflect the current eye disease of the shooting object. Therefore, based on the corrected image, the macular pigment density can be calculated, which can significantly improve the accuracy and stability of the calculation result, so as to more reliably evaluate the real macular pigment density level of the shooter.

[0138] Figure 3 is a yellow macula pigment density calculation device block diagram according to an exemplary embodiment, comprising:

[0139] The acquisition module 201 is configured to acquire a to-be-processed fundus image and a historical fundus image, the historical fundus image being at least one fundus image of a photographing object of the to-be-processed fundus image in a past preset time period;

[0140] The analysis module 202 is configured to determine an illumination component of the to-be-processed fundus image and an illumination component of the historical fundus image, and determine an average value of the illumination component of the historical fundus image to obtain a reference illumination component;

[0141] The correction module 203 is configured to correct the illumination component of the to-be-processed fundus image based on the reference illumination component to obtain a corrected image;

[0142] The determination module 204 is configured to determine a target yellow macula pigment density corresponding to the to-be-processed fundus image according to the corrected image.

[0143] As can be seen from the above, the technical scheme provided by the embodiment of the present disclosure can significantly reduce or eliminate the influence of external factors by acquiring a to-be-processed fundus image and a historical fundus image, and using the historical fundus image of the same photographing object to remove and correct the to-be-processed fundus image, thereby avoiding the problem of unstable fundus image quality, inconsistent brightness or chroma caused by differences in shooting conditions, so that the corrected image can more truly and accurately reflect the current eye lesion condition of the photographing object. Therefore, based on the corrected image, the yellow macula pigment density can be calculated, which can significantly improve the accuracy and stability of the calculation result, thereby more reliably evaluating the real yellow macula pigment density level of the photographer.

[0144] Figure 4 is a block diagram of an electronic device for yellow macula pigment density calculation according to an exemplary embodiment.

[0145] In an exemplary embodiment, a computer readable storage medium including instructions, for example, a memory including instructions, is also provided, which can be executed by a processor of an electronic device to complete the method. Optionally, the computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0146] In an exemplary embodiment, a computer program product is also provided, which, when running on a computer, causes the computer to implement the method of yellow macula pigment density calculation.

[0147] As can be seen from the above, the technical scheme provided by the embodiments of the disclosure can significantly reduce or eliminate the influence of external factors by acquiring the to-be-processed fundus image and the historical fundus image of the same photographed object, and using the historical fundus image of the same photographed object to remove and correct the to-be-processed fundus image, thereby avoiding the problems of unstable fundus image quality and inconsistent brightness or chroma caused by differences in shooting conditions, so that the corrected image can more truly and accurately reflect the current eye lesion condition of the photographed object. Therefore, based on the corrected image, the macular pigment density can be calculated, which can significantly improve the accuracy and stability of the calculation result, thereby more reliably evaluating the real macular pigment density level of the photographer.

[0148] Figure 5 is a block diagram of an apparatus 800 for macular pigment density calculation according to an exemplary embodiment.

[0149] For example, the apparatus 800 can be a mobile phone, a computer, a digital broadcast electronic device, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0150] Referring to Figure 5 The apparatus 800 can include one or more of the following components: a processing component 802, a memory component 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0151] The processing component 802 generally controls the overall operation of the apparatus 800 such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to complete all or part of steps of the methods described. In addition, the processing component 802 can include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0152] The memory component 804 is configured to store various types of data to support operations of the apparatus 800. Examples of these data include instructions for any application or method operating on the apparatus 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory component 804 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0153] Power component 807 provides power to the various components of the device 800. The power component 807 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0154] The multimedia component 808 includes a screen providing an output interface between the device 800 and the account. In some embodiments, the screen includes a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, swiping, and gestures on the touch panel. The touch sensors can not only sense a boundary of a touching or swiping action, but also detect duration and pressure related to the touching or swiping action. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0155] The audio component 810 is configured to output and / or input an audio signal. For example, the audio component 810 includes a microphone (MIC) that is configured to receive an external audio signal when the device 800 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting an audio signal.

[0156] The I / O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which can be a keyboard, a click wheel, a button, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0157] The sensor component 814 includes one or more sensors for providing status assessments for various aspects of the device 800. For example, the sensor component 814 can detect an open / closed position of the device 800, relative positioning of components, such as a display and keypad of the device 800, a change in position of the device 800 or a component of the device 800, presence or absence of an account with the device 800, orientation or acceleration / deceleration of the device 800, and temperature changes of the device 800. The sensor component 814 can include a proximity sensor configured to detect the presence of objects in proximity thereto without any physical contact. The sensor component 814 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0158] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and another device. The device 800 can access a wireless network based on a communication standard, such as WiFi, a cellular network (e.g., 2G, 3G, 4G or 5G), or a combination thereof. In an example embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 816 can further include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, infrared data association (IrDA) techniques, ultra-wideband (UWB) techniques, Bluetooth (BT) techniques, and other techniques.

[0159] In an example embodiment, the device 800 can be implemented with one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements to perform the methods of the first and second aspects.

[0160] In an example embodiment, a non-transitory computer-readable storage medium, such as the memory 804 including instructions executable by the processor 820 of the device 800 to perform the methods is also provided. Alternatively, the storage medium can be a non-transitory computer-readable storage medium, such as a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and optical data storage device, and the like.

[0161] In an example embodiment, there is also provided a computer program product containing instructions which, when executed on a computer, cause the computer to perform the method of calculating macular pigment density of any of the example embodiments.

[0162] As can be seen from the above, the technical solution provided by the embodiments of the present disclosure can significantly reduce or eliminate the influence of external factors by acquiring the to-be-processed fundus image and the historical fundus image of the same photographed object, and using the historical fundus image of the same photographed object to remove and correct the to-be-processed fundus image, thereby avoiding the problem of unstable fundus image quality, inconsistent brightness or chroma caused by differences in shooting conditions, and making the corrected image more truly and accurately reflect the current eye lesion condition of the photographed object. Therefore, calculating the macular pigment density based on the corrected image can significantly improve the accuracy and stability of the calculation result, thereby more reliably evaluating the real macular pigment density level of the photographer.

[0163] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses or adaptive changes of the present disclosure along with their equivalents which incorporate the general principles of the present disclosure and include known or customary practices in the art. The specification and examples are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0164] It should be understood that the present disclosure is not limited to the precise structures as herein described and illustrated in the drawings, and that various modifications and changes can be made without departing from its scope. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. A method of calculating macular pigment density, characterized by, The method comprises the following steps: obtaining a to-be-processed fundus image and a historical fundus image, the historical fundus image being at least one fundus image of a subject of the to-be-processed fundus image in a past preset time period; determining an illumination component of the to-be-processed fundus image and an illumination component of the historical fundus image, and determining an average value of the illumination component of the historical fundus image to obtain a reference illumination component; correcting the illumination component of the to-be-processed fundus image based on the reference illumination component to obtain a corrected image; determining a target macular pigment density corresponding to the to-be-processed fundus image according to the corrected image.

2. The macular pigment density calculation method of claim 1, wherein, The method comprises the following steps: obtaining a to-be-processed fundus image; extracting image features of the to-be-processed fundus image; performing feature matching based on the image features to obtain a historical fundus image.

3. The macular pigment density calculation method of claim 1, wherein, The method comprises the following steps: converting the illumination component of the to-be-processed fundus image and the reference illumination component into a frequency domain respectively to obtain a first frequency spectrum and a second frequency spectrum; correcting a frequency distribution of the first frequency spectrum based on the second frequency spectrum to obtain a correction map; converting the correction map into a spatial domain to obtain a corrected image.

4. The macular pigment density calculation method of claim 1, wherein, Before the step of determining the illumination component of the to-be-processed fundus image and the illumination component of the historical fundus image, the method further comprises the following steps: performing adaptive threshold segmentation on the to-be-processed fundus image to detect whether there is a shadow area in the to-be-processed fundus image; if there is, performing the steps of determining the illumination component of the to-be-processed fundus image and the illumination component of the historical fundus image.

5. The macular pigment density calculation method of claim 1, wherein, The method comprises the following steps: performing macular detection on the corrected image to determine an initial macular pigment density; obtaining a historical macular pigment density corresponding to the historical fundus image, and performing smoothing processing on the initial macular pigment density and the historical macular pigment density to obtain a target macular pigment density.

6. The macular pigment density calculation method of claim 5, wherein, The method comprises the following steps: obtaining a time interval between the historical fundus image and the to-be-processed fundus image, and determining a weight of each historical fundus image based on the time interval, the weight being negatively correlated with the time interval; performing weighted smoothing processing on the initial macular pigment density and the historical macular pigment density based on the weight to obtain a target macular pigment density.

7. The macular pigment density calculation method of claim 1, wherein, After the step of determining the target macular pigment density corresponding to the to-be-processed fundus image according to the corrected image, the method further comprises the following steps: obtaining a historical macular pigment density corresponding to a reference fundus image as a reference macular pigment density, the reference fundus image being a historical fundus image with a minimum time interval from the to-be-processed fundus image; performing calibration on the target macular pigment density based on the reference macular pigment density to obtain a calibrated macular pigment density.

8. The macular pigment density calculation method of claim 7, wherein, The calibration of the target macular pigment density based on the reference macular pigment density comprises: determining a fluctuation range based on a time interval between the reference fundus image and the to-be-processed fundus image; adding or subtracting the reference macular pigment density from or to the fluctuation range to obtain a maximum value and a minimum value; calibrating the target macular pigment density according to the maximum value and the minimum value to obtain a calibrated macular pigment density, the calibrated macular pigment density having a value between the maximum value and the minimum value.

9. A macular pigment density calculating apparatus for implementing the macular pigment density calculating method according to claim 1, characterized by, comprise: an acquisition module configured to acquire a to-be-processed fundus image and a historical fundus image, the historical fundus image being at least one fundus image of a subject of the to-be-processed fundus image in a past preset time period; an analysis module configured to determine an illumination component of the to-be-processed fundus image and an illumination component of the historical fundus image, and determine an average value of the illumination component of the historical fundus image to obtain a reference illumination component; a correction module configured to correct the illumination component of the to-be-processed fundus image based on the reference illumination component to obtain a corrected image; a determination module configured to determine a target macular pigment density corresponding to the to-be-processed fundus image according to the corrected image.

10. An electronic device, comprising: comprise: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the macular pigment density calculation method of any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, When the instructions in the computer readable storage medium are executed by the processor of the macular pigment density calculation electronic device, the macular pigment density calculation electronic device can execute the macular pigment density calculation method of any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the macular pigment density calculation method of any one of claims 1 to 8.

Images