Image display method for myopia prevention and control and related product thereof

Abstract

The invention discloses an image display method for myopia prevention and control and a related product thereof. The image display method for myopia prevention and control comprises the following steps: determining a sight fixation point when a user watches an electronic display screen; determining an out-of-focus fuzzy display area on the electronic display screen according to the sight fixation point; and performing blurring processing for simulating a myopia defocusing effect on an image positioned on the defocusing blurred display area so as to obtain an image with the myopia defocusing effect on the retina of the user. By utilizing the method disclosed by the invention, the myopia prevention and control effect can be obtained under the condition that the myopia patient does not wear the frame glasses with the myopia defocus effect, so that the myopia prevention and control cost of the user is reduced.

Description

Technical Field

[0001] This disclosure generally relates to the field of image processing. More specifically, this disclosure relates to an image display method, device, and computer-readable storage medium for myopia prevention and control. Background Art

[0002] Currently, myopia prevention and control frame glasses with the effects of myopic defocus, higher-order aberrations, and reduced contrast (collectively referred to as defocus blur) have achieved better myopia prevention and control effects in clinical trials and actual use than traditional single-vision myopia lenses. However, since these glasses all require the attachment of microlenses to the lenses, the manufacturing process complexity and price cost are relatively high, which is not conducive to the popularization of this myopia prevention and control technology. In addition, there is also a problem with frame glasses themselves, that is, the problem that the user's vision is not clear when looking at side targets by turning the eyeball. Usually, when the user looks at side targets, most will turn the eyeball. At this time, the center of the line of sight does not pass through the central window of the lens but through the peripheral defocus area, resulting in unclear targets. It can be seen that there are problems with the current use of myopia prevention and control frame glasses to prevent and improve myopia, and there is an urgent need to provide a new technical solution to effectively prevent and control myopia. Summary of the Invention

[0003] To solve at least one or more of the above-mentioned technical problems, this disclosure proposes an image display solution in multiple aspects that achieves myopia prevention and control effects by defocusing and blurring the content displayed around the line of sight. The solution of this disclosure can effectively replace myopia prevention and control frame glasses, thereby providing myopia prevention and control to electronic display users in an economical and effective manner. For this purpose, this disclosure provides solutions in the following multiple aspects.

[0004] In a first aspect, this disclosure provides an image display method for myopia prevention and control, including: determining a line-of-sight fixation point when a user views an electronic display; determining a defocus blur display area on the electronic display according to the line-of-sight fixation point; and performing defocus blurring processing on the image located in the defocus blur display area to simulate the myopic defocus effect, so as to obtain an imaging with a myopic defocus effect on the retina of the user.

[0005] In one embodiment, determining the line-of-sight fixation point when a user views an electronic display includes: positioning the user's pupil and tracking the line-of-sight direction when the user views the electronic display; and determining the line-of-sight fixation point according to the positioning of the pupil and the line-of-sight direction.

[0006] In yet another embodiment, determining the defocus blurred display area on the electronic display screen according to the line-of-sight fixation point includes: determining the clear display area on the electronic display screen according to the line-of-sight fixation point; and determining the display area outside the clear display area on the electronic display screen as the defocus blurred display area.

[0007] In another embodiment, performing defocus blurring processing on the image located in the defocus blurred display area includes: performing defocus blur filtering on the image located in the defocus blurred display area by using a defocus blur filter to obtain a defocus blurred image after defocus blurring processing.

[0008] In one embodiment, it further includes: testing the defocus blur filter by using a test image to obtain a test result regarding the imaging effect on the retina; and adjusting the working parameters of the defocus blur filter according to the test result so as to obtain a defocus blur filter that supports the defocus blurring processing.

[0009] In one embodiment, the blur filter includes a plurality of corresponding blur filters for a plurality of different colors, and each blur filter is respectively used to perform defocus blurring processing on the corresponding color of the image.

[0010] In yet another embodiment, performing defocus blurring processing on the image located in the defocus blurred display area includes: inputting the image on the defocus blurred display area into a trained deep learning network to obtain a defocus blurred image after defocus blurring processing.

[0011] In another embodiment, the loss function in the training stage of the deep learning network is based on a target retinal imaging image and a training retinal imaging image corresponding to the defocus blurred image output by the deep learning network.

[0012] In a second aspect, the present disclosure provides a device for myopia prevention and control, including: a processor; a memory that stores computer program instructions for image display for myopia prevention and control, and when the computer program instructions are executed by the processor, the image display method according to any one of the above first aspect and its multiple embodiments is implemented.

[0013] In one embodiment, the above device further includes a camera connected to the processor and used to assist in determining the line-of-sight fixation point; and an electronic display screen used to display an image after the defocus blurring processing to the user.

[0014] In a third aspect, the present disclosure provides a computer-readable storage medium having stored thereon computer program instructions for image display for myopia prevention and control. When the computer program instructions are executed by a processor, the image display method according to any one of the above first aspect and its multiple embodiments is implemented.

[0015] Through the image display solution of the present disclosure provided by the above multiple aspects and their embodiments, a user can also obtain the effect of myopic defocus without wearing glasses or myopia prevention and control glasses, thereby achieving the purpose of myopia prevention and control. Further, since the solution of the present disclosure can effectively replace myopia prevention and control frame glasses and can dynamically and real-time determine the area where defocus blurring is not performed according to the line of sight direction, the problem of blurring when looking at side targets with frame glasses is thus overcome. Additionally, when the solution of the present disclosure is implemented as a device, it can provide an electronic display screen with a myopic defocus effect, so that users (especially adolescent users) can still effectively prevent myopia without wearing expensive myopia prevention and control glasses. Finally, since the solution of the present disclosure can be implemented through software, it is relatively easy to implement and saves the cost of myopia prevention and control. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the drawings, several embodiments of the present disclosure are shown in an exemplary rather than restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0017] Figure 1 is a simplified flowchart schematically showing an image display method for myopia prevention and control according to an embodiment of the present disclosure;

[0018] Figure 2 is a detailed flowchart schematically showing an image display method for myopia prevention and control according to an embodiment of the present disclosure;

[0019] Figure 3A is a schematic diagram of retinal imaging without a microlens for myopia prevention and control;

[0020] Figure 3B is a schematic diagram of retinal imaging with a microlens for myopia prevention and control;

[0021] Figure 3C is a schematic diagram of retinal imaging using the image display method of the present disclosure;

[0022] Figure 4 is a flowchart schematically showing the determination of a filter for defocus blurring processing according to an embodiment of the present disclosure;

[0023] Figure 5 is a flowchart schematically showing the training of a deep learning network for defocus blurring processing according to an embodiment of the present disclosure;

[0024] Figure 6A is a schematic display effect diagram of an electronic screen without applying the image display method of the present disclosure;

[0025] Figure 6B is a schematic display effect diagram of an electronic screen applying the image display method of the present disclosure; and

[0026] Figure 7 is a block diagram schematically showing a system for image display according to an embodiment of the present disclosure. Detailed Description of the Invention

[0027] Hereinafter, the technical solutions in the embodiments of the present disclosure will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present disclosure.

[0028] It should be understood that the terms "comprising" and "including" used in the specification and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0029] It should also be understood that the terms used in the specification of the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the present disclosure. As used in the specification and claims of the present disclosure, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms. It should also be further understood that the term " / and / " used in the specification and claims of the present disclosure refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations.

[0030] As used in the present specification and claims, the term "if" may be interpreted as "when" or "once" or "in response to determining" or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if detecting [the described condition or event]" may be interpreted as meaning "once determined" or "in response to determining" or "once detecting [the described condition or event]" or "in response to detecting [the described condition or event]" according to the context.

[0031] The specific embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0032] Figure 1 FIG. is a simplified flowchart schematically showing an image display method 100 for myopia prevention and control according to an embodiment of the present disclosure. It can be understood that the image display method of the present disclosure can be executed by a device integrated with a general-purpose processor or a dedicated processor, for example, it can be executed by a desktop computer, a portable computer, a tablet computer, or various other electronic devices with a display screen.

[0033] As Figure 1 shown, at step S102, the line-of-sight fixation point of the user when viewing the electronic display screen is determined. As described above, the electronic display screen here can be a desktop computer, a tablet computer (such as iPad, Android tablet, Windows tablet, etc.), a display screen installed on various mobile terminals, or a dedicated display device. In one implementation scenario, the operation of determining the line-of-sight fixation point here can be jointly determined by the processor in combination with a camera installed on or adjacent to the electronic display screen. For example, when the user views the content displayed on the electronic display screen, the above-mentioned camera can observe and locate the user's line-of-sight direction and feedback the line-of-sight direction information to the processor. Thereafter, the processor can determine the position of the line-of-sight focus according to the received line-of-sight direction information, that is, the line-of-sight fixation point mentioned above.

[0034] Next, at step S104, according to the line-of-sight fixation point, a defocus blur display area on the electronic display screen is determined. In order to achieve imaging with a myopic defocus effect on the user's retina, the solution of the present disclosure sets the display area outside a certain range or area (such as a 5-degree viewing angle or 3 cm) from the line-of-sight fixation point on the screen as the defocus blur display area. It can be understood that as the line-of-sight fixation point of the user continuously changes when viewing the electronic display screen, the defocus blur display area of the present disclosure also changes dynamically accordingly, so as to maintain the expected imaging with a myopic defocus effect on the user's retina.

[0035] After determining the defocus blur display area as above, at step S106, a defocus blurring process that simulates the myopic defocus effect is performed on the image located in the defocus blur display area, so as to obtain imaging with a myopic defocus effect on the user's retina. According to different application scenarios, the defocus blurring process of the present disclosure can be implemented in different ways. In one implementation scenario, the present disclosure proposes to implement the foregoing defocus blurring process by constructing a blur filter. The operation of constructing the blur filter will be described in detail later in combination with Figure 4 to be described. In another implementation scenario, the present disclosure proposes to perform the foregoing defocus blurring process on the image in the defocus blur display area through a deep learning network. The training process of this deep learning network will be described in detail later in combination withFigure 5 A detailed description will be given.

[0036] Using the above combination Figure 1 For the image display method 100 described above, the solution of the present disclosure can provide image display effective for myopia prevention and control in a way that significantly reduces the cost of myopia prevention and control. Specifically, through the innovative defocus blurring process of the present disclosure, retinal imaging with a defocus blur effect similar to that of microlenses and other micro-optical structures can be achieved on a general electronic display device, and such a defocus blurring process is easy to implement without the need for additional components or units. Further, by means of the defocus blurring process of some areas on the electronic display screen, myopic patients can obtain the effect of myopia prevention and control without wearing frame glasses with a myopic defocus effect, thereby reducing the cost of myopia prevention and control for users.

[0037] Figure 2 FIG. is a detailed flowchart schematically showing an image display method 200 for myopia prevention and control according to an embodiment of the present disclosure. It can be understood that the method 200 herein shows more step details and can be regarded as a possible implementation manner of the method 100 described above. Therefore, the above description of the method 100 also applies to the following description of the method 200. Figure 1 As described above, the method 100 described above also applies to the following description of the method 200.

[0038] As Figure 2 shown, at step S202, when the user views the electronic display screen, the pupil of the user is located and the line-of-sight direction is tracked. Then, at step S204, the line-of-sight fixation point is determined based on the location of the pupil and the line-of-sight direction. In one implementation scenario, one or more cameras on the electronic display device, ambient light, self-supplied light, ordinary white light, or infrared light can be used to calculate the position of the pupil relative to the screen by means of perspective calculation or stereoscopic vision. For line-of-sight tracking, in one implementation scenario, the user's eyes can be detected, and the image of the user's eye region can be used as a feature input to an algorithm or neural network for determining the line-of-sight direction, so as to obtain the line-of-sight direction of the user. In another implementation scenario, the positions of the pupils in the user's two eyes can be analyzed, and then other information such as the overall face can be fused, and then the line-of-sight direction of the user can be estimated through a deep learning network. After determining the position of the pupil and the line-of-sight direction, the line-of-sight fixation point on the electronic display screen can be determined based on the two.

[0039] Next, at step S206, the clear display area on the electronic display screen is determined according to the above line-of-sight fixation point (as Figure 6Bthe area shown at 602 in). As described above, the clear display area may be an area within a predetermined range around or at a distance from the sight fixation point, such as an area within a certain angular view (e.g., 5 degrees) or a certain distance (e.g., 3 cm). The process proceeds to step S208, where the display area outside the clear display area on the electronic display screen is determined as the defocus blurred display area (which is the area other than that at 602 in Figure 6B the area other than that at 602 in). Finally, at step S210, defocus blurring processing is performed on the image located on the defocus blurred display area (the effect of which is as shown in Figure 6B the area other than that at 602 in), so as to obtain an image with a myopic defocus effect on the user's retina.

[0040] The above Figure 2 further describes the image display scheme of the present disclosure. It can be seen that when implemented on an electronic display device with a camera, the scheme of the present disclosure can achieve imaging with a myopic defocus effect on the user's retina without the need for a dedicated myopia prevention and control device. Therefore, compared with the prior art, the scheme of the present disclosure is easy to implement and effectively reduces the cost of myopia prevention and control.

[0041] To facilitate the understanding of the defocus blurring processing of the present disclosure, the principle of retina imaging related to the scheme of the present disclosure will be described below in conjunction with Figures 3A to 3C FIG.

[0042] Figure 3A is a schematic diagram of retina imaging without a microlens for myopia prevention and control. As shown in Figure 3A FIG., the screen display content (denoted as "ImageOri") 301 of the electronic display screen is focused onto the retina via a lens ("Lens") 302, obtaining an image on the retina (denoted as "ImageRetinaS") 303.

[0043] In terms of the imaging principle, "ImageOri" is the content displayed on the plane of the electronic display screen and can be represented by a two-dimensional array. Each element in the array is a pixel on the plane. For example, ImageOri(i, j) represents the value of the pixel in the i-th row and j-th column on the screen (abbreviated as "pixel value"). Here, the pixel value can be a three-element array including three color values of red ("Red"), green ("Green"), and blue ("Blue").

[0044] After passing through the optical system of the lens 302 and the eyeball, ImageOri falls on the retina, forming ImageRetinaS. Here, ImageRetinaS can also be represented by a two-dimensional array. To simplify the derivation, it is assumed that the sampling method on the retina ensures a one-to-one correspondence between the elements of ImageOri and the elements of ImageRetinaS. For example, assume that ImageOri is a 4000×3000 image, then ImageRetinaS is also a 4000×3000 image. Also, to simplify the derivation, it is assumed that the images on ImageOri are clearly imaged on the retina. Thus, the value of ImageRetinaS[i, j] is basically determined by ImageOri[i, j] and the attenuation of the entire optical system to the light intensity, and can be expressed by the following formula (1):

[0045] ImageRetinaS(i, j) = fs(i, j) × ImageOri(i, j) (1)

[0046] In the above formula (1), fs(i, j) is the attenuation value of the entire optical system to ImageOri(i, j). To simplify the derivation, fs(i, j) can be considered a constant value. That is, for different (i, j), fs(i, j) is the same, and thus can be simplified to the following formula (2):

[0047] ImageRetinaS(i, j) = fs × ImageOri(i, j) (2)

[0048] Figure 3B is a schematic diagram of retinal imaging in the presence of microlenses for myopia prevention and control. As Figure 3B shown, the screen display content of the electronic display screen (represented by "ImageOri") 304 is focused on the retina through the microlens 306 on the lens ("Lens") 305, obtaining the imaging on the retina (represented by "ImageRetinaD") 307. In terms of the operating principle, attached to the original single-vision lens (such as Figure 3AThe microlens on the 302) in it will focus a part of the light in front of the retina (close to the ocular surface side), so that the image formed on the retina is defocused and blurred in a certain way, thus achieving the effect of myopia prevention and control. In view of this, the ImageRetinaD 307 blurred by the defocus microlens is the focus of the solution disclosed herein. For this purpose, the conversion relationship between ImageRetinaS and ImageRetinaD can be determined. Then, this conversion relationship is attached to ImageOri, so that it is converted into ImageOriD. Further, the image formed by the single-vision / single-focus glasses on the retina of ImageOriD is or close to ImageRetinaD, that is, the retinal image with the myopic defocus effect. In other words, through such a conversion method, the defocus effect can be achieved without using a microlens, that is Figure 3C the situation shown.

[0049] Figure 3C is a schematic diagram of retinal imaging showing the use of the image display method disclosed herein. As Figure 3C shown, the screen display content of the electronic display screen (represented by "ImageOriD") 308 is focused on the retina through the lens ("Lens") 302, and the image formed on the retina (represented by "ImageRetinaD") 307 is obtained.

[0050] The following combines Figures 3A - 3C to analyze the relationship between ImageRetinaS and ImageRetinaD. In the absence of a microlens, ImageOri can be clearly imaged as ImageRetinaS, that is, the light emitted from ImageOri[i, j] will be converged to ImageRetinaS[i, j]. When a microlens is added to the optical system, the light emitted from ImageOri[i, j] will not be all converged to ImageRetinaD[i, j], but will also fall on some areas around ImageRetinaD[i, j], thus producing a blurred myopic defocus effect. From the perspective of digital image processing, this is a typical filtering process of a two-dimensional blur filter, which can be expressed by the following formula (3):

[0051] ImageRetinaD(i, j) = ImageRetinaS(i, j) * g(i, j) (3)

[0052] Where the symbol "*" represents convolution, and the filter g(i, j) can be calculated from the parameters of the microlens, lens, and eyeball, or simulated by optical simulation software. It should be noted here that since the eyeball is a sphere and has special optical properties, strictly speaking, the filter g(i, j) is different at different positions. Therefore, a more rigorous formula is shown in Equation (4) below:

[0053] ImageRetinaD(i, j) = ImageRetinaS(i, j) * g_ij(i, j) (4)

[0054] Where g_ij(i, j) is the blur filter calculated or simulated at the position (i, j).

[0055] Although g_ij(i, j) has different values at different positions, through theoretical analysis and simulation, it can be found that within the concerned viewing angle range (for example, the viewing angle when looking at an electronic display screen generally does not exceed 30 degrees), the difference in g_ij(i, j) is not significant. At the same time, the practical experience of the myopic defocus theory also shows that the parameters of the microlens are not a parameter that needs to be precisely controlled. Therefore, the difference in g_ij(i, j) will not have too much impact on the myopia prevention and control effect. Therefore, it can be assumed that the filter g(i, j) is independent of the position.

[0056] Based on the above discussion, the equivalent g(i, j) can be transferred to the original image of the electronic display screen. Since it has been deduced in the previous text that ImageRetinaS and ImageOri have a linear relationship, g(i, j) and fs can be interchanged, that is, the following expression:

[0057] (fs × ImageOri(i, j)) * g(i, j) = fs × (ImageOri(i, j) * g(i, j))

[0058] Based on the above discussion, the original image can be filtered with the appropriate g(i, j) filter. Then, the filtered image is imaged on the retina through a single - vision lens and the eyeball. Thus, the imaging on the retina is the same as or close to the imaging on the retina after the original image passes through a single - vision plus microlens and the eyeball. It can be understood that the same or close imaging here means the same or close myopia prevention and control effect.

[0059] It should be emphasized here that the images of the natural world can be represented by three colors, RGB (red, green, and blue), on an electronic display screen. Therefore, g(i, j) actually has three groups of RGB. Since the optical system has slightly different refraction effects on light of different wavelengths, the blur filters for the three colors of RGB are slightly different. Although the difference is very small, the solution of this disclosure optionally maintains this difference because the retina will sense and process this difference, thus affecting the effect of myopia prevention and control.

[0060] Regarding the differences between the blur filters for the three colors of RGB mentioned above, it needs to be further clarified that although these blur filters have the same calculation method in the solution of this application, due to the subtle differences in optical properties, especially the different refractive responses to light of different wavelengths (such as red, green, and blue light waves), in some application scenarios, this application proposes to specifically design the blur filters for each color. This design takes into account the refractive index differences shown by the same lens material when processing light of different wavelengths, so that the blur filter corresponding to each color can perform the best defocus adjustment for its specific light wave.

[0061] Specifically, in the solution of this application, blur filters for red light waves, green light waves, and blue light waves can be designed respectively according to the different refractive characteristics of the three color light waves of RGB. Although these blur filters are designed based on the same calculation method (such as the method discussed above), they are slightly adjusted according to the specific refractive index of each color light wave to ensure the best defocus effect on the retina, thereby effectively achieving myopia prevention and control. For example, the blur filter for red light waves may perform stronger or weaker defocus adjustment on the light of this wavelength, and the green and blue light wave filters are also adjusted slightly accordingly. Through this subtle adjustment, the solution of this application can ensure that each color light wave obtains the best visual processing, which plays a good auxiliary role in the visual effect of the overall image and the effect of myopia prevention and control.

[0062] The above derivation regarding image conversion also applies to other myopia prevention and control lenses except those with added microlens arrays. The solution of this disclosure does not limit in this regard. In addition, the system was simplified in the above derivation. These simplifications have little impact on the effect of myopia prevention and control because many parameters of the myopia prevention and control lenses themselves have a large margin, such as the degree, surface shape, quantity, and area of the microlenses.

[0063] Figure 4It is a flowchart schematically showing the determination of a filter for defocus blurring processing according to an embodiment of the present disclosure. It can be understood that the determination process 400 can be executed by an electronic display device for displaying content, or by another entity independent of the electronic display device, and the filter parameters are fed back to the electronic display device for it to perform the defocus blurring processing of the present disclosure.

[0064] As Figure 4 shown, at step S402, one or more test images ImageOri are selected. Then, at step S404, the image formed on the retina when the user wears myopia prevention and control glasses is calculated through optical simulation as ImageRetinaD. At step S406, the parameters of the filter g(i, j) are initialized. Here, the initialization can be to set the initial values of the filter parameters according to empirical values.

[0065] At step S408, the test image selected at step S402 is filtered (or filtered) using the filter g(i, j) to generate ImageOriD’. Then, at step S410, using the filtered image, the image formed on the retina is calculated again through optical simulation under a single vision lens without myopia prevention and control elements such as microlenses as ImageRetinaD’. Thereafter, at step S412, the difference between ImageRetinaD and ImageRetinaD’ is calculated. When there are multiple test images, the obtained differences can be weighted and averaged.

[0066] At step S414, the magnitude of the difference is evaluated, for example, to determine whether the obtained difference is within the threshold range. When it is determined that the difference is not within the threshold range, then at step S416, the parameters of g(i, j) are adjusted. Thereafter, steps S408 - S414 can be repeated. The aforementioned adjustment can be to determine the direction and step size of the adjustment of g(i, j) according to the direction of gradient descent. When the difference is within the threshold range, then at step S418, the current filter g(i, j) parameters are selected for use.

[0067] Through the above - combined Figure 4 description, the solution of the present disclosure can search for the optimal filter, so as to imitate the lens with myopia prevention and control effect with the best myopic defocus effect. It can be understood that the above - mentioned processing flow is also applicable to different situations when g(i, j) is in different regions. In this scenario, the optimal filter g_ij(i, j) for each region can be found zone by zone.

[0068] Figure 5It is a flowchart schematically showing the training of a deep learning network for defocus blurring processing according to an embodiment of the present disclosure. It can be understood that the training process 500 can be executed by an electronic display device for displaying content, or by another entity independent of the electronic display device, and the trained model is transmitted to the electronic display device for it to perform the defocus blurring processing of the present disclosure.

[0069] As Figure 5 shown, at step S502, image data of pairs of ImageOriD and ImageRetinaD (i.e., they have a one-to-one mapping relationship) as described above are selected as training data. Then, at step S504, each ImageOriD is input into the deep learning network to generate ImageOriD’. Thereafter, at step S506, the image formed on the retina, ImageRetinaD’, is calculated through optical simulation.

[0070] At step S508, a loss function based on ImageRetinaD’ and ImageRetinaD is constructed. For example, the loss function is constructed on the condition that the difference between the two is within a predetermined range. Then, at step S510, the deep learning network is trained using the loss function, including forward training and backward training. During the backward training process, the loss function can be used to adjust network parameters, such as weights and / or biases. Finally, at step S512, the deep learning network that minimizes the loss function is determined as the final model for generating ImageRetinaD’. Thus, instead of using the filter described in combination with Figure 4 it, the deep learning network described here can also be used to perform defocus blurring processing on the image in the defocus blurring display area, so as to obtain an image with a myopic defocus blurring effect on the retina.

[0071] Figure 6A and Figure 6B are respectively schematic diagrams showing the electronic screen display effect diagrams of not applying the present disclosure and applying the present disclosure. From Figure 6A the shown content, it can be seen that since the image display scheme of the present disclosure is not applied, the content displayed on the screen is not different from that of a conventional electronic display screen. Of course, for users wearing conventional non-myopia prevention and control glasses, such a display effect cannot achieve the effect of myopia prevention and control. In contrast, Figure 6B shows the display effect after applying the image display scheme of the present disclosure. It can be seen that, relative to Figure 6A , Figure 6B in the electronic display screen, there is a clear area and a blurred area bounded by 602. It can be understood that when a user views an image with a partially blurred image, the image formed on their retina will have a myopic defocus effect, thus achieving the effect of myopia prevention and control.

[0072] Figure 7 is a block diagram schematically showing a system 700 for image display according to an embodiment of the present disclosure. The system 700 may include a device 701 according to an embodiment of the present invention, its peripheral devices, and an external network, where the device 701 may be used to perform defocus blurring processing on an image so as to obtain an imaging with a myopic defocus effect on the retina of the user.

[0073] As Figure 7 shown, the device 701 may include a CPU 7011, which may be a general-purpose CPU, a dedicated CPU, or other execution units for information processing and program running. Further, the device 701 may also include a mass storage 7012 and a read-only memory ROM 7013, where the mass storage 7012 may be configured to store various types of data, including image data to be displayed, test image data for deriving filter parameters, intermediate data, test results, and various programs required for running a deep learning network. The ROM 7013 may be configured to store data required for power-on self-test of the device 701, initialization of each functional module in the system, driver programs for basic input / output of the system, and booting the operating system.

[0074] Further, the device 701 may also selectively include other hardware platforms or components, such as the shown tensor processing unit (“TPU”) 7014, graphics processing unit (“GPU”) 7015, field programmable gate array (“FPGA”) 7016, and machine learning unit (“MLU”) 7017. It can be understood that although various hardware platforms or components are shown in the device 700 of the present disclosure, they are merely exemplary rather than restrictive, and those skilled in the art may add or remove corresponding hardware according to actual needs. For example, the device 701 may only include a CPU as a well-known hardware platform to perform defocus blurring processing.

[0075] In some application scenarios, the device 701 of the present disclosure further includes a communication interface 7018, so that it can be connected to a local area network / wireless local area network (LAN / WLAN) 705 through the communication interface 7018, and further can be connected to a local server 706 or connected to the Internet ("Internet") 707 through the LAN / WLAN. Alternatively or additionally, the device 701 of the present invention can also be directly connected to the Internet or a cellular network based on wireless communication technology through the communication interface 7018, such as wireless communication technology based on the third generation ("3G"), fourth generation ("4G") or fifth generation ("5G"). In some application scenarios, the device 701 of the present invention can also access a server 708 and a possible database 709 of an external network as needed, so as to obtain various known neural network models, image test data and function modules, and can remotely store various data related to defocus blurring processing.

[0076] The peripheral devices of the device 701 may include a display device 702, an input device 703, and a data transmission interface 704. In one embodiment, the display device 702 may include, for example, a plurality of visual displays (such as a touch-sensitive display panel), which are configured to display the image after defocus blurring processing. The input device 703 may include, for example, a keyboard, a mouse, a microphone, a camera, or other input buttons or controls, which are configured to receive user instructions or collect external information. For example, for a camera, it can be used to collect pupil positioning information and line-of-sight direction information when the user views the display device, and feedback to the processor of the device 701. The data transmission interface 704 may include, for example, a serial interface, a parallel interface, or a universal serial bus interface ("USB"), a small computer system interface ("SCSI"), serial ATA, FireWire, PCI Express, and a high-definition multimedia interface ("HDMI"), etc., which are configured for data transmission and interaction with other devices or systems. According to the solution of the present disclosure, the data transmission interface 704 can also receive various image data for testing purposes and / or image data presented to the user, and transmit it to the device 701.

[0077] The above-mentioned CPU 7011, mass storage 7012, read-only memory ROM 7013, TPU 7014, GPU 7015, FPGA 7016, MLU 7017, and communication interface 7018 of the device 701 of the present invention can be interconnected through a bus 7019, and realize data interaction with the peripheral devices through the bus. In one embodiment, through the bus 7019, the CPU 7011 can control other hardware components and their peripheral devices in the device 701.

[0078] During operation, the CPU 7011 of the device 701 of the present invention can receive image data through the input device 703 or the data transmission interface 704, and retrieve computer program instructions or codes stored in the memory 7012 (such as codes related to image defocus blurring processing) to perform partial defocus blurring processing on the received image data, so as to obtain an imaging with myopic defocus effect on the retina of the user. At the same time, after the CPU 7011 performs the defocus blurring processing, the image after the defocus blurring processing can be displayed on the display device 702. Although shown in a separated manner in the figure, in some application scenarios, the device 701 disclosed in the present disclosure can be highly integrated with the display device 702 (such as an electronic display screen) and the input device 703 into one body, for example, forming a handheld tablet device (such as an iPad), so as to effectively protect the myopia of users, especially teenagers.

[0079] It should also be understood that any module, unit, component, server, computer, terminal or device that executes computer program instructions in the examples disclosed in the present disclosure can include or otherwise access a computer-readable medium, such as a storage medium, a computer storage medium or a data storage device (removable) and / or non-removable), such as a magnetic disk, an optical disk or a magnetic tape. The computer storage medium can include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data.

[0080] Based on the above, the present invention also discloses a computer-readable storage medium, in which program instructions are stored, and the program instructions are adapted to be loaded and executed by a processor: determining the line-of-sight fixation point of a user when viewing an electronic display screen; determining a defocus blurring display area on the electronic display screen according to the line-of-sight fixation point; performing defocus blurring processing on the image located in the defocus blurring display area to simulate the myopic defocus effect, so as to obtain an imaging with myopic defocus effect on the retina of the user. Further, when the program instructions are loaded by the processor, the process steps described in conjunction with Figure 2 - Figure 6 are also executed.

[0081] A computer-readable storage medium can be any suitable magnetic storage medium or magneto-optical storage medium, such as, resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), and so on, or any other medium that can be used to store the required information and can be accessed by an application, a module, or both. Any such computer storage medium can be part of the device or accessible or connectable to the device. Any application or module described in the present invention can be implemented using computer-readable / executable instructions that can be stored or otherwise held by such a computer-readable medium.

[0082] It should be understood that the possible terms "first", "second", etc. in the claims, the specification, and the drawings disclosed in the present invention are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" used in the specification and claims disclosed in the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0083] It should also be understood that the terms used in the specification of the present invention disclosed herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention disclosed. As used in the specification and claims of the present invention disclosed, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms. It should be further understood that the term "and / or" used in the specification and claims of the present invention disclosed refers to any combination and all possible combinations of one or more of the associated listed items and includes these combinations.

[0084] While several embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many variations, changes and alternative methods will occur to those skilled in the art without departing from the spirit and scope of the present disclosure. It should be understood that various alternatives to the embodiments of the present disclosure described herein may be employed in practicing the present disclosure. The appended claims are intended to define the scope of the present disclosure and thus cover equivalents or alternatives within the scope of these claims.

Claims

1. An image display method for myopia prevention and control, comprising: Determine the gaze fixation point of the user when viewing the electronic display screen; Determine the defocus blurred display area on the electronic display screen according to the gaze fixation point; And Perform defocus blurring processing on the image located in the defocus blurred display area to simulate the myopic defocus effect, so as to obtain an imaging with a myopic defocus effect on the retina of the user.

2. The image display method according to claim 1, wherein determining the line-of-sight fixation point when the user views the electronic display screen comprises: Locate the pupil of the user and track the gaze direction when the user views the electronic display screen; And Determine the gaze fixation point according to the positioning of the pupil and the gaze direction.

3. The image display method according to claim 1, wherein determining the defocus blur display area on the electronic display screen according to the line-of-sight fixation point comprises: Determine the clear display area on the electronic display screen according to the gaze fixation point; And Determine the display area outside the clear display area on the electronic display screen as the defocus blurred display area.

4. The image display method according to claim 1, wherein performing defocus blur processing on the image located in the defocus blur display area comprises: Perform defocus blurring filtering on the image located in the defocus blurred display area by using a blur filter to obtain a defocus blurred image after defocus blurring processing; 5. The image display method according to claim 4, further comprising: Test the defocus blur filter with a test image to obtain a test result on the imaging effect on the retina; Adjust the working parameters of the defocus blur filter according to the test result so as to obtain a blur filter that supports the defocus blurring processing.

6. The image display method according to claim 4 or 5, wherein the blur filter comprises a plurality of blur filters corresponding to a plurality of different colors, and each blur filter is respectively used to perform defocus blur processing on the corresponding color of the image.

7. The image display method according to claim 4, wherein performing defocus blur processing on the image located in the defocus blur display area comprises: Input the image on the defocus blurred display area into a trained deep learning network to obtain a defocus blurred image after blurring processing; 8. The image display method according to claim 7, wherein the loss function in the deep learning network training stage is based on the target retinal imaging image and the training retinal imaging image corresponding to the blurred image output by the deep learning network.

9. A device for myopia prevention and control, comprising: Processor; A memory that stores computer program instructions for myopia prevention and control image display, and when the computer program instructions are executed by the processor, the image display method according to any one of claims 1-8 is implemented.

10. The device according to claim 9, further comprising: A camera, which is connected to the processor and is used to assist in determining the gaze fixation point; And An electronic display screen, which is used to display an image including the defocus blurred image after the defocus blurring processing to the user.

11. A computer-readable storage medium storing computer program instructions for myopia prevention and control image display, which, when executed by a processor, enable the implementation of the image display method according to any one of claims 1-8.

Images