Vision assistance method, storage medium and vision assistance device

Abstract

The invention discloses a vision assisting method. The method comprises the following steps: presenting a visual field image to a user; determining a fixation point of the user in the view image; determining a fixation area of the user according to the fixation point; extracting the view image in the gazing area range as a view sub-image; processing the view sub-image to obtain an enhanced image with an enhanced visual effect; and presenting the enhanced image to the user. The invention further discloses a storage medium and a vision assisting device. According to the embodiment of the invention, where the eyes of the user see, the display content in the area which is seen around the eyes is enhanced in visual effect and presented to the user. Therefore, the pertinence of vision assistance for the low-vision user is remarkably improved, and the practical use requirements of the user are better met. And the user does not need to manually carry out any adjustment in the whole process, so that the two hands of the user are liberated.

Description

Technical Field

[0001] The present invention relates to the field of intelligent technology control, and more specifically to a vision assistance method for low-vision users. The present invention also relates to a storage medium storing a computer program for implementing the method and a vision assistance device. Background Art

[0002] At present, there are a huge number of people with low vision in the world. Low vision users are hindered in many aspects such as work, life, and study. Their physical and mental health and daily quality of life are reduced to varying degrees. In severe cases, they are even unable to take care of themselves, which creates a great burden on the users themselves, their families and society.

[0003] A considerable number of low vision users cannot obtain effective treatment effects through surgery and drugs. Such users can use visual aids to improve their vision. Currently, common visual aids are divided into optical visual aids and electronic visual aids. Optical visual aids are, for example, magnifying glasses and telescopes. Electronic visual aids generally include handheld, portable, desktop and head-mounted visual aids. The visual aid principle of optical visual aids is to use the magnification function of optical elements to enlarge the image (or image) and provide it to the user for viewing. The visual aid principle of electronic visual aids is to use a camera to capture the image and then the processor performs text recognition, magnification, color mode adjustment, contrast enhancement and other processing on the image before providing it to the user for viewing. Although both types of visual aids can provide users with visual assistance, their effectiveness and pertinence still need to be improved.

[0004] Therefore, it is necessary to develop relevant vision assistance methods and products for low vision users, so as to provide low vision users with more convenient and effective vision assistance in their work, life and study. Summary of the invention

[0005] The object of the present invention is to at least alleviate or solve the above-mentioned problems by providing a vision assistance method, a storage medium and a vision assistance device.

[0006] According to one aspect of the present invention, there is provided a vision assistance method, comprising: presenting a field of view image to a user; determining the user's gaze point in the field of view image; determining the user's gaze area based on the gaze point; extracting the field of view image within the gaze area as a field of view sub-image; processing the field of view sub-image to obtain an enhanced image with enhanced visual effect; and presenting the enhanced image to the user.

[0007] Preferably, the enhanced image is obtained by superimposing the visual effect of the field of view sub-image on the field of view image.

[0008] Preferably, the enhanced image is centered on the gaze point.

[0009] Preferably, the enhanced image is obtained by superimposing it on the field of view image after performing one or more of the following processing on the field of view sub-image: magnification processing; edge enhancement processing; 3D relief processing; semi-transparent or non-transparent processing; color mode reconstruction processing; and contrast reconstruction processing.

[0010] Preferably, the magnification factor during the magnification process is preset and / or set and adjusted by the user.

[0011] Preferably, the transparency of the semi-transparent or non-transparent treatment is preset, and / or set and adjusted by the user.

[0012] Preferably, the vision assist method further includes: processing the field of view image and the field of view sub-image so that the area where the field of view sub-image with enhanced visual effect is located in the enhanced image has a color pattern or contrast pattern that matches the user's vision condition with other areas.

[0013] Preferably, the vision assistance method further includes: identifying whether the field of view image includes preset image content, and when the field of view image includes the preset image content, reminding the user of the corresponding position of the preset image content.

[0014] Preferably, the preset image content includes objects that may be dangerous to the user, objects that the user is looking for, or objects that the user is familiar with.

[0015] Preferably, the vision assistance method further comprises: identifying whether the visual field sub-image contains text content, and when the visual field sub-image contains text content, automatically playing the text content in voice mode.

[0016] Preferably, the field of view image comes from an image capture device or a video signal interface.

[0017] Preferably, the gaze area is centered on the gaze point, has a circular, square or a shape that is the same as the outline of the field of view image, and has a preset size.

[0018] Preferably, when the field of view images are continuous and constitute a field of view image video data stream, the enhanced image video data stream constituted by the enhanced images is presented to the user with the same frequency and order.

[0019] Preferably, the vision assistance method further includes: identifying whether the user's voice input corresponds to a preset instruction, and when the voice input corresponds to the preset instruction, executing an operation corresponding to the preset instruction.

[0020] According to another aspect of the present invention, there is provided a storage medium having a computer program stored thereon, wherein the computer program implements the vision assistance method as described above when executed by a processor.

[0021] According to another aspect of the present invention, a vision assist device is provided, comprising: a gaze point determination module for determining a user's gaze point; an image processing module, connected to the gaze point determination module, comprising a storage medium as described above, and for executing a computer program stored on the storage medium; and a display module, connected to the image processing module, and for presenting an enhanced image to the user.

[0022] Preferably, the gaze point determination module is an eye tracking device.

[0023] Preferably, the vision assistance device further comprises: a field of view image acquisition module, which is an image capture device or a video signal interface and is connected to the image processing module.

[0024] Preferably, when the field of view image acquisition module is an image capture device, the vision aid device is a head-mounted vision aid device, the field of view image acquisition module is arranged directly in front of the head-mounted vision aid device, the eye tracking device is installed inside the head-mounted vision aid device, and the display module is a near-eye display, which is installed inside the head-mounted vision aid device and faces the user's eyes.

[0025] Preferably, the vision assistance device further comprises: a visual reminder module, for reminding the user of the corresponding position of the preset image content when the visual field image contains the preset image content.

[0026] Preferably, when the field of view image acquisition module is a video signal interface, the display module is a display screen, and the eye tracking device is arranged below the display screen.

[0027] Preferably, the vision assist device also includes: a voice playback module, which is used to automatically play the text content in voice when the field of view sub-image includes text content, wherein the voice playback module includes a voice library sub-module storing voice data and a speech speed adjustment sub-module for adjusting the speech speed of automatic playback.

[0028] Preferably, the vision assist device further comprises: a voice interaction module, which has a voice instruction library and is configured to recognize the voice input by the user and convert it into instructions executable by the image processing unit.

[0029] By means of the above technical solution of the present invention, at least the following beneficial technical effects can be achieved:

[0030] According to the embodiment of the present invention, wherever the user's eyes look, the displayed content in the area (i.e., the gaze area) surrounding the point where the eyes look (i.e., the gaze point) is visually enhanced and presented to the user. This significantly improves the pertinence of vision assistance for low-vision users and better meets the actual usage needs of users. The user does not need to make any manual adjustments during the entire process, thereby freeing the user's hands. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0032] Figure 1 is a flow chart of a vision assistance method according to an embodiment of the present invention;

[0033] Figure 2 is a schematic block diagram of a vision assistance device according to an embodiment of the present invention;

[0034] Figure 3 is a schematic block diagram of a vision assistance device according to an embodiment of the present invention;

[0035] Figure 4 is a schematic diagram of a visual field image, a fixation point, and a fixation area according to an embodiment of the present invention; and

[0036] Figure 5 Schematic diagram of a display screen type vision aid device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant inventions, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0038] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0039] Figure 1 A flowchart of a vision assistance method 100 according to an embodiment of the present invention is shown, and Figure 4 The visual field image, the fixation point and the fixation area of ​​the embodiment of the present invention are schematically shown. Figure 1 and Figure 4 As shown, according to an embodiment of the present invention, the vision assistance method 100 includes the following steps.

[0040] S110: presenting a visual field image to a user;

[0041] S120: Determine the gaze point of the user in the visual field image;

[0042] S130: Determine the user's gaze area according to the gaze point;

[0043] S140: extracting the visual field image within the gaze area as a visual field sub-image;

[0044] S150: Processing the field of view sub-image to obtain an enhanced image with enhanced visual effect; and

[0045] S160: Present the enhanced image to the user.

[0046] Figure 4 The sizes of the field of view image, fixation area, and enhanced image are only for illustrative purposes and do not represent the actual size relationship between the three.

[0047] According to an embodiment of the present invention, a gaze area is determined according to a user's gaze point in a visual field image, a visual field image located in the gaze area is extracted to obtain a visual field sub-image, the visual field sub-image is processed to obtain an enhanced image with enhanced visual effects, and the enhanced image is presented to the user. That is, in an embodiment of the present invention, by tracking the user's gaze point, the visual effect of the visual field sub-image in the gaze area around the gaze point is enhanced, and the enhanced image with enhanced visual effects is presented to the user. In this way, the display content in the area (i.e., the gaze area) surrounding the point where the eyes see (i.e., the gaze point) is visually enhanced and presented to the user. Compared with the magnification method of optical visual aids such as magnifiers and telescopes in the prior art that do not divide the area into magnifications, the above technical solution of the present invention significantly improves the pertinence of visual assistance for low-vision users, and is more in line with the actual use needs of users. In addition, throughout the entire process of the method, the user does not need to make any manual adjustments, and can be achieved by adjusting the gaze point by the eyes, thereby freeing the user's hands.

[0048] According to an embodiment of the present invention, the enhanced image may be obtained by superimposing the visual effect of the field of view sub-image on the field of view image. The field of view sub-image is processed and superimposed on the original field of view image, thereby obtaining an enhanced image with enhanced visual effect to be presented to the user. In this way, not only the display content located in the gaze area is presented to the user after visual effect enhancement, but also the display content located outside the gaze area (that is, the image content located outside the gaze area in the original field of view image) is presented to the user. This allows the user to not only see the enhanced image with enhanced visual effect, but also see the image content located outside the gaze area, for example, through peripheral vision.

[0049] According to an embodiment of the present invention, the enhanced image can be centered on the gaze point, that is, the center of the enhanced image coincides with the gaze point. When the center of the enhanced image coincides with the gaze point, the display content in the gaze area surrounding the gaze point is presented to the user with the gaze point as the center after the visual effect is enhanced, depending on where the user's eyes look. In this way, the correlation between the enhanced image and the original field of view image outside the gaze area can be better maintained. In some examples, when the user's gaze point moves, the enhanced image moves synchronously with the movement of the gaze point, and the movement direction and amplitude are consistent with the gaze point, so as to maintain the enhanced image presented to the user with the user's gaze point as the center.

[0050] According to an embodiment of the present invention, the enhanced image in step S150 can be obtained by performing one or more of the following processing on the field of view sub-image and then superimposing it on the field of view image: magnification processing; edge enhancement processing; 3D embossing processing; semi-transparent or non-transparent processing; color mode reconstruction processing; and contrast reconstruction processing. Among them, for people with low vision, magnification processing is the most basic processing requirement, and other processing is often auxiliary. When the magnification processing still cannot meet the visual needs of low vision users, further consideration is given to performing other processing on the basis of the magnification processing.

[0051] Low vision users usually use visual aids such as optical aids and electronic aids to improve their vision, but these two types of visual aids do not distinguish between regions when magnifying images, that is, each part of the image is magnified at the same magnification ratio. However, some areas in the image are not areas of concern to the user or are simply meaningless areas to the user. In an embodiment of the present invention, only the visual field sub-image located in the gaze area of ​​the user's eyes is magnified, and the original visual field image located outside the gaze area is not magnified. In this way, the display content in the area (i.e., the gaze area) surrounding where the eyes see (i.e., the gaze point) is magnified and presented to the user. Therefore, the pertinence of vision assistance for low vision users has been significantly improved, and it is more in line with the actual use needs of users.

[0052] As described above, two types of known visual aids magnify images without distinguishing regions, so after these visual aids magnify the image, the visual field range viewed by the user is inversely proportional to the magnification factor, that is, the greater the magnification factor, the smaller the visual field range viewed by the user. In an embodiment of the present invention, only the visual field sub-image located in the gaze area of ​​the user's eyes is magnified, and the original visual field image located outside the gaze area is not magnified. Since the original visual field image located outside the gaze area is not magnified, for example, when the visual field sub-image is magnified and superimposed on the visual field image to form an enhanced image, the user's visual field range can remain unchanged.

[0053] In some examples, the magnification factor when the field of view sub-image is magnified in step S150 may be preset. For example, the magnification factor may be preset to 1.2 to 10 times, such as 1.2 times, 1.5 times, 2 times, 2.5 times, 3 times, 3.5 times, 4 times, 4.5 times, 5 times, 6 times, 7 times, 8 times, 9 times, 10 times, etc., but the embodiments of the present invention are not limited thereto. In other examples, the magnification factor when the field of view sub-image is magnified in step S150 may be set and adjusted by the user. For example, in a specific scenario, the magnification factor may be set and adjusted by the user according to his or her own vision condition and needs.

[0054] In an exemplary embodiment, the processing of the field of view sub-image in step S150 may also include performing one or more of edge enhancement processing, 3D embossing processing, semi-transparency or non-transparency processing, color mode reconstruction processing, and contrast reconstruction processing on the field of view sub-image.

[0055] Edge enhancement processing, for example, refers to compensating the contour of the field of view sub-image, enhancing the edge and grayscale jump part of the field of view sub-image, and making the field of view sub-image clear. Edge enhancement processing is to highlight the edges, contours, or features of certain linear image elements in the field of view sub-image. After edge enhancement processing, specific areas of the field of view sub-image can have higher clarity and brighter colors, and the overall picture of the field of view sub-image is clearer. As a result, it is easier for users to know the position of objects in the field of view sub-image in space, which improves the user's spatial perception of objects. In terms of technical effects, it is easier for users to perceive the shape / size of objects, easier for users to grab items, and easier for users to avoid obstacles.

[0056] 3D embossing is also called three-dimensional embossing or stereo embossing. For example, it means outlining the contour of the field of view sub-image and reducing the pixel value around the contour to produce an image with a three-dimensional embossing effect. The technical effect presented is also able to enhance the user's visual perception ability.

[0057] Semi-transparent or non-transparent processing refers to, for example, adjusting the transparency of the field of view sub-image. In some examples, the transparency of the field of view sub-image during transparency processing may be preset. For example, the transparency of the field of view sub-image may be set to 0% (i.e., completely opaque) or any transparency between 0% and 100% (i.e., semi-transparent or partially transparent). In other examples, the transparency may be set and adjusted by the user. When the field of view sub-image is processed as semi-transparent, the user may see through it the image content below the field of view sub-image.

[0058] For example, when the field of view sub-image is adjusted to be semi-transparent (or partially transparent) and superimposed on the field of view image to provide an enhanced image, the user can see the original field of view image (e.g., the original field of view sub-image) located in the gaze area through the semi-transparent field of view sub-image in the enhanced image. In addition, when the field of view sub-image is processed to be semi-transparent, and for example, when the processed field of view sub-image is superimposed on the field of view image to form an enhanced image, the user can see all image content within the field of view, so that the field of view of the user is not affected.

[0059] The color mode reconstruction process, for example, refers to adjusting the color mode of the visual field sub-image within the gaze area to match the actual vision condition of the user. For example, for a red-green color blind user, the color mode reconstruction process can replace the red and green colors in the visual field sub-image with colors that the user can distinguish, such as yellow and blue. Through the color mode reconstruction process, the user can see the content of the visual field sub-image more clearly, thereby providing effective vision assistance for low-vision users.

[0060] Contrast reconstruction processing, for example, refers to contrast enhancement processing of the visual field sub-image within the gaze area to match the actual vision condition of the user. In contrast enhancement, by adjusting the grayscale distribution of the image, the distribution range of the image grayscale is increased to improve the contrast of the entire or part of the image and improve the visual effect. By contrast enhancement processing of the visual field sub-image, the user can see the content of the visual field sub-image more clearly, thereby providing effective vision assistance for low-vision users.

[0061] In an exemplary embodiment, when processing the field of view sub-image in step S150, one of a magnification process, an edge enhancement process, a 3D relief process, a semi-transparent or non-transparent process, a color mode reconstruction process, and a contrast reconstruction process may be performed on the field of view sub-image. In some examples, the field of view sub-image may be magnified only. In some examples, the field of view sub-image may be edge enhanced only. In some examples, the field of view sub-image may be 3D relief only. In some examples, the field of view sub-image may be semi-transparent or non-transparent only. In some examples, the field of view sub-image may be color mode reconstructed only. In some examples, the field of view sub-image may be contrast reconstructed only.

[0062] In an exemplary embodiment, when processing the field of view sub-image in step S150, the magnification process, edge enhancement process, 3D embossing process, semi-transparent or non-transparent process, color mode reconstruction process, and contrast reconstruction process can be combined with each other. That is, two or more of the above processes can be performed on the field of view sub-image and then superimposed on the field of view image to obtain an enhanced image.

[0063] In an exemplary embodiment, when the field of view sub-image is processed in step S150, the field of view sub-image may be magnified, and one of edge enhancement, 3D embossing, semi-transparent or non-transparent processing, color mode reconstruction, and contrast reconstruction may be performed on the field of view sub-image. In some examples, the field of view sub-image may be magnified and edge enhanced. In some examples, the field of view sub-image may be magnified and 3D embossing may be performed. In some examples, the field of view sub-image may be magnified and semi-transparent or non-transparent. In some examples, the field of view sub-image may be magnified and color mode reconstruction may be performed. In some examples, the field of view sub-image may be magnified and contrast reconstruction may be performed.

[0064] In an exemplary embodiment, when the field of view sub-image is processed in step S150, the field of view sub-image may be enlarged, and two or more of edge enhancement processing, 3D embossing processing, semi-transparent or non-transparent processing, color mode reconstruction processing, and contrast reconstruction processing may be performed on the field of view sub-image.

[0065] In an exemplary embodiment, when the field of view sub-image is processed in step S150, the field of view sub-image may be enlarged and made semi-transparent or non-transparent, and may also be subjected to one or more of edge enhancement, 3D embossing, color pattern reconstruction, and contrast reconstruction.

[0066] According to an embodiment of the present invention, the vision assistance method may further include: processing the field of view image and the field of view sub-image so that the area where the field of view sub-image with enhanced visual effect is located in the enhanced image has a color pattern or contrast pattern that matches the user's vision condition with other areas. In an embodiment of the present invention, the color pattern or contrast between the field of view sub-image located in the gaze area and the field of view image located outside other gaze areas is adjusted to match the user's actual vision condition. For example, the color between the two is switched to a color that is convenient for the user to distinguish the difference. For example, the contrast between the two is switched to a contrast that is convenient for the user to clearly recognize the field of view sub-image in the gaze area.

[0067] According to an embodiment of the present invention, the vision assistance method may further include: identifying whether the visual field image contains preset image content, and when the visual field image contains preset image content, reminding the user of the corresponding position of the preset image content. The preset image content may include objects that may cause danger to the user, objects that the user is looking for, or objects that the user is familiar with. That is, the vision assistance method according to an embodiment of the present invention also provides a recognition function. The recognition function is oriented to the entire visual field area. When objects that may cause danger, such as red lights and shuttling vehicles, are identified in the visual field area, a reminder is issued to the user, for example, the user is reminded of the location of the object. When objects that need to be found, such as bus stops, taxi license plate numbers, and specific items to be purchased in supermarkets, are identified in the visual field area, a reminder is issued to the user, for example, the user is reminded of the location of the object. When a familiar relative or friend is identified, a reminder is issued to the user. After the above-mentioned object is identified, the user is reminded, for example, in the form of a flashing red dot, so that the user moves the gaze point to the flashing red dot for further confirmation. According to this embodiment, the visual assistance effect for the user can be significantly improved, and the safety and practicality are improved.

[0068] According to an embodiment of the present invention, the vision assistance method may further include: identifying whether the visual field sub-image includes text content, and when the visual field sub-image includes text content, automatically playing the text content in a voice manner. That is, the vision assistance method according to an embodiment of the present invention also provides a voice broadcast function. When the visual field sub-image located in the gaze area includes text, the text content can be notified to the user in a voice manner through the voice broadcast function.

[0069] According to an embodiment of the present invention, in step S120, determining the gaze point of the user in the field of view image may include using an eye tracking method to track the line of sight of the user's eyes when viewing the display screen or the outside world, thereby determining the gaze point of the user's eyes on the display screen or the field of view image. In the prior art, when a user needs to observe an image of a specific area, it is necessary to manually adjust the components used for image capture on the visual aid. For example, if the user is using an optical visual aid, it is necessary to manually adjust the orientation of the magnifying glass and the telescope to obtain the image of the desired observation area; if the user is using an electronic visual aid, it is necessary to manually adjust the orientation of the camera in the electronic visual aid so that the camera is facing the desired observation area. In an embodiment of the present invention, an eye tracking method is used to determine the gaze point of the user's eyes, so that the user does not need to make any manual adjustments, thereby freeing the user's hands.

[0070] According to an embodiment of the present invention, the field of view image may come from an image capture device or a video signal interface. In some examples, the field of view image may come from an image capture device, that is, the field of view image is captured by using an image capture device. The image capture device is generally an imaging device with a lens, and is often used in scenes where the field of view image needs to be acquired in real time, such as outdoor activities. In some examples, the field of view image may come from a video signal interface, that is, the field of view image is obtained from a video signal interface. The video signal interface is generally an interface that can transmit image signals in a wired form. For example, the field of view image is acquired from a video interface on a computer or mobile phone.

[0071] According to an embodiment of the present invention, in step S130, determining the user's gaze area according to the gaze point may include: determining the gaze area with the gaze point as the center. That is, the center of the gaze area coincides with the gaze point. In some examples, the gaze area may have a circular or square shape. In some examples, the gaze area may have the same shape as the outline of the visual field image, that is, the same shape as the display screen for displaying the visual field image (see Figure 4 ). For example, when the display screen for displaying the visual field image has a rectangular shape, the gaze area also has a rectangular shape and has the same aspect ratio as the display screen. In the embodiment of the present invention, the size of the gaze area may be preset.

[0072] According to an embodiment of the present invention, when the field of view image is continuous and constitutes a field of view image video data stream, an enhanced image video data stream composed of enhanced images is presented to the user at the same frequency and order. According to this embodiment, when the user sees the field of view image as a video data stream, the corresponding enhanced image video data stream is presented to the user at the same frequency and order as the video data stream. That is, the user can use the technical solution of the present invention to watch a single static picture or a video. In fact, watching a video is a more common implementation method. In this way, the image content in the gaze area can be presented to the user in real time and dynamically.

[0073] According to an embodiment of the present invention, the vision assistance method may further include: identifying whether the user's voice input corresponds to a preset instruction, and when the voice input corresponds to the preset instruction, performing an operation corresponding to the preset instruction. In some examples, the operation corresponding to the preset instruction may be setting and adjusting the magnification when the field of view sub-image is enlarged. In other examples, the operation corresponding to the preset instruction may be setting and adjusting the transparency when the field of view sub-image is processed.

[0074] Although not shown, according to an embodiment of the present invention, there is also provided a storage medium on which a computer program is stored. When the computer program is executed by a processor, the vision assistance method according to the embodiment of the present invention described above is implemented.

[0075] Figure 2 and Figure 3 A block diagram of a vision assistance device according to an embodiment of the present invention is schematically shown.

[0076] like Figure 2 As shown, according to an embodiment of the present invention, the vision aid device 1 includes a gaze point determination module 10, an image processing module 20 and a display module 30. The gaze point determination module 10 is used to determine the gaze point of the user. The image processing module 20 is connected to the gaze point determination module 10 and includes the storage medium according to the embodiment of the present invention described above, and is used to execute the computer program stored on the storage medium. The display module 30 is connected to the image processing module and is used to present the enhanced image to the user.

[0077] According to an embodiment of the present invention, the gaze point determination module 10 may be an eye tracking device. The gaze point determination module 10 uses the eye tracking device to determine the gaze point of the user's eyes, so that the user does not need to make any manual adjustments, thereby freeing the user's hands.

[0078] like Figure 3 As shown, according to an embodiment of the present invention, the vision assist device 1 may further include a field of view image acquisition module 40. The field of view image acquisition module 40 is connected to the image processing module 20, and the field of view image acquired by the field of view image acquisition module 40 is sent to the image processing module 20. In some examples, the field of view image acquisition module 40 may be an image capture device. In some examples, the field of view image acquisition module 40 may be a video signal interface.

[0079] According to an embodiment of the present invention, when the visual field image acquisition module 40 is an image capture device, the vision aid device 1 can be a head-mounted vision aid device. The head-mounted vision aid device is convenient for use in the daily environment when the user goes out. It should be noted that the head-mounted vision aid device can also be used in fixed use places such as home and work.

[0080] In the following, the structure and working mode of the vision aid device 1 are described by taking the vision aid device 1 as a head-mounted vision aid device and the processing of the field of view sub-image as a magnification processing as an example.

[0081] In the embodiment where the vision aid device 1 is a head-mounted vision aid device (the present embodiment takes the glasses type as an example, but the present invention is not limited thereto), the gaze point determination module 10 may be an eye tracking device, the display module 30 may be a near-eye display, and the field of view image acquisition module 40 may be an image capture device such as a camera. The vision aid device 1 may also include a head-mounted display device body. The head-mounted display device body may have a shape and size that matches the face and head of a person to achieve close fit and isolate external light from entering, or the head-mounted display device body may be used in conjunction with an eye mask, one side of the eye mask may be tightly fitted with the head-mounted display device body, and the other side of the eye mask may be fitted with the skin around the human eye. The eye mask is made of a dark or black light-isolating material, and can also isolate external light from entering in terms of technical effect, and the eye mask can be replaced. The image capture device is arranged in the front part of the head-mounted display device body to obtain a field of view image for the user. The eye tracking device is installed inside the head-mounted display device body, facing the user, and is used to track the line of sight of the user's eyes and determine the gaze direction of the user's eyes. The near-eye display can be installed inside the body of the head-mounted display device and face the user's eyes to present the processed enhanced image to the user.

[0082] The eye tracking device and the image capture device are both connected to the image processing module 20. The image processing module 20 determines the user's gaze point in the visual field image according to the gaze direction of the user's eyes tracked by the eye tracking device, and amplifies the visual field image part within the gaze area centered on the gaze point according to a preset magnification ratio, while other areas of the visual field image are not amplified. The enhanced image processed by the image processing module 20 is sent to the near-eye display for display.

[0083] As described above, the fixation area may be circular or square, or have the same shape as the outline of the visual field image. The fixation area may have a preset size.

[0084] The image processing module 20 may include, for example, an image extraction unit, an image magnification unit, and an image synthesis unit. The image extraction unit extracts a gaze area around the gaze point according to the gaze point tracked by the eye tracking device. The image magnification unit magnifies the field of view sub-image located in the gaze area. The image synthesis unit superimposes the magnified field of view sub-image on the original field of view image. The superposition method may be, for example, a transparent superposition or a non-transparent superposition. For example, the user may select a transparency of 0% (i.e., completely opaque), or select a suitable transparency between 0% and 100% (i.e., semi-transparent or partially transparent).

[0085] According to an embodiment of the present invention, the vision aid device 1 may further include a visual reminder module 50. For example, when the image processing module 20 determines that the visual field image contains preset image content, the visual reminder module 50 reminds the user of the corresponding position of the preset image content.

[0086] As described above, the head-mounted vision aid device can be used in the daily environment when the user is out, and the user may encounter objects that may cause danger to the user, such as red lights and passing vehicles. In this case, the visual reminder module 50 sends a reminder to the user, such as reminding the user of the location of the object.

[0087] In the daily environment, users may also need to find some specific objects, such as bus stops, taxi license plate numbers, specific items to be purchased in supermarkets, etc. When the image processing module 20 determines that the field of view image contains these objects, the visual reminder module 50 will also remind the user of the corresponding position of the object.

[0088] In addition, in the external daily environment, when the image processing module 20 recognizes a familiar relative or friend, the visual reminder module 50 will also issue a reminder to the user.

[0089] After the object or subject is identified, the user is reminded by, for example, a flashing red dot on the display module 30 such as a near-eye display, so that the user moves his or her gaze point to the flashing red dot for further confirmation. The vision aid device 1 according to this embodiment can significantly improve the vision aid effect for the user, and improve safety and practicality.

[0090] According to an embodiment of the present invention, when the visual field image acquisition module 40 is a video signal interface, the vision aid device 1 is a display screen type vision aid device. Display screen type vision aid devices are usually used when viewing a display screen at fixed places such as home and work.

[0091] In the following, the structure and working mode of the vision aid device 1 are described by taking the vision aid device 1 as a display screen type vision aid device and the processing of the field of view sub-image as a magnification processing as an example.

[0092] Figure 5 FIG. 5 is a schematic diagram of a display screen type vision aid device 500 according to an embodiment of the present invention. Figure 5As shown, in an embodiment where the vision aid device 1 is a display screen type vision aid device 500, the display screen type vision aid device 500 includes an eye tracking device 510 and a display screen 520. The display module 30 is the display screen 520, and the gaze point determination module 10 is the eye tracking device 510 and is arranged below the display screen 520. The eye tracking device 510 is used to track the line of sight of the user's eyes and determine the gaze point of the user's eyes on the display screen 520. The vision aid device 1 also includes an image processing module 20 (not shown).

[0093] The display screen here is not limited to the display screen of a desktop computer or a laptop computer, but can be a whole display screen, a spliced ​​display screen, or a display device that uses image signals to drive related images to present images in background boards, cinemas, and other places in any place and occasion.

[0094] The image processing module 20 is a video signal interface that receives the original image. The original image here can be understood as the image signal required by the display screen to display the image, and the original image is not actually displayed on the display screen. In other words, the image processing module 20 "intercepts" the image signal required by the display screen to display the image. The image processing module 20 processes the original image displayed on the display screen according to the following rules: the image part within the gaze range centered on the gaze point is enlarged according to a preset enlargement ratio, while other areas of the original image are not enlarged. The image processed by the image processing module 20 is sent to the display screen for display.

[0095] In this embodiment, the display screen type vision aid device is used in a fixed place of use at home or work rather than in an external environment where the user is outdoors, so the vision aid device 1 may not include the visual reminder module 50 described above.

[0096] According to an embodiment of the present invention, the vision aid device 1 may further include a voice playback module 60. When the visual field sub-image includes text content, the voice playback module 60 automatically plays the text content in a voice manner. In an embodiment of the present invention, the voice playback module 60 may include a voice library submodule storing voice data and a speech rate adjustment submodule for adjusting the speech rate of automatic playback.

[0097] According to an embodiment of the present invention, the vision assist device 1 may further include a voice interaction module 70. The voice interaction module 70 may have a voice instruction library and be configured to recognize the voice input by the user and convert the voice input by the user into an instruction that can be executed by the image processing module 20. In some examples, the instruction may be to set and adjust the magnification when enlarging the field of view sub-image. In other examples, the instruction may be to set and adjust the transparency when processing the transparency of the field of view sub-image.

[0098] The vision assistance method, storage medium and vision assistance device according to the embodiments of the present invention have the following beneficial technical effects. According to the technical solution of the embodiments of the present invention, wherever the user's eyes look, the display content in the area surrounding the eyes is visually enhanced and presented to the user. This significantly improves the pertinence of vision assistance for low-vision users and is more in line with the actual use needs of users. The user does not need to make any manual adjustments throughout the process, thereby freeing the user's hands. In addition, the user's field of view remains unchanged.

[0099] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.

Claims

1. A vision assistance method, comprising: presenting a visual field image to a user; Determining a gaze point of a user in the field of view image; Determine a user's gaze area according to the gaze point; Extracting the visual field image within the gaze area as a visual field sub-image; Processing the field of view sub-image to obtain an enhanced image with enhanced visual effect; as well as The enhanced image is presented to a user.

2. The vision assistance method according to claim 1, wherein: The enhanced image is obtained by superimposing the visual effect of the field of view sub-image on the field of view image.

3. The vision assistance method according to claim 2, wherein: The enhanced image is centered on the gaze point.

4. The vision assistance method according to any one of claims 1 to 3, wherein: The enhanced image is obtained by performing one or more of the following processing on the field of view sub-image and then superimposing it on the field of view image: Amplification processing; Edge enhancement processing; 3D relief processing; Translucent or opaque treatment; Color model reconstruction processing; and Contrast reconstruction processing.

5. The vision assistance method according to claim 4, wherein: The magnification factor during the magnification process is preset and / or set and adjusted by the user.

6. The vision assistance method according to claim 4, wherein: The transparency of the semi-transparent or non-transparent processing is preset and / or set and adjusted by the user.

7. The vision assistance method according to any one of claims 1 to 3, further comprising: The visual field image and the visual field sub-image are processed so that a color pattern or a contrast pattern matching the user's vision condition exists between the region where the visual field sub-image with enhanced visual effect is located and other regions in the enhanced image.

8. The vision assistance method according to claim 1, further comprising: It is identified whether the field of view image includes preset image content, and when the field of view image includes the preset image content, a user is reminded of a corresponding position of the preset image content.

9. The vision assistance method according to claim 8, wherein: The preset image content includes objects that may be dangerous to the user, objects that the user is looking for, or objects that the user is familiar with.

10. The vision assistance method according to claim 1, further comprising: It is identified whether the field of view sub-image includes text content, and when the field of view sub-image includes text content, the text content is automatically played in a voice manner.

11. The vision assistance method according to claim 1, wherein: The visual field image comes from an image capture device or a video signal interface.

12. The vision assistance method according to claim 1, wherein: The gaze area is centered on the gaze point, has a circular shape, a square shape, or a shape that is the same as the outline of the field of view image, and has a preset size.

13. The vision assisting method according to claim 1, wherein: When the field of view images are continuous and constitute a field of view image video data stream, the enhanced image video data stream constituted by the enhanced images is presented to the user at the same frequency and order.

14. The vision assistance method according to claim 1, further comprising: It is identified whether the user's voice input corresponds to a preset instruction, and when the voice input corresponds to the preset instruction, an operation corresponding to the preset instruction is performed.

15. A storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the vision assistance method according to any one of claims 1 to 14 is implemented.

16. A vision aid device, comprising: A gaze point determination module, used to determine the user's gaze point; an image processing module, connected to the gaze point determination module, comprising a storage medium according to claim 15, and configured to execute a computer program stored on the storage medium; as well as The display module is connected to the image processing module and is used to present the enhanced image to the user.

17. The vision aid device according to claim 16, wherein: The gaze point determination module is an eye tracking device.

18. The vision assist device according to claim 17, further comprising: The visual field image acquisition module is an image capture device or a video signal interface and is connected to the image processing module.

19. The vision aid device according to claim 18, wherein: When the field of view image acquisition module is an image capture device, the vision aid device is a head-mounted vision aid device, the field of view image acquisition module is arranged directly in front of the head-mounted vision aid device, the eye tracking device is installed inside the head-mounted vision aid device, and the display module is a near-eye display, which is installed inside the head-mounted vision aid device and faces the user's eyes.

20. The vision assist device according to claim 19, further comprising: The visual reminder module is used to remind the user of the corresponding position of the preset image content when the field of view image contains the preset image content.

21. The vision aid device according to claim 18, wherein: When the visual field image acquisition module is a video signal interface, the display module is a display screen, and the eye tracking device is arranged below the display screen.

22. The vision assist device of claim 18, further comprising: A voice playing module is used to automatically play the text content in voice mode when the field of view sub-image includes text content. The voice playing module includes a voice library submodule storing voice data and a speech speed adjustment submodule for adjusting the speech speed of automatic playing.

23. The vision assist device of claim 18, further comprising: A voice interaction module has a voice instruction library and is configured to recognize the voice input by the user and convert it into an instruction executable by the image processing unit.

Images