Vision assistance system

Abstract

To provide a visual field auxiliary system which can assist a visual field of a handicapped person who has defective eyes.SOLUTION: A visual field auxiliary system includes: at least one photographing section which can continuously photograph a visual field image corresponding to a visual field of a user; a control section which can generate an image on the basis of the visual field image; a projection section which projects the image on at least one projected section; and a body section which holds the projected section in front of the eyes of the user when the user wears the system. When the control section determines that a preset specific object is photographed on the basis of the visual field image, the control section generates a specific image including the image decorating the specific object, and projects the specific image on the projected section through the projection section.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a vision assistance system using mixed reality that makes information in a virtual world appear to exist in the real world, and more particularly to a vision assistance system that assists the vision of a person with a visual impairment who has an obstacle in their vision.

Background Art

[0002] Conventionally, a vision assistance system includes, for example, a glasses-type main body that holds a lens body in front of the user's eyes when worn by the user, and a control terminal connected to the glasses-type main body. The user can project an image onto the lens body by wearing the main body and operating the control terminal, and can obtain information from the background viewed through the lens body and the image projected onto the lens body and superimposed on the background (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the vision assistance system as described above, for example, it is desired to assist the vision of a person with a visual impairment whose vision is blurred at the edges or the entire vision is blurred, and there is room for improvement in this regard.

[0005] The present invention has been made in view of such circumstances, and an object thereof is to provide a vision assistance system that can assist the vision of a person with a visual impairment who has an obstacle in their vision.

Means for Solving the Problems

[0006] The following are effective solutions for achieving the above-mentioned objectives. Explanations of their operation and other aspects will be provided as needed. Furthermore, for ease of understanding, corresponding configurations and other aspects in the embodiments of the invention will be shown as appropriate, but these are not limiting in any way.

[0007] To achieve the above-mentioned objective, the invention according to claim 1 of the present invention is: User's perspective The world At least one imaging unit capable of continuous shooting, Contents of the camera unit Based prescribed A control unit capable of generating images, The aforementioned prescribed A projection unit that projects an image onto at least one projection surface, When the user wears the device, the projection area The aforementioned A vision assistance system comprising a main unit held in front of the user's eyes, The system further includes an audio input unit that inputs the user's voice to the control unit, The control unit is the Filming content of the photography department Based identification A specific image is generated that includes an image informing the user that the object is within the user's field of view, and the object is projected via the projection unit. department Okeru The aforementioned Specific object Corresponding The specific image is projected onto the position. The function to enable, The system has a function to generate a specific image, which includes an image that informs the user of the location of an object corresponding to the specific sound, when a specific sound is input based on the aforementioned sound, and to project the specific image via the projection unit to the location on the projection unit corresponding to the object corresponding to the specific sound. A visual assistance system characterized by the following features.

[0008] According to the visual assistance system of the present invention as described in claim 1 above, the control unit is Filming content of the photography department Based identification A specific image is generated that includes an image informing the user that an object is within the user's field of view, and this image is projected via the projection unit. department A specific object in Corresponding A specific image is projected onto the location. It has the function ofTherefore, by shifting their gaze toward a specific image, the user can move a specific object to the center of their field of view where the haze is minimal, and consequently, the user can see the specific object in the center of their field of view, thus assisting the vision of people with visual impairments (see, for example, paragraphs

[0035] to

[0040] ). In addition, the vision assistance system according to claim 1 of the present invention further comprises a voice input unit that inputs the user's voice to a control unit, and the control unit has the function of generating a specific image including an image that notifies the user of the position of an object corresponding to the specific voice when a specific voice is input based on the voice, and projecting the specific image via a projection unit to the position corresponding to the object corresponding to the specific voice on the projection unit, so that the user can move their gaze toward the specific image and move the object corresponding to the specific voice to the center of their field of view, and thereby the user can see the object corresponding to the specific voice in the center of their field of view, and the vision assistance can be provided to persons with visual impairments (see, for example, paragraphs

[0041] to

[0052] ).

[0009] To achieve the above objective, the invention according to claim 2 of the present invention is: The control unit is in the presence of the specific object in front of the user. case The process generates a specific image that includes an image of a frame surrounding the specific object, and the frame is generated according to the pre-set importance of the specific object. Image Make the colors different. The visual assistance system according to feature 1.

[0010] According to the vision assistance system of the invention described in claim 2 above, the control unit detects when a specific object is in front of the user. case In addition, a specific image is generated that includes an image of a frame surrounding a specific object, and the frame is generated according to the pre-set importance of the specific object. Image To make the colors different, for example, multiple When a specific object is photographed, the user can move their viewpoint towards a frame of higher importance according to the color of the frame, thereby moving the specific object of higher importance (e.g., a traffic light or something approaching the user at a speed greater than a preset speed) to the center of the field of view where the haze is less pronounced. In turn, the user can visually confirm the specific object of higher importance in the center of their field of view, thus assisting the vision of people with visual impairments (see, for example, paragraphs

[0035] to

[0040] ).

[0011] To achieve the above objective, the invention according to claim 3 of the present invention is: The control unit is in the presence of the specific object in front of the user. case The specific range including the aforementioned specific object has been expanded. Includes images The specific image is generated by Characterized by Claim 1 The described vision assistance system.

[0012] According to the vision assistance system of the invention according to claim 3 of the present invention, when there is a specific object in front of the user case a specific range including the specific object is enlarged Includes images to generate a specific image. For example, when the user is a person with a visual impairment having fog over the entire field of view, by moving the viewpoint toward the specific image in which a specific range including the specific object is enlarged, the specific range including the specific object can be moved to the central part of the field of view. As a result, the user can visually recognize the enlarged specific object, and the vision of a person with a visual impairment can be assisted (see, for example, paragraphs

[0041] to

[0052] ).

[0013] In order to achieve the above object, the invention according to claim 4 of the present invention is when the specific object is a step case the control unit generates the specific image including a linear image along the corner of the step, characterized in that Claim 1 the described vision assistance system.

[0014] According to the vision assistance system of the invention according to claim 4 of the present invention, when the specific object is a step case the control unit generates the specific image including a linear image along the corner of the step. For example, the user can easily recognize that there are steps around, and by moving the viewpoint toward the linear image, the step can be moved to the central part of the field of view. As a result, the user can visually recognize the surrounding steps and suppress accidents such as tripping over the steps, and the vision of a person with a visual impairment can be assisted (see, for example, paragraphs

[0066] to

[0069] ).

[0015] In order to achieve the above object, the invention according to claim 5 of the present invention is when the specific object is a character or a number case the control unit generates the specific image including an image corresponding to the character or the number, Characterized by Claim 1 The described visual assistance system.

[0016] According to the visual assistance system of the invention described in claim 5 of the present invention, the control unit determines that a specific object is a letter or a number. case In order to generate a specific image that includes an image corresponding to a character or number, the user can move their gaze toward the specific image, thereby moving the character or number, which is the specific object, to the center of their field of vision, and consequently, the user can see the character or number in the center of their field of vision, thereby assisting the vision of a person with a visual impairment (see, for example, paragraphs

[0057] to

[0064] ).

[0019] To achieve the above objective, the present invention Claim 6 The invention relating to this is It further includes an audio output section for outputting audio externally. The control unit detects when the specific object is in front of the user. Something To report special The audio is generated and transmitted via the audio output unit. special Output audio to an external source. It has even more functions , Features 1 to Claim 5 A visual assistance system as described in any one of the items.

[0020] The above invention Claim 6 The vision assistance system of the invention further comprises an audio output unit that outputs sound to the outside, and a control unit that detects a specific object in front of the user. Something To report special It generates sound and outputs it via the sound output unit. special Output audio to an external source. It has even more functions Therefore, information about a specific object is obtained before the user moves their gaze toward the specific image. special The user can be notified by voice, and the user special After checking the audio special Move the viewpoint towards a specific image in response to the sound. special The object corresponding to the sound can be moved to the center of the field of view, and consequently, the user can specialThe object corresponding to the sound can be seen in the center of the field of vision, and the vision of people with visual impairments can be assisted (see, for example, paragraphs

[0041] to

[0052] ). [Effects of the Invention]

[0021] Thus, the vision assistance system of the present invention can assist the vision of disabled persons with visual impairments. [Brief explanation of the drawing]

[0022] [Figure 1] This is a perspective view showing the external appearance of the visual assistance system of this embodiment. [Figure 2] This is a block diagram of the visual assistance system of this embodiment. [Figure 3] (a) is a diagram showing the field of view of a healthy person as seen through a lens, and (b) is a diagram showing the field of view of a person with a disability as seen through a lens. [Figure 4] (a) is a diagram showing the field of view of a person with a disability as seen through a lens body onto which an image decorating a specific object is projected, and (b) is a diagram showing the field of view of a person with a disability as seen through the lens body after shifting their viewpoint toward the image decorating a specific object. [Figure 5] (a) is a diagram showing the field of view of a healthy person as seen through a lens, and (b) is a diagram showing the field of view of a person with a disability as seen through a lens. [Figure 6] (a) is a diagram showing the field of view of a person with a disability as seen through a lens body onto which an image decorating a specific object is projected, and (b) is a diagram showing the field of view of a person with a disability as seen through a lens body after the specific object has been modified. [Figure 7] (a) is a diagram showing the field of view of a healthy person as seen through a lens, and (b) is a diagram showing the field of view of a person with a disability as seen through a lens. [Figure 8] (a) is a diagram showing the field of view of a healthy person as seen through a lens, and (b) is a diagram showing the field of view of a person with a disability as seen through a lens. [Figure 9]This diagram shows the field of view of a person with a disability as seen through a lens onto which an image decorating a specific object is projected. [Figure 10] (a) is a diagram showing the field of view of a healthy person as seen through a lens, and (b) is a diagram showing the field of view of a person with a disability as seen through a lens. [Figure 11] (a) is a diagram showing the field of view of a healthy person as seen through a lens, and (b) is a diagram showing the field of view of a person with a disability as seen through a lens. [Modes for carrying out the invention]

[0023] [1. Overall configuration of the visual assistance system] The vision assistance system 1 of this embodiment will be described in detail with reference to the drawings. Figure 1 is a perspective view showing the external appearance of the vision assistance system 1 of this embodiment, and more specifically, it is a perspective view showing the external appearance of the vision assistance system 1 with the glasses-type main unit 2 and the control terminal 10 connected by a wire. Figure 2 is a block diagram showing the vision assistance system 1 of this embodiment.

[0024] As shown in Figure 1 or Figure 2, the vision assistance system 1 comprises a spectacle-type main unit 2 and a control terminal 10 connected to the main unit 2 via a connecting wire 8. The spectacle-type main unit 2 consists of a main frame 2a that holds the left and right lens bodies 3 in front of the user's eyes, and left and right subframes 2b that extend rearward from the left and right ends of the main frame 2a. The user can wear the main unit 2 by engaging the left and right subframes 2b with the upper parts of their left and right ears, thereby positioning the left and right lens bodies 3 in front of their left and right eyes. The subframes 2b may be configured to fold inward relative to the main frame 2a, which allows the main unit 2 to be made smaller when not being worn, for example, improving the convenience of carrying the main unit 2.

[0025] The main unit 2 includes left and right lens bodies 3 held in the main frame 2a, left and right depth cameras 4 provided on the upper outer surfaces of the left and right lens bodies 3, an RGB camera 4a provided on the upper inner surface of the right lens body 3, a projection unit 5 provided on the top of the main frame 2a, a microphone 6 provided in the center of the main frame 2a, left and right speakers 7 provided on the inside of the left and right subframes 2b, and a connecting line 8 provided at the end of one of the subframes 2b that connects the depth camera 4, RGB camera 4a, projection unit 5, microphone 6, speaker 7 and control terminal 10. The main unit 2 may employ, for example, mixed reality glasses such as Nreallight®. Furthermore, the main unit 2 may also be equipped with a battery to supply power to the depth camera 4, RGB camera 4a, projection unit 5, microphone 6, and speaker 7, and the connection between the depth camera 4, RGB camera 4a, projection unit 5, microphone 6, and speaker 7 and the control terminal 10 may be made wirelessly using Bluetooth® or the like instead of the connection cable 8.

[0026] The left and right lens bodies 3 are positioned in front of the user's eyes when the user wears the main unit 2, and the user can secure a field of view through the light-transmitting left and right lens bodies 3. The projection unit 5 projects images onto the left and right lens bodies 3 based on image information output from the control terminal 10, and the images projected onto the left and right lens bodies 3 can be viewed by the user through the lens bodies 3. The depth camera 4 is capable of capturing depth information in addition to vertical and horizontal information, and continuously captures images at preset time intervals, inputting the captured images as image information to the control terminal 10. The RGB camera 4a is a so-called digital camera, and like the depth camera 4, continuously captures images at preset time intervals, inputting the captured images as image information to the control terminal 10. The microphone 6 collects ambient sound and inputs the collected sound as audio information to the control terminal 10. The speaker 7 outputs sound to the outside based on the audio information output from the control terminal 10. Alternatively, instead of the microphone 6 and speaker 7 provided on the main unit 2, the control terminal 10 may be provided with the microphone 6 and speaker 7, or a headset having a microphone and speaker that is worn on the user's head or ears may be used. If a headset is used, the headset may be connected to the control terminal 10 by wire or wireless, or it may be connected to the control terminal 10 via the main unit 2 by wire.

[0027] The control terminal 10 comprises an outer frame 10a, a control unit 11 and a battery 12 located within the outer frame 10a, and a touch panel 13 attached to the outer frame 10a. The control terminal 10 may be a general-purpose terminal such as a smartphone, tablet, or notebook computer, or it may be a dedicated terminal compatible only with the vision assistance system 1. In this embodiment, a smartphone is used as the control terminal 10. The control unit 11 receives image information from the depth camera 4 and RGB camera 4a via the connection line 8 and outputs the image information to the projection unit 5. The control unit 11 also receives audio information from the microphone 6 via the connection line 8 and outputs the audio information to the speaker 7. The battery 12 supplies power to the depth camera 4, RGB camera 4a, projection unit 5, microphone 6, and speaker 7 via the connection line 8. The touch panel 13 allows the user to input setting information to the control unit 11 by operating it. The control terminal 10 may be incorporated into a glasses-type main unit 2 when further miniaturization is required.

[0028] Figure 3(a) shows the field of view of a healthy person as seen through the lens body 3, and Figure 3(b) shows the field of view of a person with a disability as seen through the lens body 3. As shown in Figure 3(a), the field of view of a healthy person as seen through the lens body 3 is formed only by a clear region X centered on the viewpoint P, free from obstructions such as haze, making it easy to distinguish objects within the field of view (e.g., traffic signals and signs). Therefore, conventionally, by using the field of view assistance system 1 having the main unit 2 and the control terminal 10 described above, the user can wear the main unit 2, operate the control terminal 10, project a desired image onto the lens body 3, and obtain multiple pieces of information from the background seen through the lens body 3 and the image projected onto the lens body 3 superimposed on the background. Specifically, for example, multiple pieces of information can be obtained simultaneously, such as incoming call information projected onto the lens body 3 and displayed at the upper end of the field of view, information that there is a pedestrian signal seen through the lens body 3, and information about the color of the signal.

[0029] However, as shown in Figure 3(b), the field of view of a person with a disability viewed through the lens body 3 has a clear area X in the central part centered on the viewpoint P, where there is no obstruction such as haze. However, outside of the clear area X, there is an unclear area Y with obstruction such as haze, making it difficult to easily distinguish objects in the unclear area Y (e.g., traffic lights or signs) or their state (e.g., the color of a traffic light or the characters on a sign). For this reason, the field of view assistance system 1 having the main body 2 and the control terminal 10 described above is desired to assist the field of view of a person with a disability who has an obstruction Y due to an eye impairment, such as haze at the edges of the field of view or haze throughout the entire field of view. The field of view assistance system 1 of this embodiment will be described below with the aim of solving these problems.

[0030] The control unit 11 of the visual assistance system 1 of this embodiment includes a storage unit 11a that stores pre-inputted setting information, image information, audio information, etc., an image determination unit 11b that determines the presence or absence of a specific object in the visual field image based on image information input from the depth camera 4 and RGB camera 4a and image information stored in the storage unit 11a, an image generation unit 11c that generates an image to decorate the specific object by referring to the image information stored in the storage unit 11a when the image determination unit 11b determines that there is a specific object in the visual field image, an audio determination unit 11d that determines the presence or absence of a specific audio input based on audio information input from the microphone 6 and audio information stored in the storage unit 11a, and an audio generation unit 11e that generates specific audio information to be output to the speaker 7 by referring to the audio information stored in the storage unit 11a when the image determination unit 11b determines that there is a specific object in the visual field image or when the audio determination unit 11d determines that there is a specific audio input.

[0031] In this embodiment, the image determination unit 11b generates sequence information by sequentially converting image information (field of view image) input from the depth camera 4 and RGB camera 4a into a sequence of numbers, and determines whether or not there are parts in the sequence information that approximate the sequence information of a specific object (for example, a signal or sign) stored in the storage unit 11a. For example, if the generated sequence information contains sequence information of a specific object at a rate of 70% or more, the image determination unit 11b determines that a specific object has been captured in the image information input from the depth camera 4 and RGB camera 4a. Furthermore, when the image determination unit 11b determines that a specific object has been captured in the image information input from the depth camera 4 and RGB camera 4a, it outputs information about the type of the specific object and information about the position of the specific object in the field of view image to the image generation unit 11c. Furthermore, the image determination unit 11b may, for example, output the sequence of numbers it has determined to be a specific object to the storage unit 11a as the sequence of numbers of the specific object, and additionally store the sequence of numbers of the specific object in the storage unit 11a. As a result, by learning from determining multiple pieces of image information, the accuracy of determining whether or not a specific object is captured in the image information input from the depth camera 4 or the RGB camera 4a can be improved.

[0032] The voice determination unit 11d determines whether a specific voice stored in the storage unit 11a has been input, based on the voice information sequentially input from the microphone 6. When the voice determination unit 11d determines that a specific voice has been input, it outputs information corresponding to that specific voice to the image generation unit 11c and the voice generation unit 11e. Specific examples of information corresponding to a specific voice will be described later.

[0033] When the image generation unit 11c receives information on the type of a specific object and the position of a specific object in the field of view image from the image determination unit 11b, it determines the form (shape, color, etc.) of the image used to decorate the specific object, which will be stored in the storage unit 11a, based on the information on the type of the specific object. Based on the information on the position of the specific object in the field of view image, it determines the size of the image used to decorate the specific object and the position where the image used to decorate the specific object will be projected onto the lens body 3. The image generation unit 11c outputs this image information and position information to the projection unit 5. A specific example of the image used to decorate a specific object generated by the image generation unit 11c will be described later. In addition, the image generation unit 11c stops outputting image information and position information to the projection unit 5, for example, based on information corresponding to a specific sound input from the sound determination unit 11d.

[0034] For example, when the voice generation unit 11e receives information on the type of a specific object and the position of a specific object in the field of view image from the image determination unit 11b, it acquires voice information of the name of the specific object stored in the storage unit 11a based on the information on the type of the specific object, and acquires voice information of the direction in which the specific object is located relative to the viewpoint P, based on the information on the position of the specific object, and integrates the voice information of the direction and the voice information of the name and outputs it to the speaker 7. In this case, the speaker 7 outputs voice to the outside (user), for example, "There is a specific object on the left, and there is a specific object on the right." However, if there is no change in the voice information of the direction and the voice information of the name to be output to the speaker 7 based on the information on the type of a specific object and the position of a specific object in the field of view image input from the image determination unit 11b, the voice generation unit 11e may not output the voice information of the direction and the voice information of the name to the speaker 7. Furthermore, when the voice generation unit 11e receives information corresponding to a specific voice from the voice determination unit 11d, for example, it retrieves the voice information stored in the storage unit 11a based on the information corresponding to the specific voice and outputs the voice information to the speaker 7. Specific examples of voice information will be described later.

[0035] [2. Operation of the Vision Assistance System] The operation of the visual assistance system 1 of this embodiment when used by a person with a visual impairment will be described in detail below. Figure 4(a) shows the field of view of a person with a visual impairment as seen through the lens body 3 onto which an image decorating a specific object is projected, and Figure 4(b) shows the field of view of a person with a visual impairment as seen through the lens body 3 after moving the viewpoint P toward the image decorating a specific object. Both figures show the field of view of a person with a visual impairment as seen through the lens body 3 when the visual assistance system 1 is in operation.

[0036] The field of view of a person with a disability as seen through the lens body 3 shown in Figure 4(a) includes, compared to the field of view of a person with a disability as seen through the lens body 3 shown in Figure 3(b), the addition of a decorative image Z1 on the left and a decorative image Z2 on the right projected onto the lens body 3. Decorative image Z1 is an image of a frame generated and projected to surround the traffic light shown in Figure 3(a), and decorative image Z2 is an image of a frame generated and projected to surround the sign shown in Figure 3(a).

[0037] In this example, when image information (field of view information) is input to the control unit 11 from the depth camera 4 and the RGB camera 4a, the image determination unit 11b first determines whether or not a specific object has been photographed based on the image information. Specifically, when image information is input, the image determination unit 11b refers to the information stored in the storage unit 11a and determines that a traffic light and a sign have been photographed in the image information. It then outputs to the image generation unit 11c and the sound generation unit 11e information indicating that a traffic light has been photographed and information indicating the position of the traffic light in the image information (field of view information), as well as information indicating that a sign has been photographed and information indicating the position of the sign in the image information (field of view information). More specifically, when image information is input, the image determination unit 11b converts the image information into numerical sequence information and determines whether there is a portion in the numerical sequence information that is similar to numerical sequence information that indicates a specific object such as a traffic light or sign stored in the storage unit 11a. If there is a similar portion, it determines that a specific object has been photographed in the image information and outputs information that a specific object has been photographed, as well as information about the position of the specific object in the image information, to the image generation unit 11c and the sound generation unit 11e.

[0038] The image generation unit 11c refers to information stored in the storage unit 11a based on information that a traffic light has been photographed and information about the location of the traffic light within the image information (field of view information), and generates image information and position information of a frame surrounding the traffic light, formed in red according to the importance of the traffic light, as a specific image to decorate a specific object. The image generation unit 11c also refers to information stored in the storage unit 11a based on information that a sign has been photographed and information about the location of the sign within the image information (field of view information), and generates image information and position information of a frame surrounding a sign, formed in blue according to the importance of the sign, as a specific image to decorate a specific object. The image information and position information of these frames are then output to the projection unit 5. As a result, the projection unit 5 projects decorative image Z1 and decorative image Z2 onto the lens body 3, and as shown in Figure 4(a), the field of view of the disabled person viewed through the lens body 3 includes the decorative image Z1 on the left surrounding the traffic light and the decorative image Z2 on the right surrounding the sign. The color of the frame of the decorative image may be unified to, for example, red, regardless of importance. Furthermore, even when the frame color is changed according to the importance of a particular object, it is desirable that the importance of that particular object be pre-set in the memory unit 11a using, for example, the microphone 6 or the touch panel 13. For example, it is desirable that information such as the highest importance (e.g., traffic lights) be red, high importance (e.g., steps) be yellow, and low importance (e.g., signs) be blue is pre-set in the memory unit 11a for each item that the user needs to identify.

[0039] Furthermore, the voice generation unit 11e refers to the information stored in the memory unit 11a based on the information that a traffic light has been photographed and the location of the traffic light within the image information (field of view information), and generates first voice information, "There is a traffic light of the highest importance to the left," as voice information for direction and voice information for name. It then refers to the information stored in the memory unit 11a based on the information that a sign has been photographed and the location of the sign within the image information (field of view information), and generates second voice information, "There is a sign of lower importance to the right," as voice information for direction and voice information for name. Finally, it integrates the first and second voice information to generate third voice information, "There is a traffic light of the highest importance to the left and a sign of lower importance to the right," and outputs the third voice information to the speaker 7. As a result, the speaker 7 outputs the voice "There is a traffic light of the highest importance to the left and a sign of lower importance to the right" to the outside (user), informing the user that there is a traffic light to the left of the user's field of view and a sign to the right of the user's field of view. Furthermore, the image determination unit 11b may not output to the voice generation unit 11e information that a traffic light has been photographed and information about the location of the traffic light in the image information (visual information), nor information that a sign has been photographed and information about the location of the sign in the image information (visual information), thereby suppressing the consumption of the battery 12.

[0040] According to the vision assistance system 1 of this embodiment, as shown in Figure 4(b), the user can move their viewpoint P toward the decorative image Z1 included in the field of view and the sound output from the speaker, thereby moving a specific object, such as a traffic light, to the clear area X, which is the central part of the field of view where the haze is small. Consequently, the user can visually confirm the traffic light, a specific object of high importance, in the clear area X, which is the central part of the field of view, thereby assisting the vision of people with visual impairments. Furthermore, even if multiple specific objects, such as traffic lights or signs, are captured by the depth camera 4 and the RGB camera 4a, the frame color and sound can be varied according to the importance of the specific object. By moving the viewpoint toward the frame of higher importance according to the frame color and sound, the user can visually confirm the specific object of high importance (for example, a traffic light or something approaching the user at a speed exceeding a preset speed) in the clear area X, which is the central part of the field of view, thereby assisting the vision of people with visual impairments.

[0041] Figure 5(a) shows the field of view of a healthy person viewed through the lens body 3, and Figure 5(b) shows the field of view of a person with a disability viewed through the lens body 3. As shown in Figure 5(a), the field of view of a healthy person viewed through the lens body 3 is formed only by a clear region X centered on the viewpoint P, free from obstructions such as haze, making it easy to distinguish objects within the field of view (for example, each displayed product and price tag on a product display shelf). In contrast, as shown in Figure 5(b), the field of view of a person with a disability viewed through the lens body 3 is formed only by an unclear region Y centered on the viewpoint P, where there are obstructions such as haze, making it difficult to easily distinguish objects within the field of view (for example, each displayed product and price tag on a product display shelf). The following describes the field of view assistance system 1 of this embodiment with the aim of solving these problems.

[0042] In the vision assistance system 1 of this embodiment, depending on the specific voice input by the user via the microphone 6 or the operation information input by the user via the touch panel 13, the system may enlarge a specific range including a specific object in the field of view through the lens body 3, or enlarge a specific range of a part of the field of view. Figure 6(a) shows the field of view of a person with a disability as seen through the lens body 3 on which an image decorating a specific object is projected, and Figure 6(b) shows the field of view of a person with a disability as seen through the lens body 3 after the specific object has been changed. Both figures show the field of view of a person with a disability as seen through the lens body 3 when the vision assistance system 1 is in operation.

[0043] The field of view of a person with a disability viewed through the lens body 3 shown in Figure 6(a) includes a decorative image Z3 projected onto the lens body 3, compared to the field of view of a person with a disability viewed through the lens body 3 shown in Figure 5(b). The decorative image Z3 is an image that has been enlarged and projected so that a specific product and its surroundings, displayed on the product shelf shown in Figure 5(a), appear larger. The decorative image Z3 may have a circular or polygonal frame into which the enlarged image is projected, or it may not have a frame and may be projected as an enlarged image over an arbitrary area.

[0044] In this example, when the specific voice command "magnify" is input to the control unit 11 via microphone 6, the voice determination unit 11d determines whether the voice information "magnify" input from microphone 6 matches the information of a specific voice stored in the storage unit 11a. When the voice determination unit 11d determines that the voice information "magnify" matches the information of a specific voice stored in the storage unit 11a, it outputs operation information corresponding to the specific voice command "magnify" to the image determination unit 11b, image generation unit 11c, and voice generation unit 11e. Furthermore, when the specific operation command "magnify" is input to the control unit 11 via touch panel 13, the control unit 11 determines whether the operation information "magnify" input from touch panel 13 matches the information of a specific operation stored in the storage unit 11a. When the control unit 11 determines that the operation information "magnify" matches the information of a specific operation stored in the storage unit 11a, it outputs operation information corresponding to the specific operation "magnify" to the image determination unit 11b, image generation unit 11c, and voice generation unit 11e.

[0045] In the image determination unit 11b, when operation information corresponding to a specific voice command such as "zoom in" or a specific operation is input, it determines whether or not a specific object has been photographed based on the image information input from the depth camera 4 and the RGB camera 4a. Specifically, when operation information is input, the image determination unit 11b refers to the information stored in the storage unit 11a based on the image information and determines that multiple products displayed in the image and their corresponding price tags have been photographed. In this example, it outputs information about a specific product and its price tag that is photographed near the center of the image, as well as information about the position of the specific product and its price tag, to the image generation unit 11c and the voice generation unit 11e. More specifically, when image information is input, the image determination unit 11b converts the central part of the image information into numerical sequence information, and determines whether there is a part in the numerical sequence information that is similar to the numerical sequence information that represents each of the multiple products stored in the storage unit 11a. If there is a similar part, it determines that a specific object is photographed in the image information, and outputs information that a specific object is photographed and information about the position of the specific object in the image information to the image generation unit 11c and the sound generation unit 11e.

[0046] The image generation unit 11c generates a specific image to decorate a specific object, based on information that a specific product and price tag have been photographed and information on the positions of the specific product and price tag within the image information (field of view information). This image generates an enlarged image including the specific product and price tag, and position information including the specific product and price tag, and outputs the image information and position information to the projection unit 5. As a result, the projection unit 5 projects the decorative image Z3 onto the lens body 3, and as shown in Figure 6(a), the field of view of the disabled person viewed through the lens body 3 includes the decorative image Z3, which enlarges the product and its price tag located in the center of the field of view. It is preferable that the decorative image Z3 includes a frame surrounding the enlarged image with lines of a predetermined color and thickness in order to differentiate it from the blurry area Y in the field of view of the disabled person viewed through the lens body 3. The magnification of the enlarged decorative image Z3 may be set in advance by the user, or it may be possible to further enlarge or reduce the decorative image Z3 after it has been projected onto the lens body 3 via the microphone 6 or touch panel 13.

[0047] Furthermore, the voice generation unit 11e generates voice information such as "The price of XX (product name) is △△ yen" as voice information for the name and voice information for the numerical value, based on the information that a specific product and price tag have been photographed and the position information of the specific product and price tag within the image information (field of view information), and outputs the voice information to the speaker 7. As a result, the speaker 7 outputs the voice "The price of XX (product name) is △△ yen" to the outside (user), and can inform the user of the name and price of the product in the center of the user's field of view. Note that the image determination unit 11b may not output information that a specific product and price tag have been photographed and the position information of the specific product and price tag within the image information (field of view information) to the voice generation unit 11e, thereby suppressing the consumption of the battery 12.

[0048] According to the visual assistance system 1 of this embodiment, as shown in Figure 6(a), the user can learn the names and prices of products lined up on the shelves through the decorative image Z3 included in the field of view and the sound output from the speaker, thereby assisting the vision of people with visual impairments. Furthermore, the method of projecting the decorative image Z3, which is an enlarged image, onto the lens body 3 as in this embodiment may be combined with decorative images Z1, etc., which are images of frames surrounding a specific object according to the importance of that specific object. For example, a decorative image in which an enlarged image of a traffic light, which has the highest importance, is surrounded by a red frame indicating the highest importance may be projected onto the lens body 3.

[0049] Furthermore, when operation information such as "Enlarge XX (product name)" is input to the image determination unit 11b, image generation unit 11c, or sound generation unit 11e via the microphone 6 or touch panel 13, the image determination unit 11b determines whether or not a specific object, XX (product name), has been photographed based on the image information input from the depth camera 4 and RGB camera 4a and the image information stored in the storage unit 11a. If the specific object has not been photographed, it outputs information that the specific object has not been photographed to the image generation unit 11c and sound generation unit 11e. On the other hand, if the specific object has been photographed, the image determination unit 11b outputs information about the specific product and price tag, and information about the position of the specific product and price tag, to the image generation unit 11c and sound generation unit 11e. When the image determination unit 11b inputs information that the specific object has not been photographed, the image generation unit 11c and sound generation unit 11e generate decorative image or sound information to inform the user of this fact and output it to the projection unit 5 or speaker 7. As a result, the user can be notified that a specific product, name XX (product name), is not captured in the image information input from the depth camera 4 and the RGB camera 4a.

[0050] In the image generation unit 11c, when information about a specific product and its price tag, and information about the position of the specific product and its price tag are input from the image determination unit 11b, the image generation unit 11c generates image information with an enlarged range including the specific product and its price tag, and position information of the range including the specific product and its price tag, as a specific image to decorate a specific object, and outputs the image information and position information to the projection unit 5. As a result, the projection unit 5 projects the decorative image Z4 onto the lens body 3, and as shown in Figure 6(b), the field of view of the disabled person viewed through the lens body 3 includes the decorative image Z4 with the user-specified product and its price tag enlarged. It is preferable that the decorative image Z4 includes a frame surrounding the enlarged image with lines of a preset color and thickness in order to differentiate it from the blurry area Y in the field of view of the disabled person viewed through the lens body 3. Furthermore, the magnification of the enlarged decorative image Z4 may be preset by the user, or it may be possible to further enlarge or reduce the decorative image Z4 after it has been projected onto the lens body 3 via the microphone 6 or touch panel 13.

[0051] The voice generation unit 11e generates voice information, "The price of the specified XX (product name) is △△ yen," as voice information for the name and voice information for the numerical value, based on information that a specific product and price tag have been photographed and information about the location of the specific product and price tag within the image information (field of view information), and outputs the voice information to the speaker 7. As a result, the speaker 7 outputs the voice "The price of the specified XX (product name) is △△ yen" to the outside (user), informing the user of the name and price of the specified product within the user's field of view. The image determination unit 11b may not output information that a specific product and price tag have been photographed and information about the location of the specific product and price tag within the image information (field of view information) to the voice generation unit 11e, thereby suppressing the consumption of the battery 12.

[0052] According to the visual assistance system 1 of this embodiment, the user can learn about a designated product on a display shelf and its price through the decorative image Z4 included in the field of view and the sound output from the speaker 7, as shown in Figure 6(b), thereby assisting the vision of a person with a visual impairment. Furthermore, the method of projecting the decorative image Z4, which is an enlarged image, onto the lens body 3 as in this embodiment may be combined with a decorative image Z1, which is an image of a frame surrounding a specific object, depending on the importance of that specific object as described above.

[0053] Figure 7(a) shows the field of view of a healthy person viewed through the lens body 3, and Figure 7(b) shows the field of view of a person with a disability viewed through the lens body 3. As shown in Figure 7(a), the field of view of a healthy person viewed through the lens body 3 is formed only by a clear region X centered on the viewpoint P, free from obstructions such as haze, making it easy to distinguish objects within the field of view (for example, each displayed product on a product display shelf). In contrast, as shown in Figure 7(b), the field of view of a person with a disability viewed through the lens body 3 is formed only by an unclear region Y centered on the viewpoint P, where obstructions such as haze are present, making it difficult to easily distinguish objects within the field of view (for example, each displayed product on a product display shelf). The following describes the field of view assistance system 1 of this embodiment with the aim of solving these problems.

[0054] In the visual assistance system 1 of this embodiment, a decorative image Z5 of a frame surrounding a specific area including a specific object is displayed in the field of view through the lens body 3, in response to specific voice input by the user via the microphone 6 or operation information input by the user via the touch panel 13. Specifically, when operation information such as "○○ (product name)" is input to the image determination unit 11b, image generation unit 11c, or sound generation unit 11e via the microphone 6 or touch panel 13, the image determination unit 11b determines whether or not the specific object, which is ○○ (product name), has been photographed based on the image information input from the depth camera 4 and RGB camera 4a and the image information stored in the storage unit 11a. If the specific object has not been photographed, information indicating that the specific object has not been photographed is output to the image generation unit 11c and sound generation unit 11e. When the image generation unit 11c or the sound generation unit 11e receives information from the image determination unit 11b that a specific object has not been photographed, it generates a decorative image or sound information to inform the user of this fact and outputs it to the projection unit 5 or speaker 7. As a result, the user can be informed that a specific product, name XX (product name), has not been photographed in the image information input from the depth camera 4 and the RGB camera 4a.

[0055] On the other hand, if a specific object is photographed, the image determination unit 11b outputs the information of the specific product in the image information and the location information of the specific product to the image generation unit 11c and the sound generation unit 11e. When the image determination unit 11c receives the information of the specific product in the image information and the location information of the specific product from the image determination unit 11b, it generates image information including a frame surrounding the specific product and location information of the area including the specific product as a specific image to decorate the specific object, and outputs the image information and location information to the projection unit 5. As a result, the projection unit 5 projects the decorative image Z5 onto the lens body 3, and as shown in Figure 7(b), the field of view of the disabled person viewed through the lens body 3 includes the decorative image Z5 of the frame surrounding the product specified by the user. It is preferable that the decorative image Z5 is composed of lines of a predetermined color and thickness in order to differentiate it from the blurry area Y in the field of view of the disabled person viewed through the lens body 3.

[0056] The voice generation unit 11e generates voice information, "The specified XX (product name) is located in the upper left of the field of view," based on information that a specific product is being photographed and information about the location of the specific product within the image information (field of view information), and outputs this voice information to the speaker 7. As a result, the speaker 7 outputs the voice message "The specified XX (product name) is located in the upper left of the field of view" to the outside (user), informing the user of the location of the product specified by the user.

[0057] Figure 8(a) shows the field of view of a healthy person viewed through the lens body 3, and Figure 8(b) shows the field of view of a person with a disability viewed through the lens body 3. As shown in Figure 8(a), the field of view of a healthy person viewed through the lens body 3 is formed only by a clear region X centered on the viewpoint P, free from obstructions such as haze, making it easy to distinguish objects within the field of view (e.g., elevator switches). In contrast, as shown in Figure 8(b), the field of view of a person with a disability viewed through the lens body 3 is formed only by an unclear region Y centered on the viewpoint P, where obstructions such as haze are present, making it difficult to easily distinguish objects within the field of view (e.g., elevator switches). The following describes the field of view assistance system 1 of this embodiment with the aim of solving these problems.

[0058] In the vision assistance system 1 of this embodiment, depending on the specific voice input by the user via the microphone 6 or the operation information input by the user via the touch panel 13, the field of view through the lens body 3 may be made clearer, including a specific area containing a specific object, or a specific area within a part of the field of view may be made clearer. Figure 9 shows the field of view of a person with a disability as seen through the lens body 3 on which an image decorating a specific object is projected.

[0059] The field of view of a person with a disability as seen through the lens body 3 shown in Figure 9 has multiple decorative images Z6 projected onto the lens body 3, compared to the field of view of a person with a disability as seen through the lens body 3 shown in Figure 8(b). The decorative images Z6 are images in which the lines of the numbers and letters of the various switches of the elevator shown in Figure 8(a) have been enlarged so that they become clearer. The decorative images Z6 may be in the form of a circular or polygonal frame into which an enlarged image is projected, or they may be in the form of images of numbers and letters with enlarged lines that are projected without a frame.

[0060] In this example, when the specific voice "number" is input to the control unit 11 via microphone 6, the voice determination unit 11d determines whether the voice information "number" input from microphone 6 matches the information of a specific voice stored in the storage unit 11a. When the voice determination unit 11d determines that the voice information "number" matches the information of a specific voice stored in the storage unit 11a, it outputs operation information corresponding to the specific voice "number" to the image determination unit 11b, image generation unit 11c, and voice generation unit 11e. Furthermore, when a specific operation "number" is input to the control unit 11 via touch panel 13, the control unit 11 determines whether the operation information "number" input from touch panel 13 matches the information of a specific operation stored in the storage unit 11a. When the control unit 11 determines that the operation information "number" matches the information of a specific operation stored in the storage unit 11a, it outputs operation information corresponding to the specific operation "number" to the image determination unit 11b, image generation unit 11c, and voice generation unit 11e.

[0061] In the image determination unit 11b, when specific voice commands such as "number" or operation information corresponding to specific operations are input, the unit determines whether a specific object (in this example, numbers or letters, etc.) has been captured based on the image information input from the depth camera 4 and the RGB camera 4a. Specifically, when operation information is input, the image determination unit 11b refers to the information stored in the storage unit 11a based on the image information and determines that multiple numbers or letters have been captured in the image information. In this example, it outputs information about each number or letter captured in the image information, as well as information about the position of each number or letter, to the image generation unit 11c and the voice generation unit 11e. More specifically, when image information is input, the image determination unit 11b converts the image information into numerical sequence information and determines whether there is a portion in the numerical sequence information that is similar to numerical sequence information indicating numbers or characters stored in the storage unit 11a. If there is a similar portion, it determines that a specific object is captured in the image information and outputs information that a specific object is captured, as well as information about the position of the specific object in the image information, to the image generation unit 11c and the sound generation unit 11e.

[0062] The image generation unit 11c generates image information and positional information for each number and character, based on the information of each number and character and the positional information of each number and character, as a specific image to decorate a specific object. This image includes an image with thickened lines for each number and character, an image of a frame surrounding each image, and outputs the image information and positional information to the projection unit 5. As a result, the projection unit 5 projects multiple decorative images Z6 onto the lens body 3, and as shown in Figure 9, the field of view of the disabled person viewed through the lens body 3 includes multiple decorative images Z6 in which the floor switches inside the elevator are clearly defined. It is preferable that the decorative images Z6 include frames surrounding each number and character with lines of a predetermined color and thickness in order to differentiate them from the blurry areas Y in the field of view of the disabled person viewed through the lens body 3, but they may also be in a form that does not include images of such frames. Furthermore, the decorative image Z6 may be an enlarged image of each number or character, the magnification may be set in advance by the user, or, for example, the decorative image Z6 may be further enlarged or reduced via the microphone 6 or touch panel 13 after it has been projected onto the lens body 3.

[0063] Furthermore, the voice generation unit 11e generates voice information such as "From top left downwards, in the order of 6, 5, 4...9, 8, 7, closed" based on the information of each number and character and the position of each number and character, and outputs the voice information to the speaker 7. As a result, the speaker 7 outputs the voice "From top left downwards, in the order of 6, 5, 4...9, 8, 7, closed" to the outside (user), and can inform the user of the information of the floor switches inside the elevator that are within the user's field of vision. Note that the image determination unit 11b may not output the information of each number and character and the position of each number and character to the voice generation unit 11e, which can suppress the consumption of the battery 12.

[0064] According to the visual assistance system 1 of this embodiment, the user can learn the numbers and letters on the floor switches inside the elevator through the decorative image Z6 included in the field of view and the sound output from the speaker 7, as shown in Figure 9, thereby assisting the vision of people with visual impairments. Furthermore, the method of projecting the decorative image Z6, which is an enlarged image, onto the lens body 3 as in this embodiment may be combined with the decorative image Z1, which is an image of a frame surrounding a specific object, depending on the importance of that specific object as described above.

[0065] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various improvements and design changes are possible without departing from the spirit of the invention, as shown below.

[0066] In other words, in the above embodiment, a decorative image Z1 enclosing a predetermined specific object in a frame was projected onto the lens body 3 via the projection unit 5. However, for example, if the specific object is stairs or steps, the image generation unit 11c may project a linear decorative image extending along the corners of the stairs or steps onto the lens body 3 via the projection unit 5. Figure 10(a) shows the field of view of a healthy person viewed through the lens body 3, and Figure 10(b) shows the field of view of a person with a disability viewed through the lens body 3. As shown in Figure 10(a), the field of view of a healthy person viewed through the lens body 3 is formed only by a clear region X centered on the viewpoint P that is free from obstructions such as haze, making it easy to distinguish objects within the field of view (for example, steps on a staircase). In contrast, as shown in Figure 10(b), the field of view of a person with a disability viewed through the lens body 3 is formed only by an unclear region Y centered on the viewpoint P that has obstructions such as haze, making it difficult to easily distinguish objects within the field of view (for example, steps on a staircase).

[0067] In this example of a visual assistance system, as shown in Figure 10(b), for example, when there is a step in the user's field of view, a linear decorative image Z7 arranged along the corner of each step is displayed in the field of view through the lens body 3, according to specific voice input by the user via the microphone 6, or operation information input by the user via the touch panel 13, or pre-entered settings. Specifically, when operation information such as "step" is input to the image determination unit 11b, image generation unit 11c, or sound generation unit 11e via the microphone 6 or touch panel 13, the image determination unit 11b determines whether or not the linear edge of the step has been photographed based on the image information input from the depth camera 4 and RGB camera 4a and the image information stored in the storage unit 11a. If the edge of the step has not been photographed, information that the edge of the step has not been photographed is output to the image generation unit 11c and sound generation unit 11e. When the image generation unit 11c or the sound generation unit 11e receives information from the image determination unit 11b that the edge of the step has not been captured, it generates a decorative image or sound information to inform the user of this fact and outputs it to the projection unit 5 or speaker 7. As a result, the user can be informed that the step has not been captured in the image information input from the depth camera 4 and the RGB camera 4a.

[0068] On the other hand, if the edge of the step is captured in the image, the image determination unit 11b outputs the step edge information and the position information of the step edge from the image information to the image generation unit 11c and the sound generation unit 11e. When the image determination unit 11b inputs the step edge information and the position information of the step edge from the image information, the image generation unit 11c generates image information including a linear portion extending linearly along the step edge and position information of the area including the step edge as a specific image to decorate the step edge, and outputs the image information and position information to the projection unit 5. As a result, the projection unit 5 projects the decorative image Z7 onto the lens body 3, and as shown in Figure 10(b), the field of view of the disabled person viewed through the lens body 3 includes the decorative image Z7 extending linearly along the step edge. It is preferable that the decorative image Z7 is composed of lines of a predetermined color and thickness in order to differentiate it from the blurry area Y in the field of view of the disabled person viewed through the lens body 3.

[0069] The voice generation unit 11e generates voice information, "The step is located ahead in the field of view," based on the information that a step has been photographed and the location of the step within the image information (field of view information), and outputs this voice information to the speaker 7. As a result, the speaker 7 outputs the voice message "The step is located ahead in the field of view" to the outside (user), informing the user of the location of the step.

[0070] Furthermore, in the above embodiment, a decorative image Z1 enclosing a predetermined specific object in a frame was projected onto the lens body 3 via the projection unit 5. However, the image determination unit 11b may, based on image information continuously input from the depth camera 4 and the RGB camera 4a, determine that there is a specific object approaching the user at a predetermined speed or higher. In this case, the image generation unit 11c may generate a decorative image enclosing the specific object in a frame and project it onto the lens body 3, and the sound generation unit 11e may generate sound to notify the user that a specific object is approaching, which is then transmitted via the speaker 7. In this case, the user can be notified that there is a specific object approaching at a predetermined speed or higher, and the field of vision of a person with a visual impairment can be assisted.

[0071] Furthermore, in the above embodiment, image information and audio information are generated according to each voice input such as "magnify" or "number," but the specific voice input may be, for example, "person," "bicycle," or "car." In this case, image information of a frame surrounding the specific object, such as a person, bicycle, or car, may be generated and projected onto the lens body 3, and audio information of the location of the person, bicycle, or car may be generated and announced via the speaker 7, thereby informing the user of the presence of a person, bicycle, or car, and assisting the field of vision of a person with a visual impairment. For example, all decorative images projected onto the lens body 3 and all audio information announced from the speaker 7 may be stopped by voice input such as "clear," or all decorative images projected onto the lens body 3 may be stopped by voice input such as "clear image," or all audio information announced from the speaker 7 may be stopped by voice input such as "clear audio."

[0072] Furthermore, for example, depending on the user's operation, a decorative image showing the route from the current location to the destination may be projected onto the lens body 3 via the projection unit 5. Figure 11(a) shows the field of view of a healthy person as seen through the lens body 3, and Figure 11(b) shows the field of view of a person with a disability as seen through the lens body 3. As shown in Figure 11(a), the field of view of a healthy person as seen through the lens body 3 is formed only by a clear region X centered on the viewpoint P that is free from obstructions such as haze, making it easy to distinguish objects within the field of view (e.g., roads and road signs). In contrast, as shown in Figure 11(b), the field of view of a person with a disability as seen through the lens body 3 is formed by a clear region X centered on the viewpoint P that is free from obstructions such as haze, and an unclear region Y formed around the clear region X that has obstructions such as haze, making it difficult to easily distinguish objects within the field of view (e.g., roads and road signs).

[0073] In the visual assistance system 1 of this example, as shown in Figure 11(b), a decorative image Z8 including an arrow indicating the route from the current location to the destination is displayed in the field of view through the lens body 3, depending on, for example, specific voice input by the user via the microphone 6, or operation information or pre-entered settings input by the user via the touch panel 13. Specifically, when operation information such as "directions to XX" is input to the control unit 11 via the microphone 6 or touch panel 13, the control unit 11 connects to an existing route guidance system via a communication line, searches for the route from the current location to XX (destination), and inputs the searched route as route information to the image determination unit 11b, the image generation unit 11c, and the voice generation unit 11e.

[0074] In the image determination unit 11b, when search information is input, it determines whether a passageway including sidewalks and stairs has been photographed based on the image information input from the depth camera 4 and RGB camera 4a and the image information stored in the storage unit 11a. If a passageway has been photographed, it outputs the location information of the passageway to the image generation unit 11c and the sound generation unit 11e. In the image generation unit 11c, when the search information and the location information of the passageway are input from the image determination unit 11b, it generates image information including an arrow extending toward the destination and location information of the area including the passageway as a specific image to decorate the passageway, and outputs the image information and location information to the projection unit 5. As a result, the projection unit 5 projects the decorative image Z8 onto the lens body 3, and as shown in Figure 11(b), the field of view of the disabled person viewed through the lens body 3 includes the decorative image Z8 consisting of an arrow extending along the passageway. It is preferable that the decorative image Z8 is composed of lines of a predetermined color and thickness in order to differentiate it from the blurry area Y in the field of view of the disabled person viewed through the lens body 3.

[0075] The voice generation unit 11e generates voice information, "The destination is located down the corridor ahead in the direction indicated by the arrow," based on the search information and the location information of the corridor, and outputs this voice information to the speaker 7. As a result, the speaker 7 outputs the voice message, "The destination is located down the corridor ahead in the direction indicated by the arrow," to the outside (user), informing the user of the location of the destination. [Explanation of symbols]

[0076] 1. Vision assistance system 2 Main body 3 Lens body 4. Depth Camera 4a RGB camera 5 Projection section 6 Microphones 7 speakers 10 Control terminals 11 Control Unit 11a Storage section 11b Image judgment section 11c Image generation section 11d Voice detection unit 11e Speech generation unit 13 Touch panel

Claims

1. At least one shooting unit capable of continuously capturing the user's field of view, A control unit capable of generating a predetermined image based on the content captured by the imaging unit, A projection unit that projects the predetermined image onto at least one projection surface, A vision assistance system comprising a main body that holds the projection portion in front of the user's eyes when worn by the user, The system further includes an audio input unit that inputs the user's voice to the control unit, The control unit has the function of generating a specific image that includes an image that informs the user that a specific object is in the user's field of view based on the content captured by the shooting unit, and projecting the specific image via the projection unit to a position on the projection unit corresponding to the specific object, The system has a function to generate a specific image, which includes an image that informs the user of the location of an object corresponding to the specific sound, when a specific sound is input based on the aforementioned sound, and to project the specific image via the projection unit to the location on the projection unit corresponding to the object corresponding to the specific sound. A visual assistance system characterized by the following features.

2. The control unit generates a specific image including an image of a frame surrounding the specific object when the specific object is in front of the user, and changes the color of the frame image according to a preset importance level of the specific object. The visual assistance system according to claim 1, characterized in that it is as described above.

3. The control unit generates the specific image, which includes an enlarged image of a specific area containing the specific object, when the specific object is in front of the user. The visual assistance system according to claim 1, characterized in that it is as described above.

4. The control unit generates the specific image, which includes a linear image along the corner of the step, when the specific object is a step. The visual assistance system according to claim 1, characterized in that it is as described above.

5. The control unit generates the specific image, which includes an image corresponding to the character or number, when the specific object is a character or a number. The visual assistance system according to claim 1, characterized in that it is as described above.

6. Further comprising an audio output unit for outputting audio to the outside, The control unit further has the function of generating a special sound to notify the user that a specific object is in front of them, and outputting the special sound to the outside via the sound output unit. A visual assistance system according to any one of claims 1 to 5, characterized by the above.

Images