Information presentation device, information presentation method, and information presentation program
Patent Information
- Application Number
- JP2023130178
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-08-09
- Publication Date
- 2025-02-21
Smart Images
Figure 2025025432000001_ABST
Abstract
Description
[Technical field]
[0001] The present application belongs to the technical fields of an information presentation device, an information presentation method, and an information presentation program. More specifically, the present application belongs to the technical fields of an information presentation device and an information presentation method, and a program for the information presentation device, which perform processing for presenting information to a visually impaired person whose vision is weaker than that of a normal person or a person accompanying the visually impaired person. [Background technology]
[0002] In recent years, with the rapid spread of portable terminal devices such as smartphones, the number of users who carry such terminal devices while traveling is also rapidly increasing. Among such users are people who have been issued a disability certificate due to visual impairment, and people who do not have a visual impairment severe enough to require a disability certificate but whose vision is weaker than that of a healthy person due to eye diseases such as cataracts and glaucoma (including people who are unaware of their weak vision). Hereinafter, the above-mentioned people who have been issued a disability certificate due to visual impairment and people whose vision is weaker than that of a healthy person will be collectively referred to as "visually impaired people."
[0003] In this case, when guiding the movement of the visually impaired person using a terminal device carried by the visually impaired person, it goes without saying that the guidance needs to take into consideration that "the terminal device is carried by a visually impaired person." For example, the following Patent Document 1 is an example of a document showing prior art that meets such a requirement.
[0004] In the prior art disclosed in Patent Document 1, a walking assistance system that assists pedestrians in their movement identifies the position of a pedestrian traffic signal based on the position of the white line of a crosswalk captured in an image taken by a wide-angle camera hanging around the pedestrian's neck, and if the traffic signal cannot be identified, changes the camera's resolution, shooting angle of view, or wide angle, etc., to improve recognition accuracy.
[0005] Here, in the prior art disclosed in the above Patent Document 1, the position of the traffic signal is only legally stipulated to be "installed at a height of 2.5m or more," and the traffic signal is not necessarily located directly in front of the pedestrian crossing. Therefore, in this case, it is important to have a wide-angle camera. [Prior art documents] [Patent documents]
[0006] [Patent Document 1] JP 2022-107967 A Summary of the Invention [Problem to be solved by the invention]
[0007] However, whether a small camera that can be hung around the neck can capture a wide-angle image while maintaining a resolution that allows image recognition of the traffic signal depends on the performance (specs) of the camera. Therefore, the prior art disclosed in Patent Document 1 has a problem that it can only be applied to a portable terminal equipped with a high-performance camera.
[0008] Therefore, the present application has been made in consideration of the above-mentioned problems, and one example of the objective thereof is to provide an information presentation device and information presentation method, as well as a program for said information presentation device, that are capable of presenting to pedestrians, such as visually impaired pedestrians, information concerning traffic lights and the like that they should check when moving around, by sound, without requiring high performance cameras. [Means for solving the problem]
[0009] In order to solve the above problem, the invention described in claim 1 includes a route information acquisition means for acquiring route information regarding a route to a user's destination, an image information acquisition means for acquiring image information showing an image captured by an image capture means that moves with the user and captures an image of the user's surroundings, an image capture range information acquisition means for acquiring image capture range information showing the image capture range of the image capture means when the image was captured, and a feature position information acquisition means for acquiring feature position information showing the position of a target traffic information feature that is a traffic information feature that the user should check when traveling along the route, based on map information in which traffic information features that are features that provide information about traffic are recorded and the acquired route information. the image capturing apparatus includes a feature position information acquisition means for acquiring the shooting range information, the acquired image information, and the acquired feature position information, a determination means for determining whether the target traffic information feature is captured in the image based on the acquired shooting range information, the acquired image information, and the acquired feature position information, and a sound emission control means for, when the determination means determines that the target traffic information feature is captured in the image, emitting a sound of the contents of the target traffic information feature to the user using a sound emitting means, and the sound emission control means is configured to, when the determination means determines that the target traffic information feature is not captured in the image, emitting a sound of a change instruction regarding a change to the shooting range to the user using the sound emitting means.
[0010] In order to solve the above problem, the invention described in claim 8 is an information presentation method executed in an information presentation device including a route information acquisition means, an image information acquisition means, a shooting range information acquisition means, a feature position information acquisition means, a determination means, and a sound emission control means, the information presentation method including a route information acquisition step of acquiring route information relating to a route to a user's destination by the route information acquisition means, an image information acquisition step of acquiring by the image information acquisition means image information showing an image captured by an image capture means that moves with the user and captures the user's surroundings, an image capture range information acquisition step of acquiring by the image capture range information acquisition means image information showing the image capture range of the image capture means when the image was captured, and a step of displaying a map information recorded with a traffic information feature that is a feature that presents information related to traffic and a sound emission control step of displaying a sound emission control signal to the user based on the acquired route information and map information in which a traffic information feature that is a feature that presents information related to traffic is recorded. The method includes a feature position information acquisition step of acquiring feature position information indicating the position of a target traffic information feature, which is the traffic information feature to be checked when traveling along the route, using the feature position information acquisition means; a determination step of determining whether the target traffic information feature is captured in the image based on the acquired shooting range information, the acquired image information, and the acquired feature position information; and a sound emission control step of, when the determination step determines that the target traffic information feature is captured in the image, causing the sound emission control means to emit a sound of the contents of the target traffic information feature to the user using a sound emitting means.In the sound emission control step, when the determination step determines that the target traffic information feature is not captured in the image, a change instruction regarding a change to the shooting range is emitted to the user using the sound emitting means.
[0011] In order to solve the above problem, the invention described in claim 9 provides a computer included in an information presentation device, comprising: a route information acquisition means for acquiring route information relating to a route to a user's destination; an image information acquisition means for acquiring image information showing an image captured by an image capture means that moves with the user and captures an image of the user's surroundings; an image capture range information acquisition means for acquiring image capture range information showing the image capture range of the image capture means when the image was captured; a feature position information acquisition means for acquiring feature position information showing the position of a target traffic information feature that is the traffic information feature that the user should check when traveling along the route, based on map information in which traffic information features that are features providing information about traffic are recorded and the acquired route information; The information presentation program functions as a determination means for determining whether the target traffic information feature is captured in the image based on shadow range information, the acquired image information, and the acquired feature position information, and as a sound emission control means for, when the determination means determines that the target traffic information feature is captured in the image, emitting a sound of the contents of the target traffic information feature to the user using a sound emission means, and is configured to cause the computer functioning as the sound emission control means to function such that, when the computer functioning as the determination means determines that the target traffic information feature is not captured in the image, a change instruction regarding a change to the shooting range is emitted to the user using the sound emission means. [Brief description of the drawings]
[0012] [Figure 1] 1 is a block diagram showing a schematic configuration of an information presentation device according to an embodiment; [Diagram 2] FIG. 2 is a block diagram showing a schematic configuration of a terminal device according to an embodiment of the present invention. [Diagram 3] 11 is a flowchart showing a guidance process according to an embodiment. [Figure 4] 11A and 11B are diagrams illustrating examples of images displayed on a terminal device in the guidance process of the embodiment. [Diagram 5] 11A and 11B are diagrams illustrating examples of images displayed on a terminal device in the guidance process of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Next, an embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram showing a schematic configuration of an information presentation device according to the embodiment.
[0014] As shown in Figure 1, the information presentation device S of the embodiment is configured to include a route information acquisition means 1, an image information acquisition means 2, a shooting range information acquisition means 3, a feature position information acquisition means 4, a determination means 5, and a sound emission control means 6.
[0015] In this configuration, the route information acquisition means 1 acquires route information regarding the route to the user's destination. The image information acquisition means 2 acquires image information indicating an image captured by the image capture means C that moves with the user and captures the user's surroundings. Meanwhile, the image capture range information acquisition means 3 acquires image capture range information indicating the image capture range of the image capture means C when the image is captured. Meanwhile, the feature position information acquisition means 4 acquires feature position information indicating the position of the target traffic information feature, which is the traffic information feature that the user should check when traveling along the route, based on map information in which traffic information features, which are features that provide information about traffic, are recorded and route information acquired by the route information acquisition means 1. Here, the term "feature" refers to facilities or equipment related to traffic and other objects on the ground (the same applies in the following explanations).
[0016] With these, the determination means 5 determines whether the target traffic information feature is captured in the image based on the shooting range information acquired by the shooting range information acquisition means 3, the image information acquired by the image information acquisition means 2, and the feature position information acquired by the feature position information acquisition means 4. After that, when the determination means 5 determines that the target traffic information feature is captured, the sound emission control means 6 causes the sound emission means SP to emit the contents of the target traffic information feature to the user.
[0017] At this time, when the determination means 5 determines that the target traffic information feature is not captured in the image, the sound emission control means 6 causes the sound emission means SP to emit a change instruction for changing the shooting range to the user.
[0018] As described above, according to the operation of the information presentation device S of the embodiment, when it is determined that the target traffic information feature is not captured in the image captured by the photographing means C, an instruction to change the photographing range of the photographing means C is issued to the user by sound. This makes it possible to present to the user, by sound, information regarding the target traffic information feature that the user should check when moving around, without requiring high performance of the photographing means C. EXAMPLES
[0019] Next, specific examples corresponding to the above-mentioned embodiment will be described with reference to Figures 2 to 5. The examples described below are examples in which the present application is applied to a case where a visually impaired person's walking movement (hereinafter, appropriately referred to as "walking movement") is guided by voice, for example, using a terminal device of the examples carried by the visually impaired person.
[0020] (I) Configuration and functions of the terminal device of the embodiment First, the configuration and functions of a terminal device according to an embodiment carried by a visually impaired person will be described with reference to Fig. 2. The terminal device is actually realized as, for example, a smartphone or tablet terminal carried by a visually impaired person and capable of performing some operations without relying on vision. Fig. 2 is a block diagram showing a schematic configuration of the terminal device according to the embodiment. In Fig. 2, the same component numbers as those of the information presentation device S of the embodiment shown in Fig. 1 are used for the components of the embodiment corresponding to the components of the information presentation device S.
[0021] As shown in FIG. 2, the terminal device T of the embodiment is composed of a processing unit 10 consisting of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., a non-volatile recording unit 11 consisting of an HDD (Hard Disc Drive) or an SSD (Solid State Drive), etc., an operation unit 12 consisting of operation buttons and a touch panel, etc., a microphone 13, a camera C consisting of a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor) imaging element, etc., a sensor unit 14 constituting a GNSS (Global Navigation Satellite System) or an autonomous navigation system, etc., a display 15 consisting of a liquid crystal display, etc., a speaker SP, and an interface 16.
[0022] The processing unit 10 is composed of a route information acquisition unit 1, an image information acquisition unit 2, a shooting range information acquisition unit 3, a feature position information acquisition unit 4, a determination unit 5, and a sound emission control unit 6. In this case, the route information acquisition unit 1, the image information acquisition unit 2, the shooting range information acquisition unit 3, the feature position information acquisition unit 4, the determination unit 5, and the sound emission control unit 6 may be realized by a hardware logic circuit constituting the processing unit 10, or may be realized in software by the processing unit 10 reading out from the recording unit 11 and executing a program corresponding to the guidance processing of the embodiment described later and pre-recorded in the recording unit 11, for example. The route information acquisition unit 1 corresponds to an example of the route information acquisition means 1 of the embodiment, the image information acquisition unit 2 corresponds to an example of the image information acquisition means 2 of the embodiment, and the shooting range information acquisition unit 3 corresponds to an example of the shooting range information acquisition unit 3 of the embodiment. Furthermore, the feature position information acquisition unit 4 corresponds to an example of the feature position information acquisition means 4 of the embodiment, the determination unit 5 corresponds to an example of the determination means 5 of the embodiment, and the sound emission control unit 6 corresponds to an example of the sound emission control means 6 of the embodiment. Furthermore, as indicated by dashed lines in Fig. 2, the route information acquisition unit 1, the image information acquisition unit 2, the shooting range information acquisition unit 3, the feature position information acquisition unit 4, the determination unit 5, and the sound emission control unit 6 configure an example of the information presentation device S of the embodiment.
[0023] In the above configuration, the recording unit 11 stores programs, voice data, map data MP, etc., necessary for the guidance process of the embodiment executed in the terminal device T in a non-volatile manner, and the data is read out by the processing unit 10 as necessary. The map data MP includes data on traffic information features such as pedestrian traffic signals at crosswalks and road signs whose display content changes depending on time, as features that provide information on traffic for people including visually impaired people. The microphone 13, under the control of the processing unit 10, collects the speech of the visually impaired person carrying the terminal device T, generates sound data corresponding to the collected sound, and outputs the generated sound data to the processing unit 10. The camera C, under the control of the processing unit 10, generates image data corresponding to the captured image of features present around the terminal device T, such as a pedestrian traffic signal at an intersection, and outputs the generated image data to the processing unit 10. At this time, the captured image included in the image data corresponds to the captured range of the camera C at that time. Furthermore, under the control of the processing unit 10, the sensor unit 14 uses the above-mentioned GNSS, etc. to detect the current position of the terminal device T, the orientation of the terminal device T, and the height of the terminal device T from the ground, etc., and generates detection result data indicating the detection results and outputs it to the processing unit 10.
[0024] Meanwhile, the display 15 displays images or maps, etc., required for the guidance processing of the embodiment under the control of the processing unit 10. The speaker SP also emits voice guidance, etc., required for the guidance processing of the embodiment under the control of the processing unit 10. Furthermore, the interface 16 controls the acquisition of various data required for the guidance processing of the embodiment via a network such as the Internet (not shown) under the control of the processing unit 10. Furthermore, when an instruction operation is performed on the terminal device T by a visually impaired person carrying it, the operation unit 12 generates an operation signal corresponding to the instruction operation and outputs it to the processing unit 10. As a result, the processing unit 10 performs the guidance processing of the embodiment while controlling each component constituting the terminal device T, such as the microphone 13, the sensor unit 14, the display 15, and the interface 16, based on the operation signal.
[0025] More specifically, in the guidance process of the embodiment as the terminal device T, first, a destination of the visually impaired person's walking movement and a request for movement guidance to the destination are input, for example, by an operation using the operation unit 12 or the voice of the visually impaired person collected by the microphone 13. Then, when the destination and the request for movement guidance are input, the processing unit 10 searches for a walking path from the current position of the terminal device T included in the detection result data from the sensor unit 14 to the destination in a manner similar to that of the conventional method, generates route data indicating the resultant movement route, and temporarily records it in the recording unit 11. At this time, the route data includes traffic signal information indicating the position, height from the ground, and direction in which the display of the traffic signal can be seen, of the traffic information feature installed on, for example, a crosswalk at an intersection on the movement route and which the visually impaired person should check when walking.
[0026] Next, the route information acquisition unit 1 of the processing unit 10 acquires the route data from the recording unit 11 in response to the request for travel guidance. Meanwhile, the image information acquisition unit 2 of the processing unit 10 acquires the image data indicating the image captured by the camera C from the camera C as imaging range data indicating the imaging range of the camera C when the image was captured. Meanwhile, the imaging range information acquisition unit 3 of the processing unit 10 acquires, as part of the imaging range data, the current position of the terminal device T (i.e., the current position of the camera C) based on the detection result data from the sensor unit 14, the imaging direction of the camera C corresponding to the orientation of the terminal device T, and height data indicating the height of the camera C from the ground corresponding to the height of the terminal device T from the ground, etc.
[0027] Next, the feature position information acquisition unit 4 of the processing unit 10 acquires, based on the route data acquired by the route information acquisition unit 1, data indicating the positions of the traffic information features that the visually impaired person carrying the terminal device T should check when traveling along the travel route, from the map data MP in which data regarding the traffic information features is recorded.
[0028] Thereafter, the determination unit 5 of the processing unit 10 determines whether the traffic signal that the visually impaired person should check is captured in the image corresponding to the image data, based on the shooting range data acquired by the shooting range information acquisition unit 3, the image data acquired from the camera C by the image information acquisition unit 2, and the data indicating the position of the traffic signal acquired by the feature position information acquisition unit 4. More specifically, the determination unit 5 determines whether the captured traffic signal is a traffic signal that the visually impaired person should check (i.e., whether the traffic signal that the visually impaired person should check is captured) based on the contents of the image data in which the traffic signal is captured, the data indicating the position of the traffic information feature, and the current position of the camera C indicated by the shooting range data, the shooting direction of the camera C, and the height of the camera C from the ground.
[0029] For example, at an intersection where multiple roads intersect, a traffic signal corresponding to each road is usually installed. If there is a glass-walled building next to the intersection, other traffic signals other than the traffic signal that the visually impaired person should check may be reflected by the glass of the building, and as a result, the other traffic signals may be reflected in the image corresponding to the image data along with the traffic signal that the visually impaired person should check. In this case, the determination unit 5 needs to select the traffic signal that the visually impaired person should check from the multiple traffic signals that appear in the image. For this reason, the determination unit 5 selects the traffic signal that the visually impaired person should check based on, for example, the clarity, size, distortion, and orientation of each traffic signal in the image, or the degree of match with template data as an image of a traffic signal.
[0030] As a result, when the sound emission control unit 6 determines that the traffic signal that the visually impaired person should check is included in the image corresponding to the image data, it causes the content of the traffic signal (i.e., for example, whether the light is green or red) to be emitted as sound using the speaker SP to the visually impaired person carrying the terminal device T.
[0031] On the other hand, when it is determined that the traffic signal that the visually impaired person should check is not captured in the image corresponding to the image data, the sound emission control unit 6 emits a change instruction to the visually impaired person by using the speaker SP, for example, as a voice, to change the shooting range of the camera C. More specifically, when it is determined that the traffic signal that the visually impaired person should check is not captured in the image corresponding to the image data, the sound emission control unit 6 emits the change instruction to the visually impaired person by sound to change the shooting range of the camera C so that the position of the traffic signal that the visually impaired person should check is included in the shooting range of the camera C, based on the data indicating the position of the traffic information feature and the current position of the camera C indicated by the shooting range data, the shooting direction of the camera C, and the height of the camera C from the ground. In this case, the change instruction is, for example, an instruction to change the shooting direction to the correct direction (upward, downward, leftward, rightward, etc.), to change the position of the visually impaired person (i.e., the position of the terminal device T), to pan shooting using the camera C (i.e., to move the shooting range horizontally), to tilt shooting using the camera C (i.e., to move the shooting range vertically up and down), or to expand the shooting range by reducing the image, which is an instruction to change the direction of the traffic signal that the visually impaired person should check. Then, when the traffic signal that the visually impaired person should check is captured in the shooting range changed in response to the change instruction, the sound emission control unit 6 emits the above-mentioned content of the traffic signal to the visually impaired person using the speaker SP.
[0032] (II) Guidance process of the embodiment Next, the guidance process of the embodiment executed in the terminal device T having the above-mentioned configuration will be specifically described with reference to Fig. 2 to Fig. 5. Fig. 3 is a flowchart showing the guidance process of the embodiment, and Fig. 4 and Fig. 5 are diagrams showing examples of images displayed on the display 15 of the terminal device T in the guidance process. In Fig. 4 and Fig. 5, the same component numbers as those of the information presentation device S of the embodiment shown in Fig. 1 are used for the components of the embodiment.
[0033] The guidance process of the embodiment is started, for example, when the power switch of the terminal device T is turned on. As shown in FIG. 3, when the guidance process of the embodiment is started, the processing unit 10 of the terminal device T generates the route data and starts the travel guidance when a request for a destination of a walking movement of the visually impaired person carrying the terminal device T and a travel guidance to the destination is input (step S1). The processing unit 10 then monitors whether the visually impaired person carrying the terminal device T has reached a pedestrian crossing as a guide point on the travel route (step S2). In the monitoring of step S2, if the visually impaired person has not reached the pedestrian crossing (step S2: NO), the processing unit 10 proceeds to step S7 described later. On the other hand, in the monitoring of step S2, if the visually impaired person has reached the pedestrian crossing (step S2: YES), the feature position information acquisition unit 4 of the processing unit 10 next acquires data indicating the position of a traffic signal that the visually impaired person should check at the pedestrian crossing as the traffic information feature based on the route data from the map data MP (step S3).
[0034] Next, the processing unit 10 judges whether or not the traffic signal that the visually impaired person should check has been photographed by the camera C by emitting an instruction such as "Please photograph the traffic signal in front" through the speaker SP (step S4). If the judgment in step S4 indicates that the photograph has not been taken because the operation has not been performed or the like (step S4: NO), the processing unit 10 proceeds to step S7 described below. On the other hand, if the judgment in step S4 indicates that the photograph has been taken (step S4: YES), the judgment unit 5 of the processing unit 10 then judges, by the above-mentioned method, whether or not the traffic signal that the visually impaired person should check is included in the image corresponding to the image data output from the camera C by the photograph (step S5). In the judgment of step S5, for example, when the traffic signal SG that the visually impaired person should check is shown on the display 15 of the terminal device T held in the hand HD of the visually impaired person as illustrated in FIG. 4 (step S5: YES), the sound emission control unit 6 uses the speaker SP to emit the contents of the traffic signal shown (step S6) to the visually impaired person. At this time, for example, as illustrated in FIG. 4, when a pedestrian speaker SGP corresponding to the traffic signal SG is provided beside the traffic signal SG that the visually impaired person should check, the sound emission control unit 6 may emit a guidance voice such as "The signal is green. Please follow the guidance sound from the speaker and cross the crosswalk carefully" from the speaker SP in addition to emitting the contents of the traffic signal SG. Also, when the camera C continues to take pictures while the person is crossing the crosswalk, the speaker SP may emit a guidance voice indicating the timing when the signal changed to red or that the green signal has started to flash (or is flashing). Thereafter, the processing unit 10 judges whether or not to end the guidance process of the embodiment, for example, because a visually impaired person carrying the terminal device T has arrived at the destination (step S7). If the judgment in step S7 is to end the guidance process of the embodiment (step S7: YES), the processing unit 10 ends the guidance process of the embodiment as is. On the other hand, if the judgment in step S7 is to continue the guidance process of the embodiment (step S7: NO), the processing unit 10 returns to step S2 and continues the guidance process described above.
[0035] In the judgment in step S5 above, if a traffic signal FSG other than the above-mentioned traffic signal SG that the visually impaired person should check is reflected in the glass GL of a building BL beside the crosswalk PC, for example as in the image of the display 15 illustrated in Figure 5 (in the case of the image illustrated in Figure 5, the traffic signal FSG is distorted horizontally), the judgment unit 5 correctly selects and judges the traffic signal SG that the visually impaired person should check, based on the size of each of the above-mentioned traffic signals SG in the image, the degree of match with the template data as an image of a traffic signal, etc.
[0036] On the other hand, in the judgment of step S5, if the traffic signal SG that the visually impaired person should check is not shown on the display 15 of the terminal device T held in the hand HD of the visually impaired person (step S5: NO), the sound emission control unit 6 makes the visually impaired person emit a sound via the speaker SP to instruct the change to the shooting range of the camera C so that the position of the traffic signal SG that the visually impaired person should check is included (step S8). After that, the processing unit 10 returns to the above step S5, and judges again based on the image data output from the camera C after the change of the shooting range whether the traffic signal that the visually impaired person should check is shown in the image corresponding to the image data (step S5).
[0037] As described above, according to the guidance process of the embodiment executed by the terminal device T, when it is determined that the traffic signal SG that the visually impaired person should check is not captured in the image captured by the camera C, a change instruction for changing the capturing range of the camera C is emitted (see step S8 in Figure 3), thereby making it possible to present information about the traffic signal SG to the visually impaired person by sound without requiring high performance of the camera C.
[0038] Furthermore, if the change instruction is any one of an instruction to correct the shooting direction, an instruction to move the shooting direction horizontally (to perform pan shooting), or an instruction to move the shooting direction vertically (to perform tilt shooting), the information about the traffic signal SG can be reliably presented to the visually impaired by sound with just the instruction to change the shooting direction.
[0039] Furthermore, since the target of information presentation to the visually impaired is the traffic signal SG, the camera C does not need to have high performance, and information about the traffic signal SG can be presented to the visually impaired by sound.
[0040] Furthermore, when determining whether the traffic signal SG is captured in an image corresponding to image data obtained from camera C based on the clarity, size, distortion, orientation, or degree of match with template data as an image of a traffic signal in the image (see Figure 5), it is possible to accurately determine whether the traffic signal SG is captured in the image.
[0041] In addition, since the guidance process of the embodiment is executed on the terminal device T carried by the visually impaired person, it is possible to provide the visually impaired person with information regarding traffic signals SG and the like that he or she should check when traveling by sound, without the need for high performance camera C.
[0042] In addition, instead of the traffic signal SG described in the embodiment, the guidance processing of the embodiment can also be applied as is to a case where the above-mentioned road signs, etc., whose display content changes depending on the time, are photographed by camera C and the content is audibly reproduced for the visually impaired. [Variations]
[0043] Next, a modified example corresponding to the above-described embodiment will be described. In the following description, the same components as those of the terminal device T in the embodiment will be described using the same component numbers.
[0044] (A) First Modification First, as a first modification, in the above embodiment, a case has been described in which the walking movement of a visually impaired person is guided based on a determination as to whether the traffic signal SG is actually captured in an image corresponding to image data acquired from a camera C. However, in addition to this, it is also possible to configure the system so that an area in which the traffic signal SG may be captured in the image is identified, and whether the traffic signal SG is captured in the image of the area is determined, and if the traffic signal SG is captured in the area, a sound is emitted to inform the visually impaired person. According to the configuration of this first modification, it is possible to reliably determine whether the traffic signal SG is captured while making the visually impaired person aware of the existence of the area.
[0045] (B) Second Modification Next, as a second modification, in the above embodiment, a case has been described in which the contents of the traffic signal SG are sounded when the traffic signal SG is captured in an image corresponding to image data acquired from the camera C. However, in addition to this, for example, when a visually impaired person has weak eyesight, if it is determined that a traffic signal SG is captured in the image, the processing unit 10 may be configured to enlarge the image of the traffic signal SG itself and display it on the display 15. In this case, the processing unit 10 corresponds to an example of the "enlarged presentation means" of the present application. According to the configuration of this second modification, the traffic signal SG is displayed to a weak-sighted person (i.e., a person who can see the contents of the enlarged image), so that the traffic signal SG can be displayed in a manner that is easy for the weak-sighted person to recognize.
[0046] (C) Third Modification Next, as a third modified example, in the above-mentioned embodiment, the case where the guidance process of the embodiment is performed only by the terminal device T carried by the visually impaired person has been described. However, in addition to this, for example, some or all of the functions of the route information acquisition unit 1, image information acquisition unit 2, shooting range information acquisition unit 3, feature position information acquisition unit 4, determination unit 5, and sound emission control unit 6 of the embodiment can be aggregated in a server device (not shown) capable of transmitting and receiving information or data to and from the terminal device T via the network, and the guidance process of the embodiment can be configured to be performed mainly by the server device. With the configuration of this third modified example, the processing load of the terminal device T can be reduced.
[0047] (D) Fourth Variation Next, as a fourth modification, in the above embodiment, a case has been described in which a visually impaired person who is the target of walking guidance carries the terminal device T and receives the guidance. However, in addition to this, a configuration may be adopted in which a companion accompanying the visually impaired person carries the terminal device T. In this case, it is preferable for the companion to photograph the surroundings using the terminal device T based on instructions from the visually impaired person who is the target of guidance. According to the configuration of this fourth modification, the present application can also be applied to a case in which a companion accompanying a visually impaired person carries the terminal device T.
[0048] (E) Fifth Variation Next, as a fifth modification, in the above embodiment, the case where the target of walking guidance is a visually impaired person has been described, but the present invention can also be applied to other cases, such as when guiding elderly people on their walking. Even in this case, information about traffic signals SG can be presented to the elderly people by sound without the need for high performance of the camera C, as in the guidance process of the embodiment.
[0049] (F) Other variations Finally, as another modified example, each program corresponding to the guidance processing of the embodiment (see FIG. 3) can be recorded on a recording medium such as an optical disk or a hard disk, or obtained via a network such as the Internet, and then read out and executed by a general-purpose microcomputer or the like, thereby causing the microcomputer to function as the processing unit 10 of the embodiment. [Explanation of symbols]
[0050] 1. Route information acquisition means (route information acquisition unit) 2. Image information acquisition means (image information acquisition unit) 3. Shooting range information acquisition means (shooting range information acquisition unit) 4 Feature location information acquisition means (feature location information acquisition unit) 5 Judgment means (judgment section) 6 Sound emission control means (sound emission control section) 10 Processing section 15 Presentation medium: Display T terminal device S Information presentation device C. Shooting means (camera) SP Sound emission means (speaker) MP Map Data SG traffic signal
Claims
1. A route information acquisition means for acquiring route information regarding a route to a destination of a user; an image information acquiring means for acquiring image information indicating an image captured by an image capturing means for capturing an image of the user's surroundings; a photographing range information acquiring means for acquiring photographing range information indicating a photographing range of the photographing means when the image was photographed; a feature position information acquisition means for acquiring feature position information indicating the position of a target traffic information feature, which is a traffic information feature that the user should check when traveling along the route, based on map information in which traffic information features, which are features that present information related to traffic, are recorded and the acquired route information; a determination means for determining whether the target traffic information feature is captured in the image based on the acquired shooting range information, the acquired image information, and the acquired feature position information; a sound emission control means for, when the determination means determines that the target traffic information feature is included in the image, emitting a sound of the content of the target traffic information feature to the user using a sound emission means; Equipped with The information presentation device is characterized in that, when the determination means determines that the target traffic information feature is not captured in the image, the sound emission control means uses the sound emission means to emit a change instruction regarding the change of the shooting range to the user.
2. 2. The information presentation device according to claim 1, The determination means identifies a candidate area in which the target traffic information feature may be captured in the image captured by the imaging means based on the imaging range information, the image information, and the feature position information, and determines whether the target traffic information feature is captured in the candidate area; The information presentation device is characterized in that, when the target traffic information feature is captured in the candidate area, the sound emission control means causes a sound emission means to emit a sound to notify the user that the target traffic information feature is captured in the candidate area.
3. 3. The information presentation device according to claim 1, The information presentation device further comprises an enlargement presentation means for enlarging an image of the target traffic information feature in the image and presenting it to the user when the determination means determines that the target traffic information feature is captured in the image.
4. The information presentation device according to any one of claims 1 to 3, the change instruction is any one of an instruction to correct the shooting direction of the shooting means, an instruction to move the shooting direction in a horizontal direction, and an instruction to move the shooting direction in a vertical direction; The information presentation device is characterized in that the determination means determines whether the target traffic information feature is captured in the image based on the image information corresponding to the changed shooting direction.
5. The information presentation device according to any one of claims 1 to 4, The information presentation device is characterized in that the target traffic information feature is a traffic signal that the user should check when traveling along the route.
6. 6. The information presentation device according to claim 5, The information presentation device is characterized in that the determination means determines whether the traffic signal is captured in the image based on the shooting range information, the image information, the feature position information, and at least one of template image information corresponding to a template of the traffic signal, size information indicating the size of the image of the traffic signal captured in the image, or clarity information indicating the clarity of the image of the traffic signal.
7. The information presentation device according to any one of claims 1 to 6, The information presentation device is characterized in that the user is a visually impaired person whose vision is weaker than that of a normal person.
8. An information presentation method executed in an information presentation device including a route information acquisition means, an image information acquisition means, a photography range information acquisition means, a feature position information acquisition means, a determination means, and a sound emission control means, a route information acquisition step of acquiring route information regarding a route to a destination of a user by the route information acquisition means; an image information acquiring step of acquiring, by an image information acquiring means, image information indicating an image captured by an image capturing means that moves with the user and captures an image of the user's surroundings; a photographing range information acquisition step of acquiring photographing range information indicating a photographing range of the photographing means when the image was photographed by the photographing range information acquisition means; a feature position information acquisition step of acquiring, by the feature position information acquisition means, feature position information indicating the position of a target traffic information feature, which is a traffic information feature that the user should check when traveling on the route, based on map information in which traffic information features, which are features that present information related to traffic, are recorded and the acquired route information; a determination means for determining whether the target traffic information feature is captured in the image based on the acquired shooting range information, the acquired image information, and the acquired feature position information; a sound emission control step of, when the determination means determines that the target traffic information feature is captured, causing the sound emission control means to emit sound of the content of the target traffic information feature to the user using the sound emission means; Including, The information presentation method is characterized in that, in the sound emission control process, when the determination means determines that the target traffic information feature is not captured in the image, a change instruction regarding a change to the shooting range is emitted to the user by the sound emission means.
9. A computer included in the information presentation device, A route information acquisition means for acquiring route information regarding a route to a user's destination; an image information acquiring means for acquiring image information indicating an image captured by an image capturing means for capturing an image of the user's surroundings; a photographing range information acquiring means for acquiring photographing range information indicating a photographing range of the photographing means when the image was photographed; a feature position information acquisition means for acquiring feature position information indicating a position of a target traffic information feature, which is a traffic information feature that the user should check when traveling along the route, based on map information in which traffic information features, which are features that present information related to traffic, are recorded and the acquired route information; A determination means for determining whether the target traffic information feature is included in the image based on the acquired shooting range information, the acquired image information, and the acquired feature position information; and a sound emission control means for, when it is determined by the determination means that the target traffic information feature is photographed, emitting a sound of the content of the target traffic information feature to the user using a sound emission means; An information presentation program that functions as A program for presenting information, characterized in that the computer functioning as the sound emission control means functions to, when the computer functioning as the judgment means determines that the target traffic information feature is not captured in the image, use the sound emission means to emit a change instruction regarding a change to the shooting range to the user.
Citation Information
Patent Citations
Walking support system
JP2022107967A