Road guidance system and portable information terminal used therefor

Abstract

To provide a road guidance system and a portable information terminal used therefor which can determine a walking direction of a visually impaired person by using only a single braille block and execute road guidance by using an inexpensive device.SOLUTION: A road guidance system comprises a portable information terminal and a braille block. The portable information terminal includes an imaging processing unit, a QR code analysis processing unit that analyzes QR codes, a voice output processing unit, and a control unit. A QR code is affixed to the braille block. The control unit obtains QR code information by analyzing the QR code affixed to the braille block imaged by the imaging processing unit through the QR code analysis processing unit, generates road guidance information generated on the basis of the walking direction of a user holding the portable information terminal from the QR code information, and voice-outputs the road guidance information from the voice output processing unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a route guidance system that uses braille blocks. [Background technology]

[0002] In recent years, consideration has been given to the needs of visually impaired people, and there are more and more opportunities to see visually impaired people using white canes and relying on braille blocks to get around.

[0003] To assist visually impaired people in moving around, braille blocks are installed in public facilities such as train stations and on sidewalks at intersections.

[0004] According to JIS T9251-2001, there are two types of Braille blocks: guiding blocks (linear protrusions) and warning blocks (dot-like protrusions).

[0005] The guiding blocks have four or more rod-shaped protrusions arranged at regular intervals to guide people in the walking direction. The longitudinal direction of the rod-shaped protrusions indicates the walking direction.

[0006] The warning blocks are made up of more than 25 dot-like protrusions arranged in a grid pattern, warning people of possible danger ahead or the need to change direction.

[0007] When visually impaired people rely on braille blocks to get around, the blocks can inform them of the direction to go, warnings of dangers and warnings of changes in walking direction, etc., but they are unable to understand the intended purpose of the braille blocks, i.e., where the blocks are located and where they are intended to guide them.

[0008] Background art in this technical field includes Patent Document 1. Patent Document 1 describes a method in which an RFID tag is attached to a braille block, information stored in the RFID tag is received by a receiving device installed in the white cane of a visually impaired person, and the information is notified by voice. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Special Publication No. 2016-541065 Summary of the Invention [Problem to be solved by the invention]

[0010] In Patent Document 1, the walking direction of a visually impaired person relative to two or more Braille blocks is clear when the person passes through those Braille blocks, but the walking direction of a visually impaired person cannot be determined from a single Braille block alone. In addition, the system requires a dedicated RFID receiving device on the white cane, which imposes an economic burden on the visually impaired.

[0011] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide a route guidance system that enables visually impaired people to determine their walking direction using only a single braille block and that can provide route guidance using inexpensive equipment, and a portable information terminal for use therewith. [Means for solving the problem]

[0012] In order to solve the above-mentioned problems, the present invention provides, as an example, a route guidance system consisting of a mobile information terminal and braille blocks, in which the mobile information terminal has a photography processing unit, a QR code analysis processing unit that analyzes a QR code (registered trademark), an audio output processing unit, and a control unit, and the braille blocks have QR codes affixed to them, and the control unit analyzes the QR code affixed to the braille blocks photographed by the photography processing unit using the QR code analysis processing unit to obtain QR code information, generates route guidance information from the QR code information based on the walking direction of a user holding the mobile information terminal, and outputs the route guidance information as audio from the audio output processing unit. Effect of the Invention

[0013] By using the technology of the present invention, the forward, backward, left and right guidance of the braille blocks can be intuitively understood, reducing mistakes regarding walking direction. [Brief description of the drawings]

[0014] [Figure 1] FIG. 4 is a schematic diagram of a warning block in the first embodiment. [Diagram 2] 1 is an external view of a smartphone according to a first embodiment. [Diagram 3] FIG. 2 is a hardware configuration diagram of a smartphone according to a first embodiment. [Figure 4] FIG. 2 is a functional block diagram of the smartphone according to the first embodiment. [Diagram 5] 4 is a flowchart showing a processing procedure of the route guidance system in the first embodiment. [Figure 6] FIG. 11 is a diagram showing the arrangement of warning blocks in the second embodiment. [Figure 7] FIG. 13 is a diagram showing the layout of warning blocks in the third embodiment. [Figure 8] FIG. 13 is a diagram showing the layout of warning blocks in the fourth embodiment. [Figure 9] FIG. 13 is a diagram showing the layout of warning blocks in the fifth embodiment. [Figure 10] FIG. 13 is a diagram showing the layout of guide blocks in the sixth embodiment. [Figure 11] FIG. 13 is a system configuration diagram according to a seventh embodiment. [Figure 12] 13 is a flowchart showing a processing procedure of a route guidance system in a seventh embodiment. [Figure 13] FIG. 13 is a schematic diagram of a smart watch with a camera in Example 8. [Figure 14] FIG. 13 is a schematic diagram of an HMD in Example 9. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that although the present embodiment is an invention that is mainly targeted at visually impaired people, it is also effective for pedestrians other than the visually impaired. EXAMPLES

[0016] In this embodiment, a QR code is attached to the braille block, and the information stored in the QR code is linked to the walking direction of the visually impaired person, in other words, the information is converted and generated based on the walking direction and transmitted to the visually impaired person. In addition, by using an existing mobile information terminal such as a smartphone, the increase in the economic burden on the visually impaired person is reduced.

[0017] Here, QR codes, which use two-dimensional codes, can store information both vertically and horizontally, unlike barcodes, which only store information horizontally, and so have become commonplace with the spread of smartphones and other devices. These QR codes have square cut-out symbols (also called position detection patterns or finder patterns) at three corners, which allow the reference direction of the QR code to be detected. In addition, while it is difficult for visually impaired people to understand the contents of the display screen of a smartphone (personal digital assistant), it has been demonstrated that with support such as voice output, it is possible for them to use smartphone functions, although it depends on their familiarity.

[0018] Fig. 1 is a schematic diagram of a warning block, which is a braille block in this embodiment. In Fig. 1, 25 dot-like protrusions 2 (height 5 mm, base diameter 22 mm) are arranged in a grid pattern with a center interval of 60 mm on a 30 cm square warning block 1.

[0019] In addition, a 10 cm square QR code 3 is attached to the center of the warning block 1. A circular notch with a diameter of 22 mm is provided in the center of the QR code 3 to prevent it from getting caught on the dot-like protrusion 2, and this can be used as a guide when attaching the QR code.

[0020] Furthermore, QR code 3 has square cut-out symbols (4, 5, 6) arranged at three corners. The arrangement of these three cut-out symbols (4, 5, 6) determines the reference direction of QR code 3 affixed to warning block 1. The reference direction of the QR code affixed to warning block 1 in Fig. 1 is the direction with cut-out symbol 5 at the top left. In other words, this means that the direction from cut-out symbol 4 to cut-out symbol 5 is the forward direction, the direction from cut-out symbol 5 to cut-out symbol 4 is the backward direction, the direction from cut-out symbol 5 to cut-out symbol 6 is the rightward direction, and the direction from cut-out symbol 6 to cut-out symbol 5 is the leftward direction.

[0021] The QR code 3 contains route guidance information for the front, back, left and right of the reference direction of the QR code. By photographing this QR code 3 with a mobile information terminal such as a smartphone, the QR code 3 can be analyzed and the route guidance information of the QR code 3 can be obtained.

[0022] The route guidance information regarding the reference direction of the acquired QR code 3 can be converted and generated into forward, backward, left and right route guidance information based on the walking direction of the visually impaired person, and output as voice from a mobile information terminal such as a smartphone, thereby providing route guidance to the visually impaired person.

[0023] In this embodiment, an example in which a smartphone is used as a portable information terminal will be described. Fig. 2 shows an external view of the smartphone in this embodiment.

[0024] In FIG. 2, smartphone 10 is composed of a front side 13 having a display screen 11 made of a touch panel and a front camera (also called an in-camera) 12 for taking selfies, a back side 15 having a rear camera (also called an out-camera or simply a camera) 14, and a side of the smartphone.

[0025] An earphone microphone terminal (not shown) is provided on the side of smartphone 10 facing the front camera 12 (the right side in FIG. 2), an external connection terminal (not shown) is provided on the side of smartphone 10 facing the opposite direction to front camera 12 (the left side in FIG. 2), a power key and volume keys (not shown) are provided on the right side of smartphone 10 facing the front camera 12 (the lower side in FIG. 2), and a card tray section (not shown) is provided on the left side of smartphone 10 facing the front camera 12 (the upper side in FIG. 2).

[0026] When the smartphone 10 is faced downward and the QR code 3 in the warning block 1 is photographed with the rear camera 14 of the smartphone 10, the screen of the QR code 3 in the warning block 1 is displayed on the display screen 11 of the smartphone 10, as shown in Fig. 2. The photographed QR code 3 is analyzed by a QR code reading application that is built into the smartphone 10 as standard.

[0027] This embodiment realizes a QR code route guidance application (hereinafter referred to as a "route guidance system") that works in conjunction with a QR code reading application.

[0028] 3 is a hardware configuration diagram of a smartphone in this embodiment. In FIG. 3, the smartphone 10 is composed of a main control device 21, a system bus 22, a storage device 40, a sensor device 50, a communication processing device 60, a video processing device 70, a sound processing device 80, and an operation input device 90.

[0029] The main control device 21 is a microprocessor unit that controls the entire smartphone 10 in accordance with a predetermined operating program such as an application.

[0030] The system bus 22 is a data communication path for transmitting and receiving various commands, data, and the like between the main control device 21 and each component block within the smartphone 10.

[0031] The storage device 40 is composed of a program section 41 that stores programs for controlling the operation of the smartphone 10, a data section 42 that stores various data such as operation setting values, detection values ​​from a sensor device described later, and objects including contents, and a rewritable program function section 43 such as a work area used in various program operations. The storage device 40 can store operation programs downloaded from a network and various data created by the operation programs. It can also store content such as videos, still images, and audio downloaded from a network. It can also store data such as videos and still images captured using a camera section. The storage device 40 needs to hold the stored information even when power is not supplied to the smartphone 10 from the outside. Therefore, for example, devices such as semiconductor element memories such as flash ROM and SSD (Solid State Drive) are used. The operation programs stored in the storage device 40 can be updated and expanded by downloading from each server device on the network.

[0032] The sensor device 50 is a sensor group of various sensors for detecting the state of the smartphone 10. The sensor device 50 is composed of a GPS (Global Positioning System) receiver 51, a geomagnetic sensor device 52, a distance sensor device 53, an acceleration sensor device 54, and a gyro sensor device 55. These sensors make it possible to detect the position, tilt, direction, movement, and movement direction of the smartphone 10. The smartphone 10 may further include other sensors such as an illuminance sensor and a proximity sensor. Furthermore, if a device that pairs with these sensors is attached to the hand or arm, the movement of the hand or arm can be detected. By comprehensively utilizing these sensors, it is possible to detect the walking direction of the user who owns the smartphone 10, the tilt and direction of the smartphone 10, and the like.

[0033] The communication processing device 60 is made up of a LAN (Local Area Network) communication unit 61 and a telephone network communication unit 62 .

[0034] The LAN communication unit 61 is connected to a network such as the Internet via an access point or the like, and transmits and receives data to and from each server device on the network. The connection to the access point or the like may be made wirelessly via Wi-Fi (registered trademark) or the like.

[0035] The telephone network communication unit 62 performs telephone communication (calls) and data transmission and reception through wireless communication with base stations of a mobile telephone communication network. The communication with the base stations may be performed by W-CDMA (Wideband Code Division Multiple Access) (registered trademark), GSM (Global System for Mobile communications) (registered trademark), LTE (Long Term Evolution), or other communication methods. The LAN communication unit 61 and the telephone network communication unit 62 each include an encoding circuit, a decoding circuit, an antenna, and the like. The communication processing device 60 may further include other communication units such as a BlueTooth (registered trademark) communication unit or an infrared communication unit.

[0036] The video processing device 70 is composed of an imaging unit 71 and a display unit 72. The imaging unit 71 is a camera unit that inputs image data of the surroundings, a QR code, or other object by converting light input from a lens into an electrical signal using an electronic device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The display unit 72 is a display device such as a liquid crystal panel, and provides the image data to the user of the smartphone 10. The display unit 72 includes a video RAM (not shown). The image data input to the video RAM is displayed on the display screen. The imaging unit 71 captures an image of the QR code.

[0037] The audio processing device 80 is composed of an audio input / output unit 81, a voice recognition unit 82, and an audio decoding unit 83. The audio input of the audio input / output unit 81 is a microphone, and the user's voice, etc. is converted into audio data and input. The audio output of the audio input / output unit 81 is a speaker or earphone, and outputs audio information, etc. required by the user. The voice recognition unit 82 analyzes the input audio information and extracts instruction commands, etc. The audio decoding unit 83 has a function of performing decoding processing (voice synthesis processing) of the encoded audio signal as necessary.

[0038] The operation input device 90 is an instruction input unit that inputs operation instructions to the smartphone 10. The operation input device 9 is composed of a touch panel on a display screen, an operation key with an arrangement of button switches, etc. Other operation devices may be further provided. The smartphone 10 may be operated using a separate mobile terminal device connected by wired or wireless communication using the communication processing device 60. In addition, the smartphone 10 may be operated by a voice command of an operation instruction using the voice recognition unit 82 of the voice processing device 80.

[0039] 3 includes many components that are not essential to this embodiment, but the effects of this embodiment are not impaired even if these components are not included. Also, components not shown, such as a digital broadcast receiving unit and an electronic money settlement unit, may be added.

[0040] Fig. 4 is a functional block diagram of the smartphone in this embodiment. In Fig. 4, the control unit 30 is mainly composed of the main control device 21, and the program unit 41 and program function unit 43 of the storage device 40. The control unit 30 executes the application of the route guidance system in this embodiment.

[0041] The various sensor information acquisition unit 31 has a function of acquiring information from various sensors of the sensor device 50, and grasps the inclination of the smartphone 10, the walking direction, and the like.

[0042] The photographing processing unit 32 has a function of photographing a QR code by the imaging unit 71 of the video processing device 70. The photographed QR code is analyzed by the QR code analysis processing unit 33, and information stored in the QR code is extracted.

[0043] The QR code information storage unit 34 has a function of storing the extracted QR code information in the various data unit 42 of the storage device 40.

[0044] The audio output processing unit 35 has a function of outputting, via the speaker or earphones of the audio input / output unit 81, audio that assists in photographing and audio that takes into account the user's walking direction regarding QR code information.

[0045] The communication processing unit 36 ​​is mainly composed of a LAN communication unit 61 of the communication processing device 60, and has the function of uploading various information of the smartphone 10 to an external network server via the Internet network, and downloading various information from an external network server via the Internet network.

[0046] The route guidance system in this embodiment is a system that determines the voice to be output by combining the QR code information obtained by the QR code analysis processing unit 33 with the walking direction of the smartphone owner obtained by the various sensor information acquisition unit 31, and outputs the voice through a speaker or earphones by the voice output processing unit 35. Of course, it goes without saying that it is also possible to display text information on the display screen without outputting voice.

[0047] Here, we will explain the procedure that a visually impaired person follows to photograph a QR code with a smartphone, obtain the QR code information, and receive audio guidance, using the flowchart in Figure 5.

[0048] Visually impaired people use their white canes or the feel of the soles of their feet to find the location of the braille blocks. Braille blocks include guiding blocks that help guide your walking direction, and warning blocks that warn of possible dangers ahead or the need to change walking direction.

[0049] When a visually impaired person determines that a braille block is a guide block, he or she walks along the longitudinal direction of the rod-shaped protrusions on the guide block.

[0050] The flowchart in Figure 5 shows the steps a visually impaired person takes from the point at which they determine that a braille block is a warning block, to obtaining QR code information and receiving audio guidance.

[0051] In FIG. 5, when the QR code processing procedure is started (S301), the visually impaired person first confirms the position of the warning block by feeling it with his or her white cane (S302).

[0052] Next, the visually impaired person confirms by touch whether the smartphone is facing the front or back, and starts the shooting operation using the shooting processing unit 32 by pointing the rear camera on the back of the smartphone at the warning block (S303). Of course, depending on the application, it goes without saying that it is also possible to shoot with the front camera on the front of the smartphone. The instruction to start shooting is given by a voice command such as "shoot" or "camera". This shooting state by the smartphone continues until the shooting ends normally or until an instruction to stop shooting is given.

[0053] Next, the audio output processing unit 35 outputs a shooting assistance sound that changes the inclination of the smartphone or the distance to the QR code as necessary so that the QR code can be properly recognized depending on the inclination of the smartphone or the distance from the smartphone to the QR code of the warning block (S304). For example, shooting assistance sound such as "Please move the camera closer to the warning block," "Please move it a little more to the right," or "Please tilt it a little further forward" is output. The visually impaired person corrects the attitude of the smartphone based on the output shooting assistance sound. Of course, shooting assistance sound is not output if it is not necessary.

[0054] Next, whether the photograph was successful or not is judged by the voice output of "Photography was successful" or "Photography could not be completed" (S305). If the QR code is not photographed normally within a specified timeout period, it is judged that the photograph was not successful. The voice output of "Photography was successful" can be omitted because it can be substituted with the shutter sound.

[0055] If the application determines in the process of S305 that photographing was not performed normally, the process ends (S308).

[0056] If the application determines in the process of S305 that the image capture was successful, the QR code analysis processing unit 33 determines whether a QR code exists (S306). If the QR code cannot be normally recognized, it is determined that no QR code exists.

[0057] If it is determined that no QR code exists, the presence or absence of a QR code is output as a voice message. For example, a voice message such as "There is no information" or "Information has been obtained" may be output. The voice message "Information has been obtained" may be omitted.

[0058] If it is determined in S306 that the QR code does not exist, the processing procedure of the route guidance system is terminated (S308).

[0059] If it is determined in S306 that a QR code exists, the QR code analysis processing unit 33 of the smartphone determines the reference direction of the warning block 1 based on the arrangement of the three cut-out symbols, analyzes the contents of the QR code, stores the information in the QR code information storage unit 34, and outputs the information as audio by the audio output processing unit 35 (S307).

[0060] When the warning block is reached without passing through the guidance block, the reference direction of the QR code may not match the walking direction. When the reference direction of the QR code and the walking direction are 0 degrees, 90 degrees to the right, 180 degrees to the right, or 90 degrees to the left, the information on front, back, left, and right is interpreted differently. When the angle is any other than these, a voice instruction is given to change the walking direction to one of 0 degrees, 90 degrees to the right, 180 degrees to the right, or 90 degrees to the left, or a voice output processing unit 35 outputs a voice in which the angle between the walking direction and the reference direction of the warning block is corrected. After that, the processing procedure of the route guidance system is terminated (S308).

[0061] Next, the information of the QR code output by the audio output processing unit 35 will be described using the example of the warning block in FIG.

[0062] The reference direction of the QR code affixed to the warning block 1 in Fig. 1 is the direction in which the cut-out symbol 5 points to the upper left. The QR code 3 also contains information about the front, back, left and right with respect to the reference direction of the QR code. For example, information such as a descending staircase 5m ahead, a fork 20m behind, an elevator 10m to the right, and platform 3 15m to the left is included. Each QR code also contains information about the distance to the next braille block in that direction where a QR code exists (hereinafter, a braille block where a QR code exists is referred to as a "QR code braille block").

[0063] The walking direction of a visually impaired person carrying the smartphone 10 is determined by a various sensor information acquisition unit 31 of the smartphone 10.

[0064] Based on the walking direction of the visually impaired person, the QR code information is output as voice by the voice output processing unit 35. For example, if the walking direction of the visually impaired person is the reference direction of the QR code (from bottom to top in FIG. 1), the voice output by the voice output processing unit 35 based on the walking direction of the visually impaired person is "There are stairs going down 5 m ahead. It is 2 m to the next QR code braille block," "There is a fork in the sidewalk 20 m behind. It is 10 m to the next QR code braille block," "There is an elevator 10 m to the right. It is 5 m to the next QR code braille block," and "Platform 3 is 15 m to the left. It is 7 m to the next QR code braille block."

[0065] For example, if the walking direction of a visually impaired person is from right to left with respect to the reference direction of the QR code, the voice output based on the walking direction of the visually impaired person will be, "There are descending stairs 5 m to the right. The next QR code tactile paving is 3 m away," "There is a fork in the sidewalk 20 m to the left. The next QR code tactile paving is 10 m away," "There is an elevator 10 m behind. The next QR code tactile paving is 5 m away," "Platform 3 is 15 m ahead. The next QR code tactile paving is 7 m away."

[0066] Furthermore, voice guidance can be provided by associating the characteristic parts of the smartphone 10 with the direction, regardless of the walking direction. For example, if the front of the smartphone 10 is facing up and the direction of the front camera 12, which is the longitudinal direction, is the right direction, the front, back, left and right can be respectively replaced with the direction of the card tray part being the front, the direction of the earphone microphone terminal being the right, the direction of the power key being the rear, and the direction of the external connection terminal being the left.

[0067] In this way, the route guidance system of this embodiment has the advantage that the visually impaired person can intuitively understand the route guidance that is based on the direction of his / her walking, and has the advantage that route guidance can be provided even when there is only one warning block.

[0068] The method of this embodiment can also be applied to able-bodied people other than the visually impaired. When applied to able-bodied people or the hearing impaired, route guidance based on the person's walking direction is displayed on the display screen as figures such as arrows indicating the direction of movement or text sentences without audio output. Of course, for able-bodied people, audio output can be used in combination.

[0069] In addition, to use the route guidance system of this embodiment, it is desirable for the user to specify in advance on the smartphone whether they are visually impaired, hearing impaired, healthy, etc., and to tune the smartphone application for each user.

[0070] In this way, in this embodiment, a QR code is attached to an existing braille block, the QR code is photographed with an existing mobile information terminal such as a smartphone, and the information stored in the QR code is output as audio in conjunction with the user's walking direction. This makes it easier for the user to intuitively understand the forward, backward, left, and right directions, reducing mistakes regarding the walking direction. In addition, since the user's walking direction can be determined by a single braille block and can be implemented with only a general-purpose mobile information terminal, a route guidance system that can perform route guidance with an inexpensive device can be provided. EXAMPLES

[0071] In the first embodiment, the case where there is one warning block is assumed, but in the present embodiment, a response to the case where there are a plurality of warning blocks will be described.

[0072] Fig. 6 is a diagram showing the layout of warning blocks in this embodiment. As shown in Fig. 6, five warning blocks are placed, and a QR code is attached to each warning block. In Fig. 6, the reference direction of the QR codes is the same for all of them.

[0073] Warning block 400 is placed in the center, warning block 401 is placed above the reference direction, warning block 402 is placed to the right of the reference direction, warning block 403 is placed below the reference direction, and warning block 404 is placed to the left of the reference direction, and each warning block has a QR code (410, 411, 412, 413, 414) affixed to it.

[0074] Each QR code stores information about the placement of the five warning blocks: QR code 410 stores information about the center, QR code 411 stores information about the top, QR code 412 stores information about the right, QR code 413 stores information about the bottom, and QR code 414 stores information about the left.

[0075] The QR code processing procedure is the same as that shown in the flowchart of FIG. 5 in the first embodiment, but the audio output is different depending on the detected warning block.

[0076] A visually impaired person follows the guiding blocks until he or she reaches a warning block that is in the walking direction.

[0077] When the walking direction is from bottom to top in FIG. 6, the first warning block that is reached is warning block 403 located below central warning block 400.

[0078] Based on the information stored in the QR code 413 affixed to the warning block 403, since it is a warning block ahead, audio such as "This is a warning block ahead," "There are stairs leading down 5 m ahead," and "There is a fork in the sidewalk 20 m behind you" is output.

[0079] When the user reaches the next warning block 400 in the walking direction, the information stored in the QR code 410 affixed to the warning block 400 indicates that it is the central warning block, and audio such as "This is the central warning block," "There are stairs leading down 5m ahead," "There is a branching path 20m behind," "There is an elevator 10m to the right," and "Platform 3 is 15m to the left" is output.

[0080] For example, if one wishes to go to platform 3, one changes walking direction to the left at the position of warning block 400, and detects the next warning block 404, which is to the left of the central warning block 400. Because the walking direction is from right to left, the information stored in the QR code 414 affixed to warning block 404 indicates that it is a warning block ahead for visually impaired people, and voices such as "This is a warning block ahead," "Platform 3 is 15 m ahead," and "There is an elevator 10 m behind" are output. Because the walking direction is fixed, the voice output of "There is an elevator 10 m behind" can be omitted.

[0081] In this way, the visually impaired person can intuitively and easily understand route guidance based on their own walking direction, and the warning blocks are multiple, so there is an advantage that the walking direction can be confirmed reliably.

[0082] The method of this embodiment can also be applied to people other than the visually impaired. When applied to hearing impaired people and able-bodied people other than the visually impaired, route guidance based on the person's walking direction can be displayed in text on the display screen without audio output. It goes without saying that for hearing impaired people and able-bodied people other than the visually impaired, the purpose can be achieved with only the information stored in the QR code 410 affixed to the central warning block 400. EXAMPLES

[0083] In the second embodiment, the reference directions of the QR codes affixed to the five warning blocks are all the same, but in the present embodiment, a case will be described in which the reference directions of the QR codes are different.

[0084] 7 is a diagram showing the layout of warning blocks in this embodiment, in which the reference directions of the QR codes affixed to the warning blocks are different. In Fig. 7, the reference directions of QR codes 415, 416, 417, and 418 affixed to warning blocks 401, 402, 403, and 404 on the front, back, left, and right sides of the central warning block 400 are all set to the direction of the central warning block 400.

[0085] Even in such a case, since information regarding the arrangement of the five warning blocks is stored in each QR code, the same effects and advantages as in the second embodiment can be obtained without depending on the reference direction of the QR code. EXAMPLES

[0086] In the second and third embodiments, it was assumed that the path branches in four directions (front, back, left, right, etc., including the walking direction) (each direction differing by 90 degrees), but in this embodiment, it will be explained that the present invention can also be applied to paths that branch in other directions other than front, back, left, and right.

[0087] 8 is a diagram showing the layout of warning blocks in this embodiment when there is no sidewalk in the straight ahead direction, but there is a sidewalk 45 degrees to the left. As shown in Fig. 8, there is no warning block in front of the central warning block 400, and warning block 405 is placed 45 degrees to the left in front of the central warning block 400 from the reference direction.

[0088] The QR code 415 of the warning block 405 stores information indicating that it is tilted forward and to the left at 45 degrees from the reference direction of the central warning block 400. Instead of expressing "forward" as in the previous embodiment, this embodiment expresses it as "45 degrees to the left." Also, at the position of the warning block 405 (position of the QR code 415), it expresses "This is a warning block 45 degrees to the left." Also, in the QR code 415 of the warning block 405, the right direction is expressed as 135 degrees to the right, the left direction is expressed as 45 degrees to the left, and the rear is expressed as 135 degrees to the left, relative to the reference direction of the central warning block 400.

[0089] According to this embodiment, the visually impaired person can recognize in advance the presence of a warning block 45 degrees to the left, and therefore can eliminate anxiety about an unknown fork in the road.

[0090] Of course, it goes without saying that for people who are not visually impaired, such as those with hearing impairments or normal eyes, it is relatively easy to obtain the QR code information for the next warning block, since all they need to do is find the guidance block with the QR code attached. EXAMPLES

[0091] In Example 1, there is no mention of the relationship between the reference direction of the QR code affixed to the warning block and the absolute directions of north, south, east, and west. In this example, however, a case is described in which the reference direction of the QR code affixed to the warning block coincides with the absolute directions of north, south, east, and west.

[0092] FIG. 9 is a schematic diagram showing an example in which the reference direction of the QR code 7 affixed to the warning block 1 in this embodiment is made to coincide with the absolute directions of north, south, east and west.

[0093] In this case, the QR code 7 contains information on the deviation angle of the warning block 1 from the absolute orientation of north, south, east, and west. FIG. 9 shows an example in which the top of the figure is north and the reference direction of the QR code 7 coincides with the north direction, resulting in the warning block 1 being tilted 45 degrees to the left. Thus, in the case of FIG. 9, information indicating that the warning block 1 is tilted 45 degrees to the left is stored in the QR code 7. Therefore, in order to convert information on front, back, left, and right for a visually impaired person walking along the guide blocks, it is necessary to convert all of the information stored in the QR code 7 into information tilted 45 degrees to the left. In other words, the 45-degree left direction of the QR code 7 is the forward direction, the 45-degree right direction of the QR code 7 is the rightward direction, the 135-degree right direction of the QR code 7 is the backward direction, and the 135-degree left direction of the QR code 7 is the leftward direction.

[0094] This allows visually impaired people to obtain the same effects and advantages as in the first embodiment described above.

[0095] For those who are not visually impaired, such as the hearing impaired and able-bodied, the absolute direction information of the QR code can be displayed on the display screen, which makes it easier to understand as it matches the image with the map. EXAMPLES

[0096] In the first to fifth embodiments, warning blocks are described, but in this embodiment, an example using guiding blocks will be described.

[0097] Fig. 10 is a schematic diagram of the guide block in this embodiment. In Fig. 10, guide block 101, which is 30 cm square, has four rod-shaped protrusions (height 5 mm, width at bottom 27 mm, length in the longitudinal direction 280 mm) 102 arranged with a center-to-center distance of 75 mm.

[0098] The QR code 103 is attached to approximately the center of the guide block 101, and its size is 37.5 mm square so that it fits between the rod-shaped protrusions 120.

[0099] Square cut-out symbols (104, 105, 106) are arranged at three corners of the QR code 103. The arrangement of these three cut-out symbols (104, 105, 106) determines the reference direction of the guidance block 101. In other words, the direction from cut-out symbol 104 to cut-out symbol 105 (the longitudinal direction of the rod-shaped protrusion) is the guidance direction of the guidance block.

[0100] The QR code 103 includes information about the front and rear with respect to the QR code reference direction. For example, information such as "a descending staircase 5 m ahead, a fork 20 m behind" is included.

[0101] It is desirable to attach QR codes to all directional blocks, but for example, if there are four consecutive directional blocks (120 cm) without QR codes, a QR code can be attached to the next directional block. In other words, QR code information can be obtained every 150 cm. A visually impaired person can reach a directional block with a QR code by tracing a maximum of five directional blocks. After that, they can skip four directional blocks and obtain the QR code information at the next directional block.

[0102] For those who are not visually impaired, such as those with hearing impairments or normal eyes, the information in the QR code can be obtained relatively easily by simply finding the guide block with the QR code attached. EXAMPLES

[0103] In the first embodiment, the route guidance system was constructed using only a smartphone, but in the present embodiment, a method for improving the effectiveness by using external information will be described.

[0104] FIG. 11 is a system configuration diagram in this embodiment in which a smartphone 10 and a route guidance server 502 that manages route guidance information are connected via a wireless router 503, a network 504, and a wireless router 505.

[0105] The owner of the smartphone 10 can receive route guidance using information from the route guidance server 502.

[0106] The route guide server 502 stores map information, facility information (including information indicating the locations of toilets, stairs, elevators, etc.), and information on the placement of braille blocks, etc. Effective examples include ticket gate and exit information in train stations, and store placement information in shopping malls.

[0107] A visually impaired person must check in advance whether information about the place he or she will be using (such as the name of a station or a facility such as a shopping mall) is available on the route guide server 502, and must download the necessary information.

[0108] By storing in the QR code attached to the braille block the absolute position information (latitude, longitude, etc.) of the QR code, unique identification information (number) that individually identifies the braille block, link information of the route guide server 502, etc., it is possible to improve consistency with the information of the route guide server 502. In addition, link information other than that of the route guide server 502 for obtaining necessary information via the network 504 can also be stored.

[0109] It is desirable for visually impaired people to download information from the route guide server 502 regarding the places they will be using to their smartphones 10 in advance.

[0110] FIG. 12 is a flow chart showing the procedure for a visually impaired person to reach a desired destination using route guidance information stored in the route guidance server 502 in this embodiment.

[0111] In FIG. 12, when the route guidance system using route guidance information stored in the route guidance server 502 is started (S601), the visually impaired person first checks his / her current location using the various sensor information acquisition unit 31 of the smartphone 10 (S602).

[0112] The position of the smartphone 10 on a plane is detected by the GPS receiver 51 of the sensor device 50. By receiving radio waves from multiple GPS satellites, the position coordinates (latitude, longitude) of the smartphone 10 can be detected. In addition to GPS information from GPS satellites, location information can also be obtained using information transmitted from Wi-Fi, Bluetooth, and mobile base stations, so when GPS information cannot be obtained, the location information of the smartphone 10 is obtained using such information. Of course, it goes without saying that such information including GPS information can also be combined. In the route guidance system of this embodiment, when location information cannot be obtained from GPS information or other information, it is complemented by information from the acceleration sensor device 54 and the gyro sensor device 55 of the sensor device 50. Then, when location information from GPS information or other information can be obtained at some point during walking, it is corrected.

[0113] Next, the destination is set in the smartphone 10 (S603). To set the destination, voice of the destination is input from the microphone of the voice input / output unit 81 of the voice processing device 80, and the destination is recognized and set by the voice recognition unit 82. If the corresponding destination is not found in the route guiding server 502, a voice saying "The desired destination cannot be found. Please set another destination" is output by the voice output processing unit 35. It is desirable to register destinations frequently used by visually impaired people in the route guiding server 502 in advance.

[0114] When the destination is set in the smartphone 10 in the process of S603, necessary route information to the destination is downloaded from the route guiding server 502 to the smartphone 10. Thereafter, the information of the route guiding server 502 stored in the smartphone 10 is used.

[0115] Of course, if information on the location to be used from the route guide server 502 is downloaded to the smartphone 10 in advance, the necessary route information to the destination can be generated by an application on the smartphone 10.

[0116] Next, based on the current location information of the smartphone 10 and the Braille block placement information obtained by the communication processing unit 36 ​​from the route guidance server 502, the application identifies the Braille block that is closest to the current location of the smartphone 10, and the distance and walking direction from the current location of the smartphone 10 are provided by audio guidance via the audio output processing unit 35 (S604).

[0117] Next, the visually impaired person uses the feel of his or her white cane to search for the distance and walking direction from the current position of the smartphone 10 or the vicinity thereof, as instructed by the application of the smartphone 10, and detects the presence of braille blocks (S605).

[0118] Next, it is determined whether the braille block detected in the process of S605 is a warning block or not by the feel of the visually impaired person's white cane (S606).

[0119] If the process of S606 determines that the detected braille block is not a warning block, it means that the detected braille block is a guide block. When the guide block is detected, the detection of the guide block is input to the smartphone 10 by voice or button, etc., and the walking direction to proceed is indicated based on the angle between the walking direction of the smartphone 10 and the guide direction (longitudinal direction) of the detected guide block from the arrangement information of the braille block obtained from the route guide server 502 (S607). After that, the process returns to the braille block search process of S605.

[0120] In addition, since the longitudinal direction of the rod-shaped protrusions of the guide blocks is the guide direction of the guide blocks, the walking direction is known, and it is considered that there is no practical problem even if the user does not search for Braille blocks with QR codes until a warning block is detected. In addition, it is not necessary to input the detection of the guide block to the smartphone 10 by voice or button, etc. Naturally, during that time, no instruction on the walking direction to proceed (processing of S607) is generated.

[0121] If the process of S606 determines that the detected braille block is a warning block, the QR code is photographed by the photographing processing unit 32 in the procedure described in the flowchart of FIG. 5 for confirmation, and the information stored in the QR code is analyzed by the QR code analysis processing unit 33 to obtain QR code information (S608). In this case, the position information of the obtained QR code is matched with the braille block arrangement information stored in the smartphone, and the physical position information of the smartphone can be corrected. This means that for all QR code braille blocks, the physical position information of the smartphone can be corrected by the position information of each QR code. The obtained information is stored by the QR code information storage unit 34.

[0122] Next, the location of the warning block is identified from the acquired QR code information, and a direction instruction is obtained (S609). That is, the application of the smartphone 10 uses the voice output processing unit 35 to give voice instructions on the walking direction to proceed.

[0123] Here, depending on whether the detected warning block is located in the front, back, left, right, or center with respect to the walking direction, the content of the audio guidance output by the audio output processing unit 35 will differ. For example, assuming that the walking direction to reach the destination is to the left of the reference direction of the QR code and the walking direction to reach the warning block matches the reference direction of the QR code, the audio output by the audio output processing unit 35 will be output as follows: (1) If the detected warning block is located in front of the central warning block, the message will say, "Go left, passing the next warning block in front of you. The next QR code braille block is 10 m away." (2) If the detected warning block is the center warning block, the message will be "Please proceed in the direction of the next warning block on your left. The next QR code braille block is 5 m away." If the detected warning block is to the left of the center warning block, the message will be "Please proceed to the left. The next QR code braille block is 7 m away." (3) If the detected warning block is to the right of the central warning block, the message will say, "Proceed in that direction, passing the next warning block on your left. The next QR code braille block is 15 m away." (4) If the detected warning block is located in front of the center warning block, the message "Proceed to the left, passing through the next warning block behind." Next, it is determined whether the destination has been reached (S610). The determination as to whether the destination has been reached is performed by an application of the smartphone 10. That is, it is determined whether the acquired QR code information is a QR code braille block of the destination.

[0124] If it is determined in the process of S610 that the destination has not yet been reached, the voice output processing unit 35 issues voice instructions on the direction and distance to the next QR code braille block, and the process returns to the braille block search process of S605.

[0125] If it is determined in the process of S610 that the destination has been reached, the route guidance system using external information is terminated (S611).

[0126] For people who are hearing impaired or able-bodied but not visually impaired, directions can be given relatively easily using text information and map information (such as arrows showing the walking direction) on the display screen of the smartphone 10 without detecting braille blocks.

[0127] As an application of this embodiment, when a device that can obtain high accuracy (on the order of centimeters) of GPS becomes available at low cost, the device will be incorporated into a smartphone, and the location of each braille block will become clearly identifiable. In that case, the location of the QR code braille block will be identified from the braille block arrangement information stored in the smartphone, and the smartphone will be guided directly above the QR code braille block. In addition, it will be possible to indicate the direction and distance to the next QR code braille block. This will further improve convenience. EXAMPLES

[0128] In the first to seventh embodiments, it is assumed that a smartphone is possessed and used, but in the present embodiment, a case other than a smartphone will be described.

[0129] Possible application examples other than smartphones include conventional mobile phones, tablet devices, notebook PCs, and other information terminals. However, taking into consideration that visually impaired people carry white canes, it would be preferable to use conventional mobile phones or tablet devices.

[0130] Of course, it goes without saying that this technology can also be applied to notebook PCs by people other than the visually impaired, such as the hearing impaired and healthy people.

[0131] Fig. 13 is a schematic diagram of a smart watch with a camera in the case where the smart watch is used as an application example other than a smartphone in this embodiment. As shown in Fig. 13, a smart watch 710 has a built-in image capturing unit in its belt 711, and can capture images using a camera lens 712.

[0132] In addition, voice guidance can be provided by associating the characteristic parts of the smart watch 710 with the direction, regardless of the walking direction. For example, if the camera lens side of the smart watch 710 is the upward direction, the front, back, left, and right can be replaced with the direction of the camera lens side.

[0133] In addition, the direction of the hour hand of a clock can be used to express front, back, left and right. For example, the front is expressed as 12 o'clock, the right as 3 o'clock, the back as 6 o'clock, and the left as 9 o'clock.

[0134] As described above, in this embodiment, the same effect as in the above-described embodiment can be obtained by photographing a QR code attached to a Braille block using a camera-equipped smart watch. EXAMPLES

[0135] In this embodiment, a case where a head mounted display (HMD) is used as an application example other than a smartphone will be described.

[0136] FIG. 14 is a schematic diagram of an HMD when the HMD is applied as an application example other than a smartphone in this embodiment.

[0137] As shown in Fig. 14, the HMD 720 is a non-transparent HMD with a built-in imaging unit that can capture images. By capturing an image of a QR code attached to a braille block, the same effect as in the above embodiment can be obtained.

[0138] By using a non-transparent HMD 720, it is possible to determine the direction in which the face of the person wearing the non-transparent HMD 720 is facing, and it is also possible to instruct the walking direction based on the forward, backward, left, and right directions relative to the face direction.

[0139] Needless to say, the see-through HMD can be used for people with hearing impairments and healthy people other than the visually impaired. For people with hearing impairments and healthy people other than the visually impaired, the see-through HMD can also display images of the real world and computer-generated images (avatars) of augmented reality (AR) superimposed on the display screen, which adds the added benefit of enabling directions to be given by the avatar.

[0140] In addition, the amount of information stored in a QR code can be increased by using QR code connections or the rectangular iQR code standard.

[0141] Although the embodiments of the present invention have been described above, it goes without saying that the configuration for realizing the technology of the present invention is not limited to the above-described embodiments, and various modifications are possible. For example, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. All of these belong to the scope of the present invention. Furthermore, the numerical values, messages, etc. appearing in the text and figures are merely examples, and the effect of the present invention will not be impaired even if different ones are used.

[0142] In the above explanation, an example using a QR code has been described, but it goes without saying that the effects of the present invention can be obtained even if a two-dimensional code called a matrix type two-dimensional code is used instead of a QR code. A matrix type two-dimensional code is surrounded by a square frame or an L-shaped frame to make it easier to detect the position of the two-dimensional code, and a characteristic mark called a finder pattern (cut-out symbol) is placed inside the symbol, so that the reference direction of the two-dimensional code can be determined by detecting this.

[0143] In addition, the functions of the present embodiment described above may be realized in part or in whole by hardware, for example, by designing them as an integrated circuit. In addition, hardware and software may be used together. The software may be stored in the program section 41 of the smartphone beforehand at the time of product shipment. It may be acquired from various server devices on the Internet after product shipment. It may also be acquired by being provided on a memory card, an optical disk, or the like. [Explanation of symbols]

[0144] 1: warning block, 2: dot-like protrusion, 3: QR code, 4-6: cut-out symbol, 10: smartphone, 30: control unit, 31: various sensor information acquisition unit, 32: photography processing unit, 33: QR code analysis processing unit, 34: QR code information storage unit, 35: audio output processing unit, 36: communication processing unit, 101: guidance block, 102: rod-like protrusion, 103: QR code, 104-106: cut-out symbol, 502: route guidance server, 710: smart watch, 720: HMD

Claims

1. A route guidance system consisting of a mobile information terminal and braille blocks, The portable information terminal has a photographing processing unit, a QR code analysis processing unit that analyzes a QR code, a voice output processing unit, and a control unit, The braille block has a QR code attached thereto, The control unit analyzes the QR code affixed to the Braille block photographed by the photographing processing unit using the QR code analysis processing unit to obtain QR code information, generates route guidance information from the QR code information based on the walking direction of the user holding the mobile information terminal, and outputs the route guidance information by voice from the audio output processing unit.

2. 2. The route guidance system according to claim 1, The control unit generates the route guidance information based on a reference direction of the QR code and a walking direction of the user.

3. 2. The route guidance system according to claim 1, A route guidance system, wherein the braille blocks are warning blocks.

4. 2. The route guidance system according to claim 1, A route guidance system, wherein the braille blocks are guide blocks.

5. 2. The route guidance system according to claim 1, A route guidance system comprising a plurality of braille blocks with the QR codes affixed thereto.

6. 2. The route guidance system according to claim 1, A server storing information relating to directions is provided; the server and the mobile information terminal are connected via a network, The control unit downloads information necessary for route guidance from the server to the mobile information terminal, and provides route guidance using the downloaded information necessary for route guidance and the generated route guidance information.

7. A mobile information terminal for use in a route guidance system including a mobile information terminal and braille blocks, The device includes a photographing unit, a QR code analysis unit that analyzes the QR code, an audio output unit, and a control unit. The braille block has a QR code attached thereto, The control unit analyzes the QR code affixed to the Braille block photographed by the photographing processing unit using the QR code analysis processing unit to obtain QR code information, generates route guidance information from the QR code information based on the walking direction of the user holding the mobile information terminal, and outputs the route guidance information by voice from the voice output processing unit.

8. The portable information terminal according to claim 7, The portable information terminal, wherein the control unit generates the route guidance information based on a reference direction of the QR code and a walking direction of the user.

9. The portable information terminal according to claim 7, The portable information terminal, wherein the control unit downloads information necessary for route guidance from a server, and performs route guidance using the downloaded information necessary for route guidance and the generated route guidance information.

Images