Mobile support device

Abstract

To provide a movement support device that can perform notification to a user with guidance blocks for visually impaired people laid on general road surfaces as indices.SOLUTION: A movement support device 10 comprises: an imaging unit 22; a processing unit 30 that executes processing based on a picked-up image acquired from the imaging unit 22; and a notification unit 40 that performs notification on the basis of a result of processing performed by the processing unit 30. The processing unit 30 executes block detection processing of detecting guidance blocks B for visually impaired people, reference point derivation processing of deriving a reference point S for grasping positions where the guidance blocks B for visually impaired people are laid, and laying direction derivation processing of deriving a direction in which the guidance blocks B for visually impaired people are laid side by side on a road surface. The notification unit 40 notifies the laying direction derived through the laying direction derivation processing.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a movement assistance device.

Background Art

[0002] Conventionally, there has been provided a movement assistance device such as a white cane with an audio guidance device described in Patent Document 1. In the white cane with an audio guidance device of Patent Document 1, when the tip of the white cane touches any part of the braille block, the camera captures the coded dot-shaped protrusions of the braille block, and the image data is sent to the control unit. Further, in the white cane with an audio guidance device of Patent Document 1, the control unit converts the image data into corresponding audio data and transmits it to the speaker, and the audio data is converted into audio by the speaker, and for example, an audio such as "There is a staircase" as warning information is transmitted.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in a movement assistance device such as the white cane with an audio guidance device described in Patent Document 1, it can be used on the premise that there is a braille block provided with coded dot-shaped protrusions. Therefore, the white cane with an audio guidance device described in Patent Document has a problem that it is useless in a place where a braille block provided with coded dot-shaped protrusions is not provided. Therefore, the white cane with an audio guidance device described in Patent Document 1 has a problem that it cannot give a notification for assisting the movement of the user using a guiding block for visually impaired persons such as a braille block laid on a general road surface as an index.

[0005] Therefore, the present invention aims to realize a mobility assistance device that can provide notifications to users using tactile paving blocks for the visually impaired, which are laid on ordinary road surfaces, as indicators. [Means for solving the problem]

[0006] (1) The mobility support device of the present invention comprises an imaging unit capable of acquiring an image of the road surface on which a user is moving, a processing unit that performs processing based on the image acquired from the imaging unit, and a notification unit that provides notification based on the results of the processing by the processing unit, wherein the processing unit performs a block detection process for detecting guiding blocks for the visually impaired included in the image, a reference point derivation process for deriving reference points for determining the installation location of at least a portion of the guiding blocks for the visually impaired detected by the block detection process, and an installation direction derivation process for deriving the installation direction of the guiding blocks for the visually impaired laid in a row on the road surface based on the direction of the extension of the line formed by connecting the reference points of the plurality of guiding blocks for the visually impaired derived by the reference point derivation process, and the notification unit provides notification of the installation direction derived by the installation direction derivation process.

[0007] The mobility support device of the present invention can derive reference points for determining the installation location of at least a portion of the guiding blocks for the visually impaired detected from an image including the road surface by performing a reference point derivation process. Furthermore, the mobility support device of the present invention can derive the installation direction of the guiding blocks for the visually impaired on the road surface based on the direction of the line formed by connecting the reference points derived by the reference point derivation process. The mobility support device of the present invention can assist the user's movement by notifying the user of the installation direction of the guiding blocks for the visually impaired derived in this way. Therefore, the mobility support device of the present invention can provide notifications to assist the user's movement based on guiding blocks for the visually impaired that are generally installed on the road surface, rather than using specially prepared guiding blocks for the visually impaired as in the prior art.

[0008] (2) The mobility support device of the present invention is preferable in which the reference point derivation process derives the center of the bounding box that includes the guidance block for the visually impaired included in the captured image as the reference point.

[0009] By having this configuration, the mobility support device of the present invention can accurately determine the location of tactile paving blocks for the visually impaired and appropriately notify the user of the direction in which the tactile paving blocks should be laid.

[0010] (3) The mobility support device of the present invention is capable of performing a branching determination process that determines whether the guidance blocks for the visually impaired are laid out in a branching manner, on the condition that a plurality of lines formed by connecting the reference points intersect, and the notification unit is capable of notifying the determination result of the branching determination process.

[0011] By having this configuration, the mobility support device of the present invention can appropriately recognize and notify the presence of a branch when the tactile paving for the visually impaired is laid out in a branching manner.

[0012] (4) The mobility support device of the present invention comprises an imaging unit capable of acquiring an image of the road surface on which a user is moving, a processing unit that performs processing based on the image acquired from the imaging unit, and a notification unit that provides notification based on the results of the processing by the processing unit, wherein the processing unit detects a group of guide blocks for the visually impaired consisting of two or more adjacent guide blocks for the visually impaired in the image and performs a block group detection process to form a bounding box encompassing the group of guide blocks for the visually impaired, and a laying direction derivation process that derives the laying direction of the guide blocks for the visually impaired laid in a row on the road surface based on the shape of the bounding box formed in the block group detection process, and the notification unit provides notification of the laying direction derived by the laying direction derivation process.

[0013] The mobility support device of the present invention can detect two or more adjacent guiding blocks for the visually impaired as a group of guiding blocks for the visually impaired by performing block group detection processing, thereby forming a bounding box. Furthermore, by performing a laying direction derivation processing, the mobility support device of the present invention can derive the laying direction of the guiding blocks for the visually impaired laid in a row on the road surface based on the shape of the bounding box. The mobility support device of the present invention can assist the user's movement by notifying the user of the laying direction of the guiding blocks for the visually impaired thus derived. Therefore, the mobility support device of the present invention can provide notifications to assist the user's movement based on guiding blocks for the visually impaired that are generally laid on the road surface, rather than using specially prepared guiding blocks for the visually impaired as in the prior art.

[0014] (5) The mobility support device of the present invention is capable of performing a branching determination process that determines whether or not the guidance blocks for the visually impaired are laid out in a branching pattern based on the aspect ratio of the bounding box, and the notification unit is capable of notifying the determination result of the branching determination process.

[0015] With this configuration, the mobility assistance device of the present invention can appropriately recognize and notify the presence of a branching path based on the aspect ratio of the bounding box when tactile paving blocks for the visually impaired are laid out in a branching pattern.

[0016] (6) The mobility support device of the present invention is capable of performing a laying direction derivation process that determines the laying direction of the guidance blocks for the visually impaired based on the aspect ratio of the bounding box, and the notification unit is capable of notifying the determination result obtained by the laying direction derivation process.

[0017] By having this configuration, the mobility support device of the present invention can appropriately determine and notify the direction in which tactile paving blocks for the visually impaired should be laid based on the aspect ratio of the bounding box.

[0018] (7) The movement assistance device of the present invention is such that the imaging unit is provided in a terminal worn by a user, and the imaging image including the road surface can be acquired in a state where the user wears the terminal, and it is preferable that the terminal is constituted by a wearable terminal that can be worn on the neck or shoulder.

[0019] By adopting such a configuration, the movement assistance device of the present invention can be used while the imaging unit captures the road surface without the user feeling a burden.

[0020] (8) The movement assistance device of the present invention is such that the imaging unit is provided in a terminal worn by a user, and the imaging image including the road surface can be acquired in a state where the user wears the terminal, and it is preferable that the terminal is constituted by a glasses-type wearable terminal.

[0021] By adopting such a configuration, the movement assistance device of the present invention can be used while the imaging unit captures the road surface without the user feeling a burden.

[0022] (9) The movement assistance device of the present invention is preferably capable of detecting and notifying the presence of an obstacle on the predicted movement path of the user based on the imaging image acquired from the imaging device.

[0023] By adopting such a configuration, the movement assistance device of the present invention can appropriately assist the user's movement by detecting the presence of an obstacle and making the user recognize it.

Effects of the Invention

[0024] According to the present invention, it is possible to provide a movement assistance device that solves the above problems.

Brief Description of the Drawings

[0025] [Figure 1] It is a perspective view of a movement assistance device according to an embodiment of the present invention. [Figure 2]Figure 1 is a block diagram showing the configuration of the mobility support device. [Figure 3] This image shows a bounding box, reference point, and line formed when the mobility support device according to the first embodiment performs a laying state detection processing operation. [Figure 4] This flowchart shows the operation of the mobility support device according to the first embodiment when it performs the installation state detection processing operation. [Figure 5] This is an image diagram showing the bounding box formed when the mobility support device according to the second embodiment performs a laying state detection processing operation. [Figure 6] This flowchart shows the operation of the mobility support device according to the second embodiment when it performs the installation state detection processing operation. [Modes for carrying out the invention]

[0026] ≪First Embodiment≫ Hereinafter, a mobility support device 10 according to the first embodiment of the present invention will be described in detail with reference to the drawings. In the following description, unless otherwise specified, positional relationships such as up, down, left, and right will be described based on the orientation shown in the drawings.

[0027] The mobility assistance device 10 shown in Figure 1 is designed to assist the mobility of visually impaired users. As shown in Figure 2, the mobility assistance device 10 is broadly comprised of a terminal 20, a processing unit 30, and a notification unit 40.

[0028] Terminal 20 is worn and used by the user. Terminal 20 is equipped with an imaging unit 22 and a battery 24. The imaging unit 22 and battery 24 are housed inside the casing that makes up terminal 20. It is desirable that terminal 20 can be worn stably by the user without causing any burden. For example, terminal 20 may be a wearable terminal that can be worn around the neck or shoulder, or a glasses-type wearable terminal. In the example shown in Figure 1, terminal 20 is a wearable terminal that can be worn around the neck. Furthermore, terminal 20 is equipped with an imaging unit 22 that can capture images of the area in front of the user when in use. Terminal 20 is equipped with a battery 24, and the power supplied from the battery 24 can operate each part that makes up the mobility support device 10, such as the imaging unit 22, the processing unit 30, and the notification unit 40.

[0029] The processing unit 30 is capable of performing processing based on the captured image acquired from the imaging unit 22 of the terminal 20. The processing unit 30 is housed inside the casing that makes up the terminal 20. The processing unit 30 is capable of performing processing based on the captured image, specifically, processing to detect the installation status of the tactile paving blocks B for the visually impaired (installation status detection processing). The installation status detection processing is performed by executing block detection processing, reference point derivation processing, installation direction derivation processing, and branch determination processing.

[0030] The block detection process detects the guidance blocks B for visually impaired individuals contained in the captured image. In the block detection process, a bounding box X encompassing the guidance blocks B for visually impaired individuals is formed in the captured image using methods such as image matching (see Figure 3).

[0031] The reference point derivation process is a process for deriving reference points S for determining the installation location of at least some of the tactile paving blocks B for the visually impaired detected by the block detection process. As shown in Figure 3, in the reference point derivation process, the center of the bounding box X formed in the block detection process is used as the reference point S.

[0032] The laying direction derivation process is a process for deriving the laying direction of the tactile paving blocks B for the visually impaired that are laid in a row on the road surface. As shown in Figure 3, in the laying direction derivation process, a line L is formed by connecting the reference points S of the multiple tactile paving blocks B for the visually impaired that were derived in the laying position derivation process. In the laying direction derivation process, the direction in which the line L extends is derived as the laying direction of the tactile paving blocks B for the visually impaired. Specifically, as shown in Figure 3(a), when the line L is formed in the vertical direction, the processing unit 30 derives the laying direction assuming that the tactile paving blocks B for the visually impaired are laid so as to extend in the front-to-back direction relative to the user. Furthermore, as shown in Figure 3(b), when line L is formed in the horizontal direction, the processing unit 30 derives the installation direction assuming that the tactile paving blocks B for the visually impaired are laid so as to extend to the left and right relative to the user.

[0033] The branching determination process is a process for determining whether or not the tactile paving blocks B for the visually impaired are laid out in a branching pattern. The branching determination process determines that the tactile paving blocks B for the visually impaired are laid out in a branching pattern if there are multiple lines L that intersect when multiple lines L are formed by connecting reference points S. Specifically, as shown in Figures 3(c) and (d), if there are multiple lines L that intersect each other, the processing unit 30 determines that the tactile paving blocks B for the visually impaired are laid out in a branching pattern. Also, as shown in Figure 3(c), if there is a line L that intersects a line L that extends in the front-to-back direction and extends to the left-to-right direction, the processing unit 30 determines that there is a branch in the left-to-right direction in front of the user. Also, as shown in Figure 3(d), if there is a line L that intersects a line L that extends in the front-to-back direction and extends to one side in the left-to-right direction (the right side in the illustrated example), the processing unit 30 determines that there is a branch that goes to one side in the left-to-right direction (the right side in the illustrated example) in front of the user.

[0034] The processing unit 30 may perform other processing based on the captured image, in addition to the processing for detecting the installation status of the visually impaired guidance blocks B described above. In this embodiment, the processing unit 30 performs processing to derive information necessary to support the user's movement (walking) based on the captured image acquired from the imaging unit 22, such as processing to detect the presence of obstacles on the user's predicted movement path (obstacle detection processing).

[0035] The notification unit 40 provides movement support notifications to assist the user's movement based on the results of processing by the processing unit 30. The notification unit 40 includes a notification information generation unit 42 and an output unit 44.

[0036] The notification information generation unit 42 performs processing to generate information related to mobility assistance notifications made by the notification unit 40. The notification information generation unit 42 can generate notification information such as notification information to inform the visually impaired person of the installation status of the guidance blocks B, and obstacle presence notifications to inform the person of the presence of an obstacle.

[0037] Specifically, the notification information generation unit 42 generates notification information to inform the visually impaired about the laying direction of the tactile paving blocks B and information about branches, based on the information regarding the laying state of the tactile paving blocks B obtained by the laying state detection process performed by the processing unit 30. More specifically, the notification information generation unit 42 generates notification information to inform the visually impaired about the laying direction of the tactile paving blocks B based on the result of the laying direction derivation process performed by the processing unit 30. In addition, the notification information generation unit 42 generates notification information to inform the visually impaired if there is a branch in the tactile paving blocks B based on the result of the branch determination process. Furthermore, the notification information generation unit 42 generates notification information to inform the visually impaired if the presence of an obstacle is detected on the user's predicted movement path by the obstacle detection process.

[0038] The output unit 44 is for outputting notification information generated by the notification information generation unit 42. The output unit 44 can output notification information by sound via a speaker, or by vibration via a vibrator, for example. In this embodiment, the output unit 44 is configured to output notification information by sound via a speaker.

[0039] The mobility support device 10 described above performs a process to detect the installation status of the tactile paving blocks B for the visually impaired (installation status detection process) in accordance with the flow shown in Figure 4, and provides notification about the installation status. The operation of the installation status detection process and notification performed by the mobility support device 10 will be described in detail below with reference to Figure 4.

[0040] ≪Operation Flowchart for Detection of Installation Status of Mobility Support Device 10≫ (Step 1-1) As shown in Figure 4, when the mobility support device 10 performs the installation state detection process, first in step 1-1, the imaging unit 22 captures an image of the area in front of the user. After that, the control flow proceeds to step 1-2.

[0041] (Steps 1-2) In step 1-2, the processing unit 30 performs the block detection process described above based on the data related to the captured image (captured image data) acquired by the imaging unit 22, thereby forming a bounding box X that includes the guidance block B for visually impaired persons. Once the bounding box X is formed, the control flow proceeds to step 1-3.

[0042] (Steps 1-3) In step 1-3, the processing unit 30 performs the process of setting the reference point S using the reference point derivation process described above. Specifically, the processing unit 30 sets the center of each of the multiple bounding boxes X formed in step 1-2 as the reference point S. Once the setting of the reference point S is complete, the control flow proceeds to step 1-4.

[0043] (Steps 1-4) In step 1-4, the processing unit 30 derives a line L connecting the multiple reference points S set in step 1-3. Once line L is formed, the control flow proceeds to step 1-5.

[0044] (Steps 1-5) In step 1-5, the processing unit 30 checks whether one or more lines L are present. If one or more lines L are present, the control flow proceeds to step 1-6. On the other hand, if no lines L are present, it means that the image acquired by the imaging unit 22 does not show any guide blocks B for the visually impaired, or only one guide block B for the visually impaired is shown. Therefore, if no lines L are present, there is no need to send a notification, and the control flow proceeds to step 1-12, which will be described later.

[0045] (Steps 1-6) When the control flow progresses from step 1-5 to step 1-6, the processing unit 30 checks whether there are two or more lines L. If there are two or more lines L, the control flow proceeds to step 1-7. On the other hand, if there are not two or more lines L, that is, if there is only one line L, the control flow proceeds to step 1-10.

[0046] (Steps 1-7) If the control flow proceeds to step 1-7, it means that there are two or more lines L. In this case, it is assumed that in the image captured by the imaging unit 22, there are cases where the guidance blocks B for the visually impaired are laid out in approximately parallel directions across multiple rows, and cases where the guidance blocks B for the visually impaired are laid out in a way that intersects and branches. Therefore, when the control flow proceeds to step 1-7, the processing unit 30 checks whether or not there is an intersection of lines L. If there is an intersection of lines L, the control flow proceeds to step 1-8, and if there is no intersection of lines L, the control flow proceeds to step 1-10.

[0047] (Steps 1-8) If the control flow proceeds to step 1-8, it means that multiple lines L are intersecting. In this case, it is assumed that the tactile paving blocks B for the visually impaired are laid out in a branching manner within the range captured by the imaging unit 22. The processing unit 30 then performs a process to derive the branching direction based on the intersection state of the lines L. Specifically, if the situation is as shown in Figure 3(c), the processing unit 30 determines that there is a branch extending to the left. Also, if the situation is as shown in Figure 3(d), the processing unit 30 determines that there is a branch extending to the right.

[0048] (Steps 1-9) When the control flow proceeds to step 1-9, notification of the branching direction is given. Specifically, in step 1-9, the notification information generation unit 42 of the notification unit 40 generates notification information to notify the branching direction derived in step 1-8. Based on this, the output unit 44 outputs the notification information generated by the notification information generation unit 42, notifying the user of the branching direction. Once the notification in step 1-9 is completed, the control flow returns to step 1-1.

[0049] (Steps 1-10) On the other hand, if it is determined in step 1-6 above that there is only one line L, or if it is determined in step 1-7 that there are two or more lines L that do not intersect, the control flow proceeds to step 1-10. In step 1-10, the direction in which line L extends is determined as the laying direction of the tactile paving blocks B for the visually impaired. After that, the control flow proceeds to step 1-11.

[0050] (Steps 1-11) When the control flow proceeds to step 1-11, notification of the installation direction of the tactile paving blocks B for the visually impaired is provided. Specifically, in step 1-11, the notification information generation unit 42 of the notification unit 40 generates notification information to notify the installation direction derived in step 1-10. Based on this, the output unit 44 outputs the notification information generated by the notification information generation unit 42, notifying the user of the installation direction of the tactile paving blocks B for the visually impaired. Once the notification in step 1-11 is completed, the control flow returns to step 1-1.

[0051] (Steps 1-12) On the other hand, if it is determined in step 1-5 above that line L does not exist, the control flow proceeds to step 1-12. As mentioned above, if line L does not exist, it means that the image acquired by the imaging unit 22 does not show any guide blocks B for the visually impaired, or that only one guide block B for the visually impaired is shown. Therefore, if line L does not exist, there is no need to give a notification. For this reason, if the control flow proceeds to step 1-12, the control flow returns to step 1-1 without any special notification.

[0052] Regarding the effects of the mobility support device 10: The mobility support device 10 described above has the following characteristic configurations (a) to (e). Therefore, the mobility support device 10 can achieve unique effects that cannot be achieved with conventional technology.

[0053] (a) A mobility support device 10 according to one aspect of the present invention includes an imaging unit 22 capable of acquiring an image of the road surface on which the user is moving, a processing unit 30 that performs processing based on the image acquired from the imaging unit 22, and a notification unit 40 that provides notification based on the results of the processing by the processing unit 30. The processing unit 30 performs a block detection process to detect tactile paving blocks B for the visually impaired included in the image, a reference point derivation process to derive reference points S for determining the installation location of at least a portion of the tactile paving blocks B for the visually impaired detected by the block detection process, and a laying direction derivation process to derive the laying direction of the tactile paving blocks B laid in a row on the road surface based on the direction in which the line L formed by connecting the reference points S of the plurality of tactile paving blocks B for the visually impaired derived by the reference point derivation process extends, and the notification unit 40 provides notification of the laying direction derived by the laying direction derivation process.

[0054] Specifically, in step 1-1 described above, the mobility support device 10 can acquire an image including the road surface on which the user is moving using the imaging unit 22. In steps 1-2, 1-8, and 1-10, the mobility support device 10's processing unit 30 performs processing based on the image acquired from the imaging unit 22. In steps 1-9 and 1-11, the mobility support device 10's notification unit 40 provides notifications based on the results of the processing performed by the processing unit 30. In step 1-2, the processing unit 30 performs block detection processing to detect tactile paving blocks B for the visually impaired included in the image. In step 1-3, the processing unit 30 performs reference point derivation processing to derive reference points S for determining the installation locations of at least some of the tactile paving blocks B for the visually impaired detected by the block detection processing. Furthermore, in steps 1-4, the processing unit 30 connects the reference points S of the multiple tactile paving blocks B for the visually impaired derived by the laying position derivation process to form a line L, and in step 1-10, it performs a process (laying direction derivation process) to derive the laying direction of the tactile paving blocks B laid in a row on the road surface based on the direction in which the line L extends. The mobility support device 10 is configured to notify the system of the laying direction derived by the laying direction derivation process using the notification unit 40.

[0055] The mobility support device 10 of this embodiment can derive reference points S for determining the installation location of at least a portion of the tactile paving blocks B for the visually impaired detected from an image including the road surface by performing a reference point derivation process. Furthermore, the mobility support device 10 can derive the installation direction of the tactile paving blocks B for the visually impaired on the road surface based on the direction of the line L formed by connecting the reference points S derived by the reference point derivation process. The mobility support device 10 can assist the user's movement by notifying the user of the installation direction of the tactile paving blocks B for the visually impaired derived in this way. Therefore, the mobility support device 10 can provide notifications to assist the user's movement based on tactile paving blocks B for the visually impaired that are generally installed on the road surface, rather than using specially prepared tactile paving blocks, as in the prior art.

[0056] (b) The mobility support device 10 described above derives a reference point from the center of the bounding box X that includes the guidance block B for visually impaired persons included in the captured image, as the reference point S.

[0057] Since the mobility support device 10 employs the configuration described in (b) above, it can accurately determine the position of the tactile paving blocks B for the visually impaired and appropriately notify the user of the direction in which the tactile paving blocks B should be laid.

[0058] (c) The mobility support device 10 described above is capable of performing a branching determination process that determines whether the tactile paving blocks B for the visually impaired are laid out in a branching pattern, provided that multiple lines L formed by connecting reference points S intersect, and the notification unit 40 is capable of notifying the determination result of the branching determination process.

[0059] Specifically, the mobility support device 10, on the condition that it is determined in step 1-7 above that two or more lines L intersect, performs a process (branching determination process) in which the processing unit 30 determines that the tactile paving blocks B for the visually impaired are laid out in a branching pattern, and notifies the determination result in step 1-9.

[0060] Furthermore, in this embodiment, when it is determined in step 1-7 that lines L intersect, not only is a process performed to determine that a branch exists (branch determination process), but a process is also performed in step 1-8 to derive the branch direction. Therefore, in step 1-9, the mobility support device 10 of this embodiment can notify not only the existence of a branch but also the direction of the branch via the notification unit 40.

[0061] By configuring the mobility support device 10 as described in (c) above, it can appropriately recognize and notify the presence of a branch when the tactile paving blocks B for the visually impaired are laid out in a branching manner.

[0062] (d) The mobility support device 10 described above has an imaging unit 22 provided on a terminal 20 worn by the user, and is capable of acquiring images including the road surface when the user is wearing the terminal 20, and the terminal 20 is made up of a wearable terminal that can be worn around the neck or shoulder.

[0063] By configuring the mobility support device 10 as described in (d) above, it can be used while the user is taking pictures of the road surface with the imaging unit 22 without feeling any burden.

[0064] (e) The above-described mobility support device 10 is capable of detecting and notifying the presence of obstacles on the user's predicted movement path based on the captured images acquired from the imaging unit 22.

[0065] By configuring the mobility support device 10 as described in (e) above, it can detect the presence of obstacles, make the user aware of them, and appropriately support the user's movement.

[0066] It should be noted that the mobility support device 10 described above is merely one embodiment of the present invention, and its configuration can be modified as appropriate without departing from the spirit of the present invention. For example, the mobility support device 10 is not limited to having all of the configurations described in (a) to (e) above, but can also be made by omitting any of these configurations or substituting them with other configurations.

[0067] Specifically, as shown in (b) above, the mobility support device 10 is shown as a reference point deriveting the center of the bounding box X that includes the guidance block B for visually impaired persons included in the captured image as the reference point S through a reference point deriving process, but the present invention is not limited to this. For example, the mobility support device 10 may also derive an appropriate position, such as a corner of the bounding box X, as the reference point S.

[0068] As shown in (c) above, the mobility support device 10 is exemplified as one that can perform branch determination processing, but the present invention is not limited thereto. For example, the mobility support device 10 may not perform branch determination processing, or it may determine the presence and direction of a branch by other methods.

[0069] Furthermore, in the above embodiment, an example was shown in which the existence of a branch is determined not only based on whether or not the line L intersects in step 1-7, but also in which the branching direction is derived in step 1-8. However, the present invention is not limited thereto. For example, the mobility support device 10 may omit step 1-8 and, instead of notifying the branching direction in step 1-9, notify that a branch exists.

[0070] As shown in (d) above, the mobility support device 10 is an example in which a wearable terminal 20 that can be worn around the neck or shoulder is used as the terminal, and an imaging unit 22 is provided on this terminal 20 so that an image including the road surface can be acquired while the terminal 20 is being worn by the user. However, the present invention is not limited to this. Specifically, in this embodiment, the mobility support device 10 may be configured such that the terminal 20 is a glasses-type wearable terminal. Even with such a configuration, the mobility support device 10 can be used without causing the user any burden.

[0071] In this embodiment, the mobility support device 10 is shown as an example in (e) above, which is capable of detecting and notifying the presence of obstacles on the user's predicted movement path based on captured images acquired from the imaging unit 22. However, the present invention is not limited to this. For example, the mobility support device 10 may be capable of detecting and notifying the presence of obstacles using, for example, various sensors instead of, or in addition to, captured images acquired from the imaging unit 22. Furthermore, the mobility support device 10 may not have a function to detect and notify the presence of obstacles on the user's predicted movement path.

[0072] ≪Second Embodiment≫ Next, the mobility support device 100 according to the second embodiment will be described in detail with reference to the drawings. In the following description, components in the mobility support device 100 according to the second embodiment that are common with the mobility support device 10 of the first embodiment described above will be denoted by the same reference numerals, and detailed explanations will be omitted.

[0073] The mobility assistance device 100 is designed to assist the mobility of visually impaired users. Like the mobility assistance device 10 described above, the mobility assistance device 100 includes a terminal 20, a processing unit 130, and a notification unit 40. While the terminal 20 and notification unit 40 of the mobility assistance device 100 have the same configuration as those of the mobility assistance device 10, the processing unit 130 differs in some of the processing capabilities it can perform from that of the processing unit 30 described above.

[0074] Specifically, the processing unit 130 is capable of performing processing based on the captured image acquired from the imaging unit 22 of the terminal 20. The processing unit 130 is housed inside the casing that makes up the terminal 20. Similar to the processing unit 30 described above, the processing unit 130 is capable of performing processing based on the captured image, specifically, processing to detect the installation status of the tactile paving blocks B for the visually impaired (installation status detection processing). The installation status detection processing is performed by executing block group detection processing, installation direction derivation processing, and branching determination processing.

[0075] The block group detection process is a process that detects a group of visually impaired guidance blocks G consisting of two or more adjacent guidance blocks B in the captured image. As shown in Figure 5, by performing the block group detection process, a bounding box Y that encompasses the group of visually impaired guidance blocks G is formed.

[0076] The laying direction derivation process is a process that derives the laying direction of the guiding blocks B for the visually impaired laid in a row on the road surface, based on the shape of the bounding box Y that encompasses the group of guiding blocks G for the visually impaired formed in the block group detection process. As shown in Figure 5(a), if the bounding box Y is formed in a vertically elongated shape (long in the vertical direction) in the captured image, it is determined that the guiding blocks B for the visually impaired are laid in the front-to-back direction relative to the user. As shown in Figure 5(b), if the bounding box Y is formed in a horizontally elongated shape (long in the left-to-right direction) in the captured image, it is determined that the guiding blocks B for the visually impaired are laid in the left-to-right direction relative to the user.

[0077] In this embodiment, the processing unit 130 determines the installation direction of the tactile paving blocks B for the visually impaired based on the aspect ratio r (r = (vertical length) / (horizontal length)), which is the ratio of the vertical length to the horizontal length of the bounding box Y. Specifically, the processing unit 130 determines that the tactile paving blocks B for the visually impaired are installed in the front-to-back direction relative to the user if the aspect ratio r is greater than a first reference value r1 (r1 > 1). Furthermore, the processing unit 130 determines that the tactile paving blocks B for the visually impaired are installed in the left-to-right direction relative to the user if the aspect ratio r is less than a second reference value r2 (r2 < 1).

[0078] The branching determination process is a process for determining whether or not the tactile paving blocks B for the visually impaired are laid out in a branching pattern. The processing unit 130 can perform the branching determination process based on the shape of the bounding box Y. Specifically, the processing unit 130 can determine whether or not the tactile paving blocks B for the visually impaired are laid out in a branching pattern based on the aspect ratio r of the bounding box Y. More specifically, if the aspect ratio r is in the range of being greater than or equal to the second reference value r2 (r2 < 1) and less than or equal to the first reference value r1 (r1 > 1), that is, if the relationship r2 ≤ r ≤ r1 holds, the processing unit 130 determines that the tactile paving blocks B for the visually impaired are laid out in a branching pattern, as shown in Figures 5(c) and 5(d).

[0079] The mobility support device 100 described above performs a process to detect the installation status of the tactile paving blocks B for the visually impaired (installation status detection process) in accordance with the flow shown in Figure 6, and provides notification about the installation status. The operation of the installation status detection process and notification performed by the mobility support device 100 will be described in detail below with reference to Figure 6.

[0080] <<Operation Flowchart for Detection of Installation Status of Mobility Support Device 100>> (Step 2-1) As shown in Figure 6, when the mobility support device 100 performs the installation state detection processing operation, first in step 2-1, the imaging unit 22 captures an image of the area in front of the user. After that, the control flow proceeds to step 2-2.

[0081] (Step 2-2) When the control flow proceeds to step 2-2, the block detection process described above is performed based on the data related to the captured image (captured image data) acquired in step 2-1, thereby forming a bounding box Y that encompasses a group of guidance blocks G for the visually impaired, consisting of multiple guidance blocks B for the visually impaired. Once the bounding box Y is formed, the control flow proceeds to step 2-3.

[0082] (Steps 2-3) In step 2-3, the processing unit 130 performs a process to derive the aspect ratio r of the bounding box Y. After that, the control flow proceeds to step 2-4.

[0083] (Steps 2-4) In step 2-4, the processing unit 130 determines whether the aspect ratio r is greater than or equal to the second reference value r2 and within the range of r2 ≤ r ≤ r1. If the aspect ratio r satisfies the relationship r2 ≤ r ≤ r1, the control flow proceeds to step 2-5. On the other hand, if the aspect ratio r does not satisfy the relationship r2 ≤ r ≤ r1, the control flow proceeds to step 2-6.

[0084] (Steps 2-5) When the control flow proceeds to step 2-5, the aspect ratio r satisfies the relationship r2 ≤ r ≤ r1, meaning that the tactile paving blocks B for the visually impaired are laid out in a branching pattern in front of the user. Therefore, in step 2-5, the notification information generation unit 42 of the notification unit 40 generates notification information to inform the user that a branch of tactile paving blocks B for the visually impaired exists in front of the user. Based on this, the output unit 44 outputs the notification information generated by the notification information generation unit 42, notifying the user of the existence of the branch. Once the notification in step 2-5 is complete, the control flow returns to step 2-1.

[0085] (Steps 2-6) On the other hand, if the control flow proceeds from step 2-4 to step 2-6 as described above, it is determined whether the aspect ratio r is greater than or equal to the second reference value r2 and greater than the first reference value r1 (r1>1). If it is determined that the aspect ratio r is greater than the first reference value r1, the control flow proceeds to step 2-7. On the other hand, if the aspect ratio r is less than the first reference value r1, the control flow proceeds to step 2-8.

[0086] (Step 2-7) When the control flow advances to Step 2-7, since the aspect ratio r is greater than the first reference value r1, the bounding box Y is vertically long, and the guiding blocks B for visually impaired persons are laid out so as to be arranged in the vertical direction (front-back direction with respect to the user). Therefore, when the control flow advances to Step 2-7, the notification information generation unit 42 of the notification unit 40 generates notification information for notifying that the laying direction of the guiding blocks B for visually impaired persons is the vertical direction. Based on this, the output unit 44 performs an output for notifying the user that the guiding blocks B for visually impaired persons are laid out in the vertical direction. When the notification in Step 2-7 is completed, the control flow returns to Step 2-1.

[0087] (Step 2-8) On the other hand, when the control flow advances to Step 2-8, it is in a state where it is determined that in Step 2-4, the aspect ratio r does not satisfy the relationship of r2 ≤ r ≤ r1, and in Step 2-6, the relationship of r > r1 is not satisfied. That is, when the control flow advances to Step 2-8, the aspect ratio r of the bounding box Y is in the relationship of r < r2, and the guiding blocks B for visually impaired persons are in a state of being laid out in the horizontal direction. Therefore, when the control flow advances to Step 2-8, the notification information generation unit 42 of the notification unit 40 generates notification information for notifying that the laying direction of the guiding blocks B for visually impaired persons is the horizontal direction. Based on this, the output unit 44 performs an output for notifying the user that the guiding blocks B for visually impaired persons are laid out in the horizontal direction. When the notification in Step 2-8 is completed, the control flow returns to Step 2-1.

[0088] ≪Effects of the Mobility Support Device 100≫ The above-described mobility support device 100 has the following characteristic configurations (f) to (h). Therefore, the mobility support device 100 can achieve unique effects that cannot be achieved by the following prior arts.

[0089] (f) A second embodiment of the mobility support device 100, which is one aspect of the present invention, comprises an imaging unit 22 capable of acquiring an image of the road surface on which the user is moving, a processing unit 130 that performs processing based on the image acquired from the imaging unit 22, and a notification unit 40 that provides notification based on the results of processing by the processing unit 130. The processing unit 130 detects a group of visually impaired guidance blocks G consisting of two or more adjacent guidance blocks B for the visually impaired in the image, and performs a block group detection process to form a bounding box Y that encompasses the group of visually impaired guidance blocks G, and a laying direction derivation process that derives the laying direction of the visually impaired guidance blocks B laid in a row on the road surface based on the shape of the bounding box Y formed in the block group detection process, and the notification unit 40 provides notification of the laying direction derived by the laying direction derivation process.

[0090] As described in step 2-2 above, the mobility support device 100 can detect two or more adjacent guiding blocks B for the visually impaired as a group of guiding blocks B for the visually impaired by performing block group detection processing, thereby forming a bounding box Y. Furthermore, by performing laying direction derivation processing after processing related to steps 2-3, 2-4, and 2-6, the mobility support device 100 can derive the laying direction of the guiding blocks B for the visually impaired laid in a row on the road surface based on the shape of the bounding box Y. The mobility support device 100 can assist the user's movement by notifying the user of the laying direction of the guiding blocks B for the visually impaired derived in this way. Therefore, the mobility support device 100 can provide notifications to assist the user's movement based on guiding blocks B for the visually impaired that are generally laid on the road surface, rather than using specially prepared guiding blocks B for the visually impaired as in the conventional technology.

[0091] (g) The mobility support device 100 described above is capable of performing a branching determination process to determine whether or not the tactile paving blocks B for the visually impaired are laid out in a branching pattern based on the aspect ratio r of the bounding box Y, and the notification unit 40 is capable of notifying the determination result of the branching determination process.

[0092] By configuring the mobility support device 100 as described in (g) above, when the tactile paving blocks B for the visually impaired are laid out in a branching pattern, the device can appropriately recognize and notify the presence of a branching pattern based on the aspect ratio r of the bounding box X.

[0093] (h) The mobility support device 100 described above is capable of performing a laying direction derivation process that determines the laying direction of the tactile paving blocks B for the visually impaired based on the aspect ratio r of the bounding box Y, and the notification unit 40 is capable of notifying the determination result obtained by the laying direction derivation process.

[0094] By configuring the mobility support device 100 as described in (h) above, it can appropriately determine and notify the direction in which the tactile paving blocks B for the visually impaired should be laid based on the aspect ratio r of the bounding box Y.

[0095] The mobility support device 100 has the same configuration as the mobility support device 10 of the first embodiment described above. Specifically, the mobility support device 10 has the configurations described in (d) and (e) above. Therefore, the mobility support device 100 can achieve the same effects as the mobility support device 10 through the configurations described in (d) and (e) above.

[0096] It should be noted that the mobility support device 100 described above is merely one embodiment of the present invention, and its configuration can be modified as appropriate without departing from the spirit of the present invention. For example, the mobility support device 100 is not limited to having all of the configurations described in (f) to (h) above, and it is also possible to omit any of these configurations or replace them with other configurations.

[0097] Specifically, in the second embodiment described above, the laying direction derivation process exemplified a method for deriving the laying method of the tactile paving blocks B for the visually impaired by determining the shape of the bounding box Y based on the aspect ratio r, but the present invention is not limited thereto. For example, the mobility support device 100 only needs to determine the shape of the bounding box Y as described in (f) above and derive a method for laying the tactile paving blocks B for the visually impaired, and it is also possible to determine the shape of the bounding box Y by a method other than the determination method based on the aspect ratio r and derive the laying state.

[0098] The above-mentioned mobility support device 100 is not limited to those that perform branching determination processing and determination of laying direction based on the aspect ratio r of the bounding box Y, as in (g) and (h) above, but may also perform branching determination processing and determination of laying direction by other methods.

[0099] The present invention is not limited to the embodiments and modifications described above, and other embodiments may be possible in the spirit and teachings thereof, without departing from the scope of the claims. The components of the embodiments described above may be arbitrarily selected and combined. Furthermore, any component of the embodiments may be arbitrarily combined with any component described in the means for solving the problem, or any component that embodies any component described in the means for solving the problem. We intend to obtain rights for these as well in amendments or divisional applications of this application. [Industrial applicability]

[0100] The present invention is suitably applicable to all types of mobility assistance devices that provide mobility assistance based on the installation status of tactile paving blocks for visually impaired persons. [Explanation of Symbols]

[0101] 10:Mobility support device 20: Terminal 22: Imaging Department 30: Processing Unit 40:Notification section 100:Mobility support device 130: Processing Unit B: Guiding blocks for pedestrians G: Group of pedestrian guidance blocks S: Reference point X: Bounding box Y: Bounding Box r: Aspect ratio

Claims

1. An imaging unit capable of acquiring images including the road surface the user is moving on, A processing unit that performs processing based on the captured image acquired from the imaging unit, A notification unit that provides notification based on the results of the processing performed by the processing unit, It is equipped with, The aforementioned processing unit, Block detection processing for detecting guidance blocks for visually impaired persons included in the captured image, A reference point derivation process is performed to derive reference points for determining the installation location of at least a portion of the tactile paving blocks for the visually impaired detected by the block detection process, A laying direction derivation process that derives the laying direction of the tactile paving blocks laid in a row on the road surface based on the direction in which the line formed by connecting the reference points of the plurality of tactile paving blocks for the visually impaired derived by the reference point derivation process, Execute, The aforementioned reference point derivation process derives the center of the bounding box that includes the guidance block for the visually impaired, which is included in the captured image, as the reference point. A mobility support device characterized in that the notification unit notifies the laying direction derived by the laying direction derivation process.

2. An imaging unit capable of acquiring images including the road surface the user is moving on, A processing unit that performs processing based on the captured image acquired from the imaging unit, A notification unit that provides notification based on the results of the processing performed by the processing unit, It is equipped with, The aforementioned processing unit, The process involves detecting a group of visually impaired guidance blocks consisting of two or more adjacent guidance blocks in the captured image, and forming a bounding box that encompasses the group of visually impaired guidance blocks. A laying direction derivation process that derives the laying direction of the tactile paving blocks for the visually impaired laid in a row on the road surface based on the shape of the bounding box formed in the block group detection process, Execute, A mobility support device characterized in that the notification unit notifies the laying direction derived by the laying direction derivation process.

Images