Mobile support device

Abstract

To realize a movement support device which can properly notify of the existence of an obstacle having high necessity of notification and suppress a fear and inconvenience due to the highly frequent occurrence of notification to the minimum level.SOLUTION: A movement support device 10 comprises: an imaging unit 22; a processing unit 30 which executes processing based on a photographed image acquired from the imaging unit 22; and a notification unit 40 which performs notification on the basis of a result of processing by the processing unit 30. The processing unit 30 can execute: block detection processing of detecting a guidance block B for a visually-impaired person included in a photographed image; region of interest setting processing of setting a region of interest R; and obstacle detection processing of detecting the existence of the obstacle. The notification unit 40 notifies of the existence of the obstacle under such a condition that the obstacle on the inner side of the region of interest R is detected with the obstacle detection processing.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a movement support device.

Background Art

[0002] Conventionally, there has been provided a movement support device such as that described in Patent Document 1. This movement support device includes an imaging unit that images a road surface, a detection unit that detects index information serving as an index of the direction in which a user can move based on the captured image of the imaging unit, and a control unit that outputs guidance information for guiding the movement of the user based on the index information detected by the detection unit. When an obstacle overlapping a visually impaired person guidance block is detected based on the captured image of the imaging unit, the movement support device of Patent Document 1 is configured to output a warning indicating the presence of the obstacle by means of an audio output unit.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When configured like the movement support device of Patent Document 1, a warning is issued each time an obstacle is detected within the captured image obtained by the imaging unit. Therefore, in the movement support device of Patent Document 1, warnings indicating the presence of obstacles may occur frequently, and there is a possibility that the user may feel an excessive sense of fear or annoyance. In addition, there is even a concern that the reliability of the warning may decrease if a warning is issued for something that does not need to be warned considering a collision with the user or the like.

[0005] On the other hand, the inventors have diligently investigated and found that even if obstacles are detected within the captured image obtained by the imaging unit, many of these obstacles may not pose a risk of collision with the user, and therefore do not require voice warnings or other notifications to the user. Furthermore, they have concluded that by accurately notifying only those obstacles that pose a high necessity for notification, taking into account the possibility of collision with the user, it may be possible to minimize the fear and annoyance caused by frequent notifications.

[0006] Therefore, the present invention aims to realize a mobility support device that can accurately notify the presence of obstacles that necessitate notification, while minimizing the fear and annoyance caused by frequent notifications. [Means for solving the problem]

[0007] (1) The mobility support device of the present invention comprises an imaging unit capable of acquiring an image of the road surface on which a user is moving, a processing unit that performs processing based on the image acquired from the imaging unit, and a notification unit that provides notification based on the results of the processing by the processing unit, wherein the processing unit is capable of performing block detection processing for detecting guidance blocks for the visually impaired included in the image, region of interest setting processing for setting a region of interest based on the presence status of the guidance blocks for the visually impaired detected by the block detection processing, and obstacle detection processing for detecting the presence of obstacles based on the image, and the notification unit is capable of performing obstacle presence notification to notify the presence of the obstacle, provided that the presence of the obstacle inside the region of interest is detected by the obstacle detection processing.

[0008] The mobility assistance device of the present invention sets a region of interest based on the presence status of guidance blocks for the visually impaired detected by block detection processing, and, on the condition that the presence of an obstacle is detected within the region of interest, executes an obstacle presence notification by the notification unit, thereby notifying the user of the presence of an obstacle. In this way, the mobility assistance device of the present invention detects and notifies the presence of obstacles only within the region of interest, rather than the entire image acquired from the imaging device. Therefore, the mobility assistance device of the present invention can minimize the fear and annoyance caused by frequent notifications. Furthermore, in the mobility assistance device of the present invention, the region of interest is set based on the presence status of guidance blocks for the visually impaired that are assumed to be used as a guide when the user moves. Therefore, the mobility assistance device of the present invention can accurately notify the user of the presence of obstacles that pose a high necessity for notification, such as those that hinder the user's movement. Accordingly, according to the present invention, it is possible to realize a mobility assistance device that can accurately notify the user of the presence of obstacles that pose a high necessity for notification, while minimizing the fear and annoyance caused by frequent notifications.

[0009] (2) The mobility support device of the present invention is preferably characterized in that, in the region of interest setting process, the processing unit sets the region of interest based on the positional relationship between the bounding box formed to include the guidance block for the visually impaired and the reference line assumed to be at a predetermined position in the captured image.

[0010] By having this configuration, the mobility assistance device of the present invention can set an appropriate region of interest based on the presence or absence of tactile paving blocks for the visually impaired. As a result, the mobility assistance device of the present invention can more accurately notify users of the presence of obstacles that necessitate notification, while minimizing the frequency of notifications.

[0011] (3) The mobility support device of the present invention is preferably characterized in that the reference line is a virtual line that passes through the center of the captured image and is set to be parallel to the user's direction of travel.

[0012] By having such a configuration, the mobility support device of the present invention can set an appropriate area of ​​interest considering the user's direction of travel. As a result, the mobility support device of the present invention can more accurately notify the presence of obstacles that necessitate notification, while minimizing the frequency of notifications.

[0013] (4) The mobility support device of the present invention is preferably characterized in that the reference line is determined based on a predicted movement path that is predicted to be the user's movement path.

[0014] By having this configuration, the mobility support device of the present invention can set an appropriate region of interest based on the prediction of the user's movement path. As a result, the mobility support device of the present invention can accurately notify the presence of obstacles that are highly likely to require notification, while minimizing the frequency of notifications.

[0015] (5) The mobility support device of the present invention is preferable such that, on the condition that the presence of the guidance block for the visually impaired is detected by the block detection process, the region of interest is set to be smaller than when the presence of the guidance block for the visually impaired is not detected.

[0016] By having this configuration, the mobility assistance device of the present invention can accurately notify users of the presence of obstacles that are highly necessary to notify, limited to the areas necessary for users who are moving using tactile paving for the visually impaired as a guide, while minimizing the frequency of notifications.

[0017] (6) The mobility support device of the present invention is preferably characterized in that, provided that the presence of the guidance block for the visually impaired is detected by the block detection process, the area outside the user's position in the width direction via the guidance block for the visually impaired is excluded from the target area of ​​the area of ​​interest.

[0018] By having this configuration, the mobility assistance device of the present invention can exclude areas outside the area of ​​interest in the width direction via the tactile paving blocks for the visually impaired, which are assumed to be of low necessity for notification to users who are moving using the tactile paving blocks as a guide. As a result, the mobility assistance device of the present invention can accurately notify the user of the presence of obstacles only in areas where notification is necessary, while minimizing the frequency of notifications.

[0019] Here, when moving, not only objects but also people can become obstacles in the vicinity of the user. Therefore, it is desirable that the mobility assistance device of the present invention described above be capable of detecting not only objects but also people as obstacles. Based on this finding, the inventors conducted diligent studies and came to the conclusion that, unlike objects, people tend to move to avoid people or objects when they are within the area they are looking at with their face turned, and therefore the necessity of notification may be low. Furthermore, they came to the conclusion that if the device can detect people within the area of ​​interest as obstacles, and also detect the direction of the recognized person's face, and then decide whether or not to notify the presence of an obstacle according to the direction of the face, the frequency of notifications can be minimized, and the fear and annoyance caused by frequent notifications can be further reduced.

[0020] (7) Based on the above-mentioned findings, the mobility support device of the present invention is characterized in that the processing unit detects a person as an obstacle within the area of ​​interest by the obstacle detection processing, and is capable of detecting the direction of the person's face, and determines whether or not to designate the person as an obstacle subject to the obstacle presence notification, using the direction of the person's face detected as an obstacle as part or all of the conditions.

[0021] (8) Similarly, the movement support device of the present invention includes an imaging unit capable of acquiring an imaging image including the road surface on which the user moves, a processing unit that executes processing based on the imaging image acquired from the imaging device, and a notification unit that performs notification based on the result of the processing by the processing unit. The processing unit is capable of executing an obstacle detection process for detecting the presence of an obstacle based on the imaging image, and the notification unit is capable of executing an obstacle presence notification for notifying the presence of the obstacle on the condition that the presence of the obstacle inside the region of interest is detected by the obstacle detection process. The processing unit is capable of detecting a person as the obstacle inside the region of interest by the obstacle detection process and detecting the direction of the face of the person, and determines whether or not to set the person as the obstacle to be the target of the obstacle presence notification with at least part of the direction of the face of the person detected as the obstacle as part or all of the conditions.

[0022] By adopting the configuration as described in (7) and (8) above, the movement support device of the present invention can appropriately notify the presence of a person who may hinder the user's movement among the persons detected as obstacles, while suppressing the notification of the presence of a person who is less likely to hinder the user's movement. Therefore, by adopting the configuration as described above, the movement support device of the present invention can minimize the frequency of notification and further reduce the sense of fear and annoyance caused by the high frequency of notification.

[0023] (9) The movement support device of the present invention is characterized in that the imaging unit is provided in a terminal worn by the user, and the imaging image including the road surface can be acquired in a state where the user wears the terminal, and the terminal is a wearable terminal that can be worn on the neck or shoulder.

[0024] By adopting such a configuration, the movement support device of the present invention can be used while the imaging unit captures the road surface without the user feeling a burden.

[0025] (10) The mobile support device of the present invention is such that the imaging unit is provided in a terminal worn by a user, and in a state where the user wears the terminal, it is possible to acquire the imaging image including the road surface. The terminal is a glasses-type wearable terminal. It is preferably characterized by this.

[0026] With such a configuration, the mobile support device of the present invention can be used while the imaging unit captures the road surface without the user feeling a burden.

Effects of the Invention

[0027] According to the present invention, it is possible to provide a mobile support device that solves the above problems.

Brief Description of the Drawings

[0028] [Figure 1] It is a perspective view of a mobile support device according to an embodiment of the present invention. [Figure 2] It is a block diagram showing the device configuration of the mobile support device of FIG. 1. [Figure 3] Examples of the region of interest, bounding box, and reference line formed by the mobile support device according to the first embodiment are shown. (a) shows the case where there is no guiding block for visually impaired persons, and (b) and (c) show examples of the case where there is no guiding block for visually impaired persons. [Figure 4] It is a flowchart showing the operation of the mobile support device according to the first embodiment. [Figure 5] It is a schematic diagram showing the relationship between the face orientation and whether to notify when the mobile support device according to the second embodiment detects a person as an obstacle. [Figure 6] It is a flowchart showing the operation of the mobile support device according to the second embodiment.

Modes for Carrying Out the Invention

[0029] The following description will detail a mobility support device 10 according to one embodiment of the present invention, with reference to the drawings. In the following description, unless otherwise specified, positional relationships such as up, down, left, and right will be described based on the orientation shown in the drawings.

[0030] The mobility assistance device 10 shown in Figure 1 is designed to assist the mobility of visually impaired users. As shown in Figure 2, the mobility assistance device 10 is broadly comprised of a terminal 20, a processing unit 30, and a notification unit 40.

[0031] Terminal 20 is worn and used by the user. Terminal 20 is equipped with an imaging unit 22 and a battery 24. The imaging unit 22 and battery 24 are housed inside the casing that makes up terminal 20. It is desirable that terminal 20 can be worn stably by the user without causing any burden. For example, terminal 20 may be a wearable terminal that can be worn around the neck or shoulder, or a glasses-type wearable terminal. In the example shown in Figure 1, terminal 20 is a wearable terminal that can be worn around the neck. Furthermore, terminal 20 is equipped with an imaging unit 22 that can capture images of the area in front of the user when in use. Terminal 20 is equipped with a battery 24, and the power supplied from the battery 24 can operate each part that makes up the mobility support device 10, such as the imaging unit 22, the processing unit 30, and the notification unit 40.

[0032] The processing unit 30 is capable of performing processing based on the captured image acquired from the imaging unit 22 of the terminal 20. The processing unit 30 is housed inside the casing of the terminal 20. By performing block detection processing, region of interest setting processing, and obstacle detection processing based on the captured image, the processing unit 30 can detect the presence of obstacles that should be notified to the user.

[0033] The block detection process detects the guidance blocks B for visually impaired individuals contained in the captured image. In the block detection process, methods such as image matching are used to detect the guidance blocks B for visually impaired individuals. In addition, the block detection process also involves forming a bounding box X that encloses the guidance blocks B for visually impaired individuals in the captured image (see Figure 3).

[0034] The region of interest setting process is the process of setting the region of interest R. The region of interest R is the target area for notification of an obstacle when its presence is detected. Therefore, in the mobility support device 10 of this embodiment, even if an obstacle exists in an area outside the region of interest R, the presence of that obstacle is not subject to notification. As shown in Figure 3, the region of interest R is set as a part of the captured image. By performing the region of interest setting process, the region of interest R is set based on the presence status of the visually impaired guidance block B detected by the block detection process (the presence or absence and location of the visually impaired guidance block B).

[0035] Specifically, if the block detection process does not detect the guidance block B for visually impaired persons, a predetermined area that forms part of the captured image is set as the region of interest R (hereinafter also referred to as the "normal region of interest Rn" as needed). In this embodiment, as shown in Figure 3(a), the normal region of interest Rn is set between the position closest to the user in the captured image (the lower position on the imaging side) and the intermediate position in the front-to-back direction relative to the user (the intermediate position in the vertical direction of the captured image). The normal region of interest Rn is also set in the intermediate region in the left-to-right direction relative to the user (the intermediate region in the left-to-right direction of the captured image). In the illustrated example, the normal region of interest Rn is set in the shape of a rectangle or trapezoid (isosceles trapezoid in this embodiment) relative to the position closest to the user in the captured image (the lower position on the imaging side).

[0036] Furthermore, if a guidance block B for the visually impaired is detected by the block detection process described above, a region of interest R (hereinafter also referred to as "region of interest Rx at block detection") is set based on the position of the guidance block B, as shown in Figures 3(b) and (c). The setting of the region of interest Rx at block detection based on the position of the guidance block B can be done by an appropriate method. In this embodiment, the region of interest Rx at block detection is set based on the positional relationship between the bounding box X formed to include the guidance block B for the visually impaired and the reference line L assumed to be at a predetermined position in the captured image.

[0037] Here, the reference line L can be set based on appropriate setting criteria. For example, the reference line L can be set as a virtual line that passes through a predetermined position (e.g., the center) of the captured image and is parallel to the user's direction of travel. Alternatively, the reference line L can be set as a predicted movement line generated by predicting the user's movement path. In this embodiment, it is assumed that tactile paving blocks B for visually impaired persons are laid in the longitudinal direction of the bounding box X. Therefore, the longitudinal direction of the bounding box X is set as the user's direction of travel, and a virtual line that passes through a predetermined position (the center in this embodiment) of the captured image while being parallel to this direction of travel is set as the reference line L.

[0038] In this embodiment, as shown in Figures 3(b) and 3(c), the region of interest R (region of interest Rx when block is detected) is set based on the positional relationship between the reference line L set as described above and the bounding box X. Specifically, when setting the region of interest Rx when block is detected, a normal region of interest Rn is assumed to be set first when the guidance block B for the visually impaired is not detected. Furthermore, when the normal region of interest Rn is divided into two regions with the reference line L as the boundary, a region in which the bounding box X (guidance block B for the visually impaired) does not exist (hereinafter also referred to as the "non-existent region Rna") and a region in which the bounding box X exists (hereinafter also referred to as the "existent region Rnb") are assumed. The non-existent region Rna is considered to be entirely part of the region of interest Rx when block is detected. Furthermore, within the presence region Rnb, the area that lies closer to the reference line L and the non-presence region Rna, relative to the location where the tactile paving blocks B for the visually impaired are laid (the area inside the tactile paving blocks B for the visually impaired), is also considered part of the region of interest Rx during block detection. On the other hand, within the presence region Rnb, the area that lies away from the reference line L and the non-presence region Rna, relative to the location where the tactile paving blocks B for the visually impaired are laid (the area outside the tactile paving blocks B for the visually impaired), is excluded from the region of interest Rx during block detection. When tactile paving blocks B for the visually impaired are detected by the block detection process, the region of interest R is set in this manner. Therefore, the region of interest R is set to a more limited range than when tactile paving blocks B are not detected, provided that tactile paving blocks B are detected.

[0039] Obstacle detection is the process of detecting the presence of obstacles based on the captured image. Similar to the block detection process described above, obstacle detection is performed using methods such as image matching to detect obstacles.

[0040] The notification unit 40 provides movement support notifications to assist the user's movement based on the results of processing by the processing unit 30. The notification unit 40 includes a notification information generation unit 42 and an output unit 44.

[0041] The notification information generation unit 42 performs processing to generate information related to mobility support notifications made by the notification unit 40. The notification information generation unit 42 can generate notification information such as obstacle presence notifications that notify the presence of obstacles detected based on the captured image.

[0042] The output unit 44 is for outputting notification information generated by the notification information generation unit 42. The output unit 44 can output notification information by sound via a speaker, or by vibration via a vibrator, for example. In this embodiment, the output unit 44 is configured to output notification information by sound via a speaker.

[0043] The mobility support device 10 described above notifies the user of the presence of obstacles that may hinder their movement, in accordance with the flow shown in Figure 4. The operation of the mobility support device 10 will be explained in detail below with reference to Figure 4.

[0044] (Step 1-1) As shown in Figure 4, the mobility support device 10 first captures an image of the area in front of the user using the imaging unit 22 in step 1-1. After that, the control flow proceeds to step 1-2.

[0045] (Steps 1-2) In step 1-2, the processing unit 30 executes the block detection process and obstacle detection process described above based on the data (image data) related to the captured image acquired by the imaging unit 22. This detects the guidance blocks B for the visually impaired and obstacles. The processing unit 30 also performs a process to form a bounding box X that includes the guidance blocks B for the visually impaired detected by the block detection process. After that, the control flow proceeds to step 1-3.

[0046] (Steps 1-3) In step 1-3, the processing unit 30 determines whether or not a tactile paving block B for the visually impaired was detected in step 1-2. If a tactile paving block B for the visually impaired is detected, the control flow proceeds to step 1-4. On the other hand, if a tactile paving block B for the visually impaired is not detected, the control flow proceeds to step 1-5.

[0047] (Steps 1-4) If the control flow proceeds from step 1-3 to step 1-4 as described above, the processing unit 30 performs the process of setting the region of interest Rx as the region of interest R at the time of block detection, based on the position of the guidance block B for the visually impaired, using the region of interest setting process described above. After that, the control flow proceeds to step 1-6.

[0048] (Steps 1-5) On the other hand, if the control flow proceeds from step 1-3 to step 1-5 as described above, the processing unit 30 performs the process of setting a predetermined normal region of interest Rn that forms part of the captured image as the region of interest R using the region of interest setting process described above. After that, the control flow proceeds to step 1-6.

[0049] (Steps 1-6) Once the process of setting the region of interest Rx during block detection or the normal region of interest Rn as the region of interest R in step 1-4 or step 1-5 described above is completed, step 1-6 determines whether or not an obstacle exists inside the region of interest R. If it is determined that an obstacle exists inside the region of interest R, the control flow proceeds to step 1-7. On the other hand, if it is determined that no obstacle exists inside the region of interest R, the control flow proceeds to step 1-8.

[0050] (Steps 1-7) If the control flow proceeds to step 1-7, it means that an obstacle exists inside the region of interest R, and there is a high necessity to inform the user of the presence of the obstacle. Therefore, in step 1-7, notification of the presence of an obstacle is made. Specifically, in step 1-7, the notification information generation unit 42 of the notification unit 40 generates notification information to notify the user of the presence of an obstacle based on the determination made in step 1-6. Based on this, the output unit 44 outputs the notification information generated by the notification information generation unit 42, notifying the user of the presence of an obstacle. Once the notification in step 1-7 is completed, the control flow returns to step 1-1.

[0051] (Steps 1-8) On the other hand, if it is determined in steps 1-6 above that there are no obstacles inside the region of interest R, there is little need to notify the user of the presence of obstacles. Therefore, if the control flow proceeds to step 1-8, the control flow returns to step 1-1 without any special notification.

[0052] Regarding the effects of the mobility support device 10: The mobility support device 10 described above has the following characteristic configurations (a) to (g). Therefore, the mobility support device 10 can achieve unique effects that cannot be achieved with conventional technology.

[0053] (a) The mobility support device 10 of this embodiment includes an imaging unit 22 capable of acquiring an image of the road surface on which the user is moving, a processing unit 30 that performs processing based on the image acquired from the imaging unit, and a notification unit 40 that provides notification based on the results of processing by the processing unit 30. The processing unit 30 is capable of performing block detection processing to detect guidance blocks B for the visually impaired included in the image, region of interest setting processing to set a region of interest R based on the presence status of the guidance blocks B for the visually impaired detected by the block detection processing, and obstacle detection processing to detect the presence of obstacles based on the image. The notification unit 40 is capable of performing obstacle presence notification to notify the user of the presence of an obstacle, provided that the presence of an obstacle inside the region of interest R is detected by the obstacle detection processing.

[0054] The mobility support device 10 sets a region of interest R based on the presence status of the guidance blocks B for visually impaired persons detected by the block detection process. Furthermore, the mobility support device 10 can perform an obstacle presence notification via the notification unit 40, provided that the presence of an obstacle is detected within the region of interest R. In this embodiment, the mobility support device 10 detects and notifies of the presence of obstacles only within the region of interest R, rather than the entire image acquired from the imaging device. Therefore, the mobility support device 10 can minimize the feeling of fear and annoyance caused by frequent notifications.

[0055] Furthermore, in the mobility support device 10, the region of interest R is set based on the presence or absence of tactile paving blocks B for the visually impaired, which are assumed to be used as a guide when the user moves. Therefore, the mobility support device 10 of this embodiment can accurately notify the user of the presence of obstacles that pose a high necessity for notification and would hinder the user's movement. Accordingly, the mobility support device 10 can accurately notify the user of the presence of obstacles that pose a high necessity for notification, while minimizing the fear and annoyance caused by frequent notifications.

[0056] (b) The mobility support device 10 is configured such that the processing unit 30 sets the region of interest R in the region of interest setting process based on the positional relationship between the bounding box X, which is formed to include the guidance block B for visually impaired persons, and the reference line L which is assumed to be at a predetermined position in the captured image.

[0057] Since the mobility support device 10 has the configuration described in (b) above, it can set the region of interest R at an appropriate location based on the presence or absence of the tactile paving blocks B for the visually impaired. As a result, the mobility support device 10 can more accurately notify users of the presence of obstacles that require notification, while minimizing the frequency of notifications.

[0058] (c) In this embodiment, the mobility support device 10 is configured such that the reference line L is a virtual line that passes through the center of the captured image and is parallel to the user's direction of travel.

[0059] The mobility support device 10 of this embodiment, by having the configuration described in (c) above, can set the region of interest R at an appropriate position considering the user's direction of travel (in this embodiment, the direction in which the guiding blocks B for visually impaired persons are laid). As a result, the mobility support device 10 can more accurately notify the presence of obstacles that require notification, while minimizing the frequency of notifications.

[0060] (d) In this embodiment, the mobility support device 10 is configured such that the reference line L is determined based on the predicted movement path that is expected to be the user's movement path.

[0061] As described above, the mobility support device 10 can set an appropriate region of interest R based on the predicted movement path (in this embodiment, the path on which the tactile paving blocks B for visually impaired persons are laid) that is expected to be the user's movement path, as shown in (d) above. This allows the mobility support device 10 of this embodiment to accurately notify the presence of obstacles that are highly likely to require notification, while minimizing the frequency of notifications.

[0062] (e) The mobility support device 10 is configured such that, on the condition that the presence of the guidance block B for the visually impaired is detected by the block detection process, the region of interest R is set to be smaller than when the presence of the guidance block B for the visually impaired is not detected.

[0063] The mobility assistance device 10 has the configuration described in (e) above. Specifically, in the mobility assistance device 10, the region of interest R when the presence of the tactile paving block B for the visually impaired is detected (region of interest Rx when block is detected) is set to be smaller than the region of interest R when the presence of the tactile paving block B for the visually impaired is not detected (normal region of interest Rn). This makes it possible to accurately notify users of the presence of obstacles that are highly necessary to notify, limited to the area necessary for users who use the tactile paving block B for the visually impaired as a guide, while minimizing the frequency of notifications.

[0064] (f) The mobility support device 10 of this embodiment is configured such that, on the condition that the presence of a guidance block B for the visually impaired is detected by the block detection process, the area outside the user's position in the width direction via the guidance block B is excluded from the target area of ​​the region of interest R.

[0065] Since the mobility support device 10 of this embodiment has the configuration described in (f) above, it is possible to exclude the area outside the width direction via the guiding blocks B for the visually impaired, which are assumed to have little need to be notified to a user moving using the guiding blocks B as a guide, from the target area of ​​the area of ​​interest R. As a result, the mobility support device 10 of this embodiment can accurately notify the user of the presence of obstacles only in the area where notification is necessary for the user (area of ​​interest Rx when blocks are detected), while minimizing the frequency of notifications.

[0066] (g) The mobility support device 10 of this embodiment has an imaging unit 22 provided on a terminal 20 worn by the user, and is capable of acquiring an image including the road surface when the user is wearing the terminal 20, and the terminal 20 is composed of a wearable terminal that can be worn around the neck or shoulder.

[0067] Since the mobility support device 10 of this embodiment has the configuration described in (g) above, the user can use it while the imaging unit 22 photographs the road surface without feeling any burden.

[0068] It should be noted that the mobility support device 10 described above is merely one embodiment of the present invention, and its configuration can be modified as appropriate without departing from the spirit of the present invention. For example, the mobility support device 10 is not limited to having all of the configurations described in (a) to (e) above, but can also be made by omitting any of these configurations or substituting them with other configurations.

[0069] Specifically, in this embodiment, the presence or absence of the tactile paving blocks B for the visually impaired is considered, in addition to the presence or absence of the tactile paving blocks B, and the installation status of the tactile paving blocks B when they are present is taken into account to narrow down the region of interest R. However, the present invention is not limited to this. For example, the mobility support device 10 may only consider the presence or absence of the tactile paving blocks B as the presence or absence of the tactile paving blocks B, and change the size of the region of interest R according to the presence or absence of the tactile paving blocks B.

[0070] In this embodiment, an example of setting the region of interest R based on the positional relationship between the bounding box X and the reference line L is shown as in (b) above, but the present invention is not limited thereto. For example, the mobility support device 10 may set the region of interest R using only the bounding box X as an indicator, without using the reference line L as an indicator.

[0071] In this embodiment, as shown in (c) above, an example is shown in which a virtual line passing through the center of the captured image and parallel to the user's direction of movement is used as the reference line L, but the present invention is not limited thereto. For example, the mobility support device 10 can be configured to set a virtual line passing off-center from the center of the captured image as the reference line L, taking into account the discrepancy between the center of the captured image and the position of the user's body.

[0072] In this embodiment, as shown in (d) above, an example is shown in which the reference line L is determined based on the predicted movement path that is expected to be the user's movement path, but the present invention is not limited thereto. For example, the mobility support device 10 can also determine the reference line L based on a criterion different from the predicted movement path. Furthermore, in the above embodiment, an example was shown in which the user's predicted movement path is a path on which tactile paving blocks B for the visually impaired are laid, and the reference line L is set based on this, but the present invention is not limited thereto. For example, the user's predicted movement path can be derived by another method, and the reference line L can be set based on this.

[0073] In this embodiment, as shown in (e) above, an example was shown in which the region of interest R is narrowed down on the condition that the presence of tactile paving blocks B for the visually impaired is detected, but the present invention is not limited thereto. Specifically, the mobility support device 10 can also be configured so that the region of interest R is not changed regardless of the presence or absence of tactile paving blocks B for the visually impaired.

[0074] In this embodiment, as shown in (f) above, an example was shown in which the region of interest R is narrowed in the width direction based on the user's position, provided that the presence of the tactile paving blocks B for the visually impaired is detected. However, the present invention is not limited to this. Specifically, the mobility support device 10 can also be configured so that the region of interest R is not changed in the width direction regardless of the presence or absence of the tactile paving blocks B for the visually impaired.

[0075] As shown in (g) above, the mobility support device 10 is an example in which a wearable terminal 20 that can be worn around the neck or shoulder is used as the terminal, and an imaging unit 22 is provided on this terminal 20 so that an image including the road surface can be acquired while the terminal 20 is being worn by the user. However, the present invention is not limited to this. Specifically, in this embodiment, the mobility support device 10 may be configured such that the terminal 20 is a glasses-type wearable terminal. Even with such a configuration, the mobility support device 10 can be used without causing any burden to the user.

[0076] ≪Second Embodiment≫ Next, the mobility support device 100 according to the second embodiment will be described in detail with reference to the drawings. In the following description, components in the mobility support device 100 according to the second embodiment that are common with the mobility support device 10 of the first embodiment described above will be denoted by the same reference numerals, and detailed explanations will be omitted.

[0077] The mobility assistance device 100 is designed to assist the mobility of visually impaired users. As shown in Figure 2, the mobility assistance device 100 includes a terminal 20, a processing unit 130, and a notification unit 40, similar to the mobility assistance device 10 described above. While the terminal 20 and notification unit 40 of the mobility assistance device 100 have the same configuration as those of the mobility assistance device 10, the processing unit 130 differs in some aspects from the processing unit 30 described above in terms of the types of processing it can perform.

[0078] Specifically, the processing unit 130 is capable of detecting people as obstacles in addition to objects and structures through obstacle detection processing. The processing unit 130 can detect objects, structures, and people as obstacles using methods such as image matching. Furthermore, the processing unit 130 is capable of detecting the direction of the face of a person detected as an obstacle. The processing unit 130 can determine whether or not to classify a person detected as an obstacle as an obstacle subject to obstacle presence notification, using the direction of the face of the person detected as an obstacle as part or all of the determination conditions.

[0079] Specifically, as shown in Figure 5(a), if the face of a person detected as an obstacle is facing away from the user, or as shown in Figures 5(b) and (f), if the face is turned directly to the side, it is assumed that the user is not within the field of view of the person detected as an obstacle. In this case, it is assumed that the likelihood of the person detected as an obstacle taking action to avoid a collision with the user is extremely low. Therefore, if the face of a person detected as an obstacle is oriented in a way that suggests the user is not within their field of view, the processing unit 130 determines that the person is an obstacle subject to obstacle presence notification. On the other hand, as shown in Figure 5(d), if the face of a person detected as an obstacle is facing the user, or as shown in Figures 5(c) and (e), if the face of a person detected as an obstacle is facing diagonally forward within a predetermined angular range relative to the user, it is assumed that the user is within the field of view of the person detected as an obstacle. In this case, it is assumed that the likelihood of the person detected as an obstacle taking action to avoid a collision with the user is high. Therefore, if the face of a person detected as an obstacle is oriented in a way that suggests the user is within the person's field of vision, the processing unit 130 determines that the person should be excluded from the list of obstacles subject to obstacle presence notification.

[0080] The mobility support device 100 described above performs the operation of detecting and notifying the presence of a person who may be an obstacle, in accordance with the flow shown in Figure 6. The operation of the mobility support device 100 will be explained in detail below with reference to Figure 6.

[0081] (Step 2-1) As shown in Figure 6, the mobility support device 100 first acquires an image by capturing the area in front of the user using the imaging unit 22 in step 2-1. The control flow then proceeds to step 2-2.

[0082] (Step 2-2) In step 2-2, the processing unit 130 determines whether or not there is a person in the captured image who could be an obstacle to the user's movement. If it is determined that there is a person who could be an obstacle, the control flow proceeds to step 2-3. On the other hand, if it is determined that there is no person who could be an obstacle, the control flow proceeds to step 2-6.

[0083] (Steps 2-3) In step 2-3, the processing unit 130 performs a process to determine the orientation of the face of the person who was determined to be a potential obstacle in step 2-2. After that, the control flow proceeds to step 2-4.

[0084] (Steps 2-4) In step 2-4, the processing unit 130 determines whether the face of the person detected in step 2-3 is facing within the field of view of the user of the mobility assistance device 100. Here, as shown in Figures 5(a), (b), and (f), if the face of the person detected as an obstacle is facing backward or directly to the side, the face of the person detected as an obstacle is not facing within the field of view of the user of the mobility assistance device 100. In this case, the control flow proceeds to step 2-5. On the other hand, as shown in Figures 5(c) to (e), if the face of the person detected as an obstacle is facing in a direction that allows the user of the mobility assistance device 100 to look directly at it, or in a direction that allows the user of the mobility assistance device 100 to be seen diagonally in front, the face of the person detected as an obstacle is facing within the field of view of the user of the mobility assistance device 100. In this case, the control flow proceeds to step 2-6.

[0085] (Steps 2-5) If the control flow proceeds to step 2-5, it is highly likely that the user of the mobility assistance device 100 is not within the field of view of the person detected as an obstacle. Therefore, in this case, a notification is issued regarding the presence of a person who may be an obstacle. After that, the control flow returns to step 2-1.

[0086] (Steps 2-6) If the control flow progresses from step 2-2 to step 2-6, no person has been detected as an obstacle, so there is no need to notify the user of the presence of a person who could be an obstacle. Also, if the control flow progresses from step 2-4 to step 2-6, it is highly likely that the user of the mobility assistance device 100 is within the field of view of the person detected as an obstacle. In this case, it is assumed that the person detected as an obstacle will likely take action to avoid a collision with the user, even without the user immediately changing direction. Therefore, if the control flow progresses to step 2-6, the notification of the presence of a person who could be an obstacle is not performed. After that, the control flow returns to step 2-1.

[0087] Regarding the effects of the mobility support device 100: The mobility support device 100 described above has the following characteristic configuration (h). Therefore, the mobility support device 100 can achieve unique effects that cannot be achieved with conventional technology, as described below.

[0088] (h) In the second embodiment of the mobility support device 100, the processing unit 130 detects a person as an obstacle inside the region of interest R by obstacle detection processing, and is also capable of detecting the direction of the person's face. The processing unit determines whether or not to consider the person as an obstacle subject to obstacle presence notification, using the direction of the person's face detected as an obstacle as part or all of the conditions.

[0089] Since the mobility support device 100 of this embodiment has the configuration described in (g) above, it can accurately notify the user of the presence of people who are detected as obstacles and who may hinder the user's movement, while suppressing notifications for people who are unlikely to hinder the user's movement. Therefore, by having the configuration described above, the mobility support device 100 of this embodiment can minimize the frequency of notifications and further reduce the fear and annoyance caused by frequent notifications.

[0090] It should be noted that the mobility support device 10 of the first embodiment and the mobility support device 100 of the second embodiment described above are merely examples of the present invention, and their configurations can be modified as appropriate without departing from the spirit of the present invention. For example, the mobility support device 10 of the first embodiment can be configured to have the configuration described in (h) above, as in the mobility support device 100 of the second embodiment. That is, the mobility support device 10 may be configured to include a processing unit 130 in addition to the processing unit 30, or the functions of the processing unit 130 may be added to the processing unit 30, so that the region of interest R can be narrowed down to a part of the image captured from the imaging device 22, the presence of an obstacle can be detected and notified, and if a person is detected as an obstacle, a determination can be made as to whether or not to include the person as an obstacle subject to obstacle presence notification according to the orientation of the person's face, and then notification can be made. Furthermore, the mobility support device 10 can, in parallel with detecting and notifying the presence of an obstacle by narrowing the region of interest R to a portion of the image captured from the imaging device 22 according to the control flow shown in Figure 4, also detect the presence of a person who could be an obstacle by targeting the entire image according to the control flow shown in Figure 6, and notify the user according to the orientation of the detected person's face.

[0091] The present invention is not limited to the embodiments and modifications described above, and other embodiments may be possible in the spirit and teachings thereof, without departing from the scope of the claims. The components of the embodiments described above may be arbitrarily selected and combined. Furthermore, any component of the embodiments may be arbitrarily combined with any component described in the means for solving the problem, or any component that embodies any component described in the means for solving the problem. We intend to obtain rights for these as well in amendments or divisional applications of this application. [Industrial applicability]

[0092] The present invention is suitably applicable to all types of mobility assistance devices that assist users in moving. [Explanation of Symbols]

[0093] 10:Mobility support device 20: Terminal 22: Imaging Department 30: Processing Unit 40:Notification section 100:Mobility support device 130: Processing Unit B: Guidance blocks for the visually impaired L:Reference line R: Area of ​​interest Rn: Typical area of ​​interest Rx: Region of interest when block is detected X: Bounding box

Claims

1. An imaging unit capable of acquiring images including the road surface the user is moving on, A processing unit that performs processing based on the captured image acquired from the imaging unit, A notification unit that provides notification based on the results of the processing performed by the processing unit, It is equipped with, The aforementioned processing unit, Block detection processing for detecting guidance blocks for visually impaired persons included in the captured image, A region of interest setting process sets a region of interest based on the presence status of the guidance blocks for the visually impaired detected by the block detection process, Obstacle detection processing that detects the presence of an obstacle based on the captured image, It is possible to do this, The notification unit can perform an obstacle presence notification to notify the presence of the obstacle, provided that the obstacle detection process detects the presence of the obstacle within the region of interest. The aforementioned processing unit, A mobility assistance device characterized in that, in the region of interest setting process, the region of interest is set based on the positional relationship between a bounding box formed to include the guidance block for the visually impaired and a reference line assumed to be at a predetermined position in the captured image.

2. The mobility support device according to claim 1, characterized in that the processing unit detects a person as an obstacle within the region of interest by the obstacle detection process, and is capable of detecting the direction of the person's face, and determines whether or not to designate the person as an obstacle subject to the obstacle presence notification, using the direction of the person's face detected as an obstacle as part or all of the conditions.

Images