Tilt determination system, tilt determination method, and movement support device

Abstract

To provide a tilt determination system, a tilt determination method and a movement support device capable of accurately performing tilt determination, without a separated sensor for tilt correction, and with a simple configuration.SOLUTION: In a tilt determination system 1, a movement support device 10 is fitted to at least any of both shoulders and a neck of a user, and used with a white stick. The movement support device comprises: a capturing unit 20 which acquires an image 21 including the white stick by capturing a front side of the user; and a white stick detection unit 25 which detects the white stick held by the user, as a bounding box having sides formed of a horizontal direction and a vertical direction in the image 21 captured and obtained by the capturing unit 20. A tilt of the movement support device 10 is determined, on the basis of a horizontal width of the bounding box.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a tilt determination system, a tilt determination method, and a movement support device. [Background technology]

[0002] Conventionally, a camera mounting angle calibration device (simply referred to as a mounting angle calibration device) that calibrates the mounting angle of a camera using an image captured by the camera is known (for example, Patent Document 1). The mounting angle calibration device described in Patent Document 1 is intended to calibrate the mounting angle of an in-vehicle camera. The mounting angle calibration device described in Patent Document 1 is intended to convert a captured image into a bird's-eye view and search for a mounting angle such that two straight line parts in the converted bird's-eye view image are parallel to each other and have the same width. As a result, the mounting angle calibration device described in Patent Document 1 is intended to calibrate the mounting angle of the in-vehicle camera based on the search result. Furthermore, in recent years, a wearable device using a camera has become known, and a mobility support device for visually impaired people and the like that uses the wearable device is also known. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-188738 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, when the wearable device described above is used as a mobility support device, there is a concern that the wearable device may shift due to vibrations caused by the user's movement, causing the wearable device to tilt excessively. In such a case, the camera mounted on the wearable device also tilts significantly, and the image captured by the camera also tilts excessively, making it difficult to provide accurate guidance to the user.

[0005] Therefore, it is possible to mount the mounting angle calibration device described in Patent Document 1 on a wearable device, but the mounting angle calibration device described in Patent Document 1 is for vehicle use, and there is a problem that mounting it on a wearable device is complicated and expensive. In addition, as described above, when the wearable device is in an excessively tilted state, it is expected that the amount of correction will be large, so there is a concern that it will be difficult to correct the tilt when the above mounting angle calibration device is used on the wearable device. In addition, there is a demand for a wearable device to be able to determine the tilt at a low cost without the need for a sensor to obtain the tilt angle. In addition, in a mobility support device for visually impaired people, it is difficult for visually impaired people to recognize that the camera is attached out of place, so it is also necessary for the mobility support device or the camera to automatically determine the tilt.

[0006] Therefore, an object of the present invention is to realize a tilt determination system, a tilt determination method, and a movement support device that are capable of performing accurate tilt determination with a simple configuration without the need for a separate sensor for tilt correction. [Means for solving the problem]

[0007] (1) The tilt determination system of the present invention provided to solve the above-mentioned problems is a tilt determination system for a mobility support device that is worn on at least one of a user's shoulders and neck and used together with a white cane, and is equipped with an imaging unit that obtains an image including the white cane by imaging the front side of the user, and a white cane detection unit that detects the white cane held by the user as a bounding box with sides in the horizontal and vertical directions within the image obtained by imaging the imaging unit, and is characterized in that it determines the tilt of the mobility support device based on the horizontal width of the bounding box.

[0008] The above-mentioned tilt determination system is adapted to perform tilt determination of a mobility support device that is attached to at least one of the shoulders and the neck of a user (visually impaired person) who uses a white cane. The above-mentioned tilt determination system is adapted to detect the white cane held by the user as a bounding box with sides in the horizontal and vertical directions in an image obtained by capturing an image with an imaging unit. That is, the above-mentioned tilt determination system can detect the entire white cane as a bounding box regardless of the inclination of the white cane in the image. Here, the horizontal width of the bounding box is minimum when the white cane is held in the vertical direction, and increases as the inclination angle of the white cane in the horizontal direction increases. Therefore, the above-mentioned tilt determination system can determine the tilt of the mobility support device based on the horizontal width of the bounding box. The above-mentioned tilt determination system can also use the tilt determination result of the mobility support device to determine whether or not the mobility support device is correctly attached to the user, for example, when the mobility support device starts to be used.

[0009] In this way, the above-mentioned tilt determination system can determine the tilt of the mobility support device without using a tilt sensor, and therefore can accurately determine the tilt without erroneous determination even if the tilt of the mobility support device (including the imaging unit) is large. Furthermore, the above-mentioned tilt determination system can accurately determine the tilt without using a complex mounting angle calibration device (including control), and therefore can suppress increases in costs. Note that the above-mentioned tilt determination system can determine whether the mobility support device is correctly attached to the user based on the tilt of the mobility support device. Here, the imaging unit can capture an image of the white cane when the user is using the white cane, i.e., when the white cane is held facing toward the ground (downward).

[0010] (2) The above-mentioned inclination determination system of the present invention may be characterized in that it determines that the inclination of the mobility support device is within a predetermined range, on the condition that the horizontal width of the bounding box is smaller than a predetermined threshold value.

[0011] The tilt determination system described above, by configuring as described above, can determine whether the tilt of the mobility support device is within a predetermined range based on a predetermined threshold. Here, the threshold may be set based on the horizontal width of a bounding box set for a white cane aligned in the vertical direction at a predetermined distance from the user (image capture unit). That is, when the horizontal width of the bounding box is smaller than the predetermined threshold, the tilt determination system described above can determine that the tilt of the mobility support device is within a predetermined range (a normal range of tilt). In this way, the tilt determination system described above can determine the tilt by comparing the horizontal width of the bounding box detected in the image with the predetermined threshold, so that there is no need to provide a separate tilt sensor. As a result, the tilt determination system described above can suppress an increase in cost.

[0012] The tilt determination system of the present invention described above may be configured to determine that the travel support device is tilted on the condition that the horizontal width of the bounding box is equal to or greater than a predetermined threshold value. With this configuration, the tilt determination system described above can determine that the travel support device is tilted.

[0013] (3) The above-mentioned tilt determination system of the present invention may be characterized in that it determines the tilt of the mobility support device based on an image obtained by capturing an image of the white cane in a hanging state due to its own weight.

[0014] The tilt determination system described above, being configured as described above, can obtain an image of the white cane in a non-tilted state, and can therefore determine the tilt of the mobility support device without any tilt of the white cane. Here, the user can, for example, hold the strap attached to the top of the white cane, thereby forming the white cane in the state of hanging down due to its own weight as described above. This allows the white cane to be positioned vertically with high accuracy and imaged.

[0015] (4) The tilt determination system of the present invention described above may have a notification unit, and may be characterized in that, if the horizontal width of the bounding box is equal to or greater than the threshold value, the notification unit issues a warning.

[0016] By configuring the above-mentioned tilt determination system in this way, when the mobility support device is tilted, the notification unit can notify a warning. As a result, the above-mentioned tilt determination system can accurately notify the user of the tilt of the mobility support device even when the mobility support device is attached in an inclined state. Note that, for example, when the mobility support device transitions from a tilted state to a state where the mobility support device is no longer tilted, a message such as "The mobility support device is attached correctly" may be notified.

[0017] (5) The tilt determination method of the present invention provided to solve the above-mentioned problems is a tilt determination method for a mobility support device that is worn on at least one of a user's shoulders and neck and used together with a white cane, and is characterized by executing the following steps: an image acquisition step of acquiring an image including the white cane by capturing an image of the front side of the user with an imaging unit; a white cane detection step of detecting the white cane in the image as a bounding box with sides in the horizontal and vertical directions; a bounding box horizontal width determination step of determining whether the horizontal width of the bounding box detected in the white cane detection step is smaller than a predetermined threshold value; and a warning notification step of determining that the mobility support device is tilted and notifying a warning, provided that the horizontal width of the bounding box in the bounding box horizontal width determination step is equal to or greater than the threshold value.

[0018] The tilt determination method described above can determine the tilt of the mobility support device by executing an image acquisition step, a white cane detection step, and a bounding box horizontal width determination step. Furthermore, if the bounding box horizontal width determination step determines that the mobility support device is tilted, the tilt determination method can notify the user of the tilt of the mobility support device immediately by executing a warning notification step.

[0019] (6) The mobility support device of the present invention provided to solve the above-mentioned problems is a mobility support device that is worn on at least one of a user's shoulders and neck and used together with a white cane, and is equipped with a shoulder movement detection sensor that acquires shoulder movement information related to the movement of at least both shoulders of the user, an imaging unit that acquires an image including the white cane by imaging the front side of the user, a white cane detection unit that detects the white cane held by the user as a bounding box with sides in the horizontal and vertical directions in the image acquired by imaging the imaging unit, and a control unit, wherein the control unit performs control to determine the inclination based on the horizontal width of the bounding box.

[0020] By configuring the above-mentioned mobility support device in this way, it is possible to perform tilt determination without providing a separate tilt sensor. This is expected to lead to a reduction in size and cost of the mobility support device. In addition, the above-mentioned mobility support device can have a tilt determination system built into the mobility support device. This is expected to lead to a further reduction in size of the mobility support device. Effect of the Invention

[0021] According to the present invention, it is possible to realize a tilt determination system, a tilt determination method, and a travel assistance device that are capable of performing accurate tilt determination with a simple configuration without providing a separate sensor for tilt correction. [Brief description of the drawings]

[0022] [Figure 1]1 is a plan view of a travel assistance device constituting an inclination determination system according to an embodiment of the present invention; [Diagram 2] 1 is a block diagram of a tilt determination system according to an embodiment of the present invention. [Diagram 3] FIG. 1 is an explanatory diagram showing an example of a method for capturing an image of a white cane in an inclination determination system according to an embodiment of the present invention. [Figure 4] 1A and 1B are explanatory diagrams of the operation of a tilt determination system according to one embodiment of the present invention, in which (a) is an explanatory diagram of a state in which the mobility support device is tilted and an image of the white cane portion, and (b) is an explanatory diagram of a state in which the mobility support device is not tilted and an image of the white cane portion. [Diagram 5] 4 is a flowchart showing the operation of the inclination determination system of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, the tilt determination system 1 according to one embodiment of the present invention will be described in detail with reference to the drawings. Note that each figure is a schematic representation for easy understanding, and may differ from the actual size. In addition, in this embodiment, a case where the tilt determination system 1 is mounted on a travel support device 10 will be described as an example. Note that the image 21 in FIG. 4 is drawn by cutting out the bounding box 26 portion.

[0024] As shown in Fig. 1, a mobility support device 10 incorporating a tilt determination system 1 of the present invention is intended to support the movement of a visually impaired user 2, for example. For this reason, in this embodiment, the mobility support device 10 is used together with a white cane 3 (see Fig. 4). The mobility support device 10 is configured as a terminal (hereinafter, the mobility support device 10 is also referred to as a terminal 10). As shown in Fig. 2, the tilt determination system 1 includes an imaging unit 20, a white cane detection unit 25, and a notification unit 40. The terminal 10 on which the tilt determination system 1 is mounted includes a shoulder movement detection sensor 15, and a control unit 30 (including a threshold setting unit 31).

[0025] The terminal 10 (mobility support device 10) is worn by the user 2 (see FIG. 4). The terminal 10 includes a battery 11. The imaging unit 20 and the battery 11 are housed inside a housing that constitutes the terminal 10 (see FIG. 1). The terminal 10 may be stably worn by the user 2 without causing any burden. In this embodiment, the terminal 10 is a wearable terminal that can be worn on at least one of both shoulders or around the neck. In the example shown in FIG. 4(a) and FIG. 4(b), the terminal 10 is a wearable terminal that can be worn around the neck and both shoulders. In this embodiment, the terminal 10 is formed in an inverted U-shape and can be worn around the neck of the user 2. The imaging unit 20 is incorporated in the terminal 10 so that the area in front of the user 2 can be photographed in the usage state. The terminal 10 is equipped with a battery 11, and the various components of the movement support device 10, such as the imaging unit 20, the control unit 30, and the notification unit 40, can be operated by power supplied from the battery 11.

[0026] The shoulder movement detection sensor 15 is, for example, configured with a gyro sensor, and can detect acceleration in three directions, the X-axis direction, the Y-axis direction, and the Z-axis direction, in an integrated manner. Here, the gyro sensor may be configured with MEMS (Micro Electro Mechanical Systems) or the like. This is expected to reduce the size of the terminal 10. The shoulder movement detection sensor 15 can acquire shoulder movement information 16 related to the movement of at least both shoulders of the user 2. Specifically, the shoulder movement detection sensor 15 can detect the movement of both shoulders in three directions, the X-axis direction, the Y-axis direction, and the Z-axis direction, as shoulder movement information 16 (see FIG. 2) by using a gyro sensor. Here, the shoulder movement information 16 is, for example, acceleration, or the movement direction of the shoulder of the user 2 obtained by arithmetic processing of the acceleration. Note that the arithmetic processing of the acceleration can be performed by at least one of the shoulder movement detection sensor 15 and the control unit 30 described later. In addition, in the tilt determination system 1 of the present invention, tilt determination can be performed independently of the shoulder movement detection sensor 15.

[0027] As shown in Fig. 1, the imaging unit 20 is composed of, for example, a camera, and can capture an image of the area in front of the user 2. The imaging unit 20 can also capture an image 21 (see Figs. 2 and 3) including the white cane 3 by capturing an image of the area in front of the user 2. Note that the imaging unit 20 can also capture an image that does not include the white cane 3 when performing mobility assistance other than tilt determination. In this embodiment, the imaging unit 20 is provided, for example, on one side of the front end of the terminal 10.

[0028] Here, in the tilt determination system 1 in this embodiment, when the user 2 wears the terminal 10, tilt determination is performed to determine whether the terminal 10 is worn correctly (without tilt). Details will be described later, but the tilt determination in the tilt determination system 1 is performed based on an image 21 obtained by imaging the white cane 3 held by the user 2 with the imaging unit 20, as shown in FIG. 3. The white cane 3 is manufactured for the purpose of supporting the walking of visually impaired people, and in this embodiment, a white cane that is straight (including a foldable one) in use is used. In addition, for example, a red marker 3A is applied near the tip of the white cane 3. The image 21 (see FIGS. 3 and 4) acquired by imaging with the imaging unit 20 is transmitted to the control unit 30 (see FIGS. 1 and 2) by wire or wirelessly, and is used for image analysis (image processing).

[0029] As shown in Figs. 3 and 4, the white cane detection unit 25 detects the white cane 3 held by the user 2 as a bounding box 26 with sides in the horizontal and vertical directions in the image 21 obtained by capturing an image by the imaging unit 20. In simple terms, when the presence of the white cane 3 in the image 21 is recognized by the image analysis of the image 21 described above, a rectangular bounding box 26 is set for the white cane 3. Here, the bounding box 26 is set so as to surround the entire white cane 3 (excluding the strap 2B) in the image 21 in a rectangular shape, regardless of whether the white cane 3 is tilted or not. The white cane detection unit 25 detects the set bounding box 26. Information related to the detected bounding box 26 (for example, the position coordinates of each vertex of the bounding box 26, etc.) is transmitted to the control unit 30 described later, and the horizontal width W1 of the bounding box 26 (also referred to as the bounding box horizontal width W1) is calculated in the control unit 30. The bounding box horizontal width W1 may be calculated by the white cane detection unit 25.

[0030] In this embodiment, the threshold setting unit 31 (see FIG. 2) is provided in the control unit 30, which will be described later. The threshold setting unit 31 sets in advance the width (e.g., the maximum width) of the white cane 3 in the image 21 at a predetermined distance as the threshold W2. Here, the threshold W2 is set based on the width of the white cane 3 when the white cane 3 is positioned along the vertical direction in the image 21 (when the white cane 3 is not tilted). In other words, the threshold W2 is obtained by converting the width of the white cane 3 at a predetermined distance (e.g., a position away from the imaging unit 20 by the arm's length of the user 2) into the width of the white cane 3 in the image 21. It is to be noted that the threshold W2 may be calculated by averaging the widths of a plurality of white canes 3.

[0031] The control unit 30 is configured with a computer or the like, and can perform calculation processing of the shoulder movement information 16, image processing of the captured image acquired from the imaging unit 20, and the like. The control unit 30 is housed inside a housing constituting the terminal 10. The control unit 30 can calculate the bounding box horizontal width W1 in the bounding box 26 based on, for example, the coordinates of each vertex of the bounding box 26 detected by the white cane detection unit 25. The control unit 30 can also determine the inclination of the mobility support device 10 (terminal 10) based on the bounding box horizontal width W1. Specifically, the control unit 30 compares the bounding box horizontal width W1 in the image 21 with a threshold value W2, thereby determining that the inclination of the terminal 10 is within a predetermined range, provided that the bounding box horizontal width W1 is smaller than the predetermined threshold value W2. The control unit 30 may also be configured to determine that the terminal 10 is inclined, provided that the bounding box horizontal width W1 is equal to or larger than the predetermined threshold value W2. With this configuration, the control unit 30 (tilt determination system 1) can determine the tilt of the movement assistance device 10 and the imaging unit 20.

[0032] The control unit 30 can perform control related to movement support in the terminal 10 based on the shoulder movement information 16, in addition to the tilt determination described above. Specifically, the control unit 30 can recognize the direction in which the user 2 is facing (traveling direction) based on the shoulder movement information 16. The control unit 30 can also detect obstacles (e.g., utility poles, cars, people, etc.) around the user 2 based on the image 21 acquired by the imaging unit 20. The control unit 30 may be provided divided into a plurality of functions. For example, a control mechanism related to movement support and a control mechanism related to tilt determination may be provided separately.

[0033] In this embodiment, the notification unit 40 is configured as a speaker (including earphones) and can issue a notification (such as an alert) by voice regarding a warning that the terminal 10 is tilted or a warning regarding an obstacle. That is, the notification unit 40 supports the movement of the user 2 by notifying the user 2 of the result of the tilt determination by the control unit 30 and the presence or absence of an obstacle. In determining the tilt of the terminal 10, the notification unit 40 issues a warning on the condition that the bounding box horizontal width W1 is greater than a threshold value W2.

[0034] The above is the configuration of the tilt determination system 1 according to one embodiment of the present invention, and the mobility support device 10 (terminal 10) of the present invention equipped with the tilt determination system 1. Next, a detailed description of an embodiment of the operation of the tilt determination system 1 will be given with reference to Figs. 3, 4(a), and 4(b). Note that in Figs. 3 and 4, for the sake of convenience, only the white cane 3 in the image 21 is described, but it should be noted that in reality, things other than the white cane 3 are also captured in the image 21. In addition, in this embodiment, the tilt determination when the terminal 10 is worn will be described, but the tilt determination may be performed at an appropriate timing other than when the terminal 10 is worn. In addition, Figs. 4(a) and 4(b) depict a state in which the user 2 is holding the grip portion of the white cane 3, but it is preferable to perform the tilt determination in a state in which the user holds the strap 2B of the white cane 3 and allows it to hang down under its own weight, as shown in Fig. 3.

[0035] 4(a) and 4(b), when the terminal 10 is attached to either one of the shoulders or the neck of the user 2 and started up, for example, a message (e.g., a voice) is issued by the notification unit 40 to perform tilt determination of the terminal 10. Also, a message is issued to capture an image of the white cane 3 for tilt determination (for example, "With your arm stretched out in front, grip the strap 2B of the white cane 3 and let the white cane 3 hang down (hang) under its own weight").

[0036] 3, the white cane 3 that is dangling down due to the user 2 gripping the strap 2B is imaged by the imaging unit 20, and an image 21 including the white cane 3 is obtained. The image 21 is then subjected to image analysis by the control unit 30, and a bounding box 26 for the white cane 3 is detected by the white cane detection unit 25.

[0037] As shown in Fig. 4(a), when the terminal 10 is worn at an angle, the white cane 3 is imaged at an angle. That is, when the terminal 10 is worn at an angle, a bounding box 26 is detected in the image 21 with the white cane 3 at an angle, and the bounding box horizontal width W1 is equal to or greater than the threshold value W2. In such a case, the notification unit 40 issues a notification (e.g., an alarm) that the terminal 10 is not worn correctly (worn at an angle). The user 2 who receives the notification corrects the angle of the terminal 10 (worn state) so that the notification is eliminated.

[0038] As shown in FIG. 4(b), when the terminal 10 is worn correctly, the white cane 3 is imaged in a state where it is aligned almost vertically (without tilt). That is, when the terminal 10 is worn correctly, the bounding box 26 is detected in the image 21 with the white cane 3 aligned vertically, and the bounding box horizontal width W1 is smaller than the threshold value W2. In such a case, no notification is given by the notification unit 40. Note that, for example, when the terminal 10 transitions from a tilted state to a state where the terminal 10 is no longer tilted, a message such as "The terminal 10 is worn correctly" may be notified.

[0039] Next, the operation (tilt determination method) of the tilt determination system 1 will be described with reference to the flowchart in Fig. 5. The tilt determination method of the present invention can be used to determine the tilt of a terminal 10 (mobility support device 10) that is worn on at least one of both shoulders and the neck of a user 2 and used together with a white cane 3. In the following description, the tilt determination of the terminal 10 that is executed at the start of wearing the terminal 10 will be described as an example.

[0040] When the terminal 10 (tilt determination system 1) is started up, the front side of the user 2 is imaged by the imaging unit 20, and an image 21 including the white cane 3 is acquired (step S10: image acquisition step). Next, the image 21 is transmitted to the control unit 30, and the control unit 30 performs image analysis (step S11: image analysis step).

[0041] When image analysis is performed in step S11, the white cane 3 is detected in the image 21 as a bounding box 26 having sides in the horizontal and vertical directions (step S12: white cane detection step).

[0042] Next, it is determined whether or not the bounding box horizontal width W1 detected in step S12 is smaller than a predetermined threshold value W2 (step S13: bounding box horizontal width determination step).

[0043] In step S13, if the bounding box horizontal width W1 is smaller than the threshold value W2, it is determined that the terminal 10 is not tilted in the tilt determination of the terminal 10 (step S14). When the process of step S14 is completed, a correction completion notification (for example, "The terminal 10 has been correctly attached.") is notified by the notification unit 40 (step S15). Furthermore, when the process of step S15 is completed, the series of processes ends.

[0044] In step S13, if the bounding box horizontal width W1 is equal to or greater than the threshold value W2, it is determined that the terminal 10 is tilted, and a warning is issued (step S20: warning notification step). Furthermore, the user 2 who has received the warning corrects the tilt of the terminal 10 (step S21). When the tilt of the terminal 10 is corrected in step S21, the process returns to the process of step S10. That is, the above series of processes are repeatedly executed until the terminal 10 is attached correctly (not tilted).

[0045] The above is an embodiment of the inclination determination system 1 and inclination determination method of the present invention, and the travel support device 10 that employs the inclination determination system 1. Next, the effects achieved by the inclination determination system 1 and inclination determination method, and the travel support device 10 of the present invention will be described below.

[0046] The above-described inclination determination system 1 has the following characteristic configurations (1) to (6). Therefore, the inclination determination system 1 can achieve the following unique effects that cannot be achieved by the conventional techniques.

[0047] (1) The tilt determination system 1 of this embodiment is a tilt determination system 1 for a mobility support device 10 that is worn on at least one of the shoulders and neck of a user 2 and used together with a white cane 3, and is equipped with an imaging unit 20 that acquires an image 21 including the white cane 3 by imaging the front side of the user 2, and a white cane detection unit 25 that detects the white cane 3 held by the user 2 as a bounding box 26 with sides in the horizontal and vertical directions within the image 21 captured by the imaging unit 20, and is characterized in that it determines the tilt of the mobility support device 10 based on the horizontal width W1 of the bounding box 26.

[0048] The above-mentioned inclination determination system 1 is adapted to perform inclination determination of a mobility support device 10 that is attached to at least one of both shoulders and the neck of a user 2 (visually impaired person) who uses a white cane 3. The above-mentioned inclination determination system 1 is adapted to detect the white cane 3 held by the user 2 as a bounding box 26 having sides in the horizontal and vertical directions in an image 21 captured by an imaging unit 20. The above-mentioned inclination determination system 1 is capable of detecting the entire white cane 3 as the bounding box 26 regardless of the inclination of the white cane 3 in the image 21. Here, the horizontal width W1 of the bounding box 26 is minimum when the white cane 3 is held in the vertical direction, and increases as the inclination angle of the white cane 3 in the horizontal direction increases. Therefore, the above-mentioned inclination determination system 1 can determine the inclination of the mobility support device 10 based on the horizontal width W1 of the bounding box 26. Furthermore, the above-described tilt determination system 1 can use the tilt determination result of the mobility support device 10 to determine whether or not the mobility support device 10 is correctly worn by the user 2, for example, when starting to use the mobility support device 10.

[0049] In this way, the tilt determination system 1 described above can determine the tilt of the mobility support device 10 without using a tilt sensor, and therefore can accurately determine the tilt without erroneous determination even if the tilt of the mobility support device 10 (including the imaging unit 20) is large. Furthermore, the tilt determination system 1 described above can accurately determine the tilt without using a complex mounting angle calibration device (including control), and therefore can suppress an increase in costs. Note that the tilt determination system 1 described above can determine whether the mobility support device 10 is correctly attached to the user 2 based on the tilt of the mobility support device 10. Here, the image of the white cane 3 in the imaging unit 20 is captured, for example, when the user 2 is using the white cane 3, that is, when the white cane 3 is held facing toward the ground (downward).

[0050] (2) The inclination determination system 1 of this embodiment is characterized in that it determines that the inclination of the mobility support device 10 is within a predetermined range, provided that the horizontal width W1 of the bounding box 26 is smaller than a predetermined threshold value W2.

[0051] The inclination determination system 1 described above, by adopting such a configuration, can determine whether or not the inclination of the mobility support device 10 is within a predetermined range, based on a predetermined threshold value W2. Here, the threshold value W2 is set based on the horizontal width W1 of the bounding box 26 set for the white cane 3 along the vertical direction at a predetermined distance from the user 2 (imaging unit 20). That is, when the horizontal width W1 of the bounding box 26 is smaller than the predetermined threshold value W2, the inclination determination system 1 described above can determine that the inclination of the mobility support device 10 is within a predetermined range (a normal range of inclination). In this way, the inclination determination system 1 described above can perform inclination determination by comparing the horizontal width W1 of the bounding box 26 detected in the image 21 with the predetermined threshold value W2, so that there is no need to provide a separate inclination sensor. As a result, the inclination determination system 1 described above can suppress an increase in cost.

[0052] The above-described tilt determination system 1 can also be configured to determine that the travel support device 10 is tilted on the condition that the horizontal width W1 of the bounding box 26 is equal to or greater than a predetermined threshold value W2. With this configuration, the above-described tilt determination system 1 can determine that the travel support device 10 is tilted.

[0053] (3) The inclination determination system 1 of this embodiment is characterized by determining the inclination of the mobility support device 10 based on an image 21 obtained by capturing an image of a white cane 3 that is hanging down due to its own weight.

[0054] With this configuration, the tilt determination system 1 described above can obtain an image 21 of the white cane 3 in an untilted state, and can therefore determine the tilt of the mobility support device 10 without any tilt of the white cane 3. Here, the user 2 can, for example, hold the strap 2B provided on the upper part of the white cane 3 to form the white cane 3 in the state where it is hanging down due to its own weight as described above. This allows the white cane 3 to be positioned vertically with high accuracy and imaged.

[0055] (4) The inclination determination system 1 of this embodiment has a notification unit 40, and is characterized in that the notification unit 40 issues a warning when the horizontal width W1 of the bounding box 26 is greater than or equal to a threshold value W2.

[0056] With such a configuration, the tilt determination system 1 described above can notify a warning by the notification unit 40 when the mobility support device 10 is tilted. As a result, the tilt determination system 1 described above can accurately notify the user 2 of the tilt of the mobility support device 10 even when the mobility support device 10 is worn in an inclined state. Note that, for example, when the mobility support device 10 transitions from a tilted state to a state where the mobility support device 10 is no longer tilted, a message such as "The mobility support device 10 is worn correctly" may be notified.

[0057] (5) The tilt determination method of this embodiment is a method for determining the tilt of a mobility support device 10 that is worn on at least one of the shoulders and the neck of a user 2 and used together with a white cane 3, and is characterized by executing the following steps: an image acquisition step of acquiring an image 21 including the white cane 3 by imaging the front side of the user 2 with an imaging unit 20; a white cane detection step of detecting the white cane 3 within the image 21 as a bounding box 26 with sides in the horizontal and vertical directions; a bounding box horizontal width determination step of determining whether the horizontal width W1 of the bounding box 26 detected in the white cane detection step is smaller than a predetermined threshold value W2; and a warning notification step of determining that the mobility support device 10 is tilted and notifying a warning, provided that the horizontal width W1 of the bounding box 26 in the bounding box horizontal width determination step is equal to or greater than the threshold value W2.

[0058] The tilt determination method described above performs an image acquisition step, a white cane detection step, and a bounding box horizontal width determination step, thereby determining the tilt of mobility support device 10. Furthermore, if the bounding box horizontal width determination step determines that mobility support device 10 is tilted, the tilt determination method described above can notify a warning by executing a warning notification step, thereby instantly communicating the tilt of mobility support device 10 to user 2.

[0059] (6) The mobility support device 10 of this embodiment is a mobility support device 10 that is worn on at least either both shoulders or the neck of a user 2 and used together with a white cane 3, and is equipped with a shoulder movement detection sensor 15 that acquires shoulder movement information 16 relating to the movement of at least both shoulders of the user 2, an imaging unit 20 that acquires an image 21 including the white cane 3 by imaging the front side of the user 2, a white cane detection unit 25 that detects the white cane 3 held by the user 2 as a bounding box 26 with sides in the horizontal and vertical directions within the image 21 acquired by imaging the imaging unit 20, and a control unit 30, and the control unit 30 is characterized in that it performs control to determine the inclination based on the horizontal width W1 of the bounding box 26.

[0060] By configuring the above-mentioned mobility support device 10 in this way, it is possible to perform tilt determination without providing a separate tilt sensor. This is expected to reduce the size and cost of the mobility support device 10. Furthermore, the above-mentioned mobility support device 10 can have the tilt determination system 1 integrated within the mobility support device 10. This is expected to further reduce the size of the mobility support device 10.

[0061] The above are the effects of the tilt determination system 1, tilt determination method, and movement support device 10 according to one embodiment of the present invention, but the tilt determination system 1, tilt determination method, and movement support device 10 of the present invention are not limited to the above-mentioned embodiment and can be modified in various ways.

[0062] In this embodiment, the mobility support device 10 (terminal 10) is attached to at least one of the shoulders and the neck of the user 2, but the present invention is not limited thereto and can be applied to various mobility support devices 10. For example, the mobility support device 10 may be attached to the head, face, or torso of the user 2. In addition, the imaging unit 20 can use various devices such as a camera or an image sensor capable of acquiring an image 21. In addition, the range of the imaging unit 20 may include the white cane 3 when performing tilt determination, and may not include the white cane 3 when performing mobility support, as necessary. In addition, the imaging unit 20 can be set in various directions such as forward, lateral, or backward of the user 2 according to the characteristics and usage mode of the mobility support device 10.

[0063] Furthermore, in this embodiment, the bounding box 26 is set with sides in the horizontal and vertical directions, but the directions of the sides forming the bounding box 26 may be changed depending on the shape and size of the white cane 3. Furthermore, in this embodiment, after the white cane 3 is detected in the image 21, the bounding box 26 is set for the detected white cane 3, but the present invention is not limited to this. Various image analysis means can be used as a method for setting the bounding box 26 and a method for detecting the white cane 3. The horizontal width W1 of the bounding box can be set to various values ​​depending on the characteristics of the imaging unit 20, the width of the white cane 3, etc. Furthermore, the threshold value W2 can be set to various values ​​depending on the white cane 3 that is the subject of tilt determination.

[0064] In this embodiment, the notification unit 40 is a speaker or an earphone, and is exemplified as a unit that notifies by voice, but the notification unit 40 is not limited to a unit that notifies by voice, and various units can be used as long as they can notify the user 2, who is the target of the movement support. In addition, in this embodiment, the tilt determination system 1 and the movement support device 10 are exemplified as being intended for visually impaired people, but the tilt determination system 1 and the movement support device 10 of the present invention can be intended for various people, not just visually impaired people. For example, the tilt determination system 1 and the movement support device 10 can be used for movement support for elderly people, etc. In such a case, a cane other than the white cane 3 may be used instead of the white cane 3. In addition, the notification unit 40 can not only notify when the bounding box horizontal width W1 is equal to or greater than the threshold value W2, but can also be configured to timely notify various kinds of notifications related to movement support. For example, the notification unit 40 can be configured to notify that the movement support device 10 is correctly attached (tilt correction is completed) when the bounding box horizontal width W1 is smaller than the threshold value W2.

[0065] The above are various embodiments and modifications of the tilt determination system, tilt determination method, and mobility support device according to the present invention. However, the present invention is not limited to the above-mentioned embodiments and modifications, and it will be easily understood by those skilled in the art that other embodiments are possible based on the teachings and spirit of the present invention without departing from the scope of the claims. [Industrial Applicability]

[0066] The present invention can be suitably used in all types of mobility assistance devices that assist users in moving. [Explanation of symbols]

[0067] 1: Tilt detection system 2: User 3: White cane 10: Mobility support device (terminal) 11: Battery 15: Shoulder movement detection sensor 20: Imaging unit 21: Image 25: White cane detector 26: Bounding box 30: Control section 31: Threshold setting unit 40:Notification section W1 : Bounding box horizontal width W2: Threshold

Claims

1. A tilt determination system for a mobility support device that is attached to at least one of both shoulders and the neck of a user and used together with a white cane, An imaging unit that captures an image including the white cane by imaging a front side of the user; a white cane detection unit that detects the white cane held by the user as a bounding box with sides in the horizontal and vertical directions within an image captured by the imaging unit; Equipped with An inclination determination system, comprising: a device that determines an inclination of the movement support device based on a horizontal width of the bounding box.

2. 2. The inclination determination system according to claim 1, wherein the inclination of the travel support device is determined to be within a predetermined range on the condition that the horizontal width of the bounding box is smaller than a predetermined threshold value.

3. 3. The inclination determination system according to claim 1, further comprising: determining an inclination of the mobility support device based on an image obtained by capturing an image of the white cane in a state where the white cane is hanging down due to its own weight.

4. A method for determining an inclination of a mobility support device that is attached to at least one of both shoulders and the neck of a user and used together with a white cane, comprising: an image acquisition step of acquiring an image including the white cane by capturing an image of a front side of the user with an imaging unit; a white cane detection step of detecting the white cane within the image as a bounding box having sides in the horizontal and vertical directions; a bounding box horizontal width determination step of determining whether or not the horizontal width of the bounding box detected in the white cane detection step is smaller than a predetermined threshold value; a warning notification step of determining that the mobility support device is tilted and notifying a warning, on the condition that the horizontal width of the bounding box is equal to or greater than the threshold value in the bounding box horizontal width determination step.

Images