Mobility support device

Abstract

To realize a mobility support device capable of accurately detecting movement of a user and appropriately supporting the movement of the user.SOLUTION: A mobility support device 1 is attached to at least one of the shoulders and the neck of a user 2, and includes a shoulder movement detection sensor 20 that acquires shoulder movement information 21 related to the movement of at least both shoulders of the user 2, and a control unit 40 that executes processing for a region of interest based on the shoulder movement information 21. The control unit 40 is capable of executing region-of-interest estimation processing that estimates a region of interest based on the shoulder movement information 21, and region-of-interest changing processing that changes the region of interest based on at least one of a plurality of region-of-interest changing patterns that is predefined based on the shoulder movement information 21, and is characterized by executing the region-of-interest estimation processing based on the shoulder movement information 21, selecting at least one region-of-interest changing pattern so as to approach the estimated region of interest, and executing the region-of-interest changing processing based on the selected region-of-interest changing pattern.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a mobility assistance device. [Background technology]

[0002] Conventionally, a method for selecting an augmented reality on a head-mounted device by a gesture with a hand is known (for example, Patent Document 1). The method described in Patent Document 1 is a method for selecting an augmented reality (AR) object on a head-mounted device (HMD), and includes the steps of: defining a region of interest (ROI) based on a gesture formed by at least one hand of a user; displaying a shape on the HMD to the user, the shape representing the outline of the ROI; displaying a plurality of AR objects to the user, each of the plurality of AR objects being associated with a target inside the ROI; and reducing the size of the ROI based on a first movement of the at least one hand of the user, the reduced size of the ROI being used to select a specific AR object from the plurality of AR objects.

[0003] As another method, it is also known to change an image displayed in a region of interest (ROI) based on the user's line of sight (for example, Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. WO2014 / 126966 [Patent Document 2] International Publication No. WO2016 / 103259 Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, the invention described in Patent Document 1 is said to reduce the size of the ROI when a user wearing a head-mounted device makes a gesture with his or her hand.

[0006] However, the method described in Patent Document 1 requires gestures to be made within a range where the gestures can be detected, and there is a concern that it requires skill. In addition, the invention described in Patent Document 1 above may lead to situations where the user does not know whether their hand has been detected, which may cause anxiety for the user, or the user may make an incorrect gesture and not perform the intended operation.

[0007] In addition, when the method described in Patent Document 2 is configured to detect the user's gaze, it is necessary to provide a gaze detection device for detecting the user's gaze. However, even if the face direction is detected through the gaze, there is a concern that the user's movement cannot be accurately detected because the face direction may change unintentionally due to environmental changes such as changes in surrounding sounds. Therefore, for example, when applied to a mobility support device that notifies the user of the presence of an obstacle, there is a concern that the user may not be accurately notified of the presence of the obstacle.

[0008] Therefore, the present invention aims to realize a mobility support device that can accurately detect a user's motion and provide an appropriate region of interest (ROI) based on the user's movement state. Another object of the present invention is to realize a mobility support device that can appropriately support the user's movement. [Means for solving the problem]

[0009] (1) The mobility support device of the present invention provided to solve the above-mentioned problems is a mobility support device worn on at least one of a user's shoulders and around the neck, and includes a shoulder movement detection sensor that acquires shoulder movement information related to the movement of at least both shoulders of the user, and a control unit that executes processing on a region of interest based on the shoulder movement information, wherein the control unit is capable of executing a region of interest estimation process that estimates the region of interest based on the shoulder movement information, and a region of interest change process that changes the region of interest based on at least one of a plurality of region of interest change patterns predefined based on the shoulder movement information, and is characterized in that the control unit executes the region of interest estimation process based on the shoulder movement information, selects at least one of the region of interest change patterns so as to approach the estimated region of interest, and executes the region of interest change process based on the selected region of interest change pattern.

[0010] The above-mentioned mobility support device can acquire shoulder movement information related to the movement of at least both shoulders of the user by the shoulder movement detection sensor. The above-mentioned mobility support device can also execute a region of interest estimation process based on the acquired shoulder movement information, select at least one region of interest change pattern so as to approach the estimated region of interest, and execute a region of interest change process based on the selected region of interest change pattern. That is, the above-mentioned mobility support device can accurately detect the user's motion based on the movement of both shoulders of the user. As a result, the above-mentioned mobility support device can provide an appropriate region of interest based on the user's moving state, and can appropriately support the user's movement.

[0011] (2) The above-mentioned mobility support device of the present invention may be characterized in that it includes an imaging unit that images the front side of the user, the area of ​​interest is set within the image captured by the imaging unit, and the control unit selects at least one area of ​​interest change pattern based on the image information captured by the imaging unit together with the shoulder movement information.

[0012] The above-mentioned mobility support device, with such a configuration, can select at least one region of interest change pattern based on image information captured by the imaging unit. That is, the above-mentioned mobility support device can change the region of interest based on both shoulder movement information and image information of the front side of the user, and can set the region of interest at an appropriate position taking into account the user's moving direction. This allows the above-mentioned mobility support device to support the user's movement with even greater precision.

[0013] (3) The above-mentioned mobility support device of the present invention may include a notification unit that notifies the result of the processing for the area of ​​interest, and the control unit is capable of executing an obstacle detection process that detects an obstacle included in the image information, and a bounding box setting process that sets a bounding box for the obstacle in the area of ​​interest, on condition that the obstacle is included in the image information, and the notification unit may be characterized in that it notifies the presence of the obstacle based on the set bounding box.

[0014] The above-mentioned mobility support device can set a bounding box for an obstacle as a region of interest by using the above-mentioned configuration. This allows the above-mentioned mobility support device to notify the presence of an obstacle based on the bounding box in the notification unit. Therefore, the above-mentioned mobility support device can appropriately support the mobility of a visually impaired person or the like.

[0015] (4) The above-mentioned mobility support device of the present invention may be characterized in that the shoulder movement detection sensor is composed of a gyro sensor, and the area of ​​interest change pattern is selected based on a change in acceleration detected by the gyro sensor.

[0016] The above-mentioned mobility support device can miniaturize the shoulder movement detection sensor by adopting such a configuration. Specifically, the above-mentioned mobility support device can detect the movement of both shoulders in three directions, the X-axis direction, the Y-axis direction, and the Z-axis direction, in an integrated manner by using a gyro sensor. Here, the gyro sensor may be configured with MEMS (Micro Electro Mechanical Systems) or the like. This is expected to reduce the size of the above-mentioned mobility support device. Note that instead of the gyro sensor, for example, acceleration sensors in the X-axis direction, the Y-axis direction, and the Z-axis direction may be provided, and the movement of both shoulders may be detected by combining the accelerations of these acceleration sensors.

[0017] (5) The above-mentioned mobility support device of the present invention may be characterized in that it has a housing that integrally contains the shoulder movement detection sensor, the control unit, and a battery for powering the shoulder movement detection sensor and the control unit, and that the housing is capable of being worn on at least one of the user's shoulders and around the neck.

[0018] The above-mentioned mobility support device, with such a configuration, can detect the movement of both shoulders of the user with high accuracy. Therefore, the above-mentioned mobility support device can appropriately support the movement of the user without relying on the movement of the user's hands. As a result, the above-mentioned mobility support device can suppress the user's anxiety regarding the detection of the movement of the hands.

[0019] (6) In the above-mentioned mobility support device of the present invention, the region of interest changing pattern may include a region widening changing pattern that changes the region of interest to widen in the width direction, on condition that the movement of the shoulders in the width direction in the shoulder movement information is greater than a predetermined movement threshold, a region depth direction changing pattern that shifts the region of interest in the depth direction, on condition that the movement speed of the shoulders in the shoulder movement information is greater than a predetermined speed threshold, and a region frontward direction changing pattern that shifts the region of interest in the frontward direction, on condition that the movement speed of the shoulders in the shoulder movement information is equal to or less than the speed threshold.

[0020] The above-mentioned mobility support device, by being configured as described above, can change the region of interest based on three patterns, namely, a region wide area change pattern, a region deep direction change pattern, and a region near direction change pattern. Therefore, the above-mentioned mobility support device can quickly and appropriately change the region of interest according to the three patterns. As a result, the above-mentioned mobility support device can shift the region of interest in the movement direction intended by the user, and therefore can provide appropriate mobility support according to the user's intention.

[0021] Here, the wide area change pattern is a pattern in which the area of ​​interest is changed to be widened in the width direction, on the condition that the movement of both shoulders in the width direction is greater than a predetermined movement threshold. As a result, even if the movement of both shoulders in the width direction of the user is large, the above-mentioned movement support device can quickly follow and support the user's movement. In addition, the area depth direction change pattern is a pattern in which the area of ​​interest is shifted in the depth direction, on the condition that the movement speed of both shoulders is greater than a predetermined speed threshold. As a result, even if the user's movement speed is fast, the user's movement support can be quickly followed and supported. In addition, the area front direction change pattern is a pattern in which the area of ​​interest is shifted in the front direction, on the condition that the movement speed of both shoulders is equal to or less than a predetermined speed threshold. As a result, when the user's movement speed is slow, appropriate movement support according to the user's movement speed can be provided.

[0022] (7) The above-mentioned mobility support device of the present invention may be characterized in that it has a shoulder movement information storage unit that stores the shoulder movement information acquired by the shoulder movement detection sensor, and the area of ​​interest change pattern is set based on learned shoulder movement information obtained by machine learning the shoulder movement information stored in the shoulder movement information storage unit.

[0023] The above-mentioned movement support device can acquire learned shoulder movement information by machine learning the shoulder movement information stored in the shoulder movement information storage unit. The above-mentioned movement support device can set a region of interest change pattern based on the learned shoulder movement information obtained by machine learning. This allows the above-mentioned movement support device to quickly set an appropriate region of interest change pattern, thereby appropriately supporting the movement of the user. Here, the shoulder movement information storage unit may be provided in the main body of the movement support device, or may be provided externally so as to be able to communicate with the movement support device. The above-mentioned movement support device may also be configured not to have a shoulder movement information storage unit. In such a case, the learned shoulder movement information may be acquired by machine learning shoulder movement information acquired in advance by a separate means.

[0024] (8) The above-mentioned mobility support device of the present invention may have a shoulder movement information storage unit that stores the shoulder movement information acquired by the shoulder movement detection sensor, and the region of interest estimation process may be executed based on learned shoulder movement information obtained by machine learning the shoulder movement information stored in the shoulder movement information storage unit.

[0025] The above-mentioned mobility support device can acquire learned shoulder movement information by machine learning the shoulder movement information stored in the shoulder movement information storage unit. The above-mentioned mobility support device can also execute a region of interest estimation process based on the learned shoulder movement information obtained by machine learning. This allows the above-mentioned mobility support device to quickly estimate an appropriate region of interest, thereby appropriately supporting the user's movement. Effect of the Invention

[0026] According to the present invention, it is possible to realize a mobility support device capable of accurately detecting a user's motion and providing an appropriate region of interest (ROI) based on the user's movement state. Also, according to the present invention, it is possible to realize a mobility support device capable of appropriately supporting the user's movement. [Brief description of the drawings]

[0027] [Figure 1] 1 is a plan view of a movement support device according to an embodiment of the present invention; [Diagram 2] 2 is a block diagram showing a device configuration of the movement support device of FIG. 1. [Diagram 3] 1 is an explanatory diagram showing the relationship between the movement direction of a user wearing a movement support device according to an embodiment of the present invention and the detection state of a shoulder movement detection sensor. FIG. [Figure 4] 1 is an explanatory diagram showing the relationship between the movement speed of a user wearing a movement support device according to an embodiment of the present invention and the detection state of a shoulder movement detection sensor. FIG. [Diagram 5] 4 is a flowchart showing control of a shoulder movement detection sensor constituting the movement support device of the present invention. [Figure 6] 4 is a flowchart showing control of an imaging unit constituting the travel support device of the present invention. [Figure 7] FIG. 13 is a block diagram showing a device configuration of a travel assistance device according to a modified example of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, a mobility support device 1 according to an embodiment of the present invention will be described in detail with reference to the drawings. Note that each figure is a schematic representation for easy understanding, and may differ from the actual size. In addition, in the following description, positional relationships such as up, down, left, and right will be described based on the posture shown in the drawings, unless otherwise specified.

[0029] The mobility support device 1 shown in Fig. 1 is for supporting the mobility of, for example, a visually impaired user 2. As shown in Fig. 2, the mobility support device 1 is configured as a terminal 10. The mobility support device 1 (terminal 10) includes a shoulder movement detection sensor 20, an imaging unit 30, a control unit 40, a notification unit 50, and the like.

[0030] The terminal 10 is worn by the user 2. The terminal 10 includes a battery 11. The imaging unit 30 and the battery 11 are housed inside a housing that constitutes the mobility support device 1. The terminal 10 may be stably worn by the user 2 without causing any burden. In this embodiment, the terminal 10 is a wearable terminal that can be worn on at least one of both shoulders or around the neck. In the example shown in FIG. 3 and FIG. 4, the terminal 10 is a wearable terminal that can be worn around the neck and around both shoulders. In this embodiment, the terminal 10 is formed in an inverted U-shape and can be hung around the neck of the user 2. In addition, the terminal 10 includes an imaging unit 30 so that the area in front of the user 2 can be photographed in the usage state. The terminal 10 includes a battery 11, and the imaging unit 30, the control unit 40, the notification unit 50, and other units that constitute the mobility support device 1 can be operated by the power supplied from the battery 11.

[0031] The shoulder movement detection sensor 20 is composed of, for example, a gyro sensor 20A, and can detect acceleration in three directions, the X-axis direction, the Y-axis direction, and the Z-axis direction, in an integrated manner. Here, the gyro sensor 20A may be composed of MEMS (Micro Electro Mechanical Systems) or the like. This is expected to reduce the size of the terminal 10. The shoulder movement detection sensor 20 can acquire shoulder movement information 21 related to the movement of at least both shoulders of the user 2. Specifically, the shoulder movement detection sensor 20 can detect the movement of both shoulders in three directions, the X-axis direction, the Y-axis direction, and the Z-axis direction, as the shoulder movement information 21 by using the gyro sensor 20A. Here, the shoulder movement information 21 is, for example, acceleration or the movement direction of the shoulders of the user 2 obtained by arithmetic processing of the acceleration. The arithmetic processing of the acceleration can be performed in at least one of the shoulder movement detection sensor 20 and the control unit 40 described later.

[0032] The imaging unit 30 is, for example, configured with a camera, and can capture an image of an area in front of the user 2. In this embodiment, the imaging unit 30 is provided, for example, on one side of the front end of the terminal 10 (see FIG. 1). Image information 31 of an image captured by the imaging unit 30 is transmitted to the control unit 40 via wired or wireless communication.

[0033] The control unit 40 is composed of a computer or the like, and can execute calculation processing of the shoulder movement information 21, image processing of the captured image acquired from the imaging unit 30, and the like. The control unit 40 is housed inside a housing constituting the terminal 10. The control unit 40 can execute processing for a region of interest (also referred to as ROI: Region of Interest) based on the shoulder movement information 21. Specifically, the control unit 40 can execute a region of interest estimation process for estimating a region of interest based on the shoulder movement information 21, and a region of interest change process for changing the region of interest based on at least one of a plurality of region of interest change patterns defined in advance based on the shoulder movement information 21.

[0034] The region of interest estimation process is a process of estimating the region of interest of user 2 based on changes in acceleration in the X-axis, Y-axis, and Z-axis directions (also referred to simply as the three-axis directions) when user 2 transitions from a state in which user 2 is facing forward as shown in FIG. 3(b) to a state in which user 2 is facing right as shown in FIG. 3(a) (arrow (3) in the figure), or, conversely, when user 2 transitions from a state in which user 2 is facing right as shown in FIG. 3(a) to a state in which user 2 is facing forward as shown in FIG. 3(b) (arrow (4) in the figure).

[0035] The region of interest estimation process is a process of estimating the region of interest of the user 2 based on the change in acceleration in the three axial directions when the user 2 transitions from a state in which the user 2 faces forward as shown in FIG. 3(b) to a state in which the user 2 faces left as shown in FIG. 3(c) (arrow (1) in the figure), or, conversely, when the user 2 transitions from a state in which the user 2 faces left as shown in FIG. 3(c) to a state in which the user 2 faces forward as shown in FIG. 3(b) (arrow (2) in the figure). In other words, the region of interest estimation process is a process of estimating a base region of interest by estimating the direction in which the user 2 faces based on the shoulder movement information 21. Here, the region of interest is a target region in which the user 2 wants to move. In this embodiment, the region of interest is set in an image captured by the imaging unit 30. The region of interest is a target region in which, when the presence of an obstacle is detected in the image information 31, a notification of the obstacle is made.

[0036] The region of interest changing process selects at least one region of interest changing pattern so as to approach the estimated region of interest, and executes the region of interest changing process based on the selected region of interest changing pattern.

[0037] In this embodiment, the region of interest changing pattern is defined as three patterns: a wide region changing pattern, a region depth direction changing pattern, and a region front direction changing pattern.

[0038] The wide area change pattern is defined as a change pattern for making a change to widen the area of ​​interest in the width direction, on the condition that the movement of both shoulders in the width direction in the shoulder movement information 21 is greater than a predetermined movement threshold. Here, the movement threshold can be set to an arbitrary value depending on, for example, the characteristics of the shoulder movement detection sensor 20 and the motion status of the user 2 to be used. Also, in the wide area change pattern, a process for making a change to widen the area of ​​interest in the width direction is executed. In other words, in the wide area change pattern, a change is executed to the settings of the shoulder movement detection sensor 20 and the wide area detection in the imaging unit 30.

[0039] As shown in FIG. 4(b), the region depth direction change pattern is defined as a change pattern for making a change to shift the region of interest in the depth direction on the condition that the movement speed of both shoulders in the shoulder movement information 21 is greater than a predetermined speed threshold Tv. Here, the speed threshold Tv can be set to an arbitrary value depending on, for example, the characteristics of the shoulder movement detection sensor 20 and the motion status of the user 2 using the shoulder movement detection sensor 20. FIG. 4(b) illustrates a case in which the movement speed of the user 2 in the X direction and the Z direction is greater than the speed threshold Tv. In addition, in the region depth direction change pattern, a process of shifting the region of interest in the depth direction is executed. In other words, in the region depth direction change pattern, a change in the setting of the region depth detection in the shoulder movement detection sensor 20 and the imaging unit 30 is executed.

[0040] The region front direction change pattern is defined as a change pattern for making a change to shift the region of interest toward the front, on condition that the movement speed of both shoulders in the shoulder movement information 21 is equal to or less than the speed threshold Tv, as shown in Fig. 4(a). Fig. 4(a) illustrates a case in which the movement speed of the user 2 in the X direction and Z direction is equal to or less than the speed threshold Tv. In addition, in the region front direction change pattern, a process of shifting the region of interest toward the front is executed. In other words, in the region front direction change pattern, a change is executed to the setting of the region front detection in the shoulder movement detection sensor 20 and the imaging unit 30.

[0041] In the mobility support device 1 of this embodiment, the above-mentioned region of interest change pattern is selected based on a change in acceleration detected by the shoulder movement detection sensor 20 (gyro sensor 20A). In the mobility support device 1 of this embodiment, the control unit 40 performs control to select at least one region of interest change pattern based on the shoulder movement information 21 and the image information 31 captured by the imaging unit 30. Specifically, the control unit 40 performs, as a process based on the captured image, an obstacle detection process to detect an obstacle included in the image information 31, and a bounding box setting process to set a bounding box for the obstacle as a region of interest on the condition that the obstacle is included in the image information 31.

[0042] In this embodiment, the notification unit 50 is configured as a speaker (including earphones) and notifies the result of processing for the region of interest by voice. Specifically, the notification unit 50 can notify the presence of an obstacle based on a set bounding box. That is, the notification unit 50 is configured to perform a movement support notification that supports the movement of the user 2 based on the result of processing by the control unit 40.

[0043] The above is the configuration of the travel support device 1 according to one embodiment of the present invention. Next, the operation of the travel support device 1 will be described with reference to the flowcharts of FIGS.

[0044] Fig. 5 is a flowchart showing the control of the shoulder movement detection sensor 20 constituting the movement support device 1, and Fig. 6 is a flowchart showing the control of the imaging unit 30 constituting the movement support device 1. Note that in this embodiment, the shoulder movement detection sensor 20 and the imaging unit 30 are controlled in parallel.

[0045] 5, when the control unit 40 starts controlling the shoulder movement detection sensor 20, the shoulder movement detection sensor 20 acquires acceleration (shoulder movement information 21) (step S10). Then, it is determined whether or not shoulder movement information 21 has been accumulated for a certain period of time (step S11).

[0046] If it is determined in step S11 that shoulder movement information 21 for a certain period of time has not been accumulated, the process returns to step S10. On the other hand, if it is determined in step S11 that shoulder movement information 21 for a certain period of time has been accumulated, an analysis of the shoulder movement information 21 is executed (step S12).

[0047] Next, it is determined whether or not the movement in the width direction in the shoulder movement information 21 is greater than a predetermined movement threshold value (step S13).

[0048] In step S13, if the widthwise movement in the shoulder movement information 21 (the amount of widthwise movement of both shoulders) is greater than a predetermined movement threshold, a wide area change pattern is selected as the area change pattern (step S14). When the wide area change pattern is selected, the wide area detection setting is changed. When the process of step S14 is completed, the process returns to the process of step S10. Note that the process can also be terminated after the process of step S14 is completed.

[0049] In step S13, if the movement in the width direction in the shoulder movement information 21 is equal to or smaller than a predetermined movement threshold, it is determined whether or not the movement speed in the shoulder movement information 21 is greater than a predetermined speed threshold Tv (step S15).

[0050] In step S15, if the movement speed (movement speed of both shoulders) in the shoulder movement information 21 is greater than a predetermined speed threshold Tv, a region depth direction change pattern is selected as the region change pattern (step S16). When the region depth direction change pattern is selected, the setting of region depth detection is changed. Specifically, if the movement speed of both shoulders is faster than the speed threshold Tv, the region of interest is shifted in the depth direction. When the process of step S16 is completed, the process returns to step S10. Note that the process can also be terminated after the process of step S16 is completed.

[0051] In step S15, if the movement speed in the shoulder movement information 21 is equal to or lower than a predetermined speed threshold Tv, a region front direction change pattern is selected as the region change pattern (step S17). When the region front direction change pattern is selected, the setting of region front detection is changed. Specifically, if the movement speed of both shoulders is slower than the speed threshold Tv, the region of interest is shifted in the front direction. When the process of step S17 is completed, the process returns to the process of step S10. Note that the process can also be terminated after the process of step S17 is completed.

[0052] The above is one embodiment of the control of shoulder movement detection sensor 20 constituting movement support device 1. Next, the control of imaging unit 30 constituting movement support device 1 will be described below.

[0053] 6, when the process starts, analysis settings in the imaging unit 30 are read (step S20). Then, imaging is performed in the imaging unit 30, and an image is acquired (step S21).

[0054] When an image is acquired in step S21, image analysis is performed (step S22). In the image analysis in step S22, obstacle detection is performed, and image information 31 including the obstacle is acquired. Note that the image analysis can be performed by various means such as image matching.

[0055] Next, a bounding box is acquired (set) for the obstacle detected in step S22 (step S23). Once the bounding box has been set, the notification unit 50 notifies the presence of the obstacle based on the set bounding box (step S24). Once the process of step S24 is complete, the process returns to step S20. Note that the process can also be terminated after the process of step S24 is complete.

[0056] The configuration and operation of the movement support device 1 according to one embodiment of the present invention have been described above. Next, the effects of the movement support device 1 will be described below.

[0057] (1) The mobility support device 1 of this embodiment is a mobility support device 1 worn on at least one of both shoulders and around the neck of a user 2, and includes a shoulder movement detection sensor 20 that acquires shoulder movement information 21 relating to the movement of at least both shoulders of the user 2, and a control unit 40 that executes processing on a region of interest based on the shoulder movement information 21. The control unit 40 is capable of executing a region of interest estimation process that estimates the region of interest based on the shoulder movement information 21, and a region of interest change process that changes the region of interest based on at least one of a plurality of region of interest change patterns predefined based on the shoulder movement information 21. The control unit 40 executes the region of interest estimation process based on the shoulder movement information 21, and selects at least one of the region of interest change patterns so as to approach the estimated region of interest, and executes the region of interest change process based on the selected region of interest change pattern.

[0058] The above-mentioned mobility support device 1 can acquire shoulder movement information 21 related to the movement of at least both shoulders of the user 2 by the shoulder movement detection sensor 20. The above-mentioned mobility support device 1 can also execute a region of interest estimation process based on the acquired shoulder movement information 21, select at least one region of interest change pattern so as to approach the estimated region of interest, and execute a region of interest change process based on the selected region of interest change pattern. That is, the above-mentioned mobility support device 1 can accurately detect the movement of the user 2 based on the movement of both shoulders of the user 2. As a result, the above-mentioned mobility support device 1 can provide an appropriate region of interest based on the moving state of the user 2, and can appropriately support the movement of the user 2.

[0059] (2) The above-mentioned mobility support device 1 is equipped with an imaging unit 30 that images the front side of the user 2, and the area of ​​interest is set within the image captured by the imaging unit 30, and the control unit 40 selects at least one area of ​​interest change pattern based on image information 31 captured by the imaging unit 30 together with shoulder movement information 21.

[0060] With this configuration, the above-mentioned mobility support device 1 can select at least one region of interest change pattern based on image information 31 captured by the imaging unit 30. That is, the above-mentioned mobility support device 1 can change the region of interest based on both shoulder movement information 21 and image information 31 of the front side of the user 2, and can therefore set the region of interest at an appropriate position taking into account the moving direction of the user 2. This allows the above-mentioned mobility support device 1 to support the movement of the user 2 with even greater accuracy.

[0061] (3) The above-mentioned mobility support device 1 is equipped with a notification unit 50 that notifies the result of the processing for the area of ​​interest, and the control unit 40 is capable of executing an obstacle detection process that detects an obstacle included in the image information 31, and a bounding box setting process that sets a bounding box for the obstacle in the area of ​​interest, on the condition that the obstacle is included in the image information 31, and the notification unit 50 is configured to notify the presence of the obstacle based on the set bounding box.

[0062] By configuring the above-described mobility support device 1 in this way, it is possible to set a bounding box for an obstacle as a region of interest. This allows the above-described mobility support device 1 to notify the presence of an obstacle based on the bounding box at the notification unit 50. Therefore, the above-described mobility support device 1 can appropriately support the mobility of a visually impaired person or the like.

[0063] (4) In the above-mentioned mobility support device 1, the shoulder movement detection sensor 20 is composed of a gyro sensor 20A, and the area of ​​interest change pattern is selected based on the change in acceleration detected by the gyro sensor 20A.

[0064] The above-described mobility support device 1, by adopting such a configuration, can miniaturize the shoulder movement detection sensor 20. Specifically, the above-described mobility support device 1 can integrally detect the movement of both shoulders in three directions, the X-axis direction, the Y-axis direction, and the Z-axis direction, by using the gyro sensor 20A.

[0065] (5) The above-mentioned mobility support device 1 has a housing that integrally contains a shoulder movement detection sensor 20, a control unit 40, and a battery 11 for driving the shoulder movement detection sensor 20 and the control unit 40, and the housing is capable of being worn on at least one of the shoulders and the neck of the user 2.

[0066] With such a configuration, the above-described mobility support device 1 can accurately detect the movement of both shoulders of the user 2. Therefore, the above-described mobility support device 1 can appropriately support the movement of the user 2 without relying on the hand movement of the user 2. As a result, the above-described mobility support device 1 can suppress the user 2 from becoming anxious about whether or not the hand movement has been detected.

[0067] (6) In the above-mentioned mobility support device 1, the region of interest changing patterns include a region widening changing pattern that changes the region of interest to widen in the width direction, on condition that the movement of the shoulders in the width direction in the shoulder movement information 21 is greater than a predetermined movement threshold, a region depth direction changing pattern that shifts the region of interest in the depth direction, on condition that the movement speed of the shoulders in the shoulder movement information 21 is greater than a predetermined speed threshold Tv, and a region frontward direction changing pattern that shifts the region of interest in the frontward direction, on condition that the movement speed of the shoulders in the shoulder movement information is equal to or less than the speed threshold Tv.

[0068] The above-mentioned mobility support device 1, by being configured as described above, can change the region of interest based on three patterns, namely, a region wide area change pattern, a region deep area direction change pattern, and a region near area direction change pattern. Therefore, the above-mentioned mobility support device 1 can quickly and appropriately change the region of interest according to the three patterns. As a result, the above-mentioned mobility support device 1 can shift the region of interest in the movement direction intended by the user 2, and therefore can provide appropriate mobility support according to the intention of the user 2.

[0069] Here, the region wide change pattern is a change that widens the region of interest in the width direction on the condition that the movement of both shoulders in the width direction is greater than a predetermined movement threshold. As a result, the above-mentioned movement support device 1 can quickly follow and support the movement of the user 2 even when the movement of both shoulders in the width direction of the user 2 is large. In addition, the region depth direction change pattern is a change that shifts the region of interest in the depth direction on the condition that the movement speed of both shoulders is greater than a predetermined speed threshold Tv. As a result, even when the movement speed of the user 2 is fast, it is possible to quickly follow and support the movement of the user 2. In addition, the region front direction change pattern is a change that shifts the region of interest in the front direction on the condition that the movement speed of both shoulders is equal to or less than a predetermined speed threshold Tv. As a result, when the movement speed of the user 2 is slow, appropriate movement support according to the movement speed of the user 2 can be performed.

[0070] The above is the action and effect of the movement support device 1 according to one embodiment of the present invention. Next, a movement support device 100 according to a modified example will be described in detail. The movement support device 100 according to the modified example has the same configuration as the movement support device 1 according to the above-mentioned embodiment, except that a shoulder movement information storage unit 110 is provided. Therefore, in the following description, a description of the configuration of similar parts will be omitted. Also, please note that in the drawings, similar reference numerals are used for members similar to those of the movement support device 1.

[0071] <<Variations>> 7, mobility support device 100 is configured as a terminal 10, similar to mobility support device 1. Mobility support device 100 includes a shoulder movement detection sensor 20, an imaging unit 30, a control unit 40, a notification unit 50, a shoulder movement information storage unit 110, and the like.

[0072] The shoulder movement information accumulation unit 110 is composed of, for example, a storage medium such as a memory. The shoulder movement information accumulation unit 110 can accumulate (store) the shoulder movement information 21 acquired by the shoulder movement detection sensor 20, for example, at predetermined time intervals. In the movement support device 100 according to the modified example, the region of interest change pattern is set based on learned shoulder movement information 22 obtained by machine learning the shoulder movement information 21 (also referred to as accumulated shoulder movement information A1 to An) accumulated in the shoulder movement information accumulation unit 110. This allows the movement support device 100 to execute a change process of the region of interest based on the region of interest change pattern set by machine learning. Note that the learned shoulder movement information 22 may be composed of one or more pieces of information.

[0073] Here, the shoulder movement information accumulation unit 110 may be provided in the main body of the movement support device 100, or may be provided externally so as to be able to communicate with the movement support device 100. The movement support device 100 described above may also be configured not to have the shoulder movement information accumulation unit 110. In such a case, shoulder movement information 21 previously acquired by a separate means may be machine-learned to acquire learned shoulder movement information 22.

[0074] Furthermore, in movement support device 100, the region of interest estimation process is executed based on learned shoulder movement information 22 obtained by machine learning of shoulder movement information 21 stored in shoulder movement information storage unit 110. In this way, movement support device 100 according to the modified example can execute a region of interest estimation process and a region of interest change process based on learned shoulder movement information 22 obtained by machine learning of a plurality of pieces of accumulated shoulder movement information A1 to An stored in shoulder movement information storage unit 110.

[0075] The above is the configuration of the movement support device 100 according to the modified example of the present invention. Next, the effects of the movement support device 100 will be described below.

[0076] (7) A movement support device 100 according to a modified example of the present invention has a shoulder movement information storage unit 110 that stores shoulder movement information 21 acquired by a shoulder movement detection sensor 20, and the area of ​​interest change pattern is set based on learned shoulder movement information 22 obtained by machine learning the shoulder movement information 21 (stored shoulder movement information A1 to An) stored in the shoulder movement information storage unit 110.

[0077] The above-mentioned mobility support device 100 can acquire learned shoulder movement information 22 by machine learning the shoulder movement information 21 stored in the shoulder movement information storage unit 110. Furthermore, the above-mentioned mobility support device 100 can set a region of interest change pattern based on the learned shoulder movement information 22 obtained by machine learning. This allows the above-mentioned mobility support device 100 to quickly set an appropriate region of interest change pattern, thereby appropriately supporting the movement of the user 2.

[0078] (8) A movement support device 100 according to a modified example of the present invention has a shoulder movement information storage unit 110 that stores shoulder movement information 21 acquired by a shoulder movement detection sensor 20, and the region of interest estimation process is executed based on learned shoulder movement information 22 obtained by machine learning the shoulder movement information 21 (stored shoulder movement information A1 to An) stored in the shoulder movement information storage unit 110.

[0079] The above-mentioned mobility support device 100 can acquire learned shoulder movement information 22 by machine learning the shoulder movement information 21 stored in the shoulder movement information storage unit 110. Furthermore, the above-mentioned mobility support device 100 can execute a region of interest estimation process based on the learned shoulder movement information 22 obtained by machine learning. This allows the above-mentioned mobility support device 100 to quickly estimate an appropriate region of interest, thereby appropriately supporting the movement of the user 2.

[0080] The configuration and effects of the mobility support device 1,100 of the present invention have been described above, but the mobility support device 1,100 of the present invention is not limited to the above-described embodiment and can be modified in various ways.

[0081] In this embodiment, the shoulder movement detection sensor 20 is integrally provided on the housing of the movement support device 1, but the shoulder movement detection sensor 20 may be provided separately from the housing. The shoulder movement detection sensor 20 may not only be formed as a single sensor, but may be provided separately on each of the shoulders, for example. In this embodiment, the movement support device 1 is formed in an inverted U-shape, but the movement support device 1 can be formed in various shapes and sizes. In this embodiment, the movement support device 1 is attached to at least one of the shoulders and the neck of the user 2, but the shoulder movement detection sensor 20 can be disposed in various positions on the user 2 as long as it can detect the movement of the shoulders of the user 2. In this embodiment, the shoulder movement detection sensor 20 is configured as the gyro sensor 20A, but the movement support device 1 of the present invention is not limited to this. For example, instead of the gyro sensor 20A, acceleration sensors in the X-axis direction, Y-axis direction, and Z-axis direction may be provided, and the movement of the shoulders may be detected by combining the accelerations of these acceleration sensors. In addition, the gyro sensor 20A may be of various types and configurations, including those using MEMS. The movement threshold and the velocity threshold Tv can be set in various ways and with various values, not only for the X-axis direction, the Y-axis direction, and the Z-axis direction, but also for the total amount of movement in these axial directions.

[0082] The set region of interest and bounding box can be formed in various forms as long as they do not impede movement support for the user 2. In the present embodiment, various controls are performed by a single control unit 40, but the control unit 40 may be formed of multiple units. For example, the control unit 40 may be provided for each function, such as image processing, analysis processing of the shoulder movement information 21, and notification processing in the notification unit 50. In the present embodiment, three patterns, a region wide area change pattern, a region depth direction change pattern, and a region front direction change pattern, are defined as the region of interest change pattern, but the region of interest change pattern can be defined as one or more various patterns.

[0083] In the present embodiment, the imaging unit 30 is configured as a camera and is provided at one end of the housing, but various imaging units capable of recognizing obstacles can be used as the imaging unit 30. The imaging unit 30 may be not only a single one, but also a plurality of ones. The imaging unit 30 can be arranged at various positions so as to be able to support the user 2. In addition, instead of or together with the imaging unit 30, for example, an obstacle detection sensor or the like may be provided to detect the obstacle. In the present embodiment, the movement support device 1 supports the movement of the user 2 based on both the shoulder movement information 21 and the image information 31, but the movement support device 1 of the present invention can support the movement of the user 2 based on the shoulder movement information 21.

[0084] In this embodiment, the notification unit 50 is a speaker or an earphone, and is exemplified as a unit that notifies by voice, but the notification unit 50 is not limited to a unit that notifies by voice, and various units can be used as long as they can notify the user 2, who is the target of the movement support. Also, in this embodiment, the movement support device 1 is exemplified as being intended for visually impaired people, but the movement support device 1 of the present invention can be intended for various people, not just visually impaired people. For example, the movement support device 1 can be used for movement support for elderly people, etc.

[0085] In addition, in the movement support device 100 according to the above-mentioned modified example, the shoulder movement information 21 (accumulated shoulder movement information A1 to An) accumulated in the shoulder movement information accumulation unit 110 is machine-learned to acquire the learned shoulder movement information 22, but the movement support device 100 of the present invention is not limited to this. For example, the shoulder movement information accumulation unit 110 may be provided externally instead of in the main body of the terminal 10, and the shoulder movement information 21 may be transmitted to the external shoulder movement information accumulation unit 110 by wireless communication with the terminal 10. In addition, the learned shoulder movement information 22 may be acquired by machine learning the shoulder movement information 21 acquired in advance by a separate means. In such a case, as in the movement support device 1, a configuration may be adopted in which the shoulder movement information accumulation unit 110 is not provided, and the region of interest estimation process and the region of interest change pattern may be set based on the learned shoulder movement information 22 acquired in advance. In addition, various machine learning means may be used for the above-mentioned machine learning.

[0086] The above are various embodiments and variations of the mobility support device according to the present invention. However, the present invention is not limited to those exemplified in the above-mentioned embodiments and variations, and it will be easily understood by those skilled in the art that other embodiments are possible based on the teachings and spirit of the present invention without departing from the scope of the claims. [Industrial Applicability]

[0087] The present invention can be suitably used in all types of mobility assistance devices that assist users in moving. [Explanation of symbols]

[0088] 1:Mobility support device 10: Terminal 11: Battery 20: Shoulder movement detection sensor 20A: Gyro sensor 21: Shoulder movement information 22: Learning shoulder movement information 30: Imaging section 31: Image information 40: Control section 50:Notification section 100:Mobility support device 110: Shoulder movement information storage unit

Claims

1. A mobility support device that is worn on at least one of the user's shoulders and the user's neck, a shoulder movement detection sensor for acquiring shoulder movement information relating to the movement of at least both shoulders of the user; A control unit that executes a process for a region of interest based on the shoulder movement information; Equipped with The control unit is A region of interest estimation process for estimating the region of interest based on the shoulder movement information; A region of interest changing process for changing the region of interest based on at least one of a plurality of region of interest changing patterns predefined based on the shoulder movement information; It is possible to A mobility support device comprising: a region of interest estimation process that estimates the region of interest based on the shoulder movement information; a region of interest change pattern that selects at least one region of interest change pattern that approaches the estimated region of interest; and a region of interest change process that estimates the region of interest based on the selected region of interest change pattern.

2. An imaging unit that images a front side of the user, the region of interest is set within an image captured by the imaging unit, The movement support device according to claim 1 , wherein the control unit selects at least one of the region of interest change patterns based on the shoulder movement information as well as image information captured by the imaging unit.

3. The region of interest change pattern is a region widening change pattern for widening the region of interest in the width direction on condition that the movement of the shoulders in the width direction in the shoulder movement information is greater than a predetermined movement threshold; a region depth direction change pattern for shifting the region of interest in a depth direction on condition that a moving speed of the both shoulders in the shoulder movement information is greater than a predetermined speed threshold; a region front direction change pattern for shifting the region of interest toward the front on condition that the movement speed of the both shoulders in the shoulder movement information is equal to or less than the speed threshold; The movement support device according to claim 1 or 2, comprising:

Images