Tilt detection system

Abstract

To provide an inclination determination system capable of accurately determining inclination without separately providing a sensor for correcting inclination.SOLUTION: An inclination determination system 1 of a movement support device 10 fitted to both the shoulders and the head of a user 2 includes an imaging section 20 for imaging a front side of the user 2, a bounding box detection section 25 for detecting a columnar object 3 extended in a vertical direction in a captured image 21 as a bounding box 26, a predetermined section setting section 30 for setting a predetermined section in advance relative to a vertical direction of the image 21, and a columnar object width threshold setting section 35 for setting width in a lower end section of the columnar object 3 as a columnar object width threshold W2 in advance, and determines the inclination of the movement support device 10 on the basis of a bounding box horizontal width W1 and the columnar object width threshold W2 in a lower end section of the bounding box 26 on condition that the lower end section of the bounding box 26 is included in the predetermined section.SELECTED DRAWING: Figure 2

Description

【Technical Field】 【0001】 The present invention relates to an inclination determination system. 【Background Art】 【0002】 Conventionally, a camera that acquires the tilt angle of a camera with a tilt detection unit (acceleration sensor) and performs tilt correction is known (for example, Patent Document 1). In recent years, wearable devices using cameras have also become known, and movement support devices for visually impaired persons and the like using such wearable devices are also known. 【Prior Art Documents】 【Patent Documents】 【0003】 【Patent Document 1】 Japanese Patent Application Laid-Open No. 2014-99832 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0004】 By the way, when using the above-described wearable device as a movement support device, there is a concern that the wearable device may shift due to vibrations or the like during the movement of the user, and the wearable device may be in an excessively tilted state. In such a case, since the camera mounted on the wearable device also tilts greatly, there is a concern that the image captured by the camera may also be in an excessively tilted state. 【0005】 【0006】 Therefore, the present invention aims to realize a tilt determination system, a tilt determination method, and a mobility support device that can accurately determine the tilt without providing a separate sensor for tilt correction. [Means for solving the problem] 【0007】 (1) The tilt determination system of the present invention, provided to solve the above-mentioned problems, is a tilt determination system for a mobility support device attached to at least one of the user's shoulders and neck, comprising: an imaging unit that images the front side of the user; a bounding box detection unit that detects columnar objects extending vertically in the image captured by the imaging unit as bounding boxes with sides in the horizontal and vertical directions; a predetermined section setting unit that sets a predetermined section in advance with respect to the vertical direction of the image; and a columnar object width threshold setting unit that sets the width at the lower end of the columnar object as a columnar object width threshold, wherein the tilt of the mobility support device is determined based on the horizontal width of the bounding box at the lower end of the bounding box and the columnar object width threshold, provided that the lower end of the bounding box is included within the predetermined section. 【0008】 The tilt determination system described above can detect columnar objects in an image captured by the imaging unit as bounding boxes with sides in the horizontal and vertical directions. In this invention, the bounding box detects the entire columnar object as a rectangular frame, regardless of the tilt of the columnar object in the image. The width of the lower end of the bounding box is obtained as the horizontal width of the bounding box. Furthermore, the tilt determination system described above can determine the tilt of the mobile support device based on the horizontal width of the bounding box at the lower end of the bounding box and the columnar object width threshold, provided that the detected lower end of the bounding box is included (located) within a predetermined section in the vertical direction of the image. In this way, the tilt determination system described above can determine the tilt of the mobile support device without using a tilt sensor, so it can accurately determine the tilt without misjudging even if the tilt of the mobile support device is large. 【0009】 Here, the predetermined section is defined with respect to the vertical direction of the image. In other words, the predetermined section corresponds to the distance from the imaging unit to the columnar object. The columnar object width threshold is set, for example, to the width of the lower end of the columnar object when it is not tilted. Therefore, the tilt determination system described above can recognize whether or not the columnar object is tilted by comparing the horizontal width of the bounding box within the predetermined section (within the predetermined distance) with the columnar object width threshold. As a result, the tilt determination system described above can determine the tilt of the movement support device based on the tilt state of the columnar object. This allows the tilt determination system described above to accurately determine the tilt without providing a separate sensor for tilt correction, thus suppressing cost increases. 【0010】 Incidentally, in the case of a columnar object that is inclined, the horizontal width of the bounding box increases. Therefore, it is thought that the inclination state of the columnar object can be determined by determining whether or not the horizontal width of the bounding box is greater than the columnar object width threshold. 【0011】 (2) Therefore, the tilt determination system of the present invention described above is characterized in that it determines that the movement support device is tilted on the condition that the horizontal width of the bounding box is greater than the columnar object width threshold. 【0012】 The tilt detection system described above can determine if a columnar object is tilted by adopting this configuration. Furthermore, since the tilt detection system can perform tilt detection based on bounding boxes detected in the image, there is no need to provide a separate tilt sensor. As a result, the tilt detection system described above can suppress cost increases. 【0013】 (3) The tilt determination system of the present invention described above is characterized in that the columnar object is one of a utility pole, a road sign post, or a streetlamp post. 【0014】 The tilt detection system described above, with this configuration, can determine the tilt of the mobility assistance device when targeting, for example, utility poles, road sign posts, and streetlamp posts (collectively referred to as utility poles, etc.) that may be obstacles on sidewalks. In other words, since it can determine the tilt of the mobility assistance device when targeting utility poles, etc. located at the user's destination, the tilt detection system described above can quickly recognize the tilt of the mobility assistance device. 【0015】 (4) The tilt determination system of the present invention described above is characterized in that it has a notification unit, and a warning is issued by the notification unit on the condition that the horizontal width of the bounding box is greater than the columnar object width threshold. 【0016】 With this configuration, the tilt detection system described above can issue a warning via the notification unit when the mobility assistance device is tilted. As a result, the tilt detection system can accurately notify the user of the tilt of the mobility assistance device, even if the device tilts against the user's will. 【0017】 (5) The tilt determination method of the present invention, provided to solve the above-mentioned problems, is a tilt determination method for a mobility support device attached to at least one of the user's shoulders and neck, comprising: an image acquisition step of acquiring an image by imaging the front side of the user with an imaging unit; a bounding box detection step of detecting a columnar object extending vertically in the image as a bounding box with sides in the horizontal and vertical directions; a bounding box coordinate acquisition step of acquiring the vertical coordinates of the lower end of the bounding box, provided that the bounding box is detected in the bounding box detection step; and the vertical coordinates are The method is characterized by performing the following steps: a vertical coordinate determination step of determining whether the image is within a predetermined interval set in advance in the vertical direction; a columnar object width determination step of determining whether the horizontal width of the bounding box at the lower end of the bounding box is greater than a columnar object width threshold set in advance as the width at the lower end of the columnar object, provided that the vertical coordinates in the vertical coordinate determination step are within the predetermined interval; and a warning notification step of determining that the movement support device is tilted and notifying a warning, provided that the horizontal width of the bounding box in the columnar object width determination step is greater than the columnar object width threshold. 【0018】 The tilt determination method described above can determine the tilt of the mobility support device by sequentially executing the following steps: image acquisition step, bounding box detection step, bounding box coordinate acquisition step, vertical coordinate determination step, and columnar object width determination step. Furthermore, if the tilt determination method determines that the mobility support device is tilted in the columnar object width determination step, it can issue a warning by executing a warning notification step, thereby immediately informing the user of the tilt of the mobility support device. 【0019】 (6) The mobile support device of the present invention provided to solve the above problems is a mobile support device worn on at least one of both shoulders and the neck of a user, and includes a shoulder movement detection sensor that acquires shoulder movement information related to the movement of at least both shoulders of the user, an imaging unit that images the front side of the user, a bounding box detection unit that detects a columnar object extending in the vertical direction in the image captured by the imaging unit as a bounding box having horizontal and vertical sides, a predetermined section setting unit that sets a predetermined section in advance with respect to the vertical direction of the image, a columnar object width threshold setting unit that sets the width at the lower end of the columnar object in advance as a columnar object width threshold, and a control unit. The control unit performs control to determine the inclination based on the horizontal width of the bounding box at the lower end of the bounding box and the columnar object width threshold on the condition that the lower end of the bounding box is included in the predetermined section. 【0020】 With such a configuration, the above-described mobile support device can determine the inclination without separately providing an inclination sensor. As a result, miniaturization of the mobile support device and cost reduction can be expected. Further, the above-described mobile support device can integrally construct an inclination determination system within the mobile support device. Thereby, further miniaturization of the mobile support device can be expected. 【Effects of the Invention】 【0021】 According to the present invention, it is possible to realize an inclination determination system, an inclination determination method, and a mobile support device that can accurately determine the inclination without separately providing a sensor for inclination correction. 【Brief Description of the Drawings】 【0022】 [Figure 1] It is a plan view of a mobile support device constituting an inclination determination system according to an embodiment of the present invention. [Figure 2] It is a block diagram of an inclination determination system according to an embodiment of the present invention. [Figure 3]It is an operation explanatory diagram of an inclination determination system according to an embodiment of the present invention. (a) represents an image of the movement assistance device in a non-inclined state, and (b) is an explanatory diagram showing the state of wearing the movement assistance device in the state of (a) on the user. [Figure 4] It is an operation explanatory diagram of an inclination determination system according to an embodiment of the present invention. (c) represents an image of the movement assistance device in an inclined state, and (d) shows the state of wearing the movement assistance device in the state of (c) on the user. [Figure 5] It is an operation explanatory diagram of an inclination determination system according to an embodiment of the present invention. (a) represents a state where the lower end of the bounding box deviates below a predetermined section and the movement assistance device is inclined, and (b) shows a state where the lower end of the bounding box deviates above a predetermined section and the movement assistance device is inclined. [Figure 6] It is an operation explanatory diagram of an inclination determination system according to an embodiment of the present invention, showing a state where the lower end of the bounding box deviates above a predetermined section. [Figure 7] It is a flowchart showing the operation of the inclination determination system of the present invention. 【Mode for Carrying Out the Invention】 【0023】 Hereinafter, the inclination determination system 1 according to an embodiment of the present invention will be described in detail with reference to the drawings. Note that each figure is schematically shown for easy understanding and may be different from the actual size. Also, in this embodiment, the case where the inclination determination system 1 is mounted on the movement assistance device 10 will be described as an example. 【0024】 As shown in Figure 1, the mobility support device 10 incorporating the tilt determination system 1 of the present invention is, for example, for assisting the movement of a user 2 who has a visual impairment. The mobility support device 10 is also configured as a terminal (hereinafter, the mobility support device 10 will also be referred to as the terminal 10). As shown in Figure 2, the tilt determination system 1 includes an imaging unit 20, a bounding box detection unit 25, a predetermined section setting unit 30, a columnar object width threshold setting unit 35, and a notification unit 45, etc. In addition to the tilt determination system 1, the terminal 10 on which the tilt determination system 1 is mounted also includes a shoulder movement detection sensor 15 and a control unit 40, etc. 【0025】 The terminal 10 (mobility support device 10) is worn and used by user 2 (see Figure 3). The terminal 10 is equipped with a battery 11. The imaging unit 20 and the battery 11 are housed inside the housing that constitutes the mobility support device 10. The terminal 10 should be able to be worn stably by user 2 without causing any burden. In this embodiment, the terminal 10 is a wearable terminal that can be worn on at least one of both shoulders or around the neck. In the example shown in Figure 3(b), the terminal 10 is a wearable terminal that can be worn around the neck and over both shoulders. In this embodiment, the terminal 10 is formed in an inverted U shape and can be worn around the neck of user 2. The terminal 10 also incorporates an imaging unit 20 so that it can capture images of the area in front of user 2 when in use. The terminal 10 is equipped with a battery 11, and the power supplied from the battery 11 can operate the imaging unit 20, the control unit 40, the notification unit 45, and other components that constitute the mobility support device 10. 【0026】 The shoulder movement detection sensor 15 is composed of, for example, a gyro sensor and can integrally detect acceleration in three directions: the X-axis, Y-axis, and Z-axis. Here, the gyro sensor is preferably composed of MEMS (Micro Electro Mechanical Systems), etc. This is expected to allow for miniaturization of the terminal 10. The shoulder movement detection sensor 15 can acquire shoulder movement information 16 related to the movement of at least both shoulders of the user 2. Specifically, by using a gyro sensor, the shoulder movement detection sensor 15 can detect the movement of both shoulders in three directions: the X-axis, Y-axis, and Z-axis, as shoulder movement information 16 (see Figure 2). Here, the shoulder movement information 16 is, for example, acceleration, or the direction of movement of the user 2's shoulder obtained by calculating acceleration. Note that the calculation of acceleration can be performed in at least one of the shoulder movement detection sensor 15 and the control unit 40, which will be described later. Furthermore, in the tilt determination system 1 of the present invention, tilt determination can be performed independently of the shoulder movement detection sensor 15. 【0027】 As shown in Figure 1, the imaging unit 20 is composed of, for example, a camera and can image the area in front of the user 2. In this embodiment, the imaging unit 20 is provided, for example, on one side of the front end of the terminal 10. By imaging the area in front of the user 2, the imaging unit 20 can acquire an image 21 as shown in Figures 3(a) and 4(c). The image 21 in this embodiment includes a columnar object 3 and a building 4. Note that the columnar object 3 includes utility poles, signposts (sign poles), and streetlamp posts (streetlamp poles), but in this embodiment, the case where the columnar object 3 is a utility pole will be described as an example. The image 21 acquired by imaging with the imaging unit 20 (see Figures 3 and 4) is transmitted to the control unit 40 by wire or wireless and used for image analysis (image processing). 【0028】 As shown in Figures 3(a) and 4(c), the bounding box detection unit 25 detects a columnar object 3 extending vertically within the image 21 captured by the imaging unit 20 as a bounding box 26 with sides in the horizontal and vertical directions. Specifically, when the presence of a columnar object 3 is recognized in the image 21 through the image analysis described above, a rectangular bounding box 26 is set for the columnar object 3. Here, the bounding box 26 is set to enclose the entire columnar object 3 in the image 21 in a rectangular shape, regardless of whether the columnar object 3 is inclined or not. The bounding box detection unit 25 detects the set bounding box 26 and obtains the coordinates of each vertex in the bounding box 26. 【0029】 Here, the coordinates of image 21 are set with the top-left vertex of image 21 as the origin (0,0). Therefore, the coordinates of the top-left vertex of the bounding box 26 can be expressed as (x1,y1), the coordinates of the top-right vertex as (x2,y1), the coordinates of the bottom-left vertex as (x1,y2), and the coordinates of the bottom-right vertex as (x2,y2) (where x1, x2, y1, and y2 are arbitrary values, and x2 > x1 and y2 > y1). The coordinates acquired by the bounding box detection unit 25 are transmitted to the control unit 40, which will be described later, and the horizontal width W1 of the bounding box at the lower end of the bounding box 26 is calculated. The horizontal width W1 of the bounding box may also be calculated by the bounding box detection unit 25. 【0030】 The predetermined interval setting unit 30 (see Figure 2) pre-sets predetermined intervals (y3 to y4) in the vertical direction of the image 21. The predetermined interval corresponds to the distance from the imaging unit 20 to the columnar object 3. In other words, the predetermined interval can be said to be the distance from the imaging unit 20 to the columnar object 3 converted into a distance in the two-dimensional image 21. Therefore, for example, y3 is set to a distance of 2m from the imaging unit 20 to the columnar object 3, and y4 is set to a distance of 5m from the imaging unit 20 to the columnar object 3. As will be explained in detail later, predetermined intervals are set because the width of the columnar object 3 changes depending on the difference in distance to the columnar object 3, and therefore it is necessary to define the criteria used for tilt determination, which will be explained later. 【0031】 The columnar object width threshold setting unit 35 (see Figure 2) pre-sets the width of the columnar object 3 at its lower end as the columnar object width threshold W2. Here, the columnar object width threshold W2 is set based on the width of the columnar object 3 when it is located vertically within a predetermined section in the image 21 (when the columnar object 3 is not tilted). In other words, the columnar object width threshold W2 is obtained by converting the width of the columnar object 3 within a predetermined distance (corresponding to a predetermined section) from the imaging unit 20 to the width of the columnar object 3 in the image 21. It is preferable to determine the columnar object width threshold W2 by averaging the widths of multiple columnar objects 3. 【0032】 The control unit 40 is composed of a computer or the like and can perform calculation processing of shoulder movement information 16 and image processing of captured images acquired from the imaging unit 20. The control unit 40 is housed inside the casing that forms the terminal 10. Based on the coordinates of each vertex of the bounding box 26 acquired by the bounding box detection unit 25, the control unit 40 can calculate the horizontal width W1 of the bounding box at the lower end of the bounding box 26. Furthermore, the control unit 40 can determine the inclination of the movement support device 10 based on the horizontal width W1 of the bounding box at the lower end of the bounding box 26 and the columnar object width threshold W2, provided that the lower end of the bounding box 26 is included (located) within a predetermined section. Specifically, the control unit 40 can determine whether or not the columnar object 3 is inclined by comparing the horizontal width W1 of the bounding box within the predetermined section with the columnar object width threshold W2. As a result, the control unit 40 (inclination determination system 1) can determine the inclination of the movement support device 10 and the imaging unit 20. 【0033】 In addition to the tilt determination described above, the control unit 40 can perform control related to mobility assistance in the mobility assistance device 10 based on the shoulder movement information 16. Specifically, the control unit 40 can recognize the direction that user 2 is facing (direction of travel) based on the shoulder movement information 16. Furthermore, the control unit 40 can detect obstacles around user 2 (e.g., columnar objects 3, cars, people, etc.) based on the image 21 acquired by the imaging unit 20. Note that the control unit 40 may be divided and provided for each of multiple functions. For example, control related to mobility assistance and control related to tilt determination may be provided separately. 【0034】 In this embodiment, the notification unit 45 is configured as a speaker (including earphones) and can provide voice notifications (such as alarms) regarding warnings that the mobility support device 10 is tilted or that there are obstacles. In other words, the notification unit 45 assists the user 2's movement by notifying the user 2 of the tilt determination results and the presence or absence of obstacles made by the control unit 40. 【0035】 The above describes the configuration of the tilt determination system 1 according to one embodiment of the present invention, and the mobility support device 10 of the present invention equipped with the tilt determination system 1. Next, with reference to Figures 3 to 6, a detailed description of one embodiment of the operation of the tilt determination system 1 will be given. Note that in Figures 3(a) and 4(c), for the sake of explanation, only the utility pole on the near side will be described, but in reality, the same process will be applied to the utility pole on the far side as well. 【0036】 As shown in Figure 3(b), if the mobility support device 10 (terminal 10) attached around the user 2's neck is properly mounted without tilting, then, as shown in Figure 3(a), image 21 is captured without tilting. Once image 21 is captured, the control unit 40 performs image analysis to determine whether or not there is a columnar object 3 (a utility pole in this embodiment) in the image 21. If a columnar object 3 is detected in image 21, a bounding box 26 is set for the columnar object 3. 【0037】 When the bounding box 26 is set, the bounding box detection unit 25 detects the bounding box 26 and obtains the coordinates of each vertex of the bounding box 26. In the example shown in Figure 3(a), it is determined that the horizontal width W1 of the bounding box is within a predetermined vertical interval (y3 to y4). Next, it is determined whether the horizontal width W1 of the bounding box is greater than the columnar object width threshold W2. In Figure 3(a), since the horizontal width W1 of the bounding box is less than or equal to the columnar object width threshold W2, the control unit 40 determines that the movement support device 10 (including the imaging unit 20) is not tilted. In this case, no warning is issued because the movement support device 10 is correctly mounted. 【0038】 Next, as shown in Figure 4(d), if the mobility support device 10 (terminal 10) is mounted at an angle, the image 21 is captured in an inclined state, as shown in Figure 4(c). Once the image 21 is captured, the control unit 40 performs image analysis to determine whether or not there is a columnar object 3 (a utility pole in this embodiment) in the image 21. If a columnar object 3 is detected in the image 21, a bounding box 26 is set for the columnar object 3. 【0039】 When the bounding box 26 is set, the bounding box detection unit 25 detects the bounding box 26 and obtains the coordinates of each vertex of the bounding box 26. In the example shown in Figure 4(c), it is determined that the horizontal width W1 of the bounding box is within a predetermined vertical interval (y3 to y4). Next, it is determined whether the horizontal width W1 of the bounding box is greater than the columnar object width threshold W2. In Figure 4(c), the horizontal width W1 of the bounding box is greater than the columnar object width threshold W2, so the control unit 40 determines that the movement support device 10 (including the imaging unit 20) is tilted. In this case, the notification unit 45 issues a warning (notification) because the movement support device 10 is mounted at an angle. 【0040】 Figure 5 illustrates a case where the lower end of the bounding box 26 in image 21 is outside the predetermined vertical section (y3~y4) and the movement support device 10 is tilted. Note that in Figures 5 and 6, the coordinates of the upper left vertex of image 21 are set as the origin (0,0), the right direction of image 21 is the positive x-axis direction, and the downward direction of image 21 is the positive y-axis direction. 【0041】 As shown in Figure 5(a), if the lower end of the bounding box 26 is greater than the vertical y4 (the distance to the columnar object 3 is shorter than the predetermined distance), the horizontal width W1 of the bounding box will be greater than the width of the columnar object 3 at the predetermined distance, regardless of whether the movement support device 10 is tilted or not, and therefore it must be excluded from the tilt determination. 【0042】 Furthermore, as shown in Figure 5(b), when the mobile support device 10 is tilted, if the lower end of the bounding box 26 is smaller than y3 in the vertical direction (the distance to the columnar object 3 is greater than a predetermined distance), the horizontal width W1 of the bounding box may be smaller than the columnar object width threshold W2, even though the mobile support device 10 is tilted. Therefore, in the tilt determination system 1 of the present invention, as described above, a predetermined vertical interval (y3 to y4) is defined as a threshold, and if the horizontal width W1 of the bounding box falls outside the predetermined interval, it is excluded from the tilt determination. As a result, the tilt determination system 1 can accurately determine the tilt of the mobile support device 10. The predetermined interval (y3 to y4) can be set to various values ​​depending on the distance between the imaging unit 20 and the columnar object 3, and the relationship with the width of the columnar object 3 in the image 21. 【0043】 Figure 6 illustrates the case in image 21 where the lower end of the bounding box 26 is outside the predetermined vertical section (y3~y4) and the mobility support device 10 is not tilted. 【0044】 As shown in Figure 6, if the lower end of the bounding box 26 is smaller than y3 in the vertical direction (the distance to the columnar object 3 is greater than a predetermined distance), the horizontal width W1 of the bounding box will also be smaller than the width of the columnar object 3 at the predetermined distance. Therefore, there is a concern that the horizontal width W1 of the bounding box may be smaller than the columnar object width threshold W2. To address this, the tilt determination system 1 of the present invention excludes bounding boxes 26 whose lower end is outside a predetermined vertical section (y3 to y4) from the tilt determination. In this way, the tilt determination system 1 described above can exclude bounding boxes 26 whose lower end is outside a predetermined section (y3 to y4) from the tilt determination by determining whether or not the lower end of the bounding box 26 is within the predetermined section (y3 to y4). Therefore, the tilt determination system 1 described above can accurately determine the tilt of the movement support device 10 and the imaging unit 20 without providing a separate tilt sensor. 【0045】 Next, the operation of the tilt determination system 1 (tilt determination method) will be explained with reference to the flowchart in Figure 7. 【0046】 As shown in Figure 7, when the mobility support device 10 (tilt determination system 1) is activated, the imaging unit 20 starts imaging the area in front of the user 2, and an image 21 is acquired (step S10: image acquisition step). Subsequently, the image 21 is transmitted to the control unit 40, and the control unit 40 performs image analysis (step S11: image analysis step). 【0047】 In step S11, when image analysis is performed, the presence or absence of columnar objects 3 in the image 21 is detected, and if columnar objects 3 are detected, a bounding box 26 is set for the columnar objects 3. In addition, the bounding box detection unit 25 detects a bounding box 26 in the image 21 with the horizontal and vertical directions as sides for columnar objects 3 that extend vertically (step S12: bounding box detection step). 【0048】 Furthermore, in step S12 (bounding box detection step), a bounding box coordinate acquisition step is performed to obtain the vertical coordinates of the lower end of the bounding box 26, provided that the bounding box 26 is detected. 【0049】 In step S12, once the detection and coordinate acquisition of the bounding box 26 is complete, a determination is made as to whether the distance to the columnar object 3 is within a predetermined interval (step S13: vertical coordinate determination step). Specifically, as shown in Figures 3 to 6, it is determined whether the vertical coordinates at the lower end of the bounding box 26 are within a predetermined interval (y3 to y4) set in advance with respect to the vertical direction of the image 21. In other words, it is determined whether the horizontal width W1 of the bounding box at the lower end of the bounding box 26 is within a predetermined interval (y3 to y4). 【0050】 In step S13, if the bounding box horizontal width W1 is within a predetermined range, it is determined whether the bounding box horizontal width W1 is greater than the columnar object width threshold W2, which has been set in advance as the width at the lower end of the columnar object 3 (step S14: columnar object width determination step). 【0051】 On the other hand, if it is determined in step S13 that the bounding box horizontal width W1 is not within a predetermined range (y3~y4), it is determined that there is no tilt in the movement support device 10 (imaging unit 20) (step S16), and the process returns to step S10. 【0052】 In step S14 (columnar object width determination step), if the bounding box horizontal width W1 is greater than the columnar object width threshold W2, it is determined that the movement support device 10 (imaging unit 20) is tilted, and a warning is issued by the notification unit 45 (step S15: warning notification step). Once the processing in step S15 is complete, the process returns to the processing in step S10. After the processing in step S15 is completed, the process can also be terminated. 【0053】 The above describes one embodiment of the tilt determination system 1, tilt determination method, and mobility support device 10 employing the tilt determination system 1 of the present invention. Next, the effects and advantages realized by the tilt determination system 1, tilt determination method, and mobility support device 10 of the present invention will be described below. 【0054】 The tilt determination system 1 described above has the following characteristic configurations (1) to (4). Therefore, the tilt determination system 1 can achieve unique effects that cannot be achieved with conventional technology. 【0055】 (1) The tilt determination system 1 of this embodiment is a tilt determination system 1 for a mobility support device 10 that is attached to at least one of the shoulders and neck of a user 2, and comprises an imaging unit 20 that images the front side of the user 2, a bounding box detection unit 25 that detects a columnar object 3 extending vertically in the image 21 captured by the imaging unit 20 as a bounding box 26 with sides in the horizontal and vertical directions, a predetermined section setting unit 30 that sets a predetermined section (y3~y4) in the vertical direction of the image 21 in advance, and a columnar object width threshold setting unit 35 that sets the width at the lower end of the columnar object 3 as a columnar object width threshold W2 in advance, and is characterized in that the tilt of the mobility support device 10 is determined based on the bounding box horizontal width W1 and the columnar object width threshold W2 at the lower end of the bounding box 26, provided that the lower end of the bounding box 26 is included in the predetermined section (y3~y4). 【0056】 The tilt determination system 1 described above can detect columnar objects 3 in the image 21 captured by the imaging unit 20 as bounding boxes 26 with sides in the horizontal and vertical directions. In this invention, the bounding box 26 detects the entire columnar object 3 as a rectangular frame, regardless of the tilt of the columnar object 3 in the image 21. The width of the lower end of the bounding box 26 is obtained as the bounding box horizontal width W1. Furthermore, the tilt determination system 1 described above can determine the tilt of the mobile support device 10 based on the bounding box horizontal width W1 at the lower end of the bounding box 26 and the columnar object width threshold W2, provided that the lower end of the detected bounding box 26 is included (located) within a predetermined section in the vertical direction of the image. In this way, the tilt determination system 1 described above can determine the tilt of the mobile support device 10 without using a tilt sensor, so it can accurately determine the tilt without misjudging even if the tilt of the mobile support device 10 is large. 【0057】 Therefore, the tilt determination system 1 described above can recognize whether the columnar object 3 is tilted or not by comparing the horizontal width W1 of the bounding box within a predetermined section (within a predetermined distance) with the columnar object width threshold W2. As a result, the tilt determination system 1 described above can determine the tilt of the movement support device 10 based on the tilt state of the columnar object 3. This allows the tilt determination system 1 described above to accurately determine the tilt without providing a separate sensor for tilt correction, thereby suppressing cost increases. 【0058】 (2) The tilt determination system 1 described above determines that the movement support device 10 is tilted on the condition that the horizontal width W1 of the bounding box is greater than the columnar object width threshold W2. 【0059】 The tilt detection system 1 described above can determine if the columnar object 3 is tilted by configuring it as shown in (2) above. Furthermore, since the tilt detection system 1 can perform tilt determination based on the bounding box 26 detected in the image 21, there is no need to provide a separate tilt sensor. As a result, the tilt detection system 1 described above can suppress cost increases. 【0060】 (3) The tilt determination system 1 described above assumes that the columnar object 3 is one of the following: a utility pole, a road sign post, or a streetlamp post. 【0061】 The tilt determination system 1 described above, by having the configuration described in (3) above, can determine the tilt of the mobility support device 10, for example, by targeting utility poles, road sign posts, and street light posts (collectively referred to as utility poles, etc.) that may be obstacles on the sidewalk. In other words, since the tilt of the mobility support device 10 can be determined by targeting utility poles, etc. located at the user 2's destination, the tilt determination system 1 described above can quickly recognize the tilt of the mobility support device 10. 【0062】 (4) The tilt determination system 1 described above has a notification unit 45, and a warning is issued by the notification unit 45 when the bounding box horizontal width W1 is greater than the columnar object width threshold W2. 【0063】 The tilt detection system 1 described above, with the configuration as shown in (4) above, can notify a warning via the notification unit 45 when the mobility support device 10 is tilted. As a result, the tilt detection system 1 described above can accurately notify the user 2 of the tilt of the mobility support device 10, even if the mobility support device 10 tilts against the user 2's will. 【0064】 The tilt determination method described above has the following characteristic configuration (5). Therefore, the tilt determination method can produce unique effects that cannot be achieved with conventional techniques. 【0065】 (5) The tilt determination method of this embodiment is a tilt determination method for a mobility support device 10 attached to at least one of the shoulders and neck of a user 2, and comprises: an image acquisition step of acquiring an image 21 by imaging the front side of the user 2 with an imaging unit 20; a bounding box detection step of detecting a columnar object 3 extending vertically in the image 21 as a bounding box 26 with sides in the horizontal and vertical directions; a bounding box coordinate acquisition step of acquiring the vertical coordinates of the lower end of the bounding box 26, provided that the bounding box 26 is detected in the bounding box detection step; and the vertical coordinates relative to the vertical direction of the image 21 The method is characterized by performing the following steps: a vertical coordinate determination step which determines whether the coordinates are within a predetermined interval (y3 to y4); a columnar object width determination step which determines whether the horizontal width W1 of the bounding box at the lower end of the bounding box 26 is greater than a columnar object width threshold W2 which has been set in advance as the width at the lower end of the columnar object 3, provided that the vertical coordinates are within the predetermined interval (y3 to y4) as determined in the vertical coordinate determination step; and a warning notification step which determines that the movement support device 10 is tilted and notifies a warning, provided that the horizontal width W1 of the bounding box is greater than the columnar object width threshold W2 as determined in the columnar object width determination step. 【0066】 The tilt determination method described above can determine the tilt of the mobility support device 10 by sequentially executing the image acquisition step, bounding box detection step, bounding box coordinate acquisition step, vertical coordinate determination step, and columnar object width determination step. Furthermore, if the mobility support device 10 is determined to be tilted in the columnar object width determination step, the above tilt determination method can issue a warning by executing the warning notification step, thereby immediately informing the user 2 of the tilt of the mobility support device 10. 【0067】 Furthermore, the mobility support device 10 described above has the following characteristic configuration (6). Therefore, the mobility support device 10 can achieve unique effects that cannot be achieved with conventional technology. 【0068】 (6) The mobility support device 10 of this embodiment is a mobility support device 10 that is attached to at least one of the shoulders and neck of the user 2, and comprises a shoulder movement detection sensor 15 that acquires shoulder movement information 16 relating to the movement of at least both shoulders of the user 2, an imaging unit 20 that images the front side of the user 2, a bounding box detection unit 25 that detects a columnar object 3 extending vertically in the image 21 captured by the imaging unit 20 as a bounding box 26 with sides in the horizontal and vertical directions, a predetermined section setting unit 30 that sets a predetermined section (y3~y4) in the vertical direction of the image 21 in advance, a columnar object width threshold setting unit 35 that sets the width at the lower end of the columnar object 3 in advance as a columnar object width threshold W2, and a control unit 40, wherein the control unit 40 performs control to determine the inclination based on the bounding box horizontal width W1 and the columnar object width threshold W2 at the lower end of the bounding box 26, provided that the lower end of the bounding box 26 is included in the predetermined section. 【0069】 The mobility support device 10 described above, by having the configuration described in (6) above, can perform tilt determination without providing a separate tilt sensor. This is expected to allow for miniaturization and cost reduction of the mobility support device 10. Furthermore, the mobility support device 10 described above can integrate the tilt determination system 1 within the mobility support device itself. This is expected to allow for even further miniaturization of the mobility support device 10. 【0070】 The above describes the effects of the tilt determination system 1, tilt determination method, and mobility support device 10 according to one embodiment of the present invention. However, the tilt determination system 1, tilt determination method, and mobility support device 10 of the present invention are not limited to the embodiments described above, and various modifications can be made. 【0071】 In this embodiment, the mobility support device 10 (terminal 10) is attached to at least one of the user 2's shoulders and neck, but the present invention is not limited to this and can be applied to various types of mobility support devices 10. For example, the mobility support device 10 may be attached to the user 2's head, face, or torso. Furthermore, the imaging unit 20 can be any type of camera or image sensor capable of acquiring images 21. In addition, the imaging range captured by the imaging unit 20 can be set to various directions such as in front of, to the side of, or behind the user 2, depending on the characteristics and usage of the mobility support device 10. 【0072】 Furthermore, in this embodiment, the bounding box 26 is set with horizontal and vertical edges, but the direction of the edges forming the bounding box 26 may be changed depending on the shape and size of the columnar object 3 to be detected. Also, in this embodiment, the bounding box 26 is set for the detected columnar object 3 after it has been detected in the image 21, but the present invention is not limited to this. Various image analysis means can be used for setting the bounding box 26 and for detecting the columnar object 3. In addition, the predetermined vertical section (y3~y4) set in the image 21 can be set to various values ​​depending on the characteristics of the imaging unit 20 and the width of the columnar object 3. In addition, the columnar object width threshold W2 can be set to various values ​​depending on the columnar object 3 to be subjected to tilt determination. Furthermore, in this embodiment, the bounding box 26 is set for a single columnar object 3, but multiple bounding boxes 26 may be set, or each of multiple types of columnar objects 3 may be set. If this is the case, you can set the bounding box horizontal width W1 and the columnar width threshold W2 for each columnar object 3. 【0073】 Furthermore, in this embodiment, the horizontal width W1 of the bounding box is set at the lower end of the columnar object 3, but the horizontal width W1 of the bounding box can be changed to any location other than the lower end of the columnar object 3 depending on the state of the acquired image 21. Also, in this embodiment, the case where the columnar object 3 is a utility pole is illustrated, but the present invention is not limited to this and can be used for various columnar objects such as road sign posts and street light posts. 【0074】 In this embodiment, the notification unit 45 is exemplified as a speaker or earphone that provides notifications by voice. However, the notification unit 45 is not limited to those that provide notifications by voice; various types of devices can be used as long as they can provide notifications to the user 2 who is the target of mobility assistance. Furthermore, in this embodiment, the tilt determination system 1 and mobility assistance device 10 are exemplified as being for visually impaired persons. However, the tilt determination system 1 and mobility assistance device 10 of the present invention can be used for various types of persons, not just visually impaired persons. For example, the tilt determination system 1 and mobility assistance device 10 can be used for mobility assistance for the elderly, etc. In addition, the notification by the notification unit 45 is not limited to notifying when the horizontal width W1 of the bounding box is greater than the columnar object width threshold W2, but can also be configured to provide various notifications related to mobility assistance in a timely manner. 【0075】 The above describes various embodiments and modifications of the tilt determination system 1, tilt determination method, and mobility support device 10 according to the present invention. However, the present invention is not limited to those exemplified in the embodiments and modifications described above, and it will be readily apparent to those skilled in the art that other embodiments may exist in the spirit and teachings thereof, without departing from the scope of the claims. [Industrial applicability] 【0076】 This invention can be used for tilt detection of various cameras and the like. Furthermore, this invention is suitably applicable to all types of mobility assistance devices that assist users in moving. [Explanation of symbols] 【0077】 1: Tilt detection system 10: Mobility support device (terminal) 11: Battery 15: Shoulder movement detection sensor 20: Imaging Department 21: Image 25: Bounding box detection unit 26: Bounding Box 30: Predetermined interval setting unit 35: Columnar object width threshold setting unit 40: Control Unit 45:Notification section W1: Bounding box horizontal width W2: Columnar object width threshold

Claims

[Claim 1] A tilt determination system for a mobility assistance device attached to at least one of the user's shoulders and neck, An imaging unit that images the area in front of the user, A bounding box detection unit detects columnar objects extending vertically within the image captured by the imaging unit as bounding boxes with sides in the horizontal and vertical directions, A predetermined section setting unit sets a predetermined section in advance in the vertical direction of the aforementioned image, A columnar object width threshold setting unit sets the width at the lower end of the columnar object in advance as the columnar object width threshold, Equipped with, An inclination determination system characterized by determining the inclination of the movement support device based on the horizontal width of the bounding box at the lower end of the bounding box and the columnar object width threshold, provided that the lower end of the bounding box is included within the predetermined section. [Claim 2] The tilt determination system according to claim 1, characterized in that it determines that the movement support device is tilted on the condition that the horizontal width of the bounding box is greater than the columnar object width threshold. [Claim 3] The tilt determination system according to claim 1 or 2, characterized in that the columnar object is one of a utility pole, a road sign post, and a streetlamp post.

Images