Movement support device

Abstract

To realize a movement support device capable of restricting a center line arranged on a road from being erroneously determined as a braille block.SOLUTION: A movement support device 1 includes: a target detection section 40a for detecting a target in an image 21 based on information obtained from image data of the image 21 photographed by the photographing section 20; a block candidate determination section 40c for determining whether the target corresponding to a braille block candidate TC exists; a crosswalk determination section 40b for determining whether the target corresponding to a crosswalk 24 exists; an inclination calculation section 40d for calculating an inclination ΔT of the braille block candidate TC and an inclination ΔL of a white line WL on the crosswalk 24; and a braille block extraction section 40e for extracting a real braille block T. The braille block extraction section 40e does not extract the braille block candidate TC as the real braille block T when a difference between the inclination ΔT of the braille block candidate TC and the inclination of ΔL of the white line WL is less than a predetermined value.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a mobility assistance device. [Background technology]

[0002] Conventionally, some braille blocks laid on roads are coded by assigning different colors to each dot-like protrusion and arranging them in a predetermined pattern. A white cane equipped with a voice guidance device is known that captures images of coded braille blocks with a camera and processes the acquired image data to provide audio guidance corresponding to the coded braille blocks (Patent Document 1). Another known technology is one that accurately recognizes white lines on a crosswalk from an image captured by a camera (Patent Document 2). In this case, even if a part of a white line forming a crosswalk is unclear, if that part is located in the area between a line connecting one end of other white lines and a line connecting the other ends of other white lines, it is determined to be a white line forming the crosswalk. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Registered Utility Model No. 3208458 [Patent Document 2] Japanese Patent Application Publication No. 2023-27471 Summary of the Invention [Problem to be solved by the invention]

[0004] Braille blocks include warning blocks with dot-like protrusions arranged in a grid pattern and guide blocks with multiple linear protrusions, both of which are colored yellow. Road center lines are colored orange, and some have linear protrusions that sound when a vehicle crosses the center line. In the above-mentioned Patent Documents 1 and 2, Braille blocks and white lines on crosswalks are recognized by the color and shape (pattern) of images captured by a camera. Therefore, when using these technologies to provide mobility assistance, there is a risk that center lines with similar shapes and colors may be mistakenly identified as Braille blocks.

[0005] Therefore, an object of the present invention is to realize a mobility assistance device that can reduce the number of times that center lines on roads are mistakenly determined to be Braille blocks. [Means for solving the problem]

[0006] (1) A mobility support device of the present invention, which is provided to solve the above-mentioned problems, is a mobility support device worn on at least one of the shoulders and the neck of a user, and includes an imaging unit that images a front side of the user, a target detection unit that detects targets in an image captured by the imaging unit based on at least brightness information and shape information obtained from image data of the image, a first determination unit that determines whether or not any of the targets detected by the target detection unit correspond to a braille block candidate, a second determination unit that determines whether or not any of the targets detected by the target detection unit correspond to a pedestrian crossing, and and a braille block extraction unit that extracts actual braille blocks from the braille block candidates, when the first determination unit determines that the braille block candidate exists and the second determination unit determines that the crosswalk exists. The braille block extraction unit is characterized in that, on condition that the difference between the slope of the braille block candidate calculated by the slope calculation unit and the slope of the white line is less than a predetermined value and the braille block candidate is located longitudinally outward of the white line, the braille block extraction unit does not extract the braille block candidate as the actual braille block.

[0007] The mobility support device described above detects targets in an image captured by an imaging unit based on at least brightness information and shape information obtained from the image data. That is, the mobility support device of the present invention detects targets based on brightness (color) and shape (pattern shape or uneven shape) from the image captured by the imaging unit. In this case, as with the technologies described in Patent Documents 1 and 2, there is a risk of erroneously determining a center line as a Braille block. However, center lines are generally located outside the longitudinal direction of the white lines of a crosswalk and extend in a direction parallel to the white lines. Therefore, the mobility support device calculates the slope of the white lines and the slope of the Braille block candidate using a slope calculation unit. If the difference between these slopes is less than a predetermined value and the Braille block candidate is located outside the longitudinal direction of the white lines, the device does not extract the Braille block candidate as a solid Braille block. In this way, the mobility support device of the present invention can prevent the Braille block candidate from being extracted as a solid Braille block even if the center line is included in the Braille block candidate. As a result, the present invention can realize a mobility assistance device that can reduce the number of times that center lines are mistakenly determined to be braille blocks.

[0008] (2) Another mobility support device of the present invention, which is provided to solve the above-mentioned problems, is a mobility support device worn on at least one of the shoulders and the neck of a user, and includes an imaging unit that images a front side of the user, a target detection unit that detects targets in the image based on at least brightness information and shape information obtained from image data of the image captured by the imaging unit, a first determination unit that determines whether or not there are targets that correspond to Braille block candidates among the targets detected by the target detection unit, a second determination unit that determines whether or not there are targets that correspond to pedestrian crossings among the targets detected by the target detection unit, and when the second determination unit determines that there is a target that corresponds to the pedestrian crossing, and a braille block extraction unit that extracts actual braille blocks from the braille block candidates. The predetermined area setting unit sets the predetermined area to be the area between a first line formed by one long side of the white line located closest to one end and an extension of the long side of that one end in a direction intersecting the longitudinal direction of the plurality of white lines that make up the crosswalk, and a second line formed by the other long side of the white line located closest to the other end and an extension of the long side of that other end. The braille block extraction unit does not extract, from the braille block candidates determined by the first determination unit, any that are within the predetermined area as actual braille blocks.

[0009] The mobility support device described above detects landmarks in an image captured by an imaging unit based on at least brightness information and shape information obtained from the image data. That is, the mobility support device of the present invention detects landmarks based on brightness (color) and shape (pattern) from the image captured by the imaging unit. In this case, as with the techniques described in Patent Documents 1 and 2, there is a risk of erroneously identifying a center line as a Braille block. However, center lines are located on the same roadway as crosswalks, while Braille blocks are located on sidewalks. Therefore, by using the configuration described in (2) above, the mobility support device of the present invention determines the extent of the roadway using the extent of the crosswalk. Specifically, the area between the first and second lines (predetermined area) can be considered to be an approximate extent of the roadway. Based on this knowledge, the mobility support device of the present invention prevents Braille block candidates located in the predetermined area from being extracted as actual Braille blocks. In this way, the mobility support device of the present invention can prevent Braille block candidates from being extracted as actual Braille blocks, even if the Braille block candidates include a center line. As a result, the present invention can realize a mobility assistance device that can reduce the number of times that center lines are mistakenly determined to be braille blocks.

[0010] (3) Furthermore, in the mobility assistance device of (2) above, the braille block extraction unit may be configured not to extract as the actual braille block those braille block candidates determined by the first determination unit that are within the specified area, are located outside the area surrounding the crosswalk by connecting the four points at both ends of the long side on one end side and the four points at both ends of the long side on the other end side with straight lines, and are located outside the longitudinal direction of the white line.

[0011] The center line is located adjacent to the crosswalk within the roadway. Based on this knowledge, the mobility assistance device according to (3) above sets an area surrounding the crosswalk by connecting the four points with straight lines, and does not extract block candidates located outside this area and outward in the longitudinal direction of the white lines as actual braille blocks. Even in this way, the mobility assistance device of the present invention can still exclude those that correspond to the center line from the braille block candidates. As a result, the present invention can realize a mobility assistance device that can reduce the number of times that a center line is mistakenly determined to be a braille block. [Effects of the Invention]

[0012] According to the present invention, it is possible to realize a mobility assistance device that can reduce the number of times that center lines marked on roads are mistakenly determined to be Braille blocks. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a plan view of a movement assistance device according to a first embodiment of the present invention. [Figure 2] 1 is a block diagram of a mobility assistance device according to a first embodiment of the present invention. [Figure 3] 4 is an example of an image captured by a camera of the mobility assistance device according to the first embodiment of the present invention. [Figure 4] These are diagrams to explain tactile paving blocks and center lines. (a) is a plan view of warning blocks among the tactile paving blocks, (b) is a plan view of guide blocks among the tactile paving blocks, and (c) is a plan view of part of the center line marked on the road. [Figure 5] 10 is a flowchart showing an example of a braille block extraction process according to the present invention. [Figure 6] FIG. 10 is a block diagram of a mobility assistance device according to a second embodiment of the present invention. [Figure 7] 10 is an example of an image captured by a camera of a mobility assistance device according to a second embodiment of the present invention. [Figure 8] FIG. 10 is a block diagram of a travel assistance device according to a modified example of the second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0014] First Embodiment Hereinafter, a mobility support device 1 according to a first embodiment of the present invention will be described in detail with reference to the drawings. Note that each drawing is a schematic representation for ease of understanding, and may differ in actual size.

[0015] As shown in Fig. 1, a mobility support device 1 of the present invention is intended to support the mobility of, for example, a visually impaired user. The mobility support device 1 is configured as a terminal (hereinafter, the mobility support device 1 may also be referred to as a terminal 1). As shown in Fig. 2, the mobility support device 1 includes an imaging unit 20, a control unit 40, a notification unit 45, etc.

[0016] The terminal 1 (mobility support device 1) is worn by a user. The terminal 1 includes a battery 11. The imaging unit 20 and the battery 11 are housed inside a housing that constitutes the mobility support device 1. The terminal 1 should be able to be worn stably without causing any strain to the user. In this embodiment, the terminal 1 is a wearable terminal that can be worn on at least one of both shoulders or around the neck. In the example shown in FIG. 1, the terminal 1 is a wearable terminal that can be worn around the neck and over both shoulders. In this embodiment, the terminal 1 is formed in an inverted U-shape and can be worn around the user's neck. The terminal 1 also includes an imaging unit 20 that can capture an image of an area in front of the user when in use. The terminal 1 includes the battery 11, and the power supplied from the battery 11 can operate the imaging unit 20, the control unit 40, the notification unit 45, and other components that constitute the mobility support device 1.

[0017] As shown in FIG. 1, the imaging unit 20 is, for example, a camera and can capture an image of the area ahead of the user. In this embodiment, the imaging unit 20 is provided, for example, on one side of the front end of the terminal 1. The imaging unit 20 captures an image of the area ahead of the user, thereby acquiring an image 21 as shown in FIG. 3 (an image 22 in a second embodiment, described later). The image 21 in this embodiment includes a crosswalk 24 having multiple white lines WL1 to WL7, a center line 25 in the center of the roadway, and other images. The image 21 (see FIG. 3) captured by the imaging unit 20 is transmitted as image data to the control unit 40 via a wired or wireless connection and subjected to image analysis (image processing). The image data includes position information (position (coordinate) information within the image) and brightness information for each pixel in the image. In the following, the multiple white lines WL1 to WL7 constituting the crosswalk 24 will be collectively referred to as white lines WL unless otherwise specified. On the other hand, when distinguishing a specific white line, it is identified by one of the white lines WL1 to WL7.

[0018] The control unit 40 is configured with a microcomputer or the like having a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), etc., and is capable of performing image processing of the captured image acquired from the imaging unit 20 and control related to mobility assistance in the mobility assistance device 1. The control unit 40 includes a landmark detection unit 40a, a crosswalk determination unit 40b, a braille block candidate determination unit 40c, a slope calculation unit 40d, a braille block extraction unit 40e, and a mobility assistance control unit 40f. The landmark detection unit 40a detects landmarks in the image 21 (see FIG. 3). The crosswalk determination unit 40b determines whether a crosswalk 24 (see FIG. 3) is present in the image 21. The braille block candidate determination unit 40c determines whether a braille block candidate TC (see FIG. 3) is present in the image 21. The slope calculation unit 40d calculates the slope of the white lines WL that make up the crosswalk 24 and the slope of the braille block candidate TC. The tactile paving block extraction unit 40e extracts actual tactile paving blocks T from among the tactile paving block candidates TC. The travel assistance control unit 40f executes travel assistance control in the travel assistance device 1. These will be described later. Note that each of the units 40a to 40f in the control unit 40 may be configured as an individual computer.

[0019] In this embodiment, the notification unit 45 is configured as a speaker (including earphones) and can guide the user to the braille blocks T and issue notifications (such as issuing an alarm) regarding warnings about obstacles by voice. In other words, the notification unit 45 supports the user's movement by guiding the user to the braille blocks T extracted by the braille block extraction unit 40e and notifying the user of the presence or absence of obstacles.

[0020] The above is the configuration of the movement support device 1 according to the first embodiment of the present invention, and next, one embodiment of the operation of the movement support device 1 will be described in detail with reference to Figures 3 and 4. In addition to the operation, each component (each of the units 40a to 40f) of the control unit 40 will also be described.

[0021] For example, when an image 21 shown in FIG. 3 is captured by the imaging unit 20 of the mobility support device 1 (terminal 1) worn around the neck of a user, image data corresponding to the image 21 is transmitted to the control unit 40. This image data includes position information (coordinates) and brightness information for each pixel. Here, the position information is, for example, information on x and y coordinates, with the horizontal direction in FIG. 3 being the x direction, the vertical direction being the y direction, and the origin being the upper left pixel.

[0022] The target object detection unit 40a of the control unit 40 detects targets in the image 21 by performing image processing on the image data transmitted from the imaging unit 20. For example, targets include white lines, braille blocks, center lines, cars, pedestrians, and utility poles. These targets can be detected by identifying their shape (linear shape or uneven shape), brightness (color), and the like based on the position information and brightness information of each pixel included in the image data.

[0023] The crosswalk determination unit 40b of the control unit 40 determines whether a crosswalk 24 is present among the targets detected by the target detection unit 40a. For example, the crosswalk determination unit 40b can determine the presence or absence of a crosswalk by pattern matching in image processing. However, the method for determining the presence or absence of a crosswalk 24 is not limited to this. The white lines WL1 to WL7 that make up the crosswalk 24 are strips of white lines of approximately the same shape arranged at equal intervals. Therefore, for example, when two or more parallel (or approximately parallel) white lines of the same shape (or approximately the same shape) are detected, the crosswalk determination unit 40b may determine that a crosswalk 24 is present based on this group of white lines. Alternatively, the crosswalk determination unit 40b may determine that a crosswalk 24 is present when a specific number of three or more parallel (or approximately parallel) white lines of the same shape (or approximately the same shape) are detected. In this way, the crosswalk determination unit 40b can determine that a crosswalk is present even when only a portion of the crosswalk is captured in the image.

[0024] The Braille block candidate determination unit 40c of the control unit 40 determines whether or not there is a Braille block candidate TC among the targets detected by the target detection unit 40a. Here, the Braille block T will be described with reference to Figures 4(a) and 4(b). As shown in Figures 4(a) and 4(b), the Braille block T includes warning blocks Ta (see Figure 4(a)) that indicate the location of dangerous areas and facilities to be guided, and guidance blocks Tb (Figure 4(b)) that indicate the direction of travel.

[0025] As shown in FIG. 4(a), the warning blocks Ta have dot-like protrusions Ta1 arranged in a grid pattern. On the other hand, as shown in FIG. 4(b), the guide blocks Tb have a plurality of linear protrusions Tb1 arranged in a row. The guide blocks Tb in FIG. 4(b) have four linear protrusions Tb1 extending parallel to the vertical direction of the page, and are arranged at equal intervals in the horizontal direction of the page. Furthermore, both the warning blocks Ta and the guide blocks Tb are formed in yellow. Here, when there is no need to distinguish between the warning blocks Ta and the guide blocks Tb, they will be collectively referred to as Braille blocks T.

[0026] The braille block candidate determination unit 40c can determine the presence or absence of braille block candidates TC by pattern matching in image processing, etc., based on the shape and color of such braille blocks T. Here, a braille block candidate TC is an object that can be determined to have a high probability of being a braille block, and as will be described later, an actual braille block T is extracted from the braille block candidates TC by the braille block extraction unit 40e.

[0027] In addition to the actual braille blocks T, a center line 25 is also a landmark that can be determined as a braille block candidate TC. As shown in FIG. 4(c), the center line 25 has a plurality of linear protrusions 25a formed within the line 25. These linear protrusions 25a generate a sound when a passing vehicle crosses the center line 25. Each linear protrusion 25a extends in a direction substantially perpendicular to the length of the center line 25 or in a direction tilted by a predetermined amount relative to the perpendicular direction, and they are arranged parallel to each other at equal intervals along the length of the center line 25. The center line 25 is colored orange.

[0028] As described above, since the center line 25 has a similar color and shape to the guide block Tb, there is a possibility that they will be determined to be the same when the target is detected based on brightness or shape. Therefore, if the center line 25 is identified as the braille block T, there is a risk that the user will be guided to the center line 25 by the mobility support control described below. Therefore, the mobility support device 1 of this embodiment focuses on the relationship between the white lines WL constituting the crosswalk 24 and the center line 25, and prevents the center line 25 determined as the braille block candidate TC from being extracted as the actual braille block T. The method for doing this will be specifically described below.

[0029] For example, FIG. 3 shows an example of an image captured by the imaging unit 20 when a user wearing the mobility assistance device 1 faces leftward relative to the traveling direction while walking on the sidewalk on the right side of the page.

[0030] This image 21 shows a crosswalk 24 made up of multiple white lines WL and a center line 25. Therefore, these targets are detected by the target detection unit 40a. When the target detection unit 40a detects these targets, the crosswalk determination unit 40b determines that there is a crosswalk 24, and the braille block candidate determination unit 40c determines that there is a braille block candidate TC based on the detection of the center line 25.

[0031] The center line 25 and the white lines WL1 to WL7 of the crosswalk 24 extend in the same direction. For example, as shown in the illustrated example, the center line 25 is parallel (or approximately parallel) to the white lines WL1 to WL7 and is located outside the white lines WL1 to WL7 in the longitudinal direction. In the mobility support device 1 according to this embodiment, if the crosswalk determination unit 40b determines that a crosswalk 24 is present in the image 21 and the braille block candidate determination unit 40c determines that a braille block candidate TC (center line 25) is present in the image 21, the slope calculation unit 40d of the control unit 40 calculates the slope ΔL of the white lines WL that make up the crosswalk 24 and the slope ΔT of the braille block candidate TC. The braille block extraction unit 40e calculates the absolute value of the difference between the slope ΔL of the white line WL and the slope ΔT of the braille block candidate TC, and if the absolute value of the difference is less than a predetermined value and the braille block candidate TC is located longitudinally outside the white line WL, the braille block candidate TC is not extracted as an actual braille block T.

[0032] Here, when the absolute value of the difference between the slope ΔL of the white line WL of the crosswalk 24 and the slope ΔT of the braille block candidate TC is less than a predetermined value, it means that the condition is met in which the braille block candidate TC (center line 25) and each white line WL of the crosswalk 24 can be considered to extend in the same direction. This predetermined value is set in advance and can be changed as appropriate within a range in which the difference in slope between the white line WL and the braille block candidate TC can be considered to extend in the same direction (for example, parallel).

[0033] The slope ΔT of the braille block candidate TC (center line 25) can be calculated, for example, as follows. That is, when the center line 25 is determined to be the braille block candidate TC, multiple bounding boxes BB are formed as shown in FIG. 3. A reference point for determining the position is derived for each bounding box BB. Furthermore, the slope of the braille block candidate TC (center line 25) is calculated based on the reference point of each bounding box BB. At this time, for example, the slope can be found by the least squares method from the x and y coordinate values ​​of each reference point of the bounding box BB.

[0034] The slope ΔL of the white line WL can be calculated, for example, as follows. For example, similar to the method for calculating the slope of the braille block candidate TC, multiple bounding boxes BB can be formed, and the slope of the white line WL can be calculated based on the reference points of each bounding box. Alternatively, the white line WL can be detected by edge detection or the like, and the direction in which the edge extends can be calculated as the slope of the white line WL. Another method for calculating the slope ΔL of the white line WL is, for example, to convert the image data to grayscale and perform edge detection (Canny method, etc.). Here, the slope ΔL of the white line WL can also be calculated by performing a Hough transform on the coordinates of each point of the detected edge.

[0035] 3, the white line WL of the crosswalk 24 and the center line 25 as the braille block candidate TC extend in the same direction, and the absolute value of the difference between the slope ΔL of the white line WL of the crosswalk 24 and the slope of the braille block candidate TC is less than a predetermined value. Therefore, the braille block candidate TC (center line 25) is not extracted as an actual braille block T.

[0036] The inclination of the white line WL that constitutes the crosswalk 24, which is the subject of calculation of the difference with the inclination ΔT of the braille block candidate TC, can be calculated using any of the calculation methods shown below (Method 1) to (Method 4), or by appropriately using the calculation methods shown below (Method 1) to (Method 4) depending on the conditions, etc. (Method 1) A method of calculating the inclination of a specific white line (for example, the white line WL1 located at the edge) among multiple white lines WL1 to WL7 as the inclination ΔL of the white line WL of the crosswalk 24, the difference of which is to be calculated from the inclination of the braille block candidate TC. (Method 2) After calculating the inclinations of all the white lines WL1 to WL7, the average value is calculated as the inclination ΔL of the white line WL of the crosswalk 24, which is the target for calculating the difference with the inclination of the tactile paving block candidate TC. (Method 3) After calculating the slopes of all the white lines WL1 to WL7, the average of the maximum and minimum values ​​is calculated as the slope ΔL of the white line WL of the crosswalk 24, which is the subject of calculation of the difference with the slope of the braille block candidate TC. (Method 4) A method of calculating the inclination of the white line WL4, which is closest to the braille block candidate TC among the white lines WL1 to WL7, as the inclination ΔL of the white line WL of the crosswalk 24, which is the subject of calculation of the difference from the inclination of the braille block candidate TC.

[0037] Whether or not the braille block candidate TC is located outside the white line WL in the longitudinal direction can be determined by, for example, comparing the XY coordinates of the vertices of the bounding box of each white line WL with the x- and y-coordinates of the vertices of the bounding box of the braille block candidate TC. Note that this determination method is just an example and can be changed as appropriate.

[0038] The travel assistance control unit 40f executes travel assistance control based on the target detection result by the target detection unit 40a, the extraction result by the tactile paving block extraction unit 40e, and the like. At this time, the travel assistance control unit 40f sets an area A in the image 21 acquired by the imaging unit 20 where travel assistance control is executed. The travel assistance control unit 40f also executes travel assistance control, such as notifying the user, for targets present in the area A. For example, in FIG. 3, a center line 25 is detected as a tactile paving block candidate TC in the area A. However, as described above, the tactile paving block candidate TC (center line 25) is not extracted as a solid tactile paving block T. Therefore, travel assistance control that would be executed if the tactile paving block candidate TC (center line 25) were extracted as a solid tactile paving block T is not executed. The area A can be changed as appropriate based on the user's movement direction, movement speed, and the like.

[0039] Next, an example of processing (braille block extraction processing) for extracting actual braille blocks T from the braille block candidates TC will be described with reference to the flowchart shown in FIG.

[0040] In step S1, the imaging unit 20 captures an image of the area in front of the user wearing the mobility assistance device 1. Image data representing the image is transmitted to the control unit .

[0041] Next, in step S2, the target detection unit 40a performs image processing on the image data acquired from the imaging unit 20 to detect targets (white lines WL, braille block candidates TC, roadways 26, obstacles, etc.) within the image.

[0042] Next, the braille block candidate determining unit 40c determines whether or not there is a braille block candidate TC among the targets detected by the target detecting unit 40a (step S3).

[0043] In step S3, if it is determined that there are no braille block candidates TC in the image acquired by the braille block candidate determination unit 40c (NO in step S3), the braille block extraction unit 40e determines that there are no actual braille blocks T in the acquired image and terminates the processing (step S10).

[0044] On the other hand, in step S3, if the braille block candidate determination unit 40c determines that there is a braille block candidate TC in the acquired image (YES in step S3), the crosswalk determination unit 40b determines whether there is a crosswalk 24 in the acquired image (step S4).

[0045] In step S4, if the crosswalk determination unit 40b determines that the crosswalk 24 is not among the targets detected by the target detection unit 40a (NO in step S4), the braille block extraction unit 40e determines that the targets determined to be braille block candidates TC are actual braille blocks T, and ends the process (step S9). As a result, the actual braille blocks T are extracted from the braille block candidates TC.

[0046] On the other hand, in step S4, if the crosswalk determination unit 40b determines that the crosswalk 24 is among the targets detected by the target detection unit 40a (YES in step S4), the slope calculation unit 40d calculates the slope ΔL of the white lines WL that make up the crosswalk 24 (step S5). Subsequently, the slope calculation unit 40d calculates the slope ΔT of the braille block candidate TC (step S6). Note that the order of steps S5 and S6 may be reversed.

[0047] Following step S6, the braille block extraction unit 40e calculates the absolute value of the difference between the slope ΔT of the braille block candidate TC calculated by the slope calculation unit 40d and the slope ΔL of the white line WL that constitutes the crosswalk 24, and then determines whether the absolute value of the difference is less than a predetermined value (step S7).

[0048] In step S7, if the absolute value of the difference is determined not to be less than the predetermined value (NO in step S7), the braille block extraction unit 40e determines that the object determined to be the braille block candidate TC is a real braille block T, and ends the process (step S9). As a result, the real braille block T is extracted from the braille block candidates TC.

[0049] On the other hand, in step S7, if the braille block extraction unit 40e determines that the absolute value of the difference is less than a predetermined value (YES in step S7), it determines whether the braille block candidate TC is located outside the white line WL in the longitudinal direction (step S11).

[0050] In step S11, if the braille block extraction unit 40e determines that the braille block candidate TC is not located outside the longitudinal direction of the white line WL (NO in step S11), it determines that the object determined to be the braille block candidate TC is an actual braille block T and terminates the processing (step S9).

[0051] On the other hand, in step S11, if the braille block extraction unit 40e determines that the braille block candidate TC is located outside the longitudinal direction of the white line WL (YES in step S11), it determines that the object determined to be the braille block candidate TC is not an actual braille block T and terminates the processing (step S8).

[0052] The above is the first embodiment of the mobility support device 1 of the present invention. Next, the effects achieved by the mobility support device 1 of the present invention will be described below.

[0053] The above-described mobility support device 1 has the following characteristic configuration: Therefore, the mobility support device 1 can achieve the following unique effects that cannot be achieved by conventional techniques.

[0054] (a) The mobility support device 1 of this embodiment is a mobility support device 1 worn on at least one of the shoulders and the neck of a user, and includes an imaging unit 20 that captures an image ahead of the user, a target detection unit 40a that detects targets in an image 21 captured by the imaging unit 20 based on at least brightness information and shape information obtained from image data of the image 21, a first determination unit (braille block candidate determination unit 40c) that determines whether or not any of the targets detected by the target detection unit 40a correspond to a braille block candidate TC, a second determination unit (crosswalk determination unit 40b) that determines whether or not any of the targets detected by the target detection unit 40a correspond to a crosswalk 24, and a determination unit (braille block candidate determination unit 40c) that determines whether or not any of the targets detected by the target detection unit 40a correspond to a crosswalk 24. When it is determined that there is a tactile paving block candidate by the first determination unit (crosswalk determination unit 40b) and when it is determined that there is a pedestrian crossing 24, the system is provided with a slope calculation unit 40d that calculates the slope ΔT of the tactile paving block candidate TC and the slope ΔL of the white lines WL1 to WL7 that make up the crosswalk 24, and a braille block extraction unit 40e that extracts actual tactile paving blocks T from the tactile paving block candidate TC, and the braille block extraction unit 40e is characterized in that it does not extract the tactile paving block candidate TC as an actual tactile paving block T on the condition that the difference between the slope ΔT of the tactile paving block candidate TC calculated by the slope calculation unit 40d and the slope ΔL of the white lines WL1 to WL7 is less than a predetermined value and the tactile paving block candidate is located outward in the longitudinal direction of the white lines.

[0055] Specifically, in step S1 described above, mobility support device 1 can acquire image 21 of the area ahead of the user using imaging unit 20. In step S2, mobility support device 1 causes target detection unit 40a to perform image processing based on at least brightness information and shape information obtained from the image data acquired by imaging unit 20 to detect targets within image 21. In step S3, mobility support device 1 causes Braille block candidate determination unit 40c to determine whether or not there is a target that can be a Braille block candidate TC among the targets detected by target detection unit 40a. In step S4, mobility support device 1 causes crosswalk determination unit 40b to determine whether or not there is a target that can be a crosswalk 24 among the targets detected by target detection unit 40a. If the targets detected by the target detection unit 40a include a target that is a crosswalk 24, in steps S5 and S6, the gradient calculation unit 40d calculates the gradient ΔL of the white lines WL1 to WL7 that make up the crosswalk 24 and the gradient ΔT of the braille block candidate TC. In step S7, the braille block extraction unit 40e determines whether the absolute value of the difference between the gradient ΔL of the white lines WL1 to WL7 calculated by the gradient calculation unit 40d and the gradient ΔT of the braille block candidate TC is less than a predetermined value. If the absolute value of the difference is less than the predetermined value, the travel support device 1 determines whether the braille block candidate TC is located outside the white lines WL1 to WL7 in the longitudinal direction (step S11). Furthermore, in step S11, if the travel assistance device 1 determines that the tactile paving block candidate TC is located outside the white lines WL1 to WL7 in the longitudinal direction, the travel assistance device 1 does not extract the tactile paving block candidate TC as a real tactile paving block T.

[0056] The mobility support device 1 configured as described above detects targets in the image 21 based on at least brightness information and shape information obtained from the image data of the image 21 captured by the imaging unit 20. That is, the mobility support device 1 of this embodiment detects targets based on brightness (color) and shape (pattern shape and uneven shape) from the image captured by the imaging unit 20. In this case, as with the techniques described in Patent Documents 1 and 2, there is a risk that the center line 25, which has a similar shape and color to the braille block T (guide block Tb), may be erroneously determined to be the braille block T. However, center lines generally extend in a direction parallel to the white lines of a crosswalk and are positioned outside the white lines in the longitudinal direction. Therefore, in this mobility support device 1, the slope calculation unit 40d calculates the slope ΔL of the white lines WL1 to WL7 and the slope ΔT of the center line 25 determined to be a braille block candidate TC. If the difference between these slopes is less than a predetermined value, it is determined whether the braille block candidate TC is located outside the white lines WL1 to WL7 in the longitudinal direction. Here, if the tactile paving block candidate TC is located outside the white lines WL1 to WL7 in the longitudinal direction, the mobility support device 1 does not extract the tactile paving block candidate TC as a solid tactile paving block T. By doing so, the mobility support device 1 of this embodiment can prevent the candidate TC from being extracted as a solid tactile paving block T even if the tactile paving block candidate TC includes the center line 25. This makes it possible to realize a mobility support device 1 that can reduce the number of times the center line 25 is erroneously determined to be a tactile paving block T.

[0057] (b) In the travel assistance device 1 of this embodiment, the slope calculation unit 40d may take the slope of the white line WL that constitutes the crosswalk 24 to be the average value of the multiple white lines WL1 to WL7 that constitute the crosswalk 24.

[0058] When calculating the slope ΔL of the white lines WL1 to WL7 that make up the crosswalk 24, if the slope of only a specific white line is calculated and used as the slope ΔL of the white lines that make up that crosswalk 24, and the calculation accuracy is low, there is a higher probability of a problem occurring in which the center line 25 that is a braille block candidate TC is extracted as an actual braille block T. However, the mobility support device 1 of this embodiment can reduce the probability of such a problem occurring by setting the slope ΔL of the white lines WL1 to WL7 that make up the crosswalk 24 to the average value of the multiple white lines WL1 to WL7.

[0059] (c) In the mobility support device 1 of this embodiment, the slope calculation unit 40d may calculate the slope ΔL of the white line WL that constitutes the crosswalk 24 as the average value of the maximum and minimum values ​​of the slopes of the multiple white lines WL1 to WL7 that constitute the crosswalk 24.

[0060] When calculating the slope ΔL of the white lines WL that make up the crosswalk 24, if only the slopes of specific white lines WL1 to WL7 are calculated and used as the slope ΔL of the white lines WL that make up that crosswalk 24, if the calculation accuracy is low, there is a higher probability of a problem occurring in which the center line 25 that is a candidate braille block TC is extracted as an actual braille block T. However, the mobility support device 1 of this embodiment can reduce the probability of such a problem occurring by setting the slope of the white lines WL that make up the crosswalk 24 to the average value of the maximum and minimum values ​​of the slopes of the multiple white lines WL1 to WL7.

[0061] (d) In the mobility support device 1 of this embodiment, the slope calculation unit 40d may calculate the slope of the white line WL4, among the multiple white lines WL1 to WL7 that make up the crosswalk 24, that is closest to the landmark that has been determined by the first determination unit (braille block candidate determination unit 40c) to contain a braille block candidate TC (center line 25), as the slope ΔL of the white lines WL that make up the crosswalk 24.

[0062] When calculating the difference between the slope ΔT of the tactile paving block candidate TC and the slope ΔL of the white lines WL that make up the crosswalk 24, it is considered that the accuracy of calculating the slope difference is high if, among the multiple white lines WL1 to WL7 that make up the crosswalk 24, the white line WL4 that is closest to the tactile paving block candidate TC (center line 25) is used as the comparison target. Therefore, by configuring the mobility support device 1 of this embodiment in this way, it is possible to increase the accuracy of calculating the difference between the slope ΔT of the tactile paving block candidate TC (center line 25) and the slope ΔL of the white lines WL that make up the crosswalk 24. As a result, the mobility support device 1 of this embodiment can reduce the probability of the occurrence of a problem in which the tactile paving block candidate TC that corresponds to the center line 25 is extracted as an actual tactile paving block T.

[0063] The mobility support device 1 of this embodiment has all of the characteristic configurations (a) to (d) described above, but the present invention is not limited to this. For example, the mobility support device 1 may not have some or all of the configurations (b) to (d) described above, or may have other configurations in addition to the configurations related to (a) to (d). Furthermore, the mobility support device 1 may be modified in such a way that the configurations related to (a) to (d) described above are changed to different ones, without departing from the spirit of the present invention.

[0064] Second Embodiment Hereinafter, a mobility assistance device 100 according to a second embodiment of the present invention will be described in detail with reference to the drawings.

[0065] The mobility support device 100 according to this embodiment differs from the mobility support device 1 of the first embodiment described with reference to FIGS. 1 to 5 in that the method of extracting the actual Braille blocks T is different, as shown in FIGS. 6 and 7. In particular, the imaging unit 200, target object detection unit 400a, pedestrian crossing determination unit 400b, Braille block candidate determination unit 400c, mobility support control unit 400f, notification unit 450, and battery 110 of this embodiment have the same configurations as the target object detection unit 40a, pedestrian crossing determination unit 40b, Braille block candidate determination unit 40c, mobility support control unit 40f, notification unit 45, and battery 110 of the first embodiment. Therefore, a description of these components will be omitted, and the following description will focus on the differences. Note that FIG. 6 is a block diagram of the mobility support device 100 according to the second embodiment of the present invention, and FIG. 7 is an example of an image captured by the camera of the mobility support device 100 according to the second embodiment of the present invention.

[0066] The mobility support device 100 according to this embodiment includes an imaging unit 200, a control unit 400, a notification unit 450, and a battery 110. The control unit 400 also includes a target detection unit 400a, a crosswalk determination unit 400b, a braille block candidate determination unit 400c, a bounding box detection unit 400d, a roadway area setting unit 400g, a braille block extraction unit 400e, and a mobility support control unit 400f.

[0067] 7 is an example of an image captured by the imaging unit 200 of a user wearing the mobility assistance device 100 standing on the sidewalk at the bottom of the page, waiting for a traffic light to cross the crosswalk 24. In this case, the forward direction of the user is a direction perpendicular (approximately perpendicular) to the length direction of each white line WL that constitutes the crosswalk 24, and is the direction toward the sidewalk at the top of the page.

[0068] This image 22 shows a crosswalk 24 made up of multiple white lines WL (WL8 to WL11) and a center line 25. Therefore, these targets are detected by the target detection unit 400a. When these targets are detected by the target detection unit 40a, the crosswalk determination unit 40b determines that the crosswalk 24 is present in the image 21. In addition, the braille block candidate determination unit 40c determines that a braille block candidate TC (braille block T, center line 25) is present in the image 21.

[0069] Incidentally, the center line 25 is placed within the roadway 26. In contrast, the braille blocks T are placed within the sidewalk. Therefore, in this embodiment, attention is paid to the positional relationship between the crosswalk 24, the braille blocks T, and the center line 25, and the actual braille blocks T are extracted from the braille block candidates TC. The extraction method will be specifically described below.

[0070] When the pedestrian crossing determination unit 400b determines that a pedestrian crossing 24 exists based on the target object detected by the target object detection unit 400a, the bounding box detection unit 400d detects each of the white lines WL8 to WL11 that make up the pedestrian crossing 24 as a bounding box BB2 with sides parallel to the x direction and the y direction on the paper. Specifically, when the pedestrian crossing determination unit 400b determines that a pedestrian crossing 24 exists, a rectangular bounding box BB2 is set for each of the white lines WL8 to WL11. Here, the bounding box BB2 is set so as to surround the entire white line WL in the image 22 in a rectangular shape (a rectangle with sides parallel to the x direction and sides parallel to the y direction), regardless of whether the longitudinal direction of each of the white lines WL8 to WL11 is inclined with respect to the x direction.

[0071] Furthermore, the bounding box detection unit 400d detects the set bounding box BB2 and acquires the coordinates of each vertex of the bounding box BB2. Note that the image 22 in Fig. 7 illustrates an example in which the length direction (longitudinal direction) of each of the white lines WL8 to WL11 is parallel to the x direction, and the width direction perpendicular to the length direction (longitudinal direction) is parallel to the y direction.

[0072] Here, the coordinates of image 22 have the upper left vertex of image 22 as the origin (0,0). Therefore, in a bounding box BB2 that encloses white line WL11, which is the farthest from the user among the multiple white lines WL8 to WL11, the coordinates of the upper left vertex A4 are expressed as (X1, Y1), and the coordinates of the upper right vertex B4 are expressed as (X2, Y1), for example. Furthermore, in a bounding box BB2 that encloses white line WL8, which is the closest to the user, the coordinates of the lower left vertex C1 are expressed as (X3, Y2), and the coordinates of the lower right vertex D1 are expressed as (X4, Y2), for example.

[0073] The white line WL11 farthest from the user can be said to be the white line WL11 located closest to one end in a direction intersecting the lengthwise direction (longitudinal direction) of each of the white lines WL8 to WL11 (for example, a direction perpendicular to the lengthwise direction of each of the white lines WL8 to WL11). In contrast, the white line WL8 closest to the user can be said to be the white line WL8 located closest to the other end in a direction intersecting the lengthwise direction (longitudinal direction) of each of the white lines WL8 to WL11 (for example, a direction perpendicular to the lengthwise direction of each of the white lines WL8 to WL11).

[0074] The roadway area setting unit 400g sets the roadway area S in the image 22 based on the coordinates of each vertex of a bounding box BB2 that surrounds each of the white lines WL8 to WL11 detected by the bounding box detection unit 400d.

[0075] Specifically, the farthest long side (long side on one end side) of the white line WL11 from the user on the farthest side (-Y side) substantially overlaps with the straight line connecting the upper left vertex A4 and the upper right vertex B4 of the bounding box BB2 of the white line WL11. Based on the coordinates of the upper left vertex A4 and the upper right vertex B4, the roadway area setting unit 400g derives an upper end line L1 (first line L1) formed by the straight line connecting these vertices A4 and B4 and an extension of that straight line. The derived upper end line L1 can be considered to substantially coincide with the line formed by the long side on one end side of the white line WL11 and the extension of that long side.

[0076] Furthermore, the long side (the other end long side) on the near side (+Y side) of the white line WL8 closest to the user substantially overlaps with the line connecting the lower left vertex C1 and the lower right vertex D1 of the bounding box BB2 of the white line WL8. Therefore, based on the coordinates of the lower left vertex C1 and the lower right vertex D1, the roadway area setting unit 400g derives a bottom line L2 (second line L2) formed by the line connecting these vertices C1 and D1 and an extension of that line. The derived bottom line L2 can be considered to substantially coincide with the line formed by the other end long side of the white line WL8 and the extension of that other end long side.

[0077] Here, the area sandwiched between the line formed by one long side of the white line WL11 and an extension of that long side, and the line formed by the other long side of the white line WL8 and an extension of that other long side, substantially coincides with the area where the roadway 26 exists. Therefore, the roadway area setting unit 400g sets the area sandwiched between the derived upper end line L1 (first line L1) and lower end line L2 (second line L2) in the image 22 as the area of ​​the roadway 26 (roadway area S: see FIG. 7).

[0078] Of the objects determined to be braille block candidates TC by the braille block candidate determination unit 400c, those located in the roadway area S set by the roadway area setting unit 400g are not extracted by the braille block extraction unit 400e as solid braille blocks. In FIG. 7, the center line 25 determined to be the braille block candidate TC is located within the roadway area S. Therefore, the braille block extraction unit 400e does not extract the braille block candidate TC (center line 25) as a solid braille block T. On the other hand, the braille block T determined to be the braille block candidate TC is located outside the roadway area S. Therefore, the braille block extraction unit 400e extracts the braille block candidate TC (braille block T) as a solid braille block T.

[0079] The above is the mobility assistance device 100 according to the second embodiment of the present invention. Next, the effects achieved by the mobility assistance device 100 of the present invention will be described below.

[0080] The above-described mobility assistance device 100 has the following characteristic configuration: Therefore, the mobility assistance device 100 can achieve the following unique effects that cannot be achieved by conventional techniques.

[0081] (e) The mobility support device 100 of this embodiment is a mobility support device 100 worn on at least one of the shoulders and the neck of a user, and includes an imaging unit 200 that images the area ahead of the user, a target detection unit 400a that detects targets in the image 22 captured by the imaging unit 200 based on at least brightness information and shape information obtained from image data of the image 22, a first determination unit (braille block candidate determination unit 400c) that determines whether or not there are targets that correspond to braille block candidates TC among the targets detected by the target detection unit 400a, a second determination unit (crosswalk determination unit 400b) that determines whether or not there are targets that correspond to a crosswalk 24 among the targets detected by the target detection unit 400a, and a predetermined area (roadway area S) that surrounds the crosswalk 24 in the image 22 when the second determination unit (crosswalk determination unit 400b) determines that there is a target that corresponds to a crosswalk 24. and a braille block extraction unit 400e that extracts braille blocks T from the braille block candidates TC. The predetermined area setting unit (roadway area setting unit 400g) sets the predetermined area (roadway area S) as an area between a first line L1 formed by one long side of the white line WL11 located closest to one end and an extension of the long side of that one end, and a second line L2 formed by the other long side of the white line WL8 located closest to the other end and an extension of the long side of that other end, in a direction intersecting with the longitudinal direction of the plurality of white lines WL8 to WL11 that make up the crosswalk 24. The braille block extraction unit 400e is characterized in that, of the braille block candidates TC determined by the first determination unit (braille block candidate determination unit 400c), those that are located in the predetermined area (roadway area S) do not extract as braille blocks T.

[0082] The mobility support device 100 described above detects targets in the image 22 based on at least brightness information and shape information obtained from the image data of the image captured by the imaging unit. That is, the mobility support device 100 of this embodiment detects targets based on brightness (color) and shape (pattern) from the image captured by the imaging unit 200. In this case, as with the techniques described in Patent Documents 1 and 2, there is a risk of erroneously determining the center line 25 as a Braille block T. However, the center line 25 is located on the same roadway 26 as the crosswalk 24, while the Braille block T is located on the sidewalk. Therefore, the mobility support device 100 of this embodiment uses the area of ​​the crosswalk 24 to determine the area of ​​the roadway 26. Specifically, the area between the first line L1 and the second line L2 (predetermined area: roadway area S) can be considered to be the approximate area of ​​the roadway 26. Based on this knowledge, the mobility support device 100 of this embodiment prevents Braille block candidates TC located in the roadway area S from being extracted as actual Braille blocks T. In this way, even if the center line 25 is included in the braille block candidate TC, the travel assistance device 100 of this embodiment can prevent the candidate TC from being extracted as the actual braille block T. As a result, this embodiment can realize the travel assistance device 100 that can reduce the number of times the center line 25 is erroneously determined to be the braille block T.

[0083] The mobility support device 100 according to this embodiment may also have the same characteristic configuration as the mobility support device 1 according to the first embodiment described above. Specifically, the mobility support device 100 may have some or all of the characteristic configurations according to (b) to (d) above. By adopting such a configuration, the mobility support device 100 can achieve the same operational effects as those described in (b) to (d) above.

[0084] <<Variation>> A modified example of the movement assistance device 100 according to the second embodiment of the present invention will be described in detail below with reference to FIGS.

[0085] The mobility support device 101 according to this example differs from the mobility support device 100 according to the second embodiment described with reference to Figures 6 and 7 in that the method for extracting the actual Braille blocks T is different. The imaging unit 200, target detection unit 400a, pedestrian crossing determination unit 400b, Braille block candidate determination unit 400c, bounding box detection unit 400d, mobility support control unit 400f, notification unit 450, and battery 110 according to this example have the same configurations as those in the second embodiment, and therefore the same reference numerals are used to denote these components, and the description will be omitted, with the differences being mainly described. Note that Figure 8 is a block diagram of the mobility support device 101 according to this example.

[0086] 8, the mobility support device 101 according to this example includes an imaging unit 200, a control unit 401, a notification unit 450, and a battery 110. The control unit 401 also includes a target detection unit 400a, a crosswalk determination unit 400b, a braille block candidate determination unit 400c, a bounding box detection unit 400d, a roadway area setting unit 400g, a crosswalk area setting unit 400h, a braille block extraction unit 401e, and a mobility support control unit 400f.

[0087] Meanwhile, the center line 25 is located within the roadway 26 and is adjacent to the crosswalk 24 on either the left or right side of the paper (x direction side: left side in FIG. 7). In contrast, the braille blocks T are placed within the sidewalk. Therefore, in this example, attention is paid to the positional relationship between the crosswalk 24, the braille blocks T, and the center line 25, and the actual braille blocks T are extracted from the braille block candidates. The extraction method will be specifically described below.

[0088] The crosswalk area setting unit 400h sets the area where the crosswalk 24 exists in the image 22 based on the coordinates (X1, Y1) of the upper left vertex A4, the coordinates (X2, Y1) of the upper right vertex B4, the coordinates (X3, Y2) of the lower left vertex C1, and the coordinates (X4, Y2) of the lower right vertex D1 detected by the bounding box detection unit 400d. Specifically, as shown in Fig. 7, the area where the crosswalk 24 exists corresponds to an area where straight lines connect four points: both ends of the long side (long side on one end side) on the farthest side (-Y side) of the white line WL11 from the user, and both ends of the long side (long side on the other end side) on the near side (+Y side) of the white line WL8 from the user. Therefore, the area surrounded by straight lines connecting the four points detected by bounding box detection unit 400d, namely, upper left vertex A4, upper right vertex B4, lower left vertex C1, and lower right vertex D1, approximately matches the area where crosswalk 24 exists in image 22. Therefore, crosswalk area setting unit 400h sets the area surrounded by the four points, namely, upper left vertex A4, upper right vertex B4, lower left vertex C1, and lower right vertex D1, as the area where the crosswalk exists.

[0089] Of the objects (FIG. 7: center line 25, braille block T) determined to be braille block candidates TC by the braille block candidate determination unit 400c, the braille block extraction unit 401e does not extract as solid braille blocks T those that are located within the roadway area S but outside the area surrounding the crosswalk 24 (area surrounded by A4, B4, C1, and D1) set by the crosswalk area setting unit 400h. Here, a braille block candidate TC located within the roadway area S corresponds to the braille block candidate TC being located outside the white line WL in the longitudinal direction. On the other hand, a braille block T determined to be a braille block candidate TC is located outside the roadway area S. Therefore, the braille block extraction unit 401e extracts the braille block candidate TC (braille block T) as a solid braille block T.

[0090] The above is the mobility assistance device 101 according to this embodiment. Next, the effects achieved by the mobility assistance device 101 of this embodiment will be described below.

[0091] The above-described mobility assistance device 101 has the following characteristic configuration: Therefore, the mobility assistance device 101 can achieve the following unique effects that cannot be achieved by conventional techniques.

[0092] (f) In the mobility support device 101 according to this example, the braille block extraction unit 401e does not extract as actual braille blocks T those (center lines 25, braille blocks T) that are within a predetermined area (roadway area S) and that are located outside the area surrounding the crosswalk 24 by connecting the four points at both ends of the long side on one end (A4, B4) and the two ends of the long side on the other end (C1, D1) with straight lines, among the braille block candidates TC (center lines 25, braille blocks T) determined by the first determination unit (braille block candidate determination unit 400c).

[0093] The center line 25 is located within the roadway 26 adjacent to the crosswalk 24 on the left and right sides of the paper. Based on this knowledge, the mobility support device 101 of this example sets an area surrounding the crosswalk 24 by connecting four points, the upper left vertex A4, the upper right vertex B4, the lower left vertex C1, and the lower right vertex D1, with straight lines, and does not extract braille block candidates TC located outside this area as actual braille blocks T. Even in this way, the mobility support device 101 of this example can exclude braille block candidates TC that correspond to the center line 25. This makes it possible to realize a mobility support device 101 that can reduce the number of times the center line 25 is mistakenly determined to be a braille block T.

[0094] The above is the mobility assistance device 101 according to the modification of the second embodiment. In each of the above-described embodiments and modifications, the crosswalk determination unit 40b, 400b (second determination unit) can determine that there is a target object corresponding to a crosswalk when two or more white lines aligned in parallel are detected.

[0095] In this way, even if the mobility support devices 1, 100, 101 according to the above-mentioned embodiments and variant examples can only detect some of the white lines WL of the crosswalk 24, they can prevent the candidate tactile paving blocks TC determined by the first determination unit (the candidate tactile paving block determination unit 40c, 400c) from extracting the candidate tactile paving block corresponding to the center line 25 as an actual tactile paving block T.

[0096] Furthermore, the mobility assistance devices 1, 100, 101 according to the above-mentioned embodiments and variations include a mobility assistance control unit 40f, 400f that executes mobility assistance control, and a mobility assistance control execution area setting unit (mobile assistance control unit 40f, 400f) that sets a mobility assistance control execution area (area A: see Figure 3) within the images 21, 22 in which mobility assistance control is executed, and the mobility assistance control unit 40f, 400f executes mobility assistance control when there is a target extracted as a real Braille block T within the mobility assistance control execution area (area A) set by the mobility assistance control execution area setting unit (mobile assistance control unit 40f, 400f).

[0097] In this way, in the above-described embodiments and modifications, the probability of erroneously determining the center line 25 as the braille block T is low, and the mobility assistance devices 1, 100, 101 that can execute appropriate mobility assistance control can be realized.

[0098] The above are the effects of the mobility support device 1,100 according to an embodiment of the present invention and the mobility support device 101 according to its modified example. However, the mobility support devices 1,100, 101 of the present invention are not limited to the above-described embodiments and modified examples, and various modifications can be made.

[0099] In this embodiment, the mobility support devices 1, 100, 101 are attached to at least one of the user's shoulders and neck, but the present invention is not limited to this and can be applied to various mobility support devices. For example, the mobility support devices 1, 100, 101 may be attached to the user's head, face, or torso. Furthermore, the imaging units 20, 200 may be various devices, such as cameras and image sensors, capable of acquiring images 21, 22. Furthermore, the imaging range of the imaging units 20, 200 can be set not only in front of the user but also in various directions, such as to the side or rear, depending on the characteristics and usage of the mobility support devices 1, 100, 101.

[0100] In this embodiment, the notification unit 45, 450 is exemplified as a speaker or earphone that notifies by voice, but the notification unit 45, 450 is not limited to a device that notifies by voice, and various devices can be used as long as they can notify the user who is the target of mobility assistance. Furthermore, in this embodiment and the modified example, the mobility assistance device 1, 100, 101 is exemplified as being intended for visually impaired persons, but the mobility assistance device 1, 100, 101 of the present invention can be intended for various persons other than visually impaired persons. For example, the mobility assistance device 1, 100, 101 can also be used to provide mobility assistance to elderly people, etc.

[0101] The above are embodiments and variations of the mobility support devices 1, 100, and 101 according to the present invention, but the present invention is not limited to the examples in the above-mentioned embodiments and variations, and it will be easily understood by those skilled in the art that other embodiments are possible based on the teachings and spirit of the present invention as long as they do not deviate from the scope of the claims. [Industrial Applicability]

[0102] The present invention can be suitably used in all types of mobility assistance devices that assist users in moving around. [Explanation of symbols]

[0103] 1,100,101:Mobility support equipment 20,200: Imaging unit 40a, 400a: Target detection unit 40b, 400b: Crosswalk judgment unit (second judgment unit) 40c, 400c: Braille block candidate determination unit (first determination unit) 40d: Tilt calculation section 40e, 400e, 401e: Braille block extraction section 400g: Roadway area setting section (predetermined area setting section) S: Road area (predetermined area) T: Braille blocks TC: Braille block candidate L1: First line L2: Second line

Claims

1. A mobility assistance device worn on at least one of the user's shoulders and neck, an imaging unit that images a front side of the user; a target detection unit that detects a target in an image captured by the imaging unit based on at least brightness information and shape information obtained from image data of the image; a first determination unit that determines whether or not there is a target corresponding to a braille block candidate among the targets detected by the target detection unit; a second determination unit that determines whether or not there is a target corresponding to a pedestrian crossing among the targets detected by the target detection unit; a gradient calculation unit that calculates a gradient of the braille block candidate and a gradient of a white line that constitutes the crosswalk when the first determination unit determines that the braille block candidate exists and the second determination unit determines that the crosswalk exists; a braille block extraction unit that extracts actual braille blocks from the braille block candidates; The braille block extraction unit does not extract the braille block candidate as the actual braille block, provided that the difference between the inclination of the braille block candidate calculated by the inclination calculation unit and the inclination of the white line is less than a predetermined value and the braille block candidate is located longitudinally outward of the white line. A mobility support device characterized by the above.

2. A mobility assistance device worn on at least one of the user's shoulders and neck, an imaging unit that images a front side of the user; a target detection unit that detects a target in an image captured by the imaging unit based on at least brightness information and shape information obtained from image data of the image; a first determination unit that determines whether or not there is a target corresponding to a braille block candidate among the targets detected by the target detection unit; a second determination unit that determines whether or not there is a target corresponding to a pedestrian crossing among the targets detected by the target detection unit; a predetermined area setting unit that sets a predetermined area surrounding the crosswalk within the image when the second determination unit determines that there is a target object corresponding to the crosswalk; a braille block extraction unit that extracts actual braille blocks from the braille block candidates; the predetermined area setting unit sets, in a direction intersecting the longitudinal direction of the plurality of white lines constituting the crosswalk, an area between a first line formed by one long side of the white line located closest to one end and an extension of the one long side, and a second line formed by the other long side of the white line located closest to the other end and an extension of the other long side, The braille block extraction unit is characterized in that, among the braille block candidates determined by the first determination unit, those that are in the specified area are not extracted as the actual braille block.

3. The mobility support device according to claim 2, characterized in that the braille block extraction unit does not extract as the actual braille block those of the braille block candidates determined by the first determination unit that are within the specified area, are located outside the area surrounding the crosswalk by connecting four points on both ends of the long side on one end side and both ends of the long side on the other end side with straight lines, and are located outside the white line in the longitudinal direction.

Images