Pedestrian assistance system, pedestrian assistance method, and pedestrian assistance program

Abstract

To provide a new technology for notifying a user of the presence or approach of an obstacle while operating a mobile terminal or immediately after operating the terminal. [Solution] A pedestrian assistance system having an acquisition unit, a detection unit, and an output unit, wherein the acquisition unit is equipped with a distance measurement sensor and acquires information related to the shape of an object using the distance measurement sensor, the detection unit excludes from detection any object whose shape is acquired by the acquisition unit and whose outline is partially shared with a boundary of a detection area, and detects any object whose entire outline is within the detection area, among the objects acquired by the acquisition unit, and the output unit outputs a warning when an object is detected by the detection unit.

Description

[Technical Field]

[0001] The present invention relates to an assistance system, a pedestrian assistance method, and a pedestrian assistance program for recognizing objects present around a pedestrian and displaying or issuing a warning.

[0002] In recent years, devices and terminals equipped with LiDAR (Light Detection and Ranging), a type of sensor that uses laser light, have become increasingly popular. LiDAR emits laser light and measures the time it takes for the laser light reflected from an object to be detected, making it possible to determine with high precision the position and shape of objects within the irradiation range, as well as their distance from the sensor. This technology is capable of obtaining information with higher accuracy than conventional camera sensors, etc. [Background technology]

[0003] For example, Patent Document 1 discloses a technology relating to a pedestrian support method using LiDAR or a similar sensor and audio guidance. This technology uses a server that stores 3D map data relating to roads and the surrounding environment and a mobile terminal equipped with a sensor to compare the map data of a created travel route with the results of object detection by the sensor, and detects differences. It discloses that a warning relating to the differences is issued by audio output. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-046298 Summary of the Invention [Problem to be solved by the invention]

[0005] Nowadays, many people use map functions and navigation apps on their mobile devices. If a person starts walking after browsing the map or starts walking while operating the mobile device, forgetting that there are obstacles around, there is a concern that they may collide with people or obstacles. The present invention proposes a novel technique for notifying a user of the presence or approach of an obstacle immediately after or during operation of such a mobile terminal. [Means for solving the problem]

[0006] [1] A pedestrian assistance system having an acquisition unit, a detection unit, and an output unit, wherein the acquisition unit is equipped with a distance measurement sensor and acquires information related to the shape of an object using the distance measurement sensor, the detection unit excludes from detection any object whose shape is acquired by the acquisition unit and whose outline is partially in contact with the boundary of a detection area, and detects as an obstacle any object whose entire outline is within the detection area, and the output unit outputs a warning when the detection unit detects an obstacle.

[0007] This configuration allows the user to know the presence of obstacles in the surrounding area without using the 3D map data described in the prior art. Also, by determining whether an object is a detection target using the object's contour, it is possible to prevent erroneous detection of an object that is not a detection target.

[0008] [2] The pedestrian assistance system according to [1], wherein the acquisition unit sets a variable masking area within the detection area, and excludes the detected object from detection when part of the contour of the detected object within the variable masking area is shared with the boundary of the detection area.

[0009] With this configuration, the determination area for objects that are not the detection target is limited, and therefore, erroneous detection of objects that are not the detection target can be reduced.

[0010] [3] The pedestrian assistance system according to [1] or [2], wherein the variable masking area is set at the lower end of the detection area.

[0011] This configuration makes it possible to detect objects located in the foreground (at the bottom of the detection area) that are within the user's awareness, reducing false alarms, and by limiting the range that is not subject to detection, it is possible to properly detect obstacles that need to be detected.

[0012] [4] The pedestrian assistance system according to any one of [1] to [3], wherein the detection unit excludes an object behind the acquisition unit from the detection target.

[0013] By adopting such a configuration, it is possible to prevent the output of a warning regarding an object that is not actually an obstacle.

[0014] [5] The pedestrian assistance system according to any one of [1] to [4], wherein the detection unit detects ground undulations, inclinations, or suspended objects from the relative positions of a plurality of different measurement points.

[0015] With this configuration, the user can be not only aware of obstacles but also of bumps and unevenness on the road surface in advance, thereby preventing tripping or falling due to road surface conditions.

[0016] [6] The pedestrian assistance system according to any one of [1] to [5], wherein the acquisition unit acquires information relating to the distance from the acquisition unit, and the output unit outputs a different warning depending on the information relating to the distance obtained by the acquisition unit.

[0017] With this configuration, it is possible to notify the user of a warning according to the degree of danger of the detected obstacle.

[0018] [7] The pedestrian assistance system according to any one of [1] to [6], wherein the output unit outputs a safety confirmation at regular intervals regardless of the detection content of the detection unit.

[0019] This configuration allows the user to be more careful and prevent collisions before they occur.

[0020] [8] The pedestrian assistance system according to any one of [1] to [7], wherein the output unit outputs a warning on the screen of a map application when the detection unit detects an object during operation or display of a map application that provides route guidance.

[0021] With this configuration, if a user of the present invention is using a map app and an obstacle approaches, a warning can be output without interrupting the display of the map app, thereby improving usability.

[0022] [9] A pedestrian assistance method using a computer having an acquisition unit, a detection unit, and an output unit, wherein the acquisition unit is equipped with a distance measurement sensor and acquires information related to the shape of an object using the distance measurement sensor, the detection unit excludes from detection any object whose shape is acquired by the acquisition unit and whose outline is partially in contact with the boundary of a detection area, and detects as an obstacle any object whose entire outline is within the detection area, and the output unit outputs a warning when the detection unit detects an obstacle.

[0023]

[10] A pedestrian assistance program that causes a computer to execute the pedestrian assistance method described in [9]. [Effects of the Invention]

[0024] According to the present invention, it is possible to provide a novel technique that allows a user to be aware of the presence of an obstacle and prevent a collision before it occurs. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 1 is a configuration diagram of an embodiment of the present invention. [Figure 2] Hardware configuration diagram. [Figure 3] 3 is a schematic diagram of the recognition range of the acquisition unit 11. [Figure 4] An example of masking processing. [Figure 5] An overview of how obstacles and ground irregularities are recognized. [Figure 6] Example warning output: [Figure 7] 3 is a flowchart of the support system according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0026] The present embodiment relates to a technology for assisting pedestrians, and in particular to a technology for recognizing the shapes of objects present around a pedestrian and outputting a warning to avoid a collision between the pedestrian and the obstacle.

[0027] In this invention, an object refers to a solid object that exists in the space around a pedestrian using this system and is large enough to be recognized by the sensor. For example, this includes objects larger than a person, such as utility poles or cars, luggage such as umbrellas or bags, and living things such as dogs or people.

[0028] In the present invention, an obstacle refers to an object that may hinder safe walking, including, for example, a pole installed on the road, a moving car, a chain hanging to indicate a site, etc.

[0029] In addition to obstacles, the present invention also recognizes road surfaces and walls that hinder walking and outputs a warning. Road surfaces and walls that hinder walking refer to ground conditions or the presence of walls that make safe walking difficult. Examples include road surfaces such as uphill and downhill steps, slopes, and walls that come too close due to a decrease in attention.

[0030] In the present invention, a warning refers to a notification to a user when an obstacle is detected, informing the user of the approach of the obstacle or urging the user to avoid the obstacle. Specifically, this includes, for example, displaying a warning on the operation screen of a device equipped with the present invention, as well as using a function used for notification such as an alarm sound or vibration.

[0031] <1. System configuration> The present invention will now be described in more detail with reference to the accompanying drawings, in which preferred embodiments are shown, but which may be embodied in many different forms and are not limited to the embodiments set forth herein.

[0032] For example, in this embodiment, the configuration, operation, etc. of the pedestrian assistance system will be described, but similar effects can be achieved by a device having similar functions, a method executed by the device, a computer program that causes a computer device to execute the method, etc. The program may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server.

[0033] In the following embodiments, the term "unit" may include, for example, a combination of hardware resources implemented by a broadly defined circuit and software information processing that can be specifically realized by these hardware resources. In this embodiment, "information" is represented by, for example, the physical value of a signal value representing voltage or current, the high or low value of a signal value as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculation can be performed on a broadly defined circuit.

[0034] A circuit in the broad sense is a circuit realized by appropriately combining a circuit, a processor, a memory, etc. For example, it is a circuit including any of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), etc.

[0035] FIG. 1 is a diagram showing the configuration of a pedestrian support system 1 according to this embodiment. The pedestrian support system 1 according to this embodiment includes a support device 2, which is configured to be able to communicate with a detection unit 3 via a network NW. In this embodiment, the network NW is an IP (Internet Protocol) network, but there are no limitations on the type of communication protocol or type of network. Note that the detection unit 3 may also be configured as part of the configuration of the support device 2 without going through the network NW.

[0036] The support device 2 acquires information relating to the shape and distance of an object through the acquisition unit 11, and outputs a warning through the output unit 12.

[0037] The detection unit 3 determines whether the detected object is an obstacle that is a detection target or an object that is not a detection target, based on information related to the shape of the object output from the acquisition unit 11, which will be described later. If the detection unit 3 determines that an obstacle exists, it outputs a detection result indicating that an obstacle exists to the output unit 12.

[0038] The acquisition unit 11 is equipped with a distance measurement sensor and a gyro sensor. The acquisition unit 11 uses the distance measurement sensor to acquire information on the shape and distance of an obstacle present within the recognition range, and then outputs the acquired information on the shape and distance of the obstacle to the detection unit 3. The gyro sensor provided in the acquisition unit 11 acquires the traveling direction and the facing angle of the support device 2, and outputs the information to the detection unit 3 or the output unit 12. The gyro sensor may be a geomagnetic sensor, an optical sensor, or the like. The distance measurement sensor is preferably a LiDAR sensor, but may also be a camera sensor, a radar sensor, an infrared sensor, or other distance measurement sensor.

[0039] The recognition range is the range in which the distance measurement sensor mounted on the acquisition unit 11 acquires point cloud data. The detection area is the range in which objects, including obstacles, are detected within the recognition range based on information related to the object's contour and distance. A variable masking area, which will be described later, may be set within the detection area. In particular, in the explanations related to Figures 3 and 4, a variable masking area is set within the detection area. Unless otherwise specified, a variable masking area is not set, and in other figures and their explanations, the entire recognition range of the acquisition unit 11 is assumed to overlap with the detection area.

[0040] The output unit 12 outputs a warning based on the detection result output from the detection unit 3. The output unit 12 may be equipped with a plurality of warning output methods. The warning output method means that when notifying the user of the presence of an obstacle, different warnings are output according to a certain scale. In this embodiment, the distance from the acquisition unit 11 is described as the scale for the warning output method, but this is not limiting and other scales may also be used.

[0041] The output unit 12 may be configured to output different warnings according to the distance between the acquisition unit 11 and an obstacle. Specifically, the detection unit 3 determines specific conditions based on the information relating to the distance to the object output from the acquisition unit 11, such as outputting an alarm if the distance is within a threshold and outputting a caution if the distance is greater than the threshold, and outputs the corresponding detection result. The output unit 12 may be configured to output a corresponding warning based on the output of the detection unit 3. In this embodiment, the detection unit 3 determines the different warning output methods based on the detection result. However, the present invention is not limited to this, and the output unit 12 may determine the warning output method based on the distance information output from the acquisition unit 11, or a determination unit that determines the warning output method separately from the detection unit 3 may be added as a component.

[0042] Furthermore, the output unit 12 may have a function to output a safety confirmation message at regular intervals, regardless of the detection results of the acquisition unit 11 and the detection unit 3, to allow the user to check for the presence of obstacles in the surrounding area.

[0043] "Safety confirmation" means ensuring the safety of walking by the user himself / herself without using the present invention, and "outputting safety confirmation" means notifying the user that a safety confirmation will be performed. Specifically, this can include outputting a message on the screen informing the user to grasp the surrounding situation.

[0044] The communication unit 13 communicates with the detection unit 3 via the network NW. The communication unit 13 may also be a part that is responsible for linking with other applications such as a map application that provides route guidance.

[0045] The memory unit 14 stores information related to the shape and distance of a detected object. It also stores threshold values ​​used by the detection unit 3 to determine the warning output method, default values ​​used when detecting ground level differences and undulations, and / or icon images used in warnings by the output unit 12, and notifications corresponding to multiple warning output methods. The memory unit 14 may be connected to the outside of the assistance device 2 or may be connected via a network NW. The threshold values ​​and warning output methods stored in the memory unit 14 may be changed by the user to any value and any format. The warning output method stored in the memory unit 14 may be set to any audio file, image file, vibration pattern, etc.

[0046] <2. Hardware configuration> Next, the hardware configuration of the pedestrian assistance system 1 in this embodiment will be described with reference to Fig. 2. A terminal device 9 (computer device) such as a personal computer, smartphone, or tablet terminal can be used as the assistance device 2. In this embodiment, the detection unit 3 is an information processing device 10 in which a computer program (pedestrian assistance program) that executes the pedestrian assistance method is installed. Furthermore, one or more information processing devices 10 (computer devices) such as a general-purpose server or personal computer can be used as the detection unit 3.

[0047] Fig. 2(a) is a hardware configuration diagram of the information processing device 10. As shown in Fig. 2, the information processing device 10 has a communication unit 101, a control unit 102, and a storage unit 103, which are used to perform the functions of each unit and each process.

[0048] The control unit 102 has a processor such as a CPU capable of executing an instruction set, and executes an OS and programs. The storage unit 103 has a volatile memory such as a RAM capable of storing an instruction set, and a non-volatile recording medium such as an HDD or SSD capable of recording the OS, a pedestrian assistance program, a DBMS, etc. The communication unit 101 has an interface for physically connecting to the network NW, and controls communication with the network NW to input and output information.

[0049] Fig. 2(b) is a hardware configuration diagram of the terminal device 9. As shown in Fig. 2, the terminal device 9 has a communication unit 901, a storage unit 902, an input unit 903, an output unit 904, a control unit 905, and a sensor unit 906, which are used to perform the functions of each unit and each process.

[0050] The control unit 905 has a processor such as a CPU capable of executing an instruction set, and executes an OS, application programs, etc. The storage unit 902 has a volatile memory such as a RAM capable of storing an instruction set, and a non-volatile recording medium such as an HDD or SSD capable of recording the OS, any application programs, etc.

[0051] The communication unit 901 has an interface for physically connecting to the network NW and controls communication with the network NW to input and output information. The input unit 903 has an operation input device capable of input processing such as a touch panel or keyboard, an audio input device capable of audio input such as a microphone, etc. The output unit 904 has a display device capable of display processing such as a display, and an audio output device such as a speaker.

[0052] The sensor unit 906 has a distance measurement sensor for acquiring information relating to the distance and shape of surrounding objects, and a gyro sensor for detecting the orientation of the terminal device 9 and the direction in which the user is moving.

[0053] <3. Functional configuration> The pedestrian assistance system 1 of this embodiment includes an assistance device 2 and a detection unit 3. The assistance device 2 includes an acquisition unit 11, an output unit 12, a communication unit 13, and a storage unit 14, and is connected to the detection unit 3 via a network NW.

[0054] The acquisition unit 11 has a distance measurement sensor and uses the distance measurement sensor to acquire point cloud data within the recognition range. The acquisition unit 11 acquires information related to the shape and distance of objects such as obstacles based on the acquired point cloud data, and outputs the information to the detection unit 3. In this embodiment, a LiDAR sensor is used as the distance measurement sensor, so point cloud data is acquired, but the format of the acquired data may differ depending on the type of distance measurement sensor.

[0055] FIG. 3 uses coordinate axes to illustrate the recognition range of the distance measurement sensor provided in the acquisition unit 11. The Z axis represents the vertical direction passing through the support device T1 equipped with the acquisition unit 11, and the Y axis represents the horizontal direction of the support device T1 on the ground. The origin is the intersection of the Z axis and the ground (Y axis). These Z axis and Y axis always exist on a vertical line passing through the support device T1 and on a horizontal plane passing through the ground. The Z axis and Y axis do not change with changes in the tilt of the support device T1 relative to the vertical direction. Only changes in the direction of the support device T1 relative to the horizontal plane are reflected, and the positive and negative directions of the Y axis change only on the horizontal plane. The direction in which the surface of the distance measurement sensor equipped in the support device T1 that has the laser light emission function faces is defined as the positive direction on the Y axis, i.e., forward, and the direction symmetrical with respect to the Z axis to the direction in which the surface of the distance measurement sensor equipped in the support device T1 that has the laser light emission function faces is defined as the negative direction on the Y axis, i.e., backward. The combined range of range A2 (gray area) and range A3 (white area) is the detection area of ​​the distance measurement sensor provided in the support device T1. Distance W1 represents the recognition limit of the acquisition unit 11 in the Y-axis direction. The dotted lines extending radially from the support device T1 represent measurement means such as infrared rays or laser light emitted from the distance measurement sensor.

[0056] A detection area is set within the recognition range, and a variable masking area may be set within the detection area. Range A3 indicates the range within the detection area excluding the variable masking area, and range A2 indicates the variable masking area within the detection area. The acquisition unit 11 performs object detection in the detection area that combines ranges A2 and A3. The variable masking area is a range of objects detected within the variable masking area that are determined to be outside the detection target range based on information related to their contours using a procedure described below. In this embodiment, the variable masking area is automatically set within a certain range from the bottom end of the detection area. In the range A2, it is preferable that the variable masking area is set in the positive direction on the Y axis from the range of Y=0 (line segment L1 in FIG. 4(a)), with the range of Y=0 being 0°, and the range from 0° to 30° with the support device T1 as the vertex. The variable masking area may be set within any range or position by the user. Furthermore, when an object is reflected within the variable masking area, the determination procedure uses information related to the shape of the outline, which will be described later, to determine whether the object is a detection target or not.

[0057] Information on obstacles detected in the detection area and changes in the shape and distance of the obstacles as pedestrians move are acquired by the acquisition unit 11 and continuously stored in the storage unit 14. Even if an object determined to be an obstacle in the detection area moves in a direction outside the recognition range or into the variable masking area, a warning is output based on the information on the object stored in the storage unit 14.

[0058] Furthermore, a fixed masking area may be set in the negative direction on the Y axis on the coordinate system in FIG. 3, i.e., in the range A1 behind the support device T1. The fixed masking area is a range in which all objects detected to be contained within the fixed masking area are automatically excluded from detection. Specifically, the support device 2 uses a gyro sensor installed inside to detect the tilt and forward / backward direction of the support device T1, and automatically sets the range to be the fixed masking area. Furthermore, the support device 2 may automatically correct the range of the fixed masking area according to the tilt and facing direction of the support device T1.

[0059] Figures 4(a) and (b) show an example of masking processing. Figure 4 shows data acquired using a camera sensor, but in reality, object recognition is performed using point cloud data.

[0060] Next, the procedure for determining an object that is not a detection target within the variable masking area will be explained using Fig. 4(a). In Fig. 4(a), a variable masking area (shaded area) is set.

[0061] In Fig. 4(a), range S1 represents the entire detection area of ​​the acquisition unit 11, range A11 represents the range of the detection area excluding the masking area (range A3 in Fig. 3), range A10 represents a variable masking area set within the detection area (range A2 in Fig. 3), object T10, object T11, object T12, object T13, and object T14 represent objects acquired by the acquisition unit 11, and line segment L1 represents the area directly below the assistance device T1 in Fig. 3, that is, the range where the value on the Y axis is 0. Furthermore, the dotted lines around object T10, object T11, object T12, object T13, and object T14 represent a schematic representation of information related to the contours.

[0062] The line segment L1 is located behind the objects T10, T11, T12, T13, and T14, and at least a portion of each of these objects is present within the range S1. The acquisition unit 11 acquires information relating to the shapes of the objects T10, T11, T12, T13, and T14 present within the detection area, and outputs the information to the detection unit 3.

[0063] The detection unit 3 determines whether an obstacle exists based on the range within the detection area and outside the variable masking area. Specifically, the object T10 is located in a range A11 of the detection area that is outside the variable masking area. In this case, the object T10 is determined to be an obstacle, and the detection result for the object T10 is output to the output unit 12.

[0064] The detection unit 3 determines that an object is not a target for detection based on the fact that the contour of an object located within the variable masking area does not form a closed shape within the variable masking area. Specifically, object T11 and object T12 share part of their contour with the boundary of range S1 within the variable masking area A10, and therefore their contours do not form a closed shape within the variable masking area. In such a case, the object is determined to be an object that is within the user's awareness and is not a target for detection. Thereafter, the detection results for object T11 and object T12 are not output to the output unit 12. When determining that an object is not a target for detection based on contour information, it is particularly preferable to determine that an object is not a target for detection based on the fact that part of the object's contour is shared with the boundary at the bottom end of the detection area (range S1).

[0065] Furthermore, when the contour of an object detected within the detection area but outside the variable masking area is shared with the boundary of the detection area, and when the contour of an object present within the variable masking area is a closed shape, the detection unit 3 determines the object to be an obstacle and outputs information related to the object to the output unit 12. Specifically, object T13 is located in range A11, which is within the detection area but outside the variable masking area, and its outline is shared with the boundary of range S1. Object T14 is located within the variable masking area, but its outline is a closed shape. These objects are not in the user's consciousness and are therefore determined to be obstacles.

[0066] Furthermore, if a part of the target object crosses the boundary of the range S1 across the range A10 and falls within the range A11, it is considered to be an obstacle.

[0067] In the present embodiment, the procedure for determining whether an object is a non-detection target has been described using only information related to the contour, but the non-detection target may be determined based on information related to the contour and the distance. Specifically, if a part of the contour of an object detected within range A10 is shared with the boundary of range S1 and the distance from the detected object to acquisition unit 11 is within a set threshold, the object may be determined to be a non-detection target.

[0068] Once an object is determined to be an obstacle, part of its contour may share the boundary of range S1 as the user faces or moves, and even if the contour does not form a closed shape within range A10, the object is determined to be an obstacle based on the continuous data within the detection area related to the object stored in memory unit 14, and a warning is output.

[0069] Furthermore, an object whose outline does not form a closed shape within the detection area of ​​the acquisition unit 11 may be determined to be an obstacle based on information related to the detected distance. Specifically, for an object larger than the recognition range acquired by the acquisition unit 11, such as a utility pole or a fence, the detection unit 3 may be configured to determine and detect the object as an obstacle if the distance between the closest part of the object and the acquisition unit 11 is equal to or greater than a reference value based on information related to the distance to the acquisition unit 11 that is initially detected.

[0070] Next, the detection of an object in a fixed masking area will be explained using Fig. 4(b), in which a variable masking area (shaded area) is set.

[0071] 4(b), range S2 represents the entire detection area of ​​the acquisition unit 11, range A12 represents the variable masking area, range A13 represents the detection area excluding the variable masking area, object T15 and object T16 represent the objects acquired by the acquisition unit 11, and line segment L2 represents the area directly below the assistance device T1 in FIG. 3, that is, the area where the value on the Y axis is 0. The dotted lines around objects T15 and T16 are a schematic representation of information related to their contours.

[0072] The acquisition unit 11 acquires information on the shapes and contours of the objects T15 and T16 present within the recognition range, and outputs the information to the detection unit 3.

[0073] Line segment L2 is located in front of object T15. Therefore, the range of the contour of object T15 does not share any points with line segment L2, and the Y coordinate is completely included in the negative area. In this case, detection unit 3 determines that object T15 is located behind the user (behind acquisition unit 11) and is therefore not an object to be detected, and does not output information related to object T15 to output unit 12.

[0074] Object T16 is positioned so as to straddle the front and rear of line segment L2. A portion of object T16 is present within the variable masking area, and is not contained within the fixed masking area. In this case, object T16 is not considered to be behind the acquisition unit 11, and the detection unit 3 makes a determination based on information related to the outline within the variable masking area. In the case of FIG. 4(b), object T16 is determined to be an object outside the detection target because its outline does not form a closed shape within the variable masking area. In addition, among objects that are determined to be obstacles and whose shape and position changes associated with continuous movement are stored, if the contour shares at least one point with line segment L2, a warning is output even if the contour shape is not a closed shape within range A12.

[0075] In addition to the embodiment described in Figures 4(a) and (b), an embodiment in which a variable masking area is not set for obstacle determination may also be adopted. When a variable masking area is not set, an object that is not a detection target is determined based on whether the boundary of the detection area and the outline of the object are shared. Specifically, if the outline of the object is shared with the boundary of the detection area, it is determined to be an object that is not a detection target, and if the entire outline of the object is present within the detection area, it is determined to be an obstacle. Furthermore, the boundary of the detection area for determining that an object is not a detection target may be limited to only the upper end or only the lower end. In particular, it is preferable to determine that an object is not a detection target based on whether the boundary of the detection area shares part of the object's outline.

[0076] Next, the procedure for determining road surfaces that hinder walking will be described using Figures 5(a) to (d). Support device T100, support device T101, support device T102, and support device T103 (hereinafter referred to as the support device group) represent devices equipped with an acquisition unit 11. The thick solid line represents the ground, and the dotted lines labeled Y and Z represent coordinate axes. Figures 5(a) to (d) schematically illustrate the measurement points of the acquisition unit 11 using coordinate axes. The Z axis represents the vertical direction passing through the support device group equipped with the acquisition unit 11, and the Y axis represents the horizontal direction of the support device group on the ground. These Z axis and Y axis always exist on a vertical line passing through the support device group and on a horizontal plane passing through the ground, and do not change with changes in the tilt of the support device group relative to the vertical direction. The direction of the Y axis only changes on the horizontal plane, reflecting only changes in the orientation of the support device group relative to the horizontal plane. The direction in which the surface of the distance measuring sensor equipped in the support device group, which has the laser light emission function, faces is defined as the positive direction on the Y axis, i.e., forward, and the direction symmetrical with respect to the Z axis to the direction in which the surface of the distance measuring sensor equipped in the support device group, which has the laser light emission function, faces is defined as the negative direction on the Y axis, i.e., backward. The intersection of the Z axis and the ground (Y axis) is defined as the origin.

[0077] 5(a) is a diagram illustrating a method for recognizing an up-step. The acquisition unit 11 acquires information relating to the distance between the measurement points P1 and P2 from the support device T100, and outputs the information to the detection unit 3.

[0078] The detection unit 3 calculates the difference between the measurement points on the vertical line from the distance information obtained by the distance measurement sensor and the direction information obtained by the gyro sensor provided in the acquisition unit 11, and determines that the step is an uphill step if the difference between the measurement points on the vertical line is a positive value equal to or greater than a specified value. Specifically, distance H1 represents the distance on the Z axis from the Y axis to measurement point P1, distance H2 represents the distance on the Z axis from the Y axis to measurement point P2, and distance W10 represents the distance on the Y axis between the two points. The detection unit 3 determines that the step is an uphill step if the difference between distance H1 and distance H2 is a positive value equal to or greater than a specified value, and outputs the detection result to the output unit 12. It is preferable that distance W10 be an arbitrary fixed value. It is also preferable that the measurement points are measured at intervals of 5 mm to 10 mm.

[0079] 5(b) is a diagram illustrating a method for recognizing a down-step. The acquisition unit 11 acquires information relating to the distance between the measurement point P3 and the measurement point P4 from the support device T101, and outputs the information to the detection unit 3.

[0080] The detection unit 3 calculates the difference between the measurement points on the vertical line from the distance information obtained by the distance measurement sensor and the direction information obtained by the gyro sensor provided in the acquisition unit 11, and determines that the step is a downhill step when the difference between the measurement points on the vertical line is a negative value equal to or greater than a specified value. Specifically, distance H3 represents the distance on the Z axis from the Y axis to measurement point P3, distance H4 represents the distance on the Z axis from the Y axis to measurement point P4, and distance W11 represents the distance on the Y axis between the two points. The detection unit 3 determines that the step is a downhill step when the difference between distance H3 and distance H4 is a negative value equal to or greater than a specified value, and outputs the detection result to the output unit 12. It is preferable that distance W11 be an arbitrary fixed value. It is also preferable that each measurement point be measured at a pitch of 5 mm to 10 mm.

[0081] 5(c) is a schematic diagram of a method for recognizing objects such as poles protruding from the ground or chains suspended between poles. The acquisition unit 11 acquires information relating to the distance between measurement points P5 and P6 from the support device T102 and outputs the information to the detection unit 3.

[0082] The detection unit 3 calculates the difference in the vertical line between the measurement points, similar to the case of the ascending steps, from the distance information obtained by the distance measurement sensor and the direction information obtained by the gyro sensor in the acquisition unit 11. If the difference is a positive value equal to or greater than a specified value, the detection unit 3 determines that the object is a protruding object or a suspended object from the ground. Specifically, if the difference between H5 and H6, which are the distances between the Y axis and the Z axis of measurement points P5 and P6, is a positive value equal to or greater than a specified value, the detection unit 3 determines the presence of a protruding object or a suspended obstacle and outputs the detection result to the output unit 12. The detection result may be combined with information related to the contour to determine whether the object is an ascending step or an obstacle, and output to the output unit 12. Alternatively, a specified value different from that for the ascending steps may be set for the determination. Furthermore, the distance W12 represents the distance on the Y axis between the two points, and is preferably a fixed value. It is preferable that each measurement point be measured at intervals of 5 mm to 10 mm.

[0083] 5(d) is a schematic diagram of a method for recognizing uphill and downhill roads and slopes. The acquisition unit 11 acquires information relating to the distance between measurement points P7 and P8 from the assistance device T103 and outputs the information to the detection unit 3.

[0084] The detection unit 3 calculates the difference in the vertical line between the measurement points, similar to the up and down steps, from the distance information obtained by the distance sensor and the direction information obtained by the gyro sensor in the acquisition unit 11. If the difference is a positive or negative value greater than or equal to a specified value, the detection unit 3 determines that the slope is an up or down slope. Specifically, the detection unit 3 determines that the slope is an up slope if the difference between H7 and H8, which are the distances between the Y axis and the Z axis of measurement points P5 and P6, is a positive value greater than or equal to a specified value. If the difference between H7 and H8 is a negative value greater than or equal to a specified value, the detection unit 3 determines that the slope is a down slope. The detection result is output to the output unit 12. The output unit 12 may output a warning different from the warning for the up and down steps based on the detection result. The distance W13 represents the distance on the Y axis between the two points and is preferably a fixed value. When based on the Building Standards Act, the specified value used for the judgment is preferably a specified value that can detect a gradient of 1 / 8 or less when calculating the quotient of the distance on the Z axis between the two measurement points and the Y axis (the difference between distance H7 and distance H8 in Figure 5(d)) and the distance on the Y axis between the two measurement points (distance W13 in Figure 5(d)).When based on the Barrier-Free Act, the specified value that can detect a gradient of 1 / 12 or less or 1 / 15 or less may be set.

[0085] In addition, to clearly distinguish between ascending and descending steps, three or more measurement points may be set and the step or slope may be determined based on the positional relationship between the measurement points. Specifically, the amount of change on the Z axis between adjacent measurement points may be calculated, and if the absolute value of the amount of change is equal to or greater than a specified value, it may be determined to be an ascending or descending step, and if it is less than the specified value, it may be determined to be an ascending or descending slope.

[0086] Each measurement point shown in Figures 5(a) to (d) represents the measurement part of a distance measuring sensor, but the number of measurement points for detecting road surfaces that hinder walking is not limited to two, and the condition of the road surface may be determined based on two or more measurement points.

[0087] The output unit 12 outputs a warning based on the detection result by the detection unit 3. D11 in FIG. 6 is an example of a warning output. When outputting a warning using an icon image or the like, a mark like D11 may be used. Furthermore, a direction to avoid may be output using a symbol such as an arrow based on the direction of the obstacle and the user's traveling direction. Specifically, the distance measurement sensor and gyro sensor included in the acquisition unit 11 detect the user's traveling direction and the presence or approaching direction of an obstacle, and output information related to the user's traveling direction, the approaching direction of the obstacle, etc. to the detection unit 3. The detection unit 3 determines the direction to avoid for the user, and outputs the detection result to the output unit 12. The output unit 12 outputs the direction to avoid in the form of an arrow or the like based on the detection result output from the detection unit 3.

[0088] The acquisition unit 11 can acquire information on multiple types of road surfaces that hinder walking by using the detection means for detecting road surfaces that hinder walking shown in Figures 5(a) to 5(d) in parallel. The output unit 12 may be configured to output different warnings for the multiple types of road surfaces that hinder walking determined by the detection unit 3. Specifically, if there is an uphill slope and a pole installed on the slope, the output unit 12 may be configured to simultaneously output two types of warnings: a warning related to the uphill slope and a warning related to the pole on the slope.

[0089] 5(a) to 5(d), the detection unit 3 does not make a determination based on the shape of the outline described above. Specifically, the road surface condition is detected by distinguishing it from an object based on the distance between the road surface and the acquisition unit 11 and the direction of entry into the variable masking area. Therefore, with regard to the detection of the road surface condition, an obstacle detection determination based on the shape of the outline within the variable masking area is not made.

[0090] The output unit 12 outputs a warning on the display of another application even if the other application is in use. Specifically, if an obstacle is detected while checking the current location with a map application or the like, a warning related to the detected obstacle may be output on the screen without interrupting the display of the map application. Also, in a map application having a function such as route guidance, a warning may be output on the operation screen of the map application when an obstacle is detected on or around the route.

[0091] Next, the processing procedure from object detection to warning output in this embodiment will be described with reference to the flowchart in FIG.

[0092] The acquisition unit 11 acquires point cloud data within the recognition range using a distance measurement sensor installed therein (S100). In this embodiment, it has been described that point cloud data is acquired because it is preferable to use a LiDAR sensor, but this is not limiting, and different data acquisition methods may be used depending on the distance measurement sensor, such as acquiring information related to objects within the recognition range based on parallax between images taken with multiple cameras.

[0093] The acquisition unit 11 acquires information relating to the orientation of the user's support device 2 and point cloud data relating to the shape and distance of an object within the detection area, and outputs the information to the detection unit 3 (S101).

[0094] The detection unit 3 determines whether the object acquired by the acquisition unit 11 is located behind the support device 2, based on the information on the orientation of the support device 2 and the point cloud data on the shape of the object (S102).

[0095] If the object is located behind the support device 2 (Yes in S102), the detection unit 3 determines that the object is not a detection target, and does not output to the output unit 12 (S103).

[0096] If the object is located in front of the support device 2 (No in S102), the detection unit 3 determines whether at least a part of the contour is shared with the boundary of the detection area based on the information regarding the shape of the object acquired by the acquisition unit 11 (S104).

[0097] If part of the contour of the detected object is shared with the boundary of the detection area (Yes in S104), the detection unit 3 determines that the object is not a detection target and does not output it to the output unit 12 (S103).

[0098] If the contour of the detected object does not share the boundary of the detection area (No in S104), the detection unit 3 determines that the object is an obstacle.

[0099] The detection unit 3 compares the information relating to the distance to the obstacle acquired by the acquisition unit 11 with a preset threshold relating to the warning output method, and determines the warning output format (S105).

[0100] If the value is equal to or less than the threshold value (Yes in S105), the detection unit 3 outputs the detection result to the output unit 12, and the output unit 12 outputs a warning regarding the alarm (S106).

[0101] If the value is greater than the threshold value (No in S105), the detection unit 3 outputs the detection result to the output unit 12, and the output unit 12 outputs a warning regarding caution (S107). [Explanation of symbols]

[0102] 1: Pedestrian assistance system 2: Support equipment 3: Detection section 9: Terminal device 10: Information processing device 11: Acquisition part 12: Output section 13: Communications Department 14: Storage section 101: Communications Department 102: Control unit 103: Storage section 901: Communications Department 902: Storage section 903: Input section 904: Output section 905: Control unit A1: Range A2: Range A3: Range A10: Range A11: Range A12: Range A13: Range D11: Warning display H1: distance H2 :Distance H3: distance H4: Distance H5: distance H6 :Distance H7: distance H8 :Distance L1: Line segment L2: Line segment P1:Measurement point P2:Measurement point P3: Measurement point P4:Measurement point P5:Measurement point P6:Measurement point P7: Measurement point P8:Measurement point S1: Range S2: Range T1: Support device T10 :Object T11:Object T12 :Object T13 :Object T14:Object T15 :Object T16 :Object T100: Support equipment T101: Support equipment T102: Support equipment T103: Support equipment W1: distance W10: distance W11 :Distance W12 :Distance W13: Distance NW: Network

Claims

1. A pedestrian assistance system having an acquisition unit, a detection unit, and an output unit, the acquisition unit is equipped with a distance measurement sensor and acquires information relating to the shape of an object by the distance measurement sensor; the detection unit excludes an object whose contour is partly shared with the boundary of the detection area from the shape of the object acquired by the acquisition unit, and detects an object whose contour is entirely within the detection area from the shape of the object acquired by the acquisition unit as an obstacle; The output unit outputs a warning when the detection unit detects an obstacle.

2. The acquisition unit has a variable masking area set within a detection area, The pedestrian assistance system according to claim 1 , wherein the detection unit excludes an object from detection when a part of the contour of the detected object is shared with a boundary of the detection area within the variable masking area.

3. The pedestrian assistance system according to claim 2 , wherein the variable masking area is set at a lower end of a detection area.

4. The pedestrian assistance system according to claim 1 , wherein the detection unit excludes an object located behind the acquisition unit from the detection target.

5. The pedestrian support system according to claim 1 , wherein the detection unit detects a road surface or a wall surface that obstructs walking from a positional relationship between a plurality of different measurement points.

6. the acquisition unit acquires information relating to a distance from the acquisition unit, The pedestrian assistance system according to claim 1 , wherein the output unit outputs a different warning depending on the distance information obtained by the acquisition unit.

7. The pedestrian assistance system according to claim 1 , wherein the output unit outputs a safety confirmation signal at regular intervals regardless of the detection result of the detection unit.

8. 2. The pedestrian assistance system according to claim 1, wherein the output unit outputs a warning on a screen of a map application that provides route guidance when the detection unit detects an object during operation or display of the map application.

9. A pedestrian assistance method using a computer having an acquisition unit, a detection unit, and an output unit, the acquisition unit is equipped with a distance measurement sensor and acquires information relating to the shape of an object by the distance measurement sensor; the detection unit excludes an object whose contour is partly shared with the boundary of the detection area from the shapes of the objects acquired by the acquisition unit, and detects an object whose entire contour is within the detection area as an obstacle, The output unit outputs a warning when the detection unit detects an obstacle.

10. A pedestrian assistance program that causes a computer to execute the pedestrian assistance method according to claim 9.

Images