Pedestrian detection device and pedestrian detection method

Abstract

Pedestrian detection apparatus (100, 150) analyzing a captured image taken by a vehicle-mounted camera (2) and thereby detecting a pedestrian appearing in the captured image, the pedestrian detection apparatus (100, 150) comprising: a roadway detection unit (103) that detects a roadway in the captured image, a target pedestrian detection unit (104) that detects a pedestrian present within a specified distance in both lateral directions from the roadway in the captured image as a target pedestrian, a gaze direction acquisition unit (105) that obtains gaze direction information from the target pedestrian, a gaze determination unit (106) that determines that the target pedestrian has gazed towards a crossing destination present on the other side of the roadway if a condition is met in which the gaze direction of the target pedestrian is in a direction of the crossing destination,which is present on the other side of the roadway, is fixed, lasts for a predetermined viewing time, a safety confirmation determination unit (107) which determines, based on the gaze direction of the target pedestrian, whether the target pedestrian has performed a safety confirmation with regard to vehicles traveling on the roadway, and a pre-crossing pedestrian detection unit (108) which detects the target pedestrian, who has performed one of the actions, looking at the crossing destination present on the other side of the roadway, and performing a safety confirmation, and then has performed the other action within a predetermined period, as a pre-crossing pedestrian who has an intention to cross the roadway.

Description

Cross-references to related registrations

[0001] The present application is based on Japanese patent application No. 2016-135404, filed on July 7, 2016, and claims priority therefrom; the descriptions thereof are incorporated herein by reference. Technical field

[0002] The present invention relates to a technology for detecting a pedestrian who is present in the periphery or vicinity of a vehicle. The present invention relates in particular to a technology for detecting a pedestrian who is attempting to cross a roadway at a stage before the pedestrian actually begins to cross. State of the art

[0003] As a result of advances in image recognition technology, a pedestrian in the vicinity of a vehicle can now be detected with sufficient accuracy from an image captured by a vehicle-mounted camera. The next development goal is to determine with sufficient accuracy whether the detected pedestrian is simply walking on a sidewalk or attempting to step off the sidewalk and cross a roadway.

[0004] In response to such a demand, a technology for determining whether a pedestrian is attempting to cross a roadway was proposed by capturing time-series variations of the pedestrian's position and time-series changes in the pedestrian's speed (PTL 1). This proposed technology determines whether a pedestrian is attempting to cross the roadway based on whether the captured time-series variations correspond to a pattern of time variations when the pedestrian attempts to cross. Citation list for patent literature

[0005] [PTL 1] JP-A-2010-102437 Summary of the invention

[0006] In the proposed technology described above, the movement of a pedestrian crossing the roadway is detected. Consequently, the pedestrian has already begun crossing at the time of detection. It is not possible to detect a pedestrian who is attempting to cross at a stage before the pedestrian has begun crossing.

[0007] It is an object of the present invention to provide a technology that makes it possible to accurately detect a pedestrian attempting to cross a roadway at a stage before the pedestrian starts to cross.

[0008] According to a first aspect of the present invention, a pedestrian detection apparatus and a pedestrian detection method detect a target pedestrian who is present within a predetermined distance from a roadway in an image captured by a vehicle-mounted camera. Based on the target pedestrian's gaze direction, it is then determined whether the target pedestrian has looked towards a crossing destination or not, and whether the target pedestrian has performed a safety check regarding vehicles traveling on the roadway or not. If the target pedestrian has performed one of the actions—looking towards the crossing destination or performing a safety check—and then performed the other action within a predetermined time period, the target pedestrian is detected as a pedestrian intending to cross the roadway.

[0009] Based on knowledge newly discovered by the inventors of the present application, a pedestrian attempting to cross a roadway performs a glance toward a destination and a safety confirmation regarding vehicles traveling on the roadway with a probability high enough to be considered 100%. Consequently, a pedestrian attempting to cross the roadway can be accurately detected at a stage before the pedestrian actually begins crossing, if a target pedestrian who has performed a glance toward the crossing destination and the safety confirmation regarding vehicles within a given time frame is detected as the pedestrian before crossing, who has an intention to cross the roadway. List of characters Fig. Figure 1 shows an explanatory diagram of a vehicle.1 , in which a pedestrian detection device 100 as shown in one embodiment, Fig. Figure 2 shows a block diagram of an internal configuration of the pedestrian detection apparatus. 100 according to the present embodiment, Fig. Figure 3 shows a flowchart of the first half of a detection process for the pedestrian before crossing, which is carried out by the pedestrian detection device. 100 according to the present embodiment, in order to detect a pedestrian attempting to cross a roadway, Fig. Figure 4 shows a flowchart of the second half of the detection process for the pedestrian before crossing, Fig. Figure 5 shows an explanatory diagram of an example of a captured image taken by a vehicle-mounted camera. 2 is recorded Fig. Figure 6 shows an explanatory diagram of a procedure for extracting a target pedestrian from the recorded image, Fig. Figure 7A shows an explanatory diagram of a procedure for determining whether the target pedestrian is facing a crossing destination or not. Fig. Figure 7B shows an explanatory diagram of a procedure for determining whether the target pedestrian is facing a crossing destination or not. Fig. Figure 7C shows an explanatory diagram of a procedure for determining whether the target pedestrian is facing a crossing destination or not. Fig. Figure 8 shows an explanatory diagram of one aspect of a security confirmation action performed by the target pedestrian. Fig. Figure 9 shows an explanatory diagram of another aspect of a security confirmation action performed by the target pedestrian. Fig. Figure 10 shows an explanatory diagram of another aspect of a security confirmation action performed by the target pedestrian. Fig. Figure 11A shows an explanatory diagram of a basic idea for extracting a pedestrian from the target pedestrians before they cross. Fig. Figure 11B shows an explanatory diagram of the basic idea for extracting a pedestrian from the target pedestrians before they cross. Fig. Figure 11C shows an explanatory diagram of the basic idea for extracting a pedestrian before target pedestrians cross, Fig. Figure 12A shows an explanatory diagram of a procedure for extracting a pedestrian from the target pedestrians prior to crossing, Fig. Figure 12B shows an explanatory diagram of the procedure for extracting a pedestrian from the target pedestrians prior to crossing, Fig. Figure 12C shows an explanatory diagram of the procedure for extracting a pedestrian prior to crossing target pedestrians, Fig. Figure 13 shows an explanatory diagram of an example of a way in which an extraction result regarding a pedestrian is output before crossing, Fig. Figure 14 shows a block diagram of an internal configuration of a pedestrian detection device. 150 in a first modification example, Fig. Figure 15A shows an explanatory diagram of a procedure by which the detection process for the pedestrian before crossing, in a second modification example, extracts a pedestrian before crossing from the target pedestrians, Fig. Figure 15B shows an explanatory diagram of the procedure by which the pre-crossing pedestrian detection process extracts a pre-crossing pedestrian from the target pedestrians in a second modification example, and Fig. Figure 15C shows an explanatory diagram of the procedure by which the detection process for pedestrians before crossing, in a second modification example, extracts a pedestrian before crossing from the target pedestrians. Description of embodiments

[0010] Embodiments are described below to clarify the details of the present invention described above. Device configuration

[0011] Fig. Figure 1 shows an overall configuration of a vehicle. 1 , in which a pedestrian detection device 100 is installed according to a present embodiment. In addition to the pedestrian detection device. 100are, as in Fig. 1 shown, a vehicle-side camera 2 , a monitor 3 , a loudspeaker 4 , a navigation device (hereinafter referred to as a navigation device) 10 and the like in the vehicle 1 appropriate.

[0012] The vehicle-side camera 2 It captures an image of a pre-state or a front-state within a specified time interval and transmits the acquired image to the pedestrian detection device. 100 out of.

[0013] The navigation device 10 It stores map data. The navigation device 10 captures the current position of the vehicle 1 and transmits the map data regarding the surroundings of the current position to the pedestrian detection device. 100 out of.

[0014] The pedestrian detection apparatus 100 analyzes the captured image from the vehicle's camera 2is received and detects a pedestrian who appears in the recorded image. Furthermore, the pedestrian detection device determines 100 The question is whether the detected pedestrian is attempting to cross a roadway at a stage before initiating a crossing. A method for determining whether a pedestrian is attempting to cross at a stage before initiating a crossing is described in detail below. A pedestrian attempting to cross a roadway but not yet initiating a crossing is referred to below as a "pre-crossing pedestrian".

[0015] The navigation device 10 is also equipped with the pedestrian detection device 100 the present example. Consequently, information about the current position of the vehicle can be obtained. 1 and map information regarding the periphery of the navigation device10 obtained and used to determine whether a pedestrian is a pedestrian before crossing or not.

[0016] Furthermore, the pedestrian detection device 100 The detection result of a pedestrian before crossing, among the pedestrians appearing in the recorded image, is transmitted to the driver using the monitor. 3 and the loudspeaker 4 out of.

[0017] Fig. Figure 2 shows an internal configuration of the pedestrian detection apparatus. 100 according to the present embodiment. The pedestrian detection device 100 indicates how in Fig. 2 shown, an Erlang unit 101 for the captured image, a pedestrian detection unit 102 , a roadway detection unit 103 , a target pedestrian detection unit 104 , a gaze direction acquisition unit 105 , a visual identification unit 106 , a security confirmation determination unit107 , and a recording unit 108 for pedestrians before crossing.

[0018] These “units” are abstract concepts into which the internal components of the pedestrian detection apparatus are divided. 100 With a focus on functions provided for detecting a pedestrian before crossing, they are classified for the sake of simplicity among pedestrians in a captured image. This does not, therefore, indicate that the pedestrian detection device 100Physically divided into these "units," these "units" can be implemented as a computer program that is processed and executed by a central processing unit (CPU). Alternatively, these "units" can be implemented as an electronic circuit featuring large-scale integration (LSI) and memory. These "units" can also be implemented through a combination of the above.

[0019] The Erlangen unit 101 The image was taken with the vehicle's camera. 2 connected. The Erlangen unit 101 for the recorded image, a recorded image obtained from the vehicle's camera 2 in front of the vehicle 1 It captures the image in a fixed time cycle and sends the captured image to the pedestrian detection unit. 102 and the road detection unit 103out of.

[0020] The pedestrian detection unit 102 analyzes the captured image, which was generated by the Erlangen unit 101 The system receives data from the captured image and thereby detects a pedestrian appearing in the captured image. Various known methods can be used to detect a pedestrian appearing in the captured image, such as detecting a section of the image that exhibits the characteristics of a pedestrian.

[0021] The road detection unit 103 analyzes the captured image, which was generated by the Erlangen unit 101 The navigation device receives information about the captured image and thereby detects a roadway that appears in the captured image. White lines are drawn on both sides of a roadway. Consequently, a roadway can be detected by the white lines captured in the captured image. 10Additionally, it records the current position of the vehicle. 1 and also stores map information that includes positional information regarding roadways. Consequently, if a roadway is detected in the captured image, the roadway detection unit can 103 the current position of the vehicle 1 and the map information from the navigation device 10 to obtain and also record the roadway using this information.

[0022] The target pedestrian detection unit 103 Detects a target pedestrian based on the pedestrian detection results from the pedestrian detection unit 102 to be obtained, and the recording results regarding roadways that were recorded by the roadway recording unit 103to be obtained. The target pedestrian here refers to a pedestrian who meets a determination regarding whether the pedestrian is a pedestrian before crossing under the area determined by the pedestrian detection unit. 102 Whether or not a pedestrian is detected is a question. For example, if a pedestrian is in a position sufficiently far from the roadway, it is obvious, even without a determination, that the pedestrian is not attempting to cross. Conversely, with regard to a pedestrian who is within a fixed distance of the roadway, it can be determined whether the pedestrian is about to cross or not. The target pedestrian detection unit 104 It is therefore recorded based on the position information of the roadway, which is provided by the roadway detection unit. 103is detected, a pedestrian (that is, a target pedestrian), which is relevant to the determination of whether the pedestrian is a pedestrian before crossing under the area determined by the pedestrian detection unit. 102 is exposed to pedestrians who were captured.

[0023] The gaze direction acquisition unit 105 The direction of view of the target pedestrian is obtained by the target pedestrian detection unit. 104 is detected. According to the present embodiment, the direction of view is determined by the acquisition unit. 101The image obtained from the captured image is analyzed with regard to the gaze direction of the target pedestrian, and the orientation of the target pedestrian's face is recorded. The pedestrian can, of course, wear a device that captures and wirelessly transmits the gaze direction, and the gaze direction transmitted wirelessly by the pedestrian can be obtained. Alternatively, a surveillance camera and an analysis device can be set up on the side of the road. The gaze direction obtained by analyzing an image of the pedestrian can be transmitted wirelessly, and the wirelessly transmitted gaze direction can be obtained.

[0024] The gaze determination unit 106Based on the gaze direction of the target pedestrian and the position information of the roadway, it determines whether the target pedestrian was looking towards a crossing destination that is on the other side of the roadway or not.

[0025] The security confirmation determination unit 107 The system additionally determines, based on the target pedestrian's gaze direction and the roadway's position information, whether the target pedestrian has confirmed their safety regarding vehicles traveling on the roadway. A procedure for determining whether the target pedestrian has looked towards a crossing destination or not, or whether the target pedestrian has confirmed their safety, is described in detail below.

[0026] If the target pedestrian has performed an action of looking towards the crossing destination and an action of confirming safety regarding vehicles within a specified time period, the detection unit records 108For the pedestrian before crossing, the target pedestrian is additionally identified as a pedestrian before crossing. Although details are described below, it is known, based on new knowledge discovered by the inventors of the present application, that a pedestrian attempting to cross a roadway performs the action of looking towards the crossing destination and the action of confirming safety regarding vehicles before crossing. If a target pedestrian who has performed these two actions within a given time period is detected as the pedestrian before crossing, then a pedestrian attempting to cross the roadway at a stage before the pedestrian begins to cross can consequently be accurately detected. The detection unit 108For the pedestrian before crossing, the result of the detection of the pedestrian before crossing, which is carried out in such a manner, using the monitor, is given. 3 or the loudspeaker 4 out of. Detection process for pedestrians before crossing

[0027] Fig. 3 and Fig. Figure 4 shows detailed flowcharts of a process in which the pedestrian detection apparatus 100 According to the present embodiment, the pedestrian is detected before crossing.

[0028] In a detection process for pedestrians before crossing, the following first occurs, as in Fig. 3 and Fig. Figure 4 shows the captured image taken by the vehicle's camera. 2 is recorded, is obtained ( S100 Then, a pedestrian is detected in the recorded image by analyzing the obtained recorded image ( S101Furthermore, a roadway is captured in the recorded image ( S102 A pedestrian can be detected in the captured image, as described above, by locating a section of the image that exhibits the characteristics of a pedestrian. Additionally, a roadway can be detected in the captured image by extracting a section where the white lines present on both sides of the roadway appear, or where a step section at the roadside or similar appears.

[0029] The target pedestrian is then extracted from the pedestrians ( S103 The target pedestrian, as described above, is a pedestrian who is subject to the determination of whether or not the pedestrian is a pedestrian prior to crossing. The procedure for extracting the target pedestrian is described using a specific example.

[0030] If the captured image is, for example, an image that appears in an example in Fig. As shown in image 5, the pedestrians are captured in the recorded image. 6a in Fig. 6 shows an aspect in which eight pedestrians, pedestrians A until H , are captured from the recorded image. Next, the roadway in the recorded image is captured. In image 6b in Fig. 6 shows the roadway captured by the image, with added hatching.

[0031] Because the vehicle-side camera 2 additionally on the vehicle 1 Once fixed, the positions of the pedestrians can be determined. A until H and the roadway in the recorded image with relative positions in relation to the vehicle 1 in an actual space. Consequently, pedestrians whose distance to the roadway is within a predetermined distance are among the pedestrians.A until H , which is in the picture 6a in Fig. Figure 6 shows the target pedestrians being extracted. One reason for this is that it is considered sufficient to extract only those pedestrians who are close to the roadway as the target pedestrians, because it cannot be assumed that a pedestrian who is in a position sufficiently far from the roadway is attempting to cross it.

[0032] As a result, the seven pedestrians A until G with the exception of pedestrian H, as shown in picture 6c in Fig. 6 shown, when the target pedestrians were extracted from the eight pedestrians A to H in the recorded image.

[0033] For example, a situation may arise where it is impossible for a pedestrian to cross the roadway, such as due to a fence located in the center of the road. Alternatively, a situation may arise where it is impossible for a pedestrian to cross the roadway for reasons such as the presence of a sidewalk on one side of the roadway but none on the opposite side. Consequently, pedestrians in such a situation can be excluded from the recorded image analysis, and other pedestrians can be extracted as the target pedestrians.

[0034] Alternatively, an area of ​​a road where it is impossible for a pedestrian to cross the roadway is indicated in the map information on the navigation device. 10stored. A pedestrian who is present within such an area of ​​a road can then be excluded, even if the pedestrian is a pedestrian who is within the specified distance from the roadway, and other pedestrians can be extracted as the target pedestrians.

[0035] If the target pedestrian is extracted from the pedestrians in the captured image in the manner described above ( S103 in Fig. 3), a single target pedestrian is selected ( S104 The direction of the target pedestrian's gaze is obtained ( S105 According to the present embodiment, the direction of view is obtained by analyzing an image of a head section of the target pedestrian in the recorded image and capturing the orientation of the face.

[0036] The direction of gaze can also be determined by using a surveillance camera and an analysis device set up at the roadside to record the pedestrian's gaze and receive the wirelessly transmitted data. Alternatively, the pedestrian's gaze can be recorded by wearing a device, and the data can be received by wirelessly transmitting the recording.

[0037] Next, it is determined whether the direction of view of the target pedestrian is directed towards the crossing destination, which is on the other side of the roadway, or not ( S106The target pedestrian, as described above, is a pedestrian located in an area whose distance to the roadway is within the specified distance. Consequently, the roadway is close to the target pedestrian. This determines whether the target pedestrian is facing the other side of the roadway or not.

[0038] Fig. 7A to Fig. Figure 7C shows a method for determining whether the target pedestrian's gaze direction is directed towards the crossing destination or not. If the gaze direction (the orientation of the face according to the present embodiment) of a target pedestrian 7a for example, as in Fig. 7A shows the roadway in a state where the roadway is directly in front of the target pedestrian's body 7a If the path runs transversely, orthogonally, it can be determined that the direction of view of the target pedestrian is 7a is directed in the direction of the crossing destination.

[0039] In cases where the body alignment of a target pedestrian 7b , as in the example in Fig. 7A shows that, in addition, the roadway is also significantly rotated or tilted when the direction of view of the target pedestrian is 7b Within a range of -45 degrees to +45 degrees with respect to the direction orthogonal to the roadway, it can be determined that the direction of view of the target pedestrian is 7b is directed in the direction of the crossing destination.

[0040] Even in cases where the body alignment of a target pedestrian 7c , as in the example in Fig. 7A shows that the direction of view of the target pedestrian is not rotated in relation to the roadway. 7b If the direction of view of the target pedestrian is not within the range of -45 degrees to +45 degrees with respect to the direction orthogonal to the roadway, it can be determined, in contrast, that the direction of view of the target pedestrian is 7cis not directed in the direction of the crossing destination.

[0041] If, in addition, a direction of view of a target pedestrian 7d , as in an example in Fig. As shown in Figure 7B, if the direction of view is directed upwards from a horizontal direction at an angle greater than a given angle (such as 20 degrees), it can be determined that the direction of view of the target pedestrian is... 7d is not directed in the direction of the crossing destination.

[0042] Alternatively, if a direction of view of a target pedestrian 7e , as in an example in Fig. As shown in Figure 7C, directed downwards from the horizontal direction at an angle greater than a predetermined angle (such as 10 degrees), it can be determined that the direction of view of the target pedestrian is 7e is not directed in the direction of the crossing destination.

[0043] In S106 in Fig. 3. It is determined in the manner above whether the direction of view of the target pedestrian is directed towards the crossing destination which is on the other side of the roadway or not.

[0044] If, as a result, it is determined that the direction of gaze is directed towards the crossing destination (Yes in S106 ), next it is determined whether the gaze direction is fixed in this direction for a predetermined viewing time (such as 100 milliseconds or longer) or longer ( S107 If it is then determined that the gaze direction is fixed for the viewing time or longer (Yes in S107 ), a visual marker or visual flag for the in S104 selected target pedestrian set to ON ( S108Here, the gaze marker is a marker or flag indicating that the pedestrian has looked towards the crossing destination on the other side of the roadway. The gaze marker is also displayed for all pedestrians. A until G in Fig. (Example 6 shown). It is generally known that when a person looks at something, their gaze is fixed for a predetermined period or longer (such as 100 milliseconds or more). Consequently, if the target pedestrian's gaze direction is fixed in the direction of the crossing destination on the other side of the roadway for the predetermined gaze time or longer, it can be assumed that the target pedestrian was looking towards the crossing destination. Therefore, in such cases, the gaze marker of the selected target pedestrian is set to AN.

[0045] If, in contrast, the direction of view is directed towards the crossing destination (Yes in S106 ), but it is determined that the viewing time has not yet expired since the gaze direction was directed in that direction (No in S107 ), the gaze marker for the selected target pedestrian is set to OFF ( S109 ).

[0046] If, in addition, the direction of view is not directed towards the crossing destination from the outset (No in S106 ), the gaze marker for the selected target pedestrian is set to OFF, without determining whether the gaze direction is fixed for the viewing time or longer ( S109 ).

[0047] Next, the pedestrian detection device determines 100 according to the present embodiment, whether the target pedestrian has carried out the safety confirmation with regard to vehicles driving on the roadway or not (S110 in Fig. 4) That is, because the roadway, as described above, is located near the target pedestrian, it is determined whether the target pedestrian has performed the safety confirmation with regard to vehicles traveling on that roadway. It is determined, based on the target pedestrian's line of sight, whether the target pedestrian has performed the safety confirmation or not, in the following manner.

[0048] For example, if a characteristic movement in which the target pedestrian's gaze direction moves, as in Fig. As shown in Figure 8, if the movement is made in a direction along one side of the roadway with respect to the target pedestrian over a predetermined angle or greater within a predetermined range of speeds, it is determined that the target pedestrian has performed the safety confirmation.

[0049] One reason for this is that if such a movement is made, it can be assumed that the safety confirmation regarding vehicles traveling on the roadway has been carried out, because such a characteristic eye movement occurs when a vehicle traveling on a roadway is followed by the eyes.

[0050] It is determined that the target pedestrian has completed the security confirmation if, in addition, as in an example in Fig. Figure 9 shows a characteristic movement in which the target pedestrian's gaze is fixed in one direction, left or right, while directed towards the roadway for the specified confirmation time or longer, and then fixed in the other direction for the same specified confirmation time or longer. Alternatively, in cases where such a gaze movement is repeated, it is determined that the target pedestrian has performed the safety confirmation.

[0051] One reason for this is that if such a gaze movement is made, it can be assumed that the safety confirmation regarding vehicles traveling on the roadway is being carried out, because such a characteristic gaze movement occurs when a pedestrian confirms safety on the left and right sides.

[0052] Here, the predetermined confirmation time, used to determine whether the pedestrian has confirmed safety on the left and right sides, can be set to the same duration as the gaze time described above (such as a period of 100 milliseconds or longer), or not. Preferably, an additional confirmation is performed to verify that the left and right directions are separated by an angle equal to or greater than a predetermined confirmation angle (typically 90 degrees).

[0053] Furthermore, it is determined that the target pedestrian has completed the security confirmation if, as in an example in Fig. Figure 10 shows a characteristic movement in which the gaze direction of the target pedestrian moves from a state in which it is directed in the direction along the roadway, beyond a state orthogonal to the roadway.

[0054] One reason for this is that it can be assumed that if such a gaze movement is made, the safety confirmation regarding vehicles traveling on the roadway is carried out, because such a characteristic gaze movement occurs when a pedestrian confirms safety behind the pedestrian. Here, confirmation is preferably carried out that the direction of gaze before such a gaze movement is made and the direction of gaze after a movement beyond the state orthogonal to the roadway are separated by an angle equal to or greater than a predetermined back-viewing angle (typically 120 degrees).

[0055] InS110 in Fig. 4. Based on the direction of view of the target pedestrian, a procedure such as the one above is used to determine whether the target pedestrian has carried out the safety confirmation with regard to vehicles traveling on the roadway or not.

[0056] If it is determined that the target pedestrian has completed the safety confirmation, a safety confirmation marker or flag will be placed for the selected target pedestrian (i.e., the one identified in the system). S104 selected target pedestrian in Fig. 3) set to ON ( S111 Here, the safety confirmation marker is a marker or flag indicating that the target pedestrian has performed the safety confirmation regarding vehicles traveling on the roadway. Similar to the visual marker described above, the safety confirmation marker is also set for each target pedestrian.

[0057] If, on the other hand, it is determined that the target pedestrian has not performed the safety confirmation, the safety confirmation marker for the selected target pedestrian will be set to OFF ( S112 ).

[0058] If the gaze marker and safety confirmation marker for the selected target pedestrian are set in this way, the gaze marker and safety confirmation marker histories set up to that point will be used to determine the next step. S113), determines whether the target pedestrian is considered the pedestrian before crossing or not. A procedure for determining whether the target pedestrian is considered the pedestrian before crossing or not is described below. Here, the pedestrian before crossing, as described above, refers to a pedestrian who intends to cross the roadway but has not yet actually started crossing.

[0059] If, as a result, it is determined that the target pedestrian is considered to be the pedestrian before crossing (Yes in S113 ), confirms that the selected target pedestrian is the pedestrian in front of the crossing ( S114 If, on the other hand, it is determined that the target pedestrian is not considered to be the pedestrian prior to crossing (No in S113 ), confirms that the selected target pedestrian is not the pedestrian before crossing ( S115 ).

[0060] If, in the manner described above, it is determined whether the selected target pedestrian is considered to be the pedestrian prior to crossing or not ( S114 , S115 ), determines whether all target pedestrians have been selected or not ( S116 That is to say, in S104 in Fig. 3. A single pedestrian is guided by the target pedestrians in step. S103 extracted. The following is extracted. S105 until S115 in Fig. 4. The series of operations described above is performed for this pedestrian. Consequently, when the operations for the selected individual target pedestrian are completed, in S116 determines whether a target pedestrian for whom the operations have not yet been completed (that is, a target pedestrian who has not been selected) exists or not.

[0061] If one of the results is a target pedestrian who has not yet been selected (No in S116 ), the pedestrian detection apparatus returns toS104 in Fig. 3. Back. After a new target pedestrian is selected, the following sequence of operations described above ( S105 until S115 in Fig. 4) performed for the selected target pedestrian.

[0062] Since such an operation or process is repeated, a determination is eventually made that all target pedestrians have been selected (Yes in S116 In this case, the pedestrian who has been confirmed to be the pedestrian before crossing is then extracted ( S117 ).

[0063] This section describes the procedure by which the pedestrian detection device 100 according to the present embodiment in the process in S113 determines whether the target pedestrian is considered the pedestrian before crossing or not.

[0064] Fig. 11A to Fig. Application 11C presents a basic idea for determining whether a destination pedestrian is considered to be the pedestrian before crossing. Firstly, there are cases where the pedestrian crossing the roadway looks towards the crossing destination immediately before initiating a crossing, and cases where the pedestrian crossing the roadway performs the safety confirmation regarding vehicles traveling on the roadway immediately before initiating a crossing. Here, based on knowledge newly discovered by the inventors of the present application, even in cases where the pedestrian looks towards the crossing destination immediately before initiating a crossing, it is not the case that the pedestrian crosses without performing the safety confirmation regarding vehicles, but rather that the pedestrian performs the safety confirmation before looking towards the crossing destination (see Fig. 11A).

[0065] A similar idea applies to cases where the pedestrian performs the safety confirmation immediately before initiating a crossing. That is, if the pedestrian performs the safety confirmation immediately before initiating a crossing, it is not the case that the pedestrian crosses without looking towards the crossing destination, but rather that the pedestrian looks towards the crossing destination before performing the safety confirmation with regard to vehicles (see Fig. 11B).

[0066] Consequently, the pre-crossing movements of over one hundred pedestrians were observed as they actually crossed a roadway at more than ten different locations. It was confirmed as a result that pedestrians crossing the roadway performed the movement of looking towards their crossing destination and confirming their safety regarding vehicles immediately before crossing with a probability of nearly 100%.

[0067] Based on the foregoing, it can be assumed that an attempt to cross the roadway is made when the two movements, that is, the movement of looking towards the crossing destination and the movement to perform the safety confirmation regarding vehicles, are carried out within a given looking-back time (such as 3 seconds) (see Fig. 11C). Additionally, the crossing can be stopped even if the attempt to cross the roadway is made. Consequently, it can be assumed that the crossing of the roadway will be stopped even if the two movements are carried out within a given time period, after a fixed period (such as 5 seconds) subsequently elapses.

[0068] The pedestrian detection apparatus 100 According to the present embodiment, it is determined, based on an idea such as the one above, whether a target pedestrian is considered to be the pedestrian before crossing or not.

[0069] Fig. 12A to Fig. Figure 12C presents explanatory diagrams of a procedure for determining whether a target pedestrian is considered to be the pedestrian prior to crossing or not. It is described in S113 in Fig. 4 mentioned, based on the histories of the gaze marker and the safety confirmation marker established up to this point, determines whether a target pedestrian is considered to be the pedestrian prior to crossing or not.

[0070] Fig. Figure 12A conceptually shows how the establishment of the gaze marker and the safety confirmation marker evolves over time for all of the target pedestrians. A until G , which in the example in Fig. The 6 shown changes. A black straight line, extending from the left side towards the right side, Fig. The extension of 12A shows that neither the viewing marker nor the safety confirmation marker is set (that is, both markers are set to OFF). A section with coarse hatching indicates that the viewing marker is set to ON. A section with fine hatching indicates that the safety confirmation marker is set to ON.

[0071] For example, a state where both markings are set to OFF lasts, with regard to the pedestrian A The status is set to ON after the gaze indicator is briefly activated. Subsequently, the safety confirmation indicator is set to ON, and this state persists until the present time.

[0072] Additionally, neither the visual marker nor the safety confirmation marker is used with regard to the pedestrian. BThe safety confirmation marker is set to AN. Furthermore, the safety confirmation marker with respect to pedestrian C is immediately set to AN after the gaze marker is set to AN. After the gaze marker is again set to AN, a state occurs in which neither the gaze marker nor the safety confirmation marker is set to AN, and this state persists until the present time.

[0073] Here, the gaze marker, as described above, is a marker indicating that a glance has been made towards the crossing destination. The safety confirmation marker is a marker indicating that a safety confirmation regarding vehicles traveling on the roadway has been performed. Because looking towards the crossing destination and performing a safety confirmation regarding vehicles cannot be done simultaneously, the gaze marker and the safety confirmation marker are not set to ON at the same time.

[0074] If the histories of the gaze marker and the safety confirmation marker for all target pedestrians (see Fig. 12A) in this way, it can be determined whether the target pedestrian is attempting to cross the roadway or not, that is, whether the pedestrian is considered to be before crossing.

[0075] For example, the safety confirmation marker is used with regard to the target pedestrian. A The gaze marker is set to ON after a short period of time has elapsed following the setting of the gaze marker to ON. Consequently, as in Fig. As shown in Figure 12B, this determines whether the gaze marker is set to ON during a period that corresponds to the specified lookback time (such as 3 seconds) back to the point in time when the safety confirmation marker is set to ON. The gaze marker is, as shown in Fig. 12B shown, with regard to the pedestrian A within the retrospective time, which goes back from time a, it is not fixed on AN. It is therefore determined that the pedestrian A The pedestrian is not considered to have crossed because, after one of the markings, safety confirmation marking and lookout marking, is set to ON, the other marking is not set to ON within the lookout time.

[0076] Immediately after the line of sight is set to AN, the safety confirmation marking regarding the pedestrian is also set. C set to ON. Consequently, the gaze marker, as in Fig. 12C is shown, set to AN within the lookback time, which goes back from a time b at which the safety confirmation marker is set to AN. As a result, it is determined that the pedestrian C as the pedestrian before crossing.

[0077] Furthermore, the line of sight to pedestrians will be marked. C even at a time point after time b within the period that corresponds to the lookback time, while the safety confirmation marker is set to AN, it is set to AN. Consequently, even after time b, it is continuously determined that the pedestrian Cas the pedestrian before crossing. The gaze marker remains set to ON for the period corresponding to the lookback time from the moment the second gaze marker is set to ON, even after the safety confirmation marker is set to OFF and the gaze marker for the second time is set to ON. Consequently, it is continuously determined that the target pedestrian C as the pedestrian before crossing. Additionally, a state in which the gaze marker is set to ON during the period corresponding to the lookback time, and a state in which the confirmation marker is set to ON, exist up to time c, even after the second gaze marker is set to OFF. Consequently, until time c, it is continuously determined that the target pedestrian C as the pedestrian before crossing.

[0078] In S113 in Fig. 4 is with regard to the selected target pedestrian (that is, the one in S104 in Fig. 3 selected pedestrian) by a procedure such as the one above, determines whether the target pedestrian is considered to be the pedestrian before crossing or not. If one result is determined to be that the selected target pedestrian is considered to be the pedestrian before crossing (Yes in S113 ), confirms that the selected target pedestrian is the pedestrian before crossing, and this confirmation is saved ( S114 If, on the other hand, it is determined that the selected target pedestrian is not considered to be the pedestrian prior to crossing (No in S113 ), confirms that the selected target pedestrian is not the pedestrian before crossing, and this confirmation is saved ( S115 ).

[0079] The process then determines, as described above, whether all target pedestrians have been selected or not ( S116). If it is determined that all target pedestrians have been selected (Yes in S116 ), the pedestrian who has been confirmed to be the pedestrian before crossing is extracted ( S117 ).

[0080] It is then determined whether the extracted pedestrian was present before crossing or not ( S118 If the pedestrian is present before crossing (Yes in S118 ), a message or notification that the pedestrian is present before crossing will be displayed using the monitor. 3 and the loudspeaker 4 , which are provided in a vehicle interior, issued ( S119 ).

[0081] Fig. Figure 13 shows an example of a method in which the extraction result regarding the pedestrian before crossing is displayed on a screen of the monitor. 3 The image captured by the vehicle's camera is displayed. 2 in front of the vehicle 1has been recorded, will be, as in Fig. 13 shown, on the monitor screen 3 displayed. The eight pedestrians A until H appear in the captured image as described above. A marker image m, indicated by a rectangle with a dashed line, is displayed to show the pedestrian. G , who is considered the pedestrian before crossing among the pedestrians A until H The extracted data overlaps. Consequently, the driver of the vehicle recognizes this. 1 immediately that the pedestrian G the pedestrian among the eight pedestrians A until H is something that requires particular attention.

[0082] If the extracted pedestrian is extracted before crossing as a result, that the pedestrian is extracted before crossing (Yes in S118 ), is present, is recorded in the pedestrian detection system. 100According to the present embodiment, an additional audio message is transmitted from the loudspeaker. 4 The message indicates that the pedestrian is present before crossing (such as "a pedestrian is attempting to cross").

[0083] If the notification that the pedestrian is present before crossing is issued in this manner ( S119 ), determines whether the in Fig. 3 and Fig. 4. The detection process shown for the pedestrian should be terminated before crossing or not ( S120 If the process is not to be terminated as a result (No in S120 ), the following series of processes described above is started after the image captured by the vehicle's camera. 2 is obtained ( S100 in Fig. 3).

[0084] If, in the meantime, a pedestrian who has been confirmed as the pedestrian before crossing, as a result that the pedestrian who has been confirmed as the pedestrian before crossing, in S117 in Fig. 4 is extracted, is not present (No in S118 ), determines whether the detection process for the pedestrian should be terminated before crossing, without issuing a notification that the pedestrian is present before crossing, or not ( S120 If the process is not to be terminated as a result (No in S120 ), the pedestrian detection apparatus returns 100 back to the beginning of the process. After the image captured by the vehicle's camera 2 is obtained ( S100 in Fig. 3) the following series of processes described above is started.

[0085] If it is additionally determined that the process will take place in S120 should be ended (Yes in S120), the detection process for the pedestrian is terminated before crossing.

[0086] The pedestrian detection apparatus 100 According to the present embodiment, a pedestrian attempting to cross the roadway (i.e., the pedestrian before crossing) can be detected in this way at a stage before the pedestrian actually begins to cross. If the pedestrian additionally, before crossing, as above with reference to Fig. 11A to Fig. As described in section 11C, the detection is based on whether the pedestrian has performed the movement of looking towards their crossing destination and the movement to perform the safety confirmation regarding vehicles on the roadway within a specified time period. Consequently, the pedestrian can be detected with high accuracy before crossing.

[0087] As a result, a pedestrian attempting to cross the road may be hit even in cases where there are many pedestrians in front of the vehicle. 1 Presences are detected accurately and at a stage before the pedestrian actually begins to cross. Consequently, the burden placed on the driver to monitor pedestrians can be significantly reduced.

[0088] For example, the eight pedestrians A until H in Fig. 13 examples shown in front of the vehicle 1 present. At this stage, however, determining which pedestrian is attempting to cross the road is difficult because none of the pedestrians have actually started to cross yet. In the pedestrian detection system 10 However, according to the present embodiment, the driver can be informed that the pedestrian G attempts to clear the roadway at a stage before the pedestrian GA pedestrian actually starts crossing the roadway. Consequently, the burden placed on the driver to monitor pedestrians can be significantly reduced.

[0089] In the pedestrian detection system 100 According to the present embodiment, if the pedestrian is present before crossing, an additional audio message is given by the loudspeaker. 4 issued (see Fig. 13) If the driver is driving while looking ahead in a normal manner, the driver can therefore immediately recognize that the pedestrian has been removed before crossing, even without the monitor. 3 to monitor. If the driver then looks at the monitor screen 3 By looking, the driver can immediately recognize from the marking image m which pedestrian is the pedestrian before crossing. Modification examples

[0090] Several modification examples can be considered with regard to the present embodiment described above. These modification examples are briefly described below, with the focus primarily on differences from the present embodiment. First modification example

[0091] In the pedestrian detection device described above 100 According to the present embodiment, it is presumed that the target pedestrian has been following a vehicle traveling on the roadway with his eyes, and it is determined that he has performed the safety confirmation with regard to vehicles if the target pedestrian's direction of gaze corresponds to the direction shown in the Fig. 8. characteristic movement shown (that is, the movement of a moving in a direction along the side of the roadway in relation to the target pedestrian over a predetermined angle or greater within a predetermined range of movement speeds).

[0092] However, the position of a vehicle traveling on the road can be detected, and it can be determined whether the target pedestrian's gaze is directed towards the vehicle or not. If the target pedestrian's gaze then also moves in line with the vehicle's position, it can be determined that the target pedestrian has given the required safety confirmation regarding vehicles.

[0093] Fig. Figure 14 shows an internal configuration of a pedestrian detection device. 150 in such a first modification example. In the pedestrian detection apparatus 150 of the in Fig. In the first modification example shown in section 14, a vehicle detection unit is used. 109 to the pedestrian detection device 100 according to the above with reference to Fig. 2. The present embodiment described above has been added. The pedestrian detection device 150However, the first modification example is similar to the pedestrian detection apparatus in other respects. 100 according to the present embodiment.

[0094] The vehicle detection unit 109 , which are in the pedestrian detection apparatus 150 The first modification example, which is provided, analyzes the image captured by the vehicle's camera. 2 is recorded, which is recorded by the Erlangen unit 101 for the captured image. The vehicle detection unit 109 It detects the position of a vehicle in the captured image and transmits the result to the security confirmation unit. 107 off. Here the vehicle detection unit is located. 109 It is described that it detects the position of a vehicle by analyzing the captured image. However, various known methods can be used as the method by which the vehicle detection unit determines the position. 109The position of a vehicle is determined. The position of a vehicle can be determined, for example, based on an output from a radar, such as a millimeter-wave radar, a sonar, or similar device.

[0095] The security confirmation determination unit 107 The gaze direction of the target pedestrian is obtained from the gaze direction acquisition unit. 105 and obtains the vehicle's position from the vehicle detection unit 109 If the target pedestrian's gaze then moves in such a way as to follow the movement of the vehicle, it is determined that the target pedestrian has carried out the safety confirmation with regard to vehicles.

[0096] One result determined is that the safety confirmation is performed when the target pedestrian's gaze is actually following a vehicle. Consequently, it can be determined even more precisely whether the target pedestrian has performed the safety confirmation with regard to vehicles or not. As a result, the pedestrian detection device 150 The modification example aims to capture the pedestrian more precisely before crossing. Second modification example

[0097] In the pedestrian detection process prior to crossing according to the present embodiment, the target pedestrian, for whom the safety confirmation marker and the gaze marker are set to AN within the period corresponding to the predetermined lookback time from the present time, is as described above with reference to Fig. 12A to Fig. 12C described when the pedestrian was hit before crossing.

[0098] However, instead of going back in time from the present and locating the target pedestrian for whom the safety confirmation marker and the gaze marker are set to AN, an effective duration for the AN state can be defined for each of the safety confirmation markers. The target pedestrian for whom the AN state persists for both markers, safety confirmation marker and gaze marker, can be identified as the pedestrian before crossing.

[0099] Fig. 15A to Fig. Figure 15C is an explanatory diagram of a method in which the pedestrian is detected by the target pedestrians before crossing in the detection process for the pedestrian before crossing in the second modification example described above. For example, the effective duration of a given period in a Fig. In the example shown in 15A, the gaze marker is set to a time d at which the gaze marker for the target pedestrian is changed from ON to OFF. While the safety confirmation marker is then changed back to ON within a period of time until the effective duration expires from time d, the target pedestrian is detected as the pedestrian before crossing.

[0100] Here in the Fig. In example 15, the safety confirmation marker remains set to AN even after the effective duration from time d has elapsed. However, because the effective duration for which the AN state of the gaze marker effectively lasts has ended after the effective duration from time d has elapsed, the target pedestrian is not detected for any longer than the pedestrian before crossing.

[0101] In a Fig. In the example shown in 15B, the effective duration of a given time period is additionally defined at a time e at which the gaze marker for the target pedestrian is set from ON to OFF. The target pedestrian is then recorded as the pedestrian before crossing, while the safety confirmation marker is set to ON within the time period until the effective duration expires, from time e. The effective duration of the safety confirmation marker is now defined from this time f when the safety confirmation marker is changed from ON to OFF. Here in Fig. Section 15B omits a description regarding the effective duration of the security confirmation mark.

[0102] Consequently, the state in which the gaze marker and the safety confirmation marker are set to ON persists until the effective duration of the gaze marker expires, even after the safety confirmation marker is set to OFF. Therefore, the target pedestrian is also detected as a pedestrian before crossing during this period.

[0103] The description above uses as an example a case where the gaze marker is initially set to AN and the safety confirmation marker is subsequently set to AN. However, a completely similar description applies to a case where the safety confirmation marker is initially set to AN and the gaze marker is subsequently set to AN.

[0104] In the description above, the gaze marker and the security confirmation marker are further described as having no difference in effective duration. However, the length of the effective duration can differ between the gaze marker and the security confirmation marker.

[0105] For example, the effective duration of the security confirmation mark, as in an example in Fig. 15C, shown to be over a shorter period than the effective duration of the gaze marking, which in the example in Fig. 15A or Fig. 15B is shown, fixed.

[0106] The gaze marker, as described above, is a marker indicating that the pedestrian has looked toward the crossing destination. The safety confirmation marker is a marker indicating that the pedestrian has confirmed their safety regarding vehicles. Once a crossing destination is confirmed, the situation is unlikely to change, even over time. Consequently, a pedestrian attempting to cross the roadway tends to initiate the crossing directly without reconfirming, even if some time passes after confirming the crossing destination.

[0107] In contrast, the situation regarding the safety confirmation for vehicles is likely to change and no longer be safe as time passes. Consequently, a pedestrian attempting to cross the roadway is more likely to initiate a crossing after re-performing the safety confirmation if time elapses since the initial confirmation.

[0108] If the effective duration of the safety confirmation marker is therefore set to a shorter period than the effective duration of the gaze marker, the pedestrian can be detected even more accurately before crossing.

[0109] The present embodiment and various modification examples are described above. However, the present invention is not limited to the described embodiment and the various modification examples and can be implemented in various ways or types without departing from the spirit of the invention. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2016135404

[0001] JP 2010102437 A

[0005]

Claims

[1] Pedestrian detection device (100, 150) which analyzes a recorded image taken by a vehicle-side camera (2) and thereby detects a pedestrian appearing in the recorded image, wherein the pedestrian detection device comprises: a roadway detection unit (103) that detects a roadway in the recorded image, a target pedestrian detection unit (104) that detects as a target pedestrian a pedestrian who is present in the recorded image within an area of ​​a predetermined distance in both lateral directions from the roadway, a gaze direction acquisition unit (105) that obtains gaze direction information from the target pedestrian, a gaze determination unit (106) which determines, based on the gaze direction of the target pedestrian, whether the target pedestrian has looked towards a crossing destination that is on the other side of the roadway or not, a safety confirmation determination unit (107) which determines, based on the direction of view of the target pedestrian, whether the target pedestrian has made a safety confirmation with regard to vehicles traveling on the roadway or not, and a pedestrian pre-crossing detection unit (108) which detects the target pedestrian who has performed one of the actions, looking towards the crossing destination and confirming safety, and has then performed the other action within a specified time period, as a pedestrian pre-crossing who has an intention to cross the roadway. [2] Pedestrian detection device according to claim 1, wherein: The detection unit for pedestrians before crossing registers the target pedestrian who has performed the safety confirmation within a first predetermined period after looking towards the crossing destination, or who has looked towards the crossing destination within a second predetermined period, which is shorter than the first predetermined period, after performing the safety confirmation, as the pedestrian before crossing. [3] Pedestrian detection device according to claim 1 or 2, wherein: The gaze direction acquisition unit obtains the gaze direction information of the target pedestrian by analyzing an image of the target pedestrian appearing in the captured image. [4] Pedestrian detection device according to any one of claims 1 to 3, wherein: The gaze determination unit determines that the target pedestrian has looked towards the crossing destination if the target pedestrian's gaze direction is fixed in a direction towards the crossing destination for a predetermined gaze time. [5] Pedestrian detection device according to claim 4, wherein: The gaze determination unit determines that the target pedestrian is not looking towards the crossing destination if the direction of gaze of the target pedestrian is not within an angle range that is within 45 degrees from a direction perpendicular to the roadway. [6] Pedestrian detection device according to claim 4 or 5, wherein: The gaze determination unit determines that the target pedestrian is not looking towards the crossing destination if the target pedestrian's gaze direction is directed upwards at a predetermined upper limit angle or greater with respect to a horizontal direction. [7] Pedestrian detection device according to any one of claims 4 to 6, wherein: The gaze determination unit determines that the target pedestrian is not looking towards the crossing destination if the target pedestrian's gaze direction is directed downwards at a specified lower limit angle or greater with respect to a horizontal direction. [8] Pedestrian detection device according to any one of claims 1 to 7, wherein: The safety confirmation determination unit determines that the target pedestrian has performed the safety confirmation with respect to vehicles traveling on the roadway when the target pedestrian's line of sight moves in a direction along one side of the roadway in relation to the target pedestrian over a specified angle or greater within a specified range of movement speeds. [9] Pedestrian detection device according to claim 8, further comprising: a vehicle detection unit (109) that detects a vehicle traveling on the roadway, wherein The safety confirmation determination unit determines that the target pedestrian has performed the safety confirmation with respect to the vehicle if the target pedestrian's line of sight moves in conjunction with a movement of the vehicle traveling on the roadway. [10] Pedestrian detection device according to any one of claims 1 to 9, wherein The safety confirmation determination unit determines that the target pedestrian has performed the safety confirmation with regard to vehicles traveling on the roadway if the target pedestrian's gaze direction moves in one direction, left and right, when directed towards the roadway, and subsequently moves in the other direction. [11] Pedestrian detection device according to claim 10, wherein: The safety confirmation determination unit determines that the target pedestrian has performed the safety confirmation with regard to vehicles traveling on the roadway if the target pedestrian's line of sight is directed in one of the directions, left and right, and if the line of sight is directed in the other direction, the lines of sight are separated by 90 degrees or more. [12] Pedestrian detection device according to any one of claims 1 to 11, wherein: The safety confirmation determination unit determines that the target pedestrian has carried out the safety confirmation with regard to vehicles traveling on the roadway when the target pedestrian's direction of gaze moves from a direction along the roadway to a direction that is no longer orthogonal to the roadway, via the direction orthogonal to the roadway. [13] Pedestrian detection method for analyzing a recorded image obtained from a vehicle-mounted camera (2) in order to detect a pedestrian appearing in the recorded image, wherein the pedestrian detection method comprises: a step (S102) to detect a roadway in the recorded image, a step (S103) to detect a pedestrian who is present within an area of ​​a predetermined distance in both lateral directions from the roadway in the recorded image, as a target pedestrian, a step (S105) to obtain gaze direction information of the target pedestrian, a step (S106, S107) to determine, based on the direction of the target pedestrian's gaze, whether the target pedestrian has looked towards the crossing destination located on the other side of the roadway or not, a step (S110) to determine, based on the target pedestrian's direction of gaze, whether the target pedestrian has made a safety confirmation regarding vehicles traveling on the roadway or not, and a step (S113) to detect the target pedestrian who has performed one of the actions, looking towards the crossing destination and confirming safety, and has then performed the other action within a specified time period, as a pre-crossing pedestrian who has an intention to cross the roadway.