Dangerous behavior prediction apparatus, dangerous behavior prediction method, dangerous behavior prediction program, dangerous behavior prediction system, and vehicle

Abstract

To provide a technique capable of improving safety and predicting a dangerous behavior.SOLUTION: An exemplary dangerous behavior prediction apparatus acquires detection information of a dangerous behavior that has occurred in a peripheral area of a place where a prediction target of the dangerous behavior exists, and sets a determination criterion used when predicting the dangerous behavior of the prediction target based on an acquisition status of the detection information.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a technique for predicting risky behavior, such as a pedestrian running out into a traffic lane. [Background technology]

[0002] BACKGROUND ART Conventionally, there is known a technique for predicting whether or not a pedestrian will run out into a vehicle traffic area (a detailed example is a roadway) (see, for example, Patent Document 1).

[0003] The pedestrian jumping out prediction device disclosed in Patent Document 1 acquires time-series changes in the position and movement speed of a pedestrian in front of the vehicle. The pedestrian jumping out prediction device also acquires surrounding information. The device compares the acquired time-series changes in the position and movement speed with pre-determined patterns of time-series changes in the position and movement speed when the pedestrian jumps out onto the roadway. The device also compares the acquired surrounding information with pre-determined surrounding information when the pedestrian jumps out onto the roadway. The device then uses the results of these comparisons to predict whether the pedestrian will jump out onto the roadway on which the vehicle is traveling. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-1023437 Summary of the Invention [Problem to be solved by the invention]

[0005] In social psychology, it is known that people show a psychological tendency to move in extreme directions as soon as they move from being individuals to being in a group. This psychological tendency is called a risky shift, and a specific example is the belief that "it's not scary to cross the street when the light is red if we all do it together." If someone engages in risky behavior, such as ignoring the traffic light at a crosswalk, there is a possibility that others will be inspired to run out into the street. However, conventional predictions of people running out into the street do not take into account such risky shifts. For this reason, it was thought that there was room for improvement in conventional predictions of people running out into the street from a safety perspective.

[0006] In view of the above, an object of the present invention is to provide a technology that can predict risky behavior while improving safety. [Means for solving the problem]

[0007] An exemplary risky behavior prediction device of the present invention acquires detection information of risky behavior that has occurred in the surrounding area of ​​a location where a target of risky behavior is located, and sets judgment criteria to be used when predicting the risky behavior of the target based on the acquisition status of the detection information. [Effects of the Invention]

[0008] An exemplary risky behavior prediction device of the present invention is configured to be able to change the setting state of the judgment criteria used to predict risky behavior depending on the acquisition status of detection information of risky behavior occurring in the surrounding area of ​​a target (pedestrian, etc.) of risky behavior prediction. With such a configuration, the judgment criteria can be set so that when risky behavior of a person around a pedestrian, etc. is detected, the probability of determining that the pedestrian, etc. will engage in risky behavior is higher than when the risky behavior is not detected. In other words, according to the exemplary present invention, it is possible to set judgment criteria that take risky shifts into consideration, thereby making it possible to predict risky behavior with increased safety. [Brief explanation of the drawings]

[0009] [Figure 1] Block diagram showing the general configuration of the risky behavior prediction system [Figure 2] A block diagram showing the general configuration of an information collection device. [Figure 3] Schematic diagram showing an example of a table that aggregates information on the detection of jumping out [Figure 4] A block diagram showing the general configuration of an in-vehicle device. [Figure 5] Schematic diagram illustrating the state of an intersection [Figure 6] 1 is a flowchart illustrating a flow of processing executed by an information collection device; [Figure 7] 1 is a flowchart illustrating a flow of processing executed by an in-vehicle device (risky behavior prediction device); [Figure 8] A flowchart showing a detailed example of the process of step S14 in FIG. 7. [Figure 9] A block diagram showing a schematic configuration of a risky behavior prediction system according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.

[0011] <1. Risky behavior prediction system> FIG. 1 is a block diagram showing a schematic configuration of a risky behavior prediction system 100 according to an embodiment of the present invention. The risky behavior prediction system 100 is a system configured to predict risky behaviors performed by people. Targets of risky behavior prediction are, for example, pedestrians and bicycles (cyclists). Specific examples of risky behaviors include running into a traffic area (hereinafter sometimes simply referred to as "running out" or "running into the roadway"), ignoring traffic signals, and illegal crossing without following traffic rules.

[0012] As shown in FIG. 1, a risky behavior prediction system 100 of this embodiment includes a camera 1, an information collection device 2, and an in-vehicle device 3.

[0013] The camera 1 is a device for detecting risky behavior by people. Note that, since it is only necessary to detect risky behavior by people, a configuration may be adopted in which a sensor such as a millimeter wave radar or a Lidar (Light Detection and Ranging) is provided instead of or in addition to the camera 1.

[0014] The camera 1 may be installed, for example, in infrastructure equipment installed on a road or in a vehicle traveling on the road. The camera 1 may be installed in at least one of a vehicle and infrastructure equipment installed on a road. The infrastructure equipment is, for example, a traffic light or a sign. Hereinafter, in this embodiment, it is assumed that the camera 1 is installed in the infrastructure equipment. Furthermore, the cameras 1 included in the risky behavior prediction system 100 are installed at multiple locations, for example, at each intersection. At one intersection, the cameras 1 may be installed at multiple locations.

[0015] The information collection device 2 collects risky behavior detection information obtained from images captured by the camera 1. In this embodiment, the information collection device 2 is provided so as to be able to communicate with the camera 1 and acquires the captured images from the camera 1. The information collection device 2 processes the acquired captured images to collect detection information of risky behavior such as running out into the street. Note that in this embodiment, the information collection device 2 is configured to process the captured images of the camera 1 to acquire risky behavior detection information, but this is an example. The risky behavior detection information may be acquired by the camera 1 itself, or by an image processing device (not shown) that is capable of communicating with the camera 1, processing the captured images. In this case, the information collection device 2 is configured to acquire risky behavior detection information from an external source.

[0016] In this embodiment, the information collection device 2 is provided in infrastructure equipment. The information collection device 2 is capable of communicating with the camera 1 via wired or wireless communication. The information collection device 2 may be configured to be provided at each intersection, for example, in which case the risky behavior prediction system 100 will include multiple information collection devices 2. However, the information collection device 2 may also be configured as a server device such as a cloud server, in which case the risky behavior prediction system 100 may include a single or multiple information collection devices 2.

[0017] The in-vehicle device 3 is mounted on each vehicle V. In other words, the vehicle V is equipped with the in-vehicle device 3. Note that the vehicle V is typically an automobile such as a passenger car. The automobile is not limited to a four-wheeled automobile, but may also be a two-wheeled automobile (motorcycle). In this embodiment, the in-vehicle device 3 is a drive recorder and includes a camera capable of capturing images in front of the vehicle V. The camera may constitute the above-mentioned camera 1. Note that the in-vehicle device 3 may be a device other than a drive recorder, or may simply be a computer device that processes information. In this case, the vehicle V may be configured to be equipped with an in-vehicle camera that is provided as a device separate from the in-vehicle device 3.

[0018] In this embodiment, the in-vehicle device 3 is provided so as to be able to communicate with the information collection device 2. The communication between the in-vehicle device 3 and the information collection device 2 is, in detail, wireless communication. The in-vehicle device 3 predicts risky behavior using set criteria. That is, the in-vehicle device 3 constitutes a risky behavior prediction device. The criteria will be described later.

[0019] The in-vehicle device 3 predicts risky behavior when a risky behavior prediction target exists. In this embodiment, the risky behavior prediction target is a pedestrian present in front of the vehicle V (host vehicle) on which the in-vehicle device 3 is mounted, and the risky behavior is running out into the traffic area of ​​the vehicle V. That is, in this embodiment, the in-vehicle device 3 predicts a pedestrian running out. A case where a risky behavior prediction target exists corresponds to a case where a pedestrian is detected in front of the vehicle V by the camera provided in the in-vehicle device 3. Note that the risky behavior prediction target may include, for example, a bicycle (bicyclist) instead of or in addition to a pedestrian.

[0020] Specifically, the in-vehicle device 3 appropriately uses information transmitted from the information collection device 2 when predicting risky behavior. This will be described in detail later. Note that FIG. 1 shows only one in-vehicle device 3 (vehicle V) provided in the risky behavior prediction system 100. However, the risky behavior prediction system 100 actually has a plurality of in-vehicle devices 3 (vehicles V).

[0021] <2. Information gathering device> Fig. 2 is a block diagram showing a schematic configuration of an information collection device 2 according to an embodiment of the present invention. Note that Fig. 2 shows components necessary for explaining the features of the information collection device 2 according to the embodiment, and omits a description of general components. The information collection device 2 according to this embodiment is a computer device.

[0022] As shown in FIG. 2, the information collection device 2 includes a first controller 21, a first memory 22, and a first communication unit .

[0023] The first controller 21 is configured to include an arithmetic circuit that performs arithmetic processing. More specifically, the first controller 21 is equipped with a processor that performs arithmetic processing and the like. The processor may be configured to include, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The first controller 21 may be configured with one processor or multiple processors. When configured with multiple processors, the processors may be connected to each other so that they can communicate with each other.

[0024] The first memory 22 is configured to include a volatile memory and a nonvolatile memory. The volatile memory is specifically a RAM (Random Access Memory). The nonvolatile memory is specifically a ROM (Read Only Memory). The nonvolatile memory may also include a flash memory, a hard disk drive, or the like. The nonvolatile memory stores computer-readable programs (computer programs) and data.

[0025] The first communication unit 23 includes a communication device that enables communication with the camera 1 and the in-vehicle device 3. The first communication unit 23 includes a communication device that enables wired communication with the camera 1. The first communication unit 23 also includes a communication device that enables wireless communication with the in-vehicle device 3. As described above, communication with the camera 1 may also be wireless communication, in which case the first communication unit 23 is configured to include a communication device that enables wireless communication with the camera 1.

[0026] The first controller 21 includes, as its functions, an acquisition unit 211, an aggregation unit 212, and a distribution unit 213. In this embodiment, the functions of the first controller 21 are realized by a processor executing arithmetic processing in accordance with a program stored in the first memory 22. The program that realizes the functions of the first controller 21 may be composed of a single program or multiple programs.

[0027] The program stored in the first memory 22 may be provided by, for example, a computer-readable nonvolatile recording medium. The nonvolatile recording medium may be, for example, an optical recording medium (for example, an optical disk), a magneto-optical recording medium (for example, a magneto-optical disk), a USB memory, an SD card, or the like, in addition to the nonvolatile memory described above. As another example, the program stored in the first memory 22 may be configured to be provided from a program providing server via a communication line such as the Internet (a so-called configuration provided by download).

[0028] In addition, in this embodiment, the functions of the first controller 21 are realized by an arithmetic circuit (processor) executing arithmetic processing according to a program, i.e., by software, but this is merely an example and the functions may be realized by other methods. At least some of the functions of the first controller 21 may be realized using, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). That is, at least some of the functions of the first controller 21 may be realized by hardware using a dedicated IC or the like. At least some of the functions of the first controller 21 may also be realized by a combination of software and hardware.

[0029] Furthermore, each of the functional units 211 to 213 is a conceptual component. A function executed by one component may be distributed among multiple components. Furthermore, functions possessed by multiple components may be integrated into one component.

[0030] The acquisition unit 211 acquires information from an external element communicatively connected to the information collection device 2. In detail, an example of the external element is the camera 1 described above. The acquisition unit 211 may be configured to acquire information from only one camera 1, but is preferably configured to acquire information from multiple cameras 1. For example, a configuration in which information is acquired from multiple cameras 1 installed at various locations and angles at an intersection makes it possible to thoroughly detect risky behavior such as running out into the street. The acquisition unit 211 acquires captured images from the camera 1 at regular intervals. In other words, the acquisition unit 211 acquires video captured by the camera 1.

[0031] The aggregation unit 212 appropriately processes the captured images of the camera 1 acquired by the acquisition unit 211 and aggregates the processed information. In detail, the aggregation unit 212 performs image processing on the acquired captured images of the camera 1 to detect people jumping out. The method for detecting people jumping out is not particularly limited, and the detection of people jumping out may be performed using a known image processing method. For example, people, traffic lights, and roads are detected from the captured images (video) of the camera 1 using a known object detection method. Then, the color of the detected traffic lights is determined. The detected people are tracked to track their movement relative to the road. When the movement of a person crossing the road even though the traffic light is red is detected from the video obtained by collecting captured images over a certain period of time, the aggregation unit 212 detects people jumping out. When people jumping out is detected, the aggregation unit 212 aggregates the detection information of people jumping out in a table 221 (see FIG. 3 described later) and stores it in the first memory 22.

[0032] The detection of objects jumping out may be performed, for example, using an AI model for detecting objects jumping out, which is generated by performing deep learning using as training data a large amount of video data that includes a collection of various scenes in which objects actually jump out.

[0033] 3 is a schematic diagram showing an example of table 221 that aggregates information on detection of jumping out. As shown in FIG. 3, the information items held in table 221 that aggregates information on detection of jumping out include the detection information number, the time of occurrence, the location of occurrence, and the action taken at the time of jumping out. The registered information for each item in table 221 is updated as appropriate when a jumping out is detected. Furthermore, information registered in table 221 that has been stored for a certain period of time may be deleted as appropriate.

[0034] The "detection information number" item stores a number that serves as identification information for identifying the detection information of unauthorized crossing. The "detection information number" functions as the primary key of table 221. A data record is generated for each detection information number. The data record stores information corresponding to each detection information number, such as "time of occurrence," "location of occurrence," and "action details."

[0035] The item "time of occurrence" stores the time when the person jumped out. The item "location of occurrence" stores the location where the person jumped out. The item "action details" stores the specific action details at the time of the person jumping out. In the example shown in FIG. 3, the item "action details" stores, as an example, the direction of movement of the person who jumped out.

[0036] The distribution unit 213 (see FIG. 2) performs distribution processing of the detection information of people stepping out. Through this distribution processing, the detection information of people stepping out is broadcast via the first communication unit 23. The distribution unit 213 may be configured to distribute all of the detection information of people stepping out collected in the table 221, or may be configured to distribute only a portion of the information. In this embodiment, the distribution unit 213 distributes only the detection information of people stepping out that has occurred most recently (for example, within 5 or 10 minutes). The purpose of distributing only the necessary information is to reduce the processing load. The detection information of people stepping out distributed by the distribution unit 213 is acquired by the in-vehicle device 3 provided in the vehicle V and used as appropriate, and this point will be described in detail later.

[0037] <3.In-vehicle equipment> Fig. 4 is a block diagram showing a schematic configuration of an in-vehicle device 3 according to an embodiment of the present invention. Note that Fig. 4 shows components necessary for explaining the features of the in-vehicle device 3 according to the embodiment, and a description of general components is omitted. In this embodiment, as shown in Fig. 4, the in-vehicle device 3 constitutes, as an example, a risky behavior prediction device.

[0038] As shown in FIG. 4, the in-vehicle device 3 includes a second controller 31, a second memory 32, a second communication unit 33, and a camera .

[0039] The second controller 31 is configured to include an arithmetic circuit that performs arithmetic processing. More specifically, the second controller 31 is equipped with a processor that performs arithmetic processing and the like. The processor may be configured to include, for example, a CPU or a GPU. The second controller 31 may be configured with one processor or multiple processors. When the second controller 31 is configured with multiple processors, the processors may be connected to each other so that they can communicate with each other.

[0040] The second memory 32 is configured to include volatile memory and nonvolatile memory. The volatile memory is specifically RAM. The nonvolatile memory is specifically ROM. The nonvolatile memory may also include flash memory, a hard disk drive, or the like. The nonvolatile memory stores computer-readable programs (computer programs) and data.

[0041] The second communication unit 33 includes a communication device that enables wireless communication with the information collection device 2. The communication device may be a communication device that enables V2X communication. Note that the "V" in V2X refers to an automobile as the host vehicle, and "X" can refer to various entities other than the host vehicle, such as pedestrians, other vehicles, infrastructure facilities, networks, and servers. In this embodiment, communication between the second communication unit 33 of the in-vehicle device 3 and the first communication unit 23 of the information collection device 2 corresponds to V2I communication (road-to-vehicle communication).

[0042] The camera 34 is a camera that captures images in front of the vehicle V. Image information captured by the camera 34 is transmitted to the second controller 31. Communication between the camera 34 and the second controller 31 may be wireless or wired. As described above, the in-vehicle device 3 does not need to be equipped with the camera 34. In this case, the in-vehicle device 3 may be configured such that image information of the area in front of the vehicle is input to the in-vehicle device 3 from an in-vehicle camera provided separately from the in-vehicle device 3.

[0043] The second controller 31 includes, as its functions, an acquisition unit 311, a risky behavior prediction unit 312, a driving support processing unit 313, and a judgment criterion setting unit 314. In this embodiment, the functions of the second controller 31 are realized by a processor executing arithmetic processing in accordance with a program stored in the second memory 32. The program that realizes the functions of the second controller 31 may be composed of a single program or multiple programs.

[0044] The program stored in the second memory 32 may be provided by a computer-readable non-volatile recording medium or the like, similar to the program stored in the first memory 22. Also, similar to the first controller 21, at least some of the functions of the second controller 31 may be realized by hardware rather than software, or a combination of software and hardware.

[0045] The acquisition unit 311 acquires information from the camera 34 and the information collection device 2. The acquisition unit 311 acquires captured images from the camera 34 at regular intervals. In other words, the acquisition unit 311 acquires video captured by the camera 34. The acquisition unit 311 also acquires risky behavior detection information from the information collection device 2. That is, the in-vehicle device (risky behavior prediction device) 3 acquires detection information of risky behavior that has occurred in the past. In this embodiment, the risky behavior detection information is detection information of running out, and the running out detection information includes information such as the location and time of the running out, as described above. The acquisition unit 311 stores the acquired information in the second memory 32 as appropriate.

[0046] When a target for which risky behavior is predicted exists, the risky behavior prediction unit 312 predicts whether the target will perform risky behavior using a set criterion. In this embodiment, as described above, the risky behavior is a pedestrian running out into the road. That is, when a pedestrian who is a target for which risky behavior is predicted exists, the risky behavior prediction unit 312 (in-vehicle device 3) predicts whether the pedestrian will run out into the road using a set criterion.

[0047] The risky behavior prediction unit 312 detects a pedestrian who is predicted to jump out, using image information acquired from the camera 34. A known object detection method may be used as the detection method. The known object detection method may be, for example, a method that uses an object detection model configured as an AI model.

[0048] When the risky behavior prediction unit 312 detects a pedestrian in an image captured by the camera 34, it performs a running-out prediction for the detected pedestrian. Running-out prediction can be performed, for example, using time-series change information on the pedestrian's position and movement speed obtained using captured images for a predetermined period of time. In addition to the time-series change information on the pedestrian's position and movement speed, road conditions, the status of surrounding vehicles, pedestrian information, or surrounding environment information may be used when running-out prediction. Pedestrian information may include, for example, information on the pedestrian's state, such as the orientation of the pedestrian's body or face, or pedestrian characteristic information, such as the pedestrian's age. Surrounding environment information may include, for example, information on surrounding facilities, traffic light status, or weather information.

[0049] For example, the following well-known method may be used to predict whether a pedestrian will run out into the roadway. First, the time-series changes in the pedestrian's position and movement speed are acquired from images captured by the camera 34. The acquired time-series change information is then compared with a predetermined pattern of time-series changes in the position and movement speed when the pedestrian runs out into the roadway to determine the degree of match. If the degree of match is equal to or greater than a set threshold, it is predicted that the pedestrian will run out into the roadway. If the degree of match is less than the set threshold, it is predicted that the pedestrian will not run out into the roadway.

[0050] The previously determined patterns of time-series changes in position and time-series changes in moving speed when a pedestrian runs out onto the roadway may be stored as a database of cases of pedestrians running out into the roadway in the second memory 32. The degree of match obtained by the comparison may be the highest degree of match obtained by comparing the time-series change patterns of a plurality of cases of pedestrians running out into the roadway.

[0051] Furthermore, the degree of agreement between the time-series changes in the position and movement speed of a pedestrian obtained from the captured image and the time-series change patterns of the position and movement speed when the pedestrian actually runs out into the roadway may be determined using an AI model. The AI ​​model may be generated by performing deep learning using as training data the time-series change patterns of the position and movement speed of a pedestrian obtained from a large amount of video data collecting various scenes in which pedestrians actually run out into the roadway.

[0052] The risky behavior prediction unit 312 may also input video information of a captured pedestrian into an AI model, calculate the degree of coincidence between the pedestrian's behavior in the video and the pedestrian's behavior immediately before running out into the roadway that is assumed based on past cases of pedestrians running out into the roadway, and predict the running out of the roadway by comparing the degree of coincidence with a judgment criterion. In this case, the AI ​​model may be an AI model generated by performing deep learning using as training data a large amount of video data that includes a collection of various scenes in which pedestrians actually ran out into the roadway.

[0053] The driving support processing unit 313 performs driving support processing when it predicts that the risky behavior prediction target will perform risky behavior. In detail, the driving support processing unit 313 (in-vehicle device 3) performs driving support processing when it predicts that a pedestrian will jump out into the road. The driving support processing is, for example, a notification processing that notifies the driver of the vehicle V that there is a risk of a pedestrian jumping out into the road, or a processing that causes the vehicle V to automatically perform a risk avoidance operation.

[0054] Specifically, the notification process is a process of causing a notification means provided in the vehicle V to perform a notification operation to notify that there is a risk of a person running out into the road. The notification means is, for example, a display device that displays a message notifying of the risk, a buzzer or speaker that notifies of the risk by sound or voice, a vibration device that notifies of the risk by vibration, a light-emitting device that notifies of the risk by light, etc.

[0055] Furthermore, the process of automatically causing the vehicle V to perform a danger avoidance operation is specifically a process of instructing a driving control unit (not shown) that controls the driving of the vehicle V to perform a danger avoidance operation. The danger avoidance operation is, for example, deceleration or stopping by automatic braking, or changing the direction of travel by automatic steering (automatic steering), etc.

[0056] The judgment criterion setting unit 314 sets judgment criteria used when predicting risky behavior. An example of the judgment criterion is the above-mentioned judgment threshold. The judgment criterion setting unit 314 changes the judgment criterion as appropriate. The reason for providing the judgment criterion setting unit 314 that changes the judgment criterion will be described with reference to FIG. 5.

[0057] FIG. 5 is a schematic diagram illustrating the state of an intersection. At the intersection in FIG. 5, the direction of arrow X is a red light, and the direction of arrow Y is a green light. In FIG. 5, vehicle V1 is a taxi, and it has stopped in response to a signal from pedestrian P1. This causes traffic to stagnate in the left lane in the direction of arrow X. Taking advantage of this traffic stagnation, pedestrian P2 crosses diagonally in the direction of solid thick arrow A1. Because the light in the direction of arrow X is red, pedestrian P2's crossing is illegal, ignoring the traffic signal, and constitutes a jumping-out act.

[0058] In such a case, for example, pedestrian P3 who witnesses the behavior of pedestrian P2 may be tempted by pedestrian P2's behavior and illegally cross the street in the direction of dashed thick arrow A2. The behavior of pedestrian P3 can be said to be behavior based on the risky shift described above. In this embodiment, the judgment criterion setting unit 314 is provided with the aim of preventing collision accidents caused by dangerous pedestrian behavior based on such risky shift. By providing the judgment criterion setting unit 314, it is possible to change the judgment criterion depending on whether it is necessary to consider risky shift or not.

[0059] The judgment criterion setting unit 314 sets the judgment criterion used when predicting risky behavior based on the acquisition status of detection information of risky behaviors previously performed by other people in the surrounding area of ​​a location where a risky behavior prediction target exists. That is, the in-vehicle device (risky behavior prediction unit) 3 sets the judgment criterion used when predicting risky behavior based on the acquisition status of detection information of risky behaviors that have occurred in the surrounding area of ​​a location where a risky behavior prediction target exists. In this embodiment, the risky behavior is a pedestrian stepping out into the traffic area of ​​the vehicle V, and the detection information of the risky behavior is detection information of a pedestrian stepping out. To this end, in detail, the judgment criterion setting unit 314 sets the judgment threshold used when predicting a pedestrian stepping out based on the acquisition status of detection information of a pedestrian stepping out that has occurred in the surrounding area of ​​a location where a pedestrian whose stepping out prediction is to be made exists.

[0060] With this configuration, the setting of the determination criteria can be changed depending on whether a pedestrian is detected in the area surrounding the pedestrian whose jump-out is to be predicted or not. As a result, this configuration makes it possible to perform risky shift-based prediction of a pedestrian jumping out with increased safety.

[0061] The judgment criteria setting unit 314 (in-vehicle device (risky behavior prediction device) 3) sets the judgment criteria so that the probability of determining that a pedestrian (prediction target) will engage in risky behavior is higher when risky behavior detection information is acquired than when risky behavior detection information is not acquired. In this embodiment, once the judgment criteria are set, it is determined whether or not the pedestrian will jump out in front of the road in accordance with the set judgment criteria.

[0062] In this embodiment, the determination criterion is a threshold value set for determining the possibility that a pedestrian (a target of risky behavior prediction) will engage in risky behavior. More specifically, the determination criterion is a threshold value set for determining whether or not a pedestrian will engage in risky behavior. More specifically, the threshold value is the determination threshold value for the above-mentioned jumping-out prediction determination.

[0063] The determination threshold is set to a different value when the detection information of a pedestrian jumping out is acquired and when it is not acquired. Specifically, the determination threshold is set to a lower value when the detection information of a pedestrian jumping out is acquired than when it is not acquired. With this configuration, it is possible to perform a pedestrian jumping out prediction that takes into account a risky shift and improves safety, and the following effects are also obtained. That is, when no other person jumps out around the pedestrian and there is no need to consider a risky shift, the determination threshold can be set to a high value, so that the probability of determining that a pedestrian will jump out is not increased more than necessary. As a result, it is possible to prevent situations such as excessive warnings being issued based on the prediction of a pedestrian jumping out.

[0064] In this embodiment, when the judgment criteria setting unit 314 acquires detection information of a person jumping out from the information collection device 2, it specifically judges the following points regarding the acquired detection information and decides whether to change the setting of the judgment threshold value.

[0065] The judgment criteria setting unit 314 (risky behavior prediction device) determines whether to change the judgment criteria setting based on the presence or absence of detection information of a pedestrian jumping out in the surrounding area of ​​the pedestrian who is the target of the jumping out prediction. This allows the judgment threshold to be set taking into consideration the presence or absence of a risky shift, and allows for appropriate jumping out prediction. The surrounding area of ​​the pedestrian is an area within a certain distance range from the pedestrian. The certain distance range is, for example, a range of 50 to 200 meters square.

[0066] The judgment criterion setting unit 314 (risky behavior prediction device) determines whether to change the judgment criterion setting based on the presence or absence of detection information of a person running out during a period from the current time back to a predetermined time. This allows the judgment threshold to be set taking into consideration the presence or absence of a risky shift, and makes it possible to appropriately predict a person running out. The predetermined time is, for example, 5 to 10 minutes. The detection information of a person running out during a period from the current time back to a predetermined time can be said to be detection information of a person running out that occurred immediately in the past, based on the current time.

[0067] A detailed example of the process of setting the determination threshold (change determination process) by the determination criterion setting unit 314 will be described later.

[0068] <4. Processing flow for predicting risky behavior> Next, the flow of processing related to risky behavior prediction executed by the risky behavior prediction system 100 will be described.

[0069] [4-1. Processing performed by the information collection device] FIG. 6 is a flowchart illustrating the flow of processing executed by the information collection device 2. The flowchart shows the technical contents of a computer program that causes a computer to realize processing related to the collection of risky behavior detection information. The processing shown in FIG. 6 is constantly executed when the information collection device 2 is in a state where it can acquire photographic information from the camera 1. The camera 1 is provided to photograph the situation at an intersection, etc., and is basically in a constant state of photographing, periodically transmitting the photographed information to the information collection device 2. In the example shown in FIG. 6, the risky behavior is assumed to be running out into the roadway.

[0070] In step S1, the first controller 21 (acquisition unit 211) acquires a captured image from the camera 1. As described above, the acquisition unit 211 periodically acquires captured images from the camera 1. Note that the acquisition unit 211 may be configured to acquire captured images from multiple cameras 1. Once the captured images are acquired, the process proceeds to the next step S2.

[0071] In step S2, the first controller 21 (aggregating unit 212) processes the captured images acquired from the cameras 1 as appropriate to aggregate information. More specifically, the aggregating unit 212 processes the acquired captured images to perform a pop-out detection process. More specifically, the pop-out detection process is performed using not only the most recently acquired captured image, but also the captured images acquired earlier. Note that in a configuration in which captured images are acquired from multiple cameras 1, the aggregating unit 212 performs a pop-out detection process on the captured images from each camera 1. If the aggregating unit 212 detects a pop-out, it adds the detected information to a table 221 (see FIG. 3) for aggregating detection information. Note that there may be cases in which a pop-out is not detected from the captured image, in which case the detection information is not added to the table 221. When the information aggregation process for adding the pop-out detection information to the table 221 is completed, the process proceeds to the next step S3.

[0072] In step S3, the first controller 21 (distribution unit 213) determines whether or not it is necessary to distribute the detection information of jumping out. If the information aggregated in the table 221 contains detection information of jumping out that occurred within a period from the current time to a predetermined time going back, the distribution unit 213 determines that it is necessary to distribute the detection information of jumping out. On the other hand, if the information aggregated in the table 221 does not contain detection information of jumping out that occurred within a period from the current time to a predetermined time going back, the distribution unit 213 determines that it is not necessary to distribute the detection information of jumping out. Note that the period from the current time to a predetermined time going back is preferably within 5 to 10 minutes from the current time, as described above. If it is determined that it is necessary to distribute the detection information of jumping out (Yes in step S3), the process proceeds to the next step S4. If it is determined that it is not necessary to distribute the detection information of jumping out (No in step S3), the process returns to step S1, and the processes from step S1 onwards are performed.

[0073] In step S4, the first controller 21 (distribution unit 213) broadcasts detection information of jumping out that has occurred within a period from the present time to a predetermined time point. The detection information of jumping out includes the location and time of the jumping out, and the action details at the time of the jumping out (such as the jumping out direction). When the distribution process of step S4 is completed, the process returns to step S1, and the processes from step S1 onwards are performed.

[0074] The detection information of the jumping out that is broadcast by the distribution process of step S4 is received by the second communication unit 33 of the in-vehicle device 3 provided in the vehicle V that is present in the vicinity of the information collection device 2 that broadcast the detection information. In other words, the in-vehicle device 3 of the vehicle V receives the detection information that has occurred in the vicinity of the vehicle. As will be described below, the in-vehicle device 3 appropriately uses the received detection information of the jumping out to perform a prediction process of the jumping out.

[0075] It is also possible to configure the system so that the processing of step S3 is not performed and the current contents of table 221 are distributed when the information aggregation processing is completed (when the processing of step S2 is completed). However, in a configuration in which the processing of step S3 is performed, it is possible to prevent information on detection of jumping out that occurred prior to a predetermined time period from the current time from being broadcast. This makes it possible to avoid the distribution of unnecessary data that will not be used by the receiving side, thereby reducing the burden on communication processing.

[0076] [4-2. Processing performed by the in-vehicle device] FIG. 7 is a flowchart illustrating the flow of processing executed by the in-vehicle device (risky behavior prediction device) 3. The flowchart shows the technical contents of a computer program that causes a computer to realize a risky behavior prediction method. The processing shown in FIG. 7 starts when the vehicle V equipped with the in-vehicle device 3 starts to start and various processes for predicting risky behavior become executable. At the start of the processing in FIG. 7, the acquisition unit 311 included in the second controller 31 is able to acquire photographic information from the camera 34 and also able to acquire detection information of running out into the roadway distributed from the information collection device 2. Note that in the example shown in FIG. 7 as well, the risky behavior is assumed to be running out into the roadway, as in the example shown in FIG. 6.

[0077] In step S11, the second controller 31 (risky behavior prediction unit 312) attempts to detect a pedestrian from the image captured by the camera 34, and determines whether or not a pedestrian has been detected. The pedestrian detected in step S11 is a pedestrian who is a target for predicting whether or not a pedestrian will jump out. In addition, multiple pedestrians may be detected in step S11. In such a case, the processing from step S12 onwards is executed for each detected pedestrian. If it is determined that a pedestrian has been detected (Yes in step S11), the processing proceeds to the next step S12. If it is not determined that a pedestrian has been detected (No in step S11), the processing of step S11 is repeated.

[0078] In step S12, the second controller 31 (risky behavior prediction unit 312) acquires time-series change information of the position and movement speed of the detected pedestrian using the captured images. The time-series change information is acquired by collecting captured images for a predetermined period of time. Once the time-series change information has been acquired, the process proceeds to the next step S13.

[0079] In step S13, the second controller 31 (risky behavior prediction unit 312) calculates the degree of coincidence between the time-series change information acquired in step S12 and the time-series change patterns of the position and moving speed at the time of running out, which have been collected in advance as running out cases. Once the degree of coincidence is calculated, the process proceeds to the next step S14.

[0080] In step S14, the second controller 31 (the judgment criterion setting unit 314) determines whether or not it is necessary to change the judgment threshold for jumping out judgment. In detail, the judgment criterion setting unit 314 determines whether or not it is necessary to change the judgment threshold based on the current state of the judgment threshold and the jumping out detection information acquired from the information collection device 2.

[0081] The process of determining whether to change the judgment criteria will be described with reference to Fig. 8. Fig. 8 is a flowchart showing a detailed example of the process of step S14 in Fig. 7. The explanation of Fig. 8 is based on the following premise: The judgment criteria setting unit 314 sets different judgment threshold values ​​for cases where there is no appearance of another person, which requires consideration of a risky shift, and cases where there is. The former judgment threshold is called the normal threshold, and the latter threshold is called the alert-needed threshold. The alert-needed threshold is set to a smaller value than the normal threshold.

[0082] 8, the determination criterion setting unit 314 determines whether the current determination threshold is the normal threshold. If the current determination threshold is the normal threshold (Yes in step S141), the process proceeds to the next step S142. If the current determination threshold is not the normal threshold, that is, if the current threshold is the alert-required threshold (No in step S141), the process proceeds to step S145.

[0083] In step S142, the determination criterion setting unit 314 determines whether or not predetermined jumping out detection information is present in the information acquired from the information collection device 2. In this embodiment, the predetermined jumping out detection information is detection information of a pedestrian jumping out that has occurred in the surrounding area of ​​the pedestrian who is the subject of jumping out prediction and that has occurred within a predetermined time period going back from the current time. As described above, the surrounding area of ​​the pedestrian who is the subject of jumping out prediction is, for example, an area within 50 to 200 m from the pedestrian who is the subject of prediction. In addition, the predetermined time period is 5 to 10 minutes. If the predetermined jumping out detection information is present (Yes in step S142), the process proceeds to step S143. If the predetermined jumping out detection information is not present (No in step S142), the process proceeds to step S144.

[0084] In step S143, the determination criterion setting unit 314 determines (decides) that the current determination threshold needs to be changed. In detail, since it is necessary to take into account a risky shift, it is determined that the normal threshold needs to be changed to the alert-required threshold.

[0085] In step S144, the judgment criterion setting unit 314 judges (determines) that it is not necessary to change the current judgment threshold. In detail, since it is not necessary to take into account a risky shift, it is judged that it is not necessary to change the threshold in order to maintain the normal threshold.

[0086] In step S145, similarly to step S142, the determination criterion setting unit 314 determines whether or not predetermined jumping out detection information is included in the information acquired from the information collection device 2. If the predetermined jumping out detection information is included (Yes in step S145), the process proceeds to step S146. If the predetermined jumping out detection information is not included (No in step S145), the process proceeds to step S147.

[0087] In step S146, the judgment criterion setting unit 314 judges (determines) that it is not necessary to change the current judgment threshold. In detail, since it is necessary to take into consideration the risky shift, it is judged that it is not necessary to change the threshold in order to maintain the threshold when caution is required.

[0088] In step S147, the determination criterion setting unit 314 determines (decides) that the current determination threshold needs to be changed. In detail, since there is no need to consider a risky shift, it is determined that the threshold needs to be changed from the alert-required threshold to the normal threshold.

[0089] In this example, the determination of the setting of the judgment threshold (whether or not to change it) is made depending on whether or not predetermined jumping out detection information is present, but this is merely an example. For example, the risky behavior prediction device may be configured to set the judgment criteria based on the number of acquired detection information. For example, the judgment threshold that takes risky shifting into consideration may be set only when the number of acquired predetermined jumping out detection information is equal to or greater than a predetermined threshold. This prevents the judgment threshold from becoming the alert-required threshold even when risky shifting does not need to be considered. As a result, it is possible to prevent driving assistance such as issuing an alarm more than necessary.

[0090] As another example, the risky behavior prediction device may be configured to set the judgment criteria based on the direction of a person who has run out. The direction of a person who has run out can be obtained from the behavior details included in the running-out detection information. For example, in the example shown in FIG. 5, the traveling direction of vehicle V2 approaching the intersection is the X direction. Risky shifting increases the likelihood of running out in the X direction, which is the same risky behavior as ignoring a traffic light, but has little impact on pedestrians who want to proceed in the Y direction. Therefore, vehicle V2 (traveling in the X direction) is considered unlikely to collide with a pedestrian who has run out. Therefore, when predicting a running-out pedestrian in vehicle V2, if the only recent running-out pedestrian is in the X direction, the judgment threshold may be set to a normal threshold rather than a warning threshold that takes risky shift into consideration. Setting the judgment threshold in this way, taking the running-out direction into consideration, can prevent the judgment threshold from becoming the warning threshold even when risky shifting does not need to be considered. As a result, driving assistance such as issuing an alarm can be prevented from being performed more frequently than necessary.

[0091] 7, if it is determined in step S14 that the determination threshold needs to be changed (Yes in step S14), the process proceeds to the next step S15. If it is determined that the determination threshold does not need to be changed (No in step S14), the process proceeds to step S16.

[0092] In step S15, the second controller 31 (determination criterion setting unit 314) changes the determination threshold for jumping out determination from the current value. If the current determination threshold is the normal threshold, the determination criterion setting unit 314 changes the determination threshold to the alert-required threshold. If the current determination threshold is the alert-required threshold, the determination criterion setting unit 314 changes the determination threshold to the normal threshold. When the determination threshold change process is completed, the process proceeds to the next step S16.

[0093] In step S16, the second controller 31 (risky behavior prediction unit 312) compares the degree of match calculated in step S13 with a set judgment threshold. If the degree of match calculated in step S13 is equal to or greater than the judgment threshold, it is predicted that the pedestrian being predicted will run out into the roadway. If the degree of match calculated in step S13 is less than the judgment threshold, it is predicted that the pedestrian being predicted will not run out into the roadway. When the running out judgment process in step S16 is completed, the process returns to step S11, and the processes from step S11 onwards are carried out.

[0094] When the prediction process for pedestrians running out into the road is completed and it is predicted that a pedestrian will run out into the roadway, the driving support processing unit 313 (see FIG. 4 ) performs driving support processes such as issuing a warning (alarm) and applying the automatic brakes, as described above. In this embodiment, as described above, the vigilance-needed threshold is set smaller than the normal threshold, so that when the vigilance-needed threshold is set, it is easier to predict that a pedestrian will run out into the roadway even if the degree of match with past cases of pedestrians running out into the roadway is low. In other words, according to this example, in a state where the risk of a pedestrian running out into the roadway due to being tempted by the running out of the roadway by others (due to a risky shift) is high, it is easier to predict that a pedestrian will run out into the roadway, making it easier to take preventative measures such as issuing a warning. As a result, the probability of an accident occurring due to a pedestrian running out into the roadway can be reduced.

[0095] In the above, the determination threshold is set to either the normal threshold or the alert threshold, but this is merely an example. The determination threshold may be configured to be switched between three or more values ​​in stages depending on the acquisition status of the jump-out detection information. For example, the determination threshold may be configured to be gradually lowered as the number of jump-out detection information acquisitions increases. Furthermore, the threshold may be configured to be gradually switched depending on the elapsed time since the latest occurrence of a predetermined jump-out detection information. For example, the determination threshold may be configured to be the smallest at the most recent jump-out occurrence time, and to be gradually increased as the elapsed time increases.

[0096] <5. Points to note> Various technical features disclosed in the description of the present invention may be modified in various ways without departing from the spirit of the technical creation. Furthermore, multiple embodiments and modifications disclosed in the description of the present invention may be combined to the extent possible.

[0097] For example, in the above-described embodiment, the determination threshold is set in the in-vehicle device 3, but this is merely an example. The determination threshold may be set in the information collecting device 2. The determination threshold set in the information collecting device 2 may then be distributed to the in-vehicle device 3 provided in each vehicle V. In such a configuration, the risky behavior prediction device is made up of the information collecting device 2 and the in-vehicle device 3. Furthermore, a computer program for realizing the risky behavior prediction function is provided in a distributed manner in multiple devices.

[0098] Furthermore, in the above-described embodiment, the determination criterion is set by changing the determination threshold. However, instead of changing the determination threshold, the degree of match calculated by the risky behavior prediction unit 312 may be multiplied by a coefficient corresponding to the presence or absence of a risky shift. For example, in a situation where there is no risky shift, the coefficient by which the degree of match is multiplied may be 1, and in a situation where there is a risky shift, the coefficient by which the degree of match is multiplied may be 1.25. By adopting such a configuration, it becomes easier to predict that a pedestrian will run out even if the degree of match with past cases of pedestrians running out may be low. Furthermore, the configuration for gradually switching the determination threshold described above may be replaced with a configuration for changing the coefficient by which the multiplication is performed to multiple values.

[0099] In the above-described embodiment, the risky behavior prediction system 100 is configured to include the information collection device 2 provided separately from the in-vehicle device 3, but this is also merely an example. The risky behavior prediction system may be configured not to include the information collection device 2 as shown in the above-described embodiment.

[0100] FIG. 9 is a block diagram showing a schematic configuration of a modified risky behavior prediction system 100A. The modified risky behavior prediction system 100A does not include an information collection device 2. Although not shown, the modified risky behavior prediction system 100A includes a camera. The risky behavior prediction system 100A uses a camera provided in an in-vehicle device 3A as a camera for detecting pedestrians running out and as a camera for detecting pedestrians who are the target of running out prediction. The camera provided in the vehicle VA may be provided in the vehicle VA as a device separate from the in-vehicle device 3A.

[0101] In Fig. 9, the in-vehicle device 3A provided in each vehicle VA is configured to be able to communicate with the in-vehicle devices 3A provided in other vehicles VA by V2V communication (vehicle-to-vehicle communication). Note that, for convenience of explanation, the number of in-vehicle devices 3A (vehicles VA) is three in Fig. 9, but it is preferable that the number of in-vehicle devices 3A provided in the risky behavior prediction system 100A is greater than three.

[0102] 9, the in-vehicle device 3A has a function of distributing detection information of a vehicle jumping out and a function of predicting a vehicle jumping out. Note that the in-vehicle device 3A may be configured as a single device, or may be configured as a collection of multiple devices.

[0103] For example, suppose that an in-vehicle device 3A of a vehicle VA1 detects a person jumping out from a camera image. In this case, vehicle VA1 broadcasts detection information of the person jumping out from the vehicle VA1 using V2V communication. As a result, each in-vehicle device 3A provided in a vehicle VA2 and a vehicle VA3 present around vehicle VA1 acquires the detection information of the person jumping out from the vehicle VA1. The detection information of the person jumping out from the vehicle VA1 includes the location and time of the person jumping out from the vehicle VA1.

[0104] Each in-vehicle device 3A that has acquired the detection information of a pedestrian running out designates a certain range area (e.g., 200m square) from the position where the pedestrian ran out as a warning area for a predetermined period of time. When predicting whether a pedestrian in the warning area will run out, each in-vehicle device 3A changes the judgment threshold used to determine whether the pedestrian will run out from the normal threshold to the warning threshold, and predicts whether the pedestrian will run out. Even in the case of this modified example, it is possible to perform running out prediction taking into account a risky shift, thereby making it possible to perform running out prediction with increased safety. [Explanation of symbols]

[0105] 1, 34... Camera 2. Information gathering device 3. In-vehicle device (risky behavior prediction device) 100 Risky behavior prediction system P: Pedestrian (target of risky behavior prediction) V, VA...vehicle

Claims

1. Obtaining detection information of risky behavior occurring in the surrounding area of ​​a location where a target of risky behavior is predicted; setting a determination criterion to be used when predicting the risky behavior of the prediction target based on the acquisition status of the detection information; Risky behavior prediction device.

2. The determination criterion is set so that the probability that the prediction target is determined to perform risky behavior is higher when the detection information is acquired than when the detection information is not acquired. The risky behavior prediction device according to claim 1 .

3. the determination criterion is a threshold value set for determining the possibility that the prediction target will perform a risky behavior, The threshold is set to a different value depending on whether the detection information is acquired or not acquired. The risky behavior prediction device according to claim 1 .

4. determining whether to change the setting of the determination criteria based on whether or not there is detection information of the risky behavior occurring within a period from the present time to a time point preceding a predetermined time; The risky behavior prediction device according to claim 1 .

5. setting the determination criteria based on the number of acquired pieces of detection information; The risky behavior prediction device according to claim 1 .

6. setting the determination criteria based on the elapsed time from the occurrence of the risky behavior in the detection information; The risky behavior prediction device according to claim 1 .

7. the risky behavior detection information is detection information of a vehicle running into a traffic area, The determination criterion is set based on the direction in which the person who jumped out jumped out. The risky behavior prediction device according to claim 1 .

8. A risky behavior prediction method executed by a device, Obtaining detection information of risky behavior occurring in the surrounding area of ​​a location where a target of risky behavior is predicted; setting a determination criterion to be used when predicting the risky behavior of the prediction target based on the acquisition status of the detection information; Methods for predicting risky behavior.

9. A program for causing a computer to execute a risky behavior prediction method, The computer Obtaining detection information of risky behavior occurring in a surrounding area of ​​a location where a target of risky behavior is predicted; setting a determination criterion to be used when predicting the risky behavior of the prediction target based on the acquisition status of the detection information; A risky behavior prediction program that serves as a means to

10. a camera installed on at least one of a vehicle and infrastructure equipment installed on a road; an information collection device that collects risky behavior detection information obtained from the images captured by the camera; an in-vehicle device that predicts risky behavior using a set determination criterion; Equipped with In either the information collection device or the in-vehicle device, the determination criteria are set based on detection information of the risky behavior occurring in a peripheral area of ​​a location where the target of the risky behavior is present. Risky behavior prediction system.

11. A vehicle equipped with an on-board device, The in-vehicle device When a pedestrian who is a target for predicting whether or not a vehicle will suddenly jump out is present, predicting whether or not the pedestrian will suddenly jump out using a preset determination criterion; When it is predicted that the pedestrian will jump out into the road, a driving assistance process is performed; When detection information of a pedestrian running out in the vicinity of a location where the pedestrian is present is acquired, the determination criterion is set so that the probability of the pedestrian running out is higher than when the detection information is not acquired, and the determination is made as to whether the pedestrian will run out. vehicle.

Images