Moving body obstruction detection device

Abstract

The purpose is to provide a moving object obstruction detection device, system, method, and storage medium that can more accurately determine the crossing of a moving object compared to the method of estimating the movement state by detecting the leg opening of a pedestrian. The moving object obstruction detection unit (46) includes: an object detection unit (56) that performs the process of detecting objects such as vehicles, people, and bicycles existing in an image and enclosing the detected objects with bounding boxes; and an object state estimation unit (58) that estimates the state of the moving object (e.g., the state of the moving object related to crossing a road, such as crossing, waiting to cross, or near a pedestrian crossing) based on the position of the bottom edge of the bounding box (70) of the moving object detected by the object detection unit (56).

Description

Technical Field

[0001] This disclosure relates to a moving object obstruction detection device, a moving object obstruction detection system, a moving object obstruction detection method, and a storage medium for detecting obstructions relative to various moving objects such as pedestrians and bicycles. Background Technology

[0002] Japanese Patent Application Publication No. 2007-264778 discloses a pedestrian recognition device that detects pedestrians outside the vehicle, detects the state of the detected pedestrians, and determines whether the pedestrian will enter the vehicle's path of travel based on the state of the pedestrian's legs.

[0003] In Japanese Patent Application Publication No. 2007-264778, edge detection is used to estimate the movement state by detecting the opening of a pedestrian's left and right legs. However, depending on clothing, the legs may be obscured, sometimes making it impossible to accurately estimate the movement state. Furthermore, when a vehicle is turning left or right at an intersection and a pedestrian approaches from the inside of the crosswalk, the angle relative to the pedestrian (facing them) makes it difficult to accurately calculate the leg opening, sometimes resulting in an inability to estimate the movement state. Moreover, since the detection of moving objects such as bicycles is not considered, there is room for improvement. Summary of the Invention

[0004] Therefore, this disclosure provides a moving object obstruction detection device, a moving object obstruction detection system, a moving object obstruction method, and a storage medium that can more accurately determine the crossing of a moving object compared to the case of presuming the movement state by detecting the opening of a pedestrian's legs.

[0005] The first aspect of this disclosure is a moving object obstruction detection device, comprising: a detection unit for detecting a predetermined moving object in an image captured by a camera unit installed on a vehicle; and an estimation unit for estimating the moving object's movement state related to road crossing based on the position of a bounding box surrounding the moving object detected by the detection unit.

[0006] According to the first scheme, in the detection unit, a predetermined moving object is detected in the image captured by the camera unit installed on the vehicle.

[0007] In the estimation unit, the movement state of the moving body related to road crossing is estimated based on the position of the bounding box surrounding the moving body detected by the detection unit. By estimating the movement state of the moving body related to road crossing based on the position of the bounding box surrounding the moving body, the crossing of the moving body can be determined without detecting the leg opening of the pedestrian. Therefore, compared with the case where the movement state is estimated by detecting the leg opening of the pedestrian, the crossing of the moving body can be determined more accurately.

[0008] It should be noted that the estimation unit can estimate the movement state of the moving body based on the position of the bottom edge of the bounding box. By estimating the movement state based on the position of the bottom edge of the bounding box, the movement state can be estimated for moving bodies other than pedestrians, such as bicycles, and thus the crossing can be determined for moving bodies other than pedestrians.

[0009] Additionally, the system may include: a distance estimation unit that estimates the distance from the vehicle to the moving body; a behavior determination unit that determines the vehicle's behavior based on vehicle information indicating the vehicle's state; and a determination unit that determines obstruction to the moving body based on the moving body state estimated by the estimation unit, the distance estimated by the distance estimation unit, and the vehicle's behavior determined by the behavior determination unit. By estimating the distance from the vehicle to the moving body and determining the vehicle's behavior in this way, and determining obstruction to the moving body based on the moving body state, the distance from the vehicle to the moving body, and the vehicle's behavior, it is possible to determine whether or not obstruction to the moving body exists.

[0010] The second aspect of this disclosure can be a moving object obstruction detection system, comprising the aforementioned moving object obstruction detection device and a vehicle equipped with the aforementioned camera unit.

[0011] The third aspect of this disclosure is a moving object obstruction detection method, comprising the following steps: detecting a predetermined moving object in an image captured by a camera unit installed on a vehicle, and estimating the moving object's movement state related to road crossing based on the position of the bounding box surrounding the detected moving object.

[0012] The fourth aspect of this disclosure is a non-transitory storage medium storing a program that enables a computer to perform a moving object obstruction detection process, wherein the moving object obstruction detection process includes the following steps: detecting a predetermined moving object in an image captured by a camera unit located on a vehicle, and estimating the moving object's movement state related to road crossing based on the position of the bounding box surrounding the detected moving object.

[0013] As described above, according to this disclosure, a moving object obstruction detection device, a moving object obstruction detection system, a moving object obstruction method, and a storage medium are provided that can more accurately determine the crossing of a moving object compared to the case of presuming the movement state by detecting the opening of a pedestrian's legs. Attached Figure Description

[0014] Figure 1 This is a diagram showing the general structure of the dangerous driving detection system of this embodiment.

[0015] Figure 2This is a functional block diagram illustrating the functional structure of the vehicle-mounted device and the dangerous driving data collection server in the dangerous driving detection system of this embodiment.

[0016] Figure 3 This is a block diagram showing the structure of the control unit and the central processing unit.

[0017] Figure 4 This is a block diagram showing the detailed structure of the moving body obstruction detection unit of the dangerous driving data collection server in the dangerous driving detection system of this embodiment.

[0018] Figure 5 This is a diagram showing an example of a bounding box surrounding a vehicle and pedestrian as moving objects.

[0019] Figure 6 This is a diagram illustrating the driver's obligations when approaching a pedestrian crossing.

[0020] Figure 7 This is a flowchart illustrating an example of the processing flow performed by the moving body obstruction detection unit of the dangerous driving data collection server in the dangerous driving detection system of this embodiment.

[0021] Figure 8 This is a functional block diagram illustrating a modified example of the functional structure of the vehicle-mounted device and the dangerous driving data collection server in the dangerous driving detection system of this embodiment. Detailed Implementation

[0022] Hereinafter, an example of an embodiment of the present disclosure will be described in detail with reference to the accompanying drawings. Figure 1 This is a diagram showing the general structure of the dangerous driving detection system of this embodiment.

[0023] In the dangerous driving detection system 10 of this embodiment, the vehicle-mounted device 16 mounted on the vehicle 14 and the dangerous driving data collection server 12 are connected via a communication network 18. In this dangerous driving detection system 10, image information obtained by multiple vehicle-mounted devices 16 and vehicle information indicating the status of each vehicle are sent to the dangerous driving data collection server 12, which accumulates the image information and vehicle information. Then, the dangerous driving data collection server 12 performs processing to detect dangerous driving based on the accumulated image information and vehicle information. In this embodiment, as an example of detected dangerous driving, dangerous driving includes at least one of rapid acceleration and rapid deceleration, dangerous driving that fails to maintain a safe following distance, dangerous driving that obstructs movement, and dangerous driving that exceeds the speed limit.

[0024] Figure 2This is a functional block diagram illustrating the functional structure of the vehicle-mounted device 16 and the dangerous driving data collection server 12 in the dangerous driving detection system 10 of this embodiment.

[0025] The vehicle-mounted unit 16 includes a control unit 20, a vehicle information detection unit 22, a camera unit 24, a communication unit 26, and a display unit 28.

[0026] The vehicle information detection unit 22 detects vehicle information related to vehicle 14. Examples of vehicle information include the detection of vehicle 14's position, speed, acceleration, steering angle, accelerator opening, distance to obstacles around the vehicle, and path. Specifically, the vehicle information detection unit 22 can utilize various sensors and devices to obtain information indicating the condition of the vehicle 14's surrounding environment. Examples of sensors and devices include vehicle speed sensors and acceleration sensors mounted on vehicle 14, GNSS (Global Navigation Satellite System) devices, vehicle-mounted communication devices, navigation systems, and radar devices. The GNSS device receives GNSS signals from multiple GNSS satellites to determine the position of vehicle 14. The accuracy of the GNSS device increases as the number of GNSS signals it can receive increases. The vehicle-mounted communication device is a communication device that, via the communication unit 26, enables at least one of the following: vehicle-to-vehicle communication with other vehicles 14 and road-to-road communication with roadside vehicles. The navigation system includes a map information storage unit that stores map information. Based on the location information obtained from the GNSS device and the map information stored in the map information storage unit, it processes the data to display the location of the vehicle 14 on the map and guide the user along the path to the destination. Additionally, the radar device includes multiple radars with different detection ranges. It detects pedestrians, other vehicles 14, and other objects around the vehicle 14, and obtains the relative position and relative speed of the detected objects relative to the vehicle 14. Furthermore, the radar device has a built-in processing unit that processes the detection results of surrounding objects. This processing unit excludes noise, roadside objects such as guardrails, etc., from the monitored objects based on changes in relative position and relative speed with each object in the most recent multiple detection results, and tracks and monitors pedestrians, bicycles, other vehicles 14, etc. The radar device outputs information such as the relative position and relative speed with each monitored object.

[0027] In this embodiment, the camera unit 24 is mounted on the vehicle and captures images of the vehicle's surroundings, such as the front of the vehicle, generating image data representing a moving image. For example, a camera such as a dashcam can be used as the camera unit 24. It should be noted that the camera unit 24 can also further capture images of at least one of the vehicle's surroundings, namely the sides and rear of the vehicle 14. Additionally, the camera unit 24 can also further capture images of the vehicle interior.

[0028] The communication unit 26 establishes communication with the dangerous driving data collection server 12 via the communication network 18, and transmits and receives information such as image information obtained by the camera unit 24 and vehicle information detected by the vehicle information detection unit 22.

[0029] The display unit 28 provides various information to the occupants by displaying information. In this embodiment, it displays information such as information provided by the dangerous driving data collection server 12.

[0030] like Figure 3 As shown, the control unit 20 is composed of a typical microcomputer including a CPU (Central Processing Unit) 20A, ROM (Read Only Memory) 20B, RAM (Random Access Memory) 20C, memory 20D, interface (I / F) 20E, and bus 20F. Furthermore, the control unit 20 performs control functions such as uploading image information representing images captured by the imaging unit 24 and vehicle information detected by the vehicle information detection unit 22 during image capture to the dangerous driving data collection server 12.

[0031] On the other hand, the dangerous driving data collection server 12 has a central processing unit 30, a central communications unit 36, and a database 38.

[0032] like Figure 3 As shown, the central processing unit 30 is composed of a general-purpose microcomputer including a CPU 30A, ROM 30B and RAM 30C, a memory 30D, an interface (I / F) 30E, and a bus 30F. The central processing unit 30 has the functions of an information collection unit 40, a rapid acceleration / deceleration detection unit 42, a vehicle distance maintenance failure detection unit 44, a moving object obstruction detection unit 46 (an example of a moving object obstruction detection device), an overspeed detection unit 48, and a dangerous driving detection collection unit 50. It should be noted that each function of the central processing unit 30 is implemented by the CPU 30A executing programs stored in the ROM 30B, etc.

[0033] The information gathering unit 40 obtains vehicle information such as vehicle speed, acceleration, and position information from the DB38, as well as animation frames that are image information captured by the imaging unit 24. It performs time alignment between the vehicle information and the animation frames to synchronize the vehicle information and the animation frames, thereby gathering the information. It should be noted that, hereinafter, the gathered information will sometimes be referred to as the gathered information.

[0034] The rapid acceleration / deceleration detection unit 42 detects dangerous driving based on the collected information gathered by the information gathering unit 40, specifically the rapid acceleration and rapid deceleration. For example, it detects whether the vehicle speed or acceleration matches a predetermined dangerous driving condition and whether the surrounding conditions correspond to such a condition based on image information and vehicle information. Alternatively, it may detect dangerous driving based solely on vehicle information, specifically the vehicle speed and acceleration that match a predetermined dangerous driving condition.

[0035] The vehicle distance non-maintaining detection unit 44 detects dangerous driving where the vehicle distance is not maintained within a predetermined distance based on the collected information gathered by the information gathering unit 40. For example, it detects a vehicle ahead based on image information and vehicle information, and detects that the distance to the vehicle ahead is within a predetermined distance, thereby detecting dangerous driving where the vehicle distance is not maintained.

[0036] The moving object obstruction detection unit 46 detects dangerous driving that obstructs moving objects such as pedestrians or bicycles based on the collected information gathered by the information gathering unit 40. For example, based on image information and vehicle information, it detects pedestrian crossings ahead and pedestrians meeting predetermined conditions, and detects whether they cross without stopping or slowing down, thereby detecting dangerous driving that obstructs moving objects. Regarding pedestrians meeting predetermined conditions, for example, it detects pedestrians who are crossing the pedestrian crossing, pedestrians near the pedestrian crossing, or pedestrians who are about to cross the pedestrian crossing.

[0037] The speeding detection unit 48 detects dangerous speeding based on the collected information gathered by the information collection unit 40. For example, based on image information and vehicle information, it identifies signs through image recognition and detects speeds exceeding a predetermined speed limit according to the identified signs, thereby detecting dangerous speeding. Alternatively, it can determine whether it is a regular road or a highway based on location information and detect speeds exceeding a predetermined speed limit on each road.

[0038] The dangerous driving detection aggregation unit 50 aggregates the dangerous driving detected by the rapid acceleration / deceleration detection unit 42, the inter-vehicle distance maintenance detection unit 44, the moving object obstruction detection unit 46, and the speeding detection unit 48, and comprehensively judges dangerous driving. For example, when detecting each dangerous driving, the danger level can be calculated within the range of 0 to 1, the average danger level of each dangerous driving can be calculated, and the average value above a predetermined threshold can be comprehensively judged as dangerous driving. Alternatively, the presence or absence of each dangerous driving can be detected as 0 (no detection) or 1 (detection), and the sum of the detection results can be derived as the comprehensive danger level. Alternatively, when detecting each dangerous driving, a score for each dangerous driving can be derived, the sum of the scores can be calculated, and the sum of the scores above a predetermined threshold can be comprehensively judged as dangerous driving. Alternatively, regarding the detection of each dangerous driving, no detection can be 0, detection can be 1, the detection results of each dangerous driving can be added, and the result of 1 or higher or above a predetermined threshold can be judged as dangerous driving.

[0039] It should be noted that when detecting each of the four types of dangerous driving, the driving scenario can be determined based on the collected information. The detection thresholds and weights for dangerous driving can then be adjusted according to the driving scenario to detect dangerous driving corresponding to that scenario. For example, when driving on a highway, the weight of the "failure to maintain a safe distance" judgment can be increased to raise the level of danger. Additionally, in rainy conditions, the weight of the "speeding" judgment can be increased to raise the level of danger. Furthermore, the detection threshold for "pedestrian obstruction" can be lowered in situations with poor visibility, such as at night or in fog (e.g., lowering the speed threshold from below 20 km / h to 10 km / h) to make it easier to detect. Furthermore, the detection thresholds for each type of dangerous driving can be adjusted based on the past accident rate at the same driving location to facilitate detection. Moreover, when combining different driving scenarios, the weighting can be further increased. For example, in rainy weather at night, the weighting of dangerous driving can be increased or the threshold for dangerous driving judgment can be lowered to facilitate detection.

[0040] The central communications unit 36 ​​establishes communication with the vehicle-mounted device 16 via the communication network 18 to send and receive information such as image information and vehicle information.

[0041] DB38 receives image information and vehicle information from the vehicle-mounted device 16, establishes a correspondence for each received image information and vehicle information, and stores it.

[0042] In the dangerous driving detection system 10 configured as described above, the image information captured by the camera unit 24 of the vehicle-mounted device 16, along with vehicle information, is sent to the dangerous driving data collection server 12 and stored in the DB38.

[0043] The dangerous driving data collection server 12 processes data to detect dangerous driving based on image and vehicle information stored in the DB38. Furthermore, the dangerous driving data collection server 12 provides various services, such as providing feedback on the detection results to the driver.

[0044] Here, the detailed structure of the moving object obstruction detection unit 46 described above will be explained. Figure 4 This is a block diagram showing the detailed structure of the moving body obstruction detection unit 46 of the dangerous driving data collection server 12 in the dangerous driving detection system 10 of this embodiment.

[0045] like Figure 4 As shown, the moving object obstruction detection unit 46 has the functions of an acquisition unit 52, a horizon detection unit 54, an object detection unit 56 (an example of a detection unit), an object state estimation unit 58 (an example of an estimation unit), a distance estimation unit 60, a vehicle behavior detection unit 62 (an example of a behavior determination unit), and a moving object obstruction determination unit 64 (an example of a determination unit). Furthermore, it is assumed that pre-extracted regression formulas (details described later) are stored in the memory, DB38, of the dangerous driving data collection server 12.

[0046] The acquisition unit 52 acquires the collected information obtained by the information collection unit 40, which collects image information and vehicle information. The image information is output to the horizon detection unit 54, and the vehicle information is output to the distance estimation unit 60 and the vehicle behavior detection unit 62.

[0047] The horizon detection unit 54 sequentially acquires image information from the collected information and detects the horizon in the image. The detected horizon is used to correct the vehicle's tilt in the longitudinal direction, which is a result of the installation error of the imaging unit 24, when estimating the distance to an object in the captured image.

[0048] As a method for detecting the horizon by the horizon detection unit 54, for example, all straight lines existing in the image are extracted, and road-related lines are extracted from the extracted lines. Then, the vanishing point is derived from the intersection of the extracted lines, and the y-coordinate of the vanishing point is used as the horizon for detection. It should be noted that the horizontal direction of the image captured by the imaging unit 24 is set as the x-axis, and the direction orthogonal to the x-axis is set as the y-axis.

[0049] In detail, the horizon detection unit 54 includes image preprocessing, line extraction within the image, horizon estimation, and time series processing steps. In the image preprocessing step, the image is converted to grayscale and contour lines are extracted using edge detection. In the line extraction step, lines are extracted using a probabilistic Hough transform, and a threshold is set on the tilt of the lines to exclude lines such as buildings and power lines, thus extracting only straight lines from roads. In the horizon estimation step, intersection points are derived from the combination of all extracted lines, deviation values ​​are removed by setting a threshold on the coordinates of the intersection points, and the y-coordinate value of the horizon is calculated based on the average of all intersection points. In the time series processing step, the mode of the horizon values ​​from past frames is calculated and set as the horizon value for the current frame.

[0050] The object detection unit 56 performs various well-known object detection processes to detect objects such as vehicles, people, and bicycles present in the image and encloses the detected objects with bounding boxes. Furthermore, when detecting objects, it determines the type of object within the bounding box. For example, ... Figure 5 As shown by the dashed lines, a bounding box 70 is generated to surround blocks (objects) that meet predetermined conditions. The type of object within the bounding box 70 is determined, and the movement of objects such as vehicles, people, and bicycles is detected. It should be noted that... Figure 5 This is a diagram showing an example of a bounding box 70 that surrounds vehicles and pedestrians that are moving objects.

[0051] The object state estimation unit 58 estimates the state of the moving body (e.g., crossing a pedestrian crossing, waiting to cross at a pedestrian crossing, or near a pedestrian crossing, etc.) based on the position of the bottom edge of the bounding box 70 of the moving body detected by the object detection unit 56. The object state estimation unit 58 estimates whether the moving body is crossing a pedestrian crossing, waiting to cross at a pedestrian crossing, or near a pedestrian crossing, etc., based on the position and changes of the bounding box. It should be noted that, as a driver's obligation when approaching a pedestrian crossing, for example, there are... Figure 6The three scenarios are shown. The first is slowing down to a point where a person is present near the crosswalk, allowing for a temporary halt. The second is temporarily stopping to allow passage when a person is about to cross or is currently crossing. The third is temporarily stopping to allow passage when overtaking a vehicle that is stopped near the crosswalk. Therefore, the object state estimation unit 58 estimates the state of moving objects related to crossing, such as crossing, waiting to cross, or near the crosswalk, based on the position and changes of the bottom edge of the boundary frame 70. For example, if the bottom edge of the boundary frame 70 is moving on the crosswalk, it is estimated that the object is crossing. Furthermore, if the bottom edge of the boundary frame 70 is stopped within a predetermined distance from the crosswalk, it is estimated that the object is waiting to cross. Additionally, if the bottom edge of the boundary frame 70 is moving towards the crosswalk, it is estimated that the object near the crosswalk may be about to cross.

[0052] In addition, the object state estimation unit 58 estimates whether there is an intention to cross based on the actions and orientation of moving objects such as pedestrians.

[0053] The distance estimation unit 60 estimates the distance from the vehicle 14 to the moving object detected by the object detection unit 56 based on the image captured by the imaging unit 24. For example, the distance to the object is estimated using a correspondence for estimating the distance to the object based on the position coordinates of the bottom edge of the bounding box 70. This correspondence is pre-derived using a set of data containing the position coordinates of the bottom edge of the bounding box 70 surrounding the moving object detected by the object detection unit 56 and the positive value of the distance from the vehicle (or from the imaging unit 24). In this embodiment, a regression formula is used as an example of the correspondence, and the distance from the shooting position of the imaging unit 24 to the moving object is estimated by setting the position coordinates of the bottom edge of the bounding box 70 as input. That is, the position of the bottom edge of the bounding box 70 on the image becomes the position corresponding to the distance to the moving object, so the distance to the moving object can be estimated by a regression formula pre-derived based on the position of the bottom edge of the bounding box 70. It should be noted that the regression formula, which is derived in advance as a set of data representing the position coordinates of the bottom edge of the bounding box 70 of the object and the positive value of the distance from the vehicle, is, for example, the regression formula shown below. The following regression formula is pre-saved in memory, DB38, etc., and the distance to the object is estimated by inputting the y-coordinate of the position coordinates of the bounding box 70 into the following regression formula. In the following regression formula, since the position coordinates of the bottom edge of the bounding box 70 are corrected using the position coordinates of the horizon, the tilt of the camera unit 24 in the vehicle's longitudinal direction, which contributes to the installation error of the camera unit 24, can be corrected.

[0054] height_cor = video_H / 720

[0055] Distance = 15.87 * math.exp(-(0.021 / height_cor) * (y - horizon * height_cor))

[0056] It should be noted that video_H is set to the vertical pixel count of the camera 24, height_cor is set to the correction value of the vertical pixel count corresponding to the camera 24, y is set to the y coordinate of the bottom edge of the bounding box 70, and horizon is set to the y coordinate of the horizon.

[0057] Furthermore, the distance estimation unit 60 calculates the arrival time of the moving body. For example, the arrival time is calculated using the vehicle speed and estimated distance contained in the vehicle information obtained by the acquisition unit 52.

[0058] The vehicle behavior detection unit 62 determines whether the vehicle 14 has temporarily stopped or slowed down in front of the pedestrian crossing based on the vehicle information (vehicle speed, brake pressure, etc.) obtained by the acquisition unit 52, thereby detecting the vehicle behavior.

[0059] The moving object obstruction determination unit 64 determines whether there is an obstruction to the moving object based on the estimated state of the moving object and the actions (vehicle behavior) of the vehicle 14. For example, if the moving object does not stop temporarily while crossing the road and proceeds straight, or if a moving object is detected near a pedestrian crossing but the moving object does not slow down and proceeds straight, it is determined that there is an obstruction.

[0060] Next, the specific processing performed by the moving body obstruction detection unit 46 of the dangerous driving data collection server 12 in the dangerous driving detection system 10 of this embodiment, configured as described above, will be explained. Figure 7 This is a flowchart illustrating an example of the processing flow performed by the moving body obstruction detection unit 46 of the dangerous driving data collection server 12 in the dangerous driving detection system 10 of this embodiment. It should be noted that... Figure 7 The processing begins, for example, at predetermined intervals or whenever the vehicle information and image information sent from the vehicle unit 16 and stored in the DB38 reaches a predetermined amount of data. Specifically, the CPU 30A executes a program stored in the ROM 30B, etc., causing each part of the central processing unit 30 to operate as follows.

[0061] In step 100, the acquisition unit 52 acquires vehicle information and image information from the collected information collected by the information collection unit 40, and moves to step 102.

[0062] In step 102, the object detection unit 56 detects moving objects such as vehicles, pedestrians, and bicycles, and moves to step 104. For example, the object detection unit 56 performs various well-known object detection processes to detect objects such as vehicles, people, and bicycles present in the image and encloses the detected objects with bounding boxes 70. In addition, when detecting objects, it determines the type of object within the bounding box 70, and detects the moving objects by identifying the type of object such as vehicles, pedestrians, and bicycles.

[0063] In step 104, the object state estimation unit 58 estimates the state of the detected moving object and moves to step 106. That is, the state of the moving object is estimated based on the position and changes of the bottom edge of the bounding box 70 of the moving object detected by the object detection unit 56 (e.g., the state of the moving object related to crossing, such as crossing, waiting to cross, or near a pedestrian crossing). In this embodiment, the state of a pedestrian or bicycle is estimated.

[0064] In step 106, the object state estimation unit 58 determines whether a moving object is present on or near the pedestrian crossing. This determination is based on the estimation result of the moving object state in step 104 (e.g., moving object state related to crossing, such as crossing, waiting to cross, or near the pedestrian crossing). If the determination is positive, the process proceeds to step 108; otherwise, the processing of the moving object obstruction detection unit 46 ends.

[0065] In step 108, the object state estimation unit 58 determines whether the moving object is on the pedestrian crossing based on the estimation result of the moving object's state. If the determination is affirmative, it proceeds to step 110. On the other hand, if the moving object is near the pedestrian crossing, the determination is negative and it proceeds to step 120.

[0066] In step 110, the distance estimation unit 60 estimates the distance to the moving object and moves to step 112. That is, the distance to the moving object is estimated using a regression formula, which is derived in advance using a set of data of the position coordinates of the bottom edge of the bounding box 70 of the moving object detected by the object detection unit 56 and the positive value of the distance from the vehicle (or the distance from the shooting unit 24).

[0067] In step 112, the distance estimation unit 60 estimates the arrival time of the moving body and the pedestrian crossing, and moves to step 114. For example, the arrival time is calculated using the vehicle speed and estimated distance included in the vehicle information obtained by the acquisition unit 52.

[0068] In step 114, the vehicle behavior detection unit 62 determines whether the estimated arrival time is below a predetermined threshold. If the determination is positive, the process proceeds to step 116; otherwise, the processing of the moving object obstruction detection unit 46 ends.

[0069] In step 116, the vehicle behavior detection unit 62 determines whether the vehicle 14 is parked. This determination is based on vehicle information from the collected information obtained by the acquisition unit 52. If the determination is negative, the process proceeds to step 118; if positive, the processing of the moving object obstruction detection unit 46 ends.

[0070] In step 118, the moving object obstruction determination unit 64 determines that the moving object obstructs dangerous driving and ends the processing of the moving object obstruction detection unit 46.

[0071] On the other hand, in step 120, the vehicle behavior detection unit 62 determines whether the vehicle 14 is moving slowly. This determination is based on vehicle information in the aggregated information obtained by the acquisition unit 52. If the determination is negative, the process proceeds to step 118; if positive, the processing of the moving object obstruction detection unit 46 ends.

[0072] Thus, in this embodiment, the movement state of the moving body related to crossing is estimated based on the position of the bounding box 70 surrounding the moving body. As a result, the crossing of the moving body can be determined without detecting the leg opening of the pedestrian, and therefore the crossing of the moving body can be determined more accurately than the case where the movement state is estimated by detecting the leg opening of the pedestrian.

[0073] Furthermore, in this embodiment, since the state of the moving body is estimated based on the position of the bottom edge of the bounding box 70, the moving body state can be estimated by including moving bodies such as bicycles other than pedestrians, and the crossing of the road can be determined by including moving bodies other than pedestrians.

[0074] It should be noted that the above embodiment illustrates an example where the processing of dangerous driving detection is performed on the dangerous driving data collection server 12, but it is not limited to this. For example, it can also be configured as follows: Figure 2 The functions of the central processing unit 30 are as follows: Figure 8 The control unit 20, which is provided on the side of the vehicle-mounted device 16 as shown, performs [operations]. Figure 7 The processing can be handled by the control unit 20. Specifically, the functions of the information collection unit 40, the rapid acceleration / deceleration detection unit 42, the inter-vehicle distance maintenance detection unit 44, the moving object obstruction detection unit 46, the speeding detection unit 48, and the dangerous driving detection collection unit 50 can be integrated into the control unit 20. In this case, the information collection unit 40 obtains vehicle information such as vehicle speed, acceleration, and position information from the vehicle information detection unit 22, and obtains animation frames from the imaging unit 24. Alternatively, these functions can be configured to reside on other external servers, etc.

[0075] Furthermore, in the above embodiments, the movement state of the moving body is estimated based on the position of the bottom edge of the bounding box, but it is not limited to this. For example, the movement state of the moving body can also be estimated based on the position of other edges of the bounding box besides the bottom edge.

[0076] Furthermore, in the above-described embodiments, four types of dangerous driving—rapid acceleration / deceleration, failure to maintain a safe following distance, obstruction of pedestrians, and speeding—were used as examples, but the description is not limited to these. For example, it could include two or three of the four types. Alternatively, it could include other types of dangerous driving besides the four. Examples of other dangerous driving include, for instance, failing to stop temporarily, running red lights, road rage, dangerous tailgating, cutting in line, failing to use turn signals when changing lanes or turning left or right, driving without lights at night, driving against traffic, obstructing the road ahead (such as the overtaking lane), parking outside of a parking space, occupying a disabled parking space, parking on the side of the road, inattentive driving, fatigued driving, and distracted driving.

[0077] Furthermore, in the above embodiment, a regression equation was described as an example of using a correspondence to estimate the distance of an object based on the position coordinates of the bottom edge of the bounding box 70. However, the correspondence is not limited to a regression equation, and correspondences other than regression equations can also be used. For example, a table pre-derived based on a regression equation can also be used as a correspondence.

[0078] Furthermore, the processing performed by the moving body obstruction detection unit 46 of the dangerous driving data collection server 12 in the above embodiments has been described as software processing performed by the CPU 30A, but it is not limited to this. For example, it may also be performed using hardware such as a processor, i.e., a dedicated circuit, with a circuit structure specifically designed for performing specific processing, such as a GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), and FPGA (Field-Programmable Gate Array). It may be executed using one of these various processors, or it may be executed using a combination of two or more processors of the same or different types (e.g., multiple FPGAs and a combination of CPU and FPGA). In addition, the hardware structure of these various processors is more specifically a circuit composed of circuit elements such as semiconductor elements. Alternatively, it may be configured as a combination of both software and hardware processing. In addition, when the software is configured for processing, the program can be stored on various storage media such as CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), and USB (Universal Serial Bus) for circulation.

[0079] Moreover, this disclosure is not limited to the above, and can of course be implemented in various variations without departing from its main purpose.

Claims

1. A moving object obstruction detection device, comprising: The detection unit detects predetermined moving objects in images captured by a camera unit installed on the vehicle. The estimation unit estimates the movement state of the moving body in relation to the crossing of the road's crosswalk, including the movement state of the moving body that is crossing the crosswalk, the movement information of the moving body waiting to cross the crosswalk, and the movement state of the moving body that may cross the crosswalk, based on the position of the bounding box surrounding the moving body detected by the detection unit. The horizon detection unit detects the horizon in the image; The distance estimation unit uses the detection results of the horizon detection unit to correct the installation error of the shooting unit, and estimates the distance from the vehicle to the moving body based on the position of the bounding box; The behavior determination unit determines the behavior of the vehicle based on vehicle information indicating the state of the vehicle. as well as The determination unit determines whether there is any obstruction to the moving body based on the state of the moving body estimated by the estimation unit, the distance estimated by the distance estimation unit, and the behavior of the vehicle determined by the behavior determination unit. The determination unit determines that the driver violated their duty when approaching the pedestrian walkway. The detection unit detects objects such as vehicles, people, and bicycles present in the image and encloses the detected objects using bounding boxes. The estimation unit estimates the state of the moving body based on the position of the bottom edge of the bounding box of the moving body detected by the detection unit. The distance estimation unit uses a correspondence to estimate the distance to the object based on the position coordinates of the bottom edge of the bounding box. This correspondence is derived in advance using a data set of positive values ​​of the distance from the vehicle to the bottom edge of the bounding box surrounding the moving body detected by the detection unit.

Images