Control device, mobile body, control method, and computer-readable storage medium

Abstract

The control device includes: a first determination section that determines a position of an object located ahead of a moving body in a forward direction; a time calculation section that calculates a time until the moving body reaches the position of the object determined by the first determination section; a second determination section that determines a position of an object in a direction intersecting the forward direction from an image captured by a camera provided to the moving body; a judgment section that judges whether or not a difference between the position of the moving body in the direction intersecting the forward direction and the position of the object is within a predetermined range, in a case where it is judged that the time calculated by the time calculation section is shorter than a predetermined threshold and the position of the moving body reaches the position determined by the first determination section in the forward direction; and a notification control section that performs notification control in a case where it is judged by the judgment section that the difference between the position of the moving body in the direction intersecting the forward direction and the position of the object is within the predetermined range.

Description

TECHNICAL FIELD

[0001] The present application relates to a control device, a moving body, a control method, and a computer-readable storage medium. BACKGROUND

[0002] In Patent Literature 1, it is described that, in a case where a time at which an acceleration exceeds a threshold value is within a collision prediction allowable time, the acceleration being sensed by a front collision occupant protection acceleration sensor for sensing a head-on collision and causing a safety airbag or the like to operate, it is determined that an object collides with the host vehicle, and the center is notified.

[0003] Patent Literature 1: Japanese Patent Application Publication No. 2020-169016 SUMMARY

[0004] In the first aspect, a control device is provided. The control device includes a first determination section that determines a position of an object that is located in front of a moving body in a forward direction. The control device includes a time calculation section that calculates a time until the moving body reaches the position of the object determined by the first determination section. The control device includes a second determination section that determines a position of the object in a direction intersecting the forward direction from an image captured by a camera provided to the moving body. The control device includes a judgment section that judges whether or not a difference between the position of the moving body in the direction intersecting the forward direction and the position of the object is within a predetermined range, in a case where it is judged that the time calculated by the time calculation section is shorter than a predetermined threshold value and the position of the moving body reaches the position determined by the first determination section in the forward direction. The control device includes a notification control section that performs notification control in a case where it is judged by the judgment section that the difference between the position of the moving body in the direction intersecting the forward direction and the position of the object is within the predetermined range.

[0005] The notification control section can perform the notification control in a case where it is judged by the judgment section that the difference between the position of the moving body in the direction intersecting the forward direction and the position of the object is within the predetermined range, and the time until the moving body reaches the position of the object determined by the first determination section is shorter than the predetermined threshold value.

[0006] The time calculation section can calculate the time until the moving body reaches the position of the object based on a time rate of change of a size of an image of the object extracted from an image captured by the camera.

[0007] The control device can further include an angular velocity acquisition section that acquires angular velocity information of the moving body from a sensor that is provided to the moving body and that detects rotational movement of the moving body. The second determination section can calculate the position of the moving body in the direction intersecting the direction of travel based on the angular velocity information of the moving body and the speed information of the direction of travel of the moving body.

[0008] The control device can further include a communication control section that controls reception of the position of a mobile terminal from the mobile terminal, the mobile terminal being in the position determined by the first determination section. The determination section can correct the predetermined range based on the position of the mobile terminal received from the mobile terminal.

[0009] The moving body can be a vehicle.

[0010] The notification control section can control a call to a call center with which a passenger of the vehicle can communicate.

[0011] The notification control section can perform notification control in a case where the vehicle is stopped.

[0012] The notification control section can perform notification control also in a case where an airbag provided to the vehicle is deployed.

[0013] After a collision mitigation brake provided to the vehicle starts to operate, the first determination section, the time calculation section, and the second determination section can also continue to operate, and the determination section can determine whether the difference between the position of the vehicle in the direction intersecting the direction of travel and the position of the object is within the predetermined range.

[0014] In the second aspect, a moving body is provided. The moving body includes a control device.

[0015] In the third aspect, a control method is provided. The control method includes a first determination step of determining a position of an object that is ahead in a direction of travel of a moving body. The control method includes a calculation step of calculating a time until the moving body reaches the position of the object determined in the first determination step. The control method includes a second determination step of determining a position of the object in a direction intersecting the direction of travel from an image captured by a capturing section provided to the moving body. The control method includes a determination step of determining whether a difference between the position of the moving body in the direction intersecting the direction of travel and the position of the object is within a predetermined range, in a case where it is determined that the time calculated in the calculation step is shorter than a predetermined threshold value and the position of the moving body reaches the position determined in the first determination step in the direction of travel. The control method includes a notification control step of performing notification control in a case where it is determined in the determination step that the difference between the position of the moving body in the direction intersecting the direction of travel and the position of the object is within the predetermined range.

[0016] In the fourth aspect, a program is provided. The program causes a computer to function as a first determination section that determines a position of an object located ahead of a moving body in a direction of travel of the moving body. The program causes the computer to function as a time calculation section that calculates a time until the moving body reaches the position of the object determined by the first determination section. The program causes the computer to function as a second determination section that determines a position of an object in a direction intersecting the direction of travel from an image captured by a capturing section provided to the moving body. The program causes the computer to function as a judgment section that judges whether or not a difference between the position of the moving body in the direction intersecting the direction of travel and the position of the object is within a predetermined range, in a case where it is judged that the time calculated by the time calculation section is shorter than a predetermined threshold value and the position of the moving body reaches the position determined by the first determination section in the direction of travel. The program causes the computer to function as a notification control section that performs notification control in a case where it is judged by the judgment section that the difference between the position of the moving body in the direction intersecting the direction of travel and the position of the object is within the predetermined range.

[0017] In addition, the above summary of the invention does not list all the essential features of the invention. Furthermore, combinations of branch features among the above features can also be inventions. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A use scene of the notification system 10 of one embodiment is schematically shown.

[0019] Figure 2 The system structure of the vehicle 20 is shown.

[0020] Figure 3 is a diagram for one example of a processing flow realized by the control device 40.

[0021] Figure 4 is a diagram for explaining processing when the vehicle 20 approaches the pedestrian 80.

[0022] Figure 5 An execution procedure of the control method executed by the control device 40 is shown.

[0023] Figure 6 An example of a computer 2000 that can embody the plurality of embodiments of the present invention in whole or in part is shown. DETAILED DESCRIPTION

[0024] Hereinafter, the present invention will be described by embodiments of the invention, but the following embodiments are not intended to limit the invention addressed by the claims. Furthermore, all combinations of features described in the embodiments are not essential to the solution of the invention.

[0025] Figure 1 The illustration schematically depicts a scenario where a notification system 10 according to one embodiment is used. The notification system 10 includes a vehicle 20 and a call center 70. The vehicle 20 is an example of a "moving body." A pedestrian 80 is an example of an "object" identified by the vehicle 20.

[0026] In the notification system 10, the vehicle 20 includes a sensor 29 and a control device 40. The sensor 29 includes, for example, a camera for capturing images of the front of the vehicle 20, and a yaw sensor. Here, the camera or yaw sensor can be separately positioned at different locations on the vehicle 20. For example, the sensor 29 is not limited to being configured at... Figure 1 The camera on the front of the vehicle 20 is positioned at a location where it can capture images of the front of the vehicle 20, such as the upper part of the windshield, the edge of the roof, or the roof itself. Images captured by the camera on the sensor 29 are continuously acquired, and pedestrian 80 is identified from the acquired images. Corresponding to the forward movement of the control device 40, the distance between the pedestrian 80 and the vehicle 20 decreases. Consequently, the image of the pedestrian 80 reflected in the image on the sensor 29 becomes larger. The control device 40 calculates the time until the vehicle 20 reaches the position of the pedestrian 80 based on the change in the size of the pedestrian 80's image and the speed of the vehicle 20.

[0027] If the time until the vehicle 20 reaches the pedestrian 80 is less than a predetermined threshold, the control device 40 calculates the position of the pedestrian 80 in a direction intersecting the vehicle 20's direction of travel from images continuously acquired by the camera equipped with sensor 29. Furthermore, the control device 40 calculates the position in a direction intersecting the vehicle 20's direction of travel based on information obtained from a yaw sensor. This direction intersecting the vehicle 20's direction of travel is, for example, orthogonal to the vehicle 20's direction of travel and substantially parallel to the road's travel surface. For ease of explanation, the direction intersecting the vehicle 20's direction of travel is sometimes referred to as the "lateral direction," and the vehicle 20's direction of travel is sometimes referred to as the "longitudinal direction."

[0028] If the control device 40 determines that the position of the vehicle 20 overlaps with the position of the pedestrian 80 in the longitudinal direction, it further determines whether the lateral position of the vehicle 20 overlaps with the lateral position of the pedestrian 80. When the control device 40 determines that the lateral position of the vehicle 20 overlaps with the lateral position of the pedestrian 80, it notifies the call center 70 via the network 90. ​​Thus, information obtained from the camera or yaw sensor of the sensor 29 can be used, without relying on the forward-leaning occupant protection accelerometer, to appropriately determine whether to notify the call center 70.

[0029] For example, in a system that determines whether to notify a call center based on the magnitude of acceleration detected by an accelerometer used for forward occupant protection, sometimes contact is not detected when a low-speed vehicle comes into contact with an object, thus failing to notify the call center. Conversely, when a large acceleration is detected while traveling on a road with poor conditions, a call center notification may sometimes be unnecessary.

[0030] In contrast, according to the control device 40, information acquired from the camera or yaw sensor of the sensor 29 can be used to appropriately determine whether to notify the call center 70, taking into account the respective positions of the vehicle 20 and the pedestrian 80 in the lateral direction. Therefore, for example, if it is determined that the positions of the vehicle 20 and the pedestrian 80 overlap in the lateral direction, notification to the call center 70 can be made, thereby improving safety. Furthermore, if it is determined that the positions of the vehicle 20 and the pedestrian 80 do not overlap in the lateral direction, useless notifications to the call center 70 can be avoided.

[0031] Figure 2 The system structure of vehicle 20 is shown. Vehicle 20 includes sensor 29, display device 32, communication device 34, and AEB 30.

[0032] The communication device 34 communicates with the call center 70 via the network 90. ​​The display device 32 notifies the passengers of the vehicle 20. The display device 32 may include devices that perform the display functions of HMI (Human Machine Interface), IVI (in-vehicle infotainment system), or MID (Multi Information Display).

[0033] Sensor 29 includes a camera 22, a vehicle speed sensor 24, and a yaw sensor 26. The camera 22 is an example of a capturing unit that takes pictures of the vehicle 20 in its forward direction to generate image information. The vehicle speed sensor 24 is mounted on a transmission or similar device and generates information indicating the vehicle speed of the vehicle 20. The yaw sensor 26 generates information indicating the yaw of the vehicle 20.

[0034] AEB30 is an Autonomous Emergency Braking system. AEB30 performs automatic braking based on information detected by sensor 29.

[0035] The control device 40 includes a processing unit 200 and a storage unit 280. The processing unit 200 is implemented, for example, by an arithmetic processing device including a processor. The storage unit 280 is implemented as a non-volatile storage medium. The processing unit 200 performs processing using information stored in the storage unit 280. The processing unit 200 can be implemented by an ECU (Electronic Control Unit) equipped with a microcomputer, which includes a CPU, ROM, RAM, I / O, and a bus, etc.

[0036] The processing unit 200 includes a first determination unit 210, a time calculation unit 230, a judgment unit 240, an angular velocity acquisition unit 250, a second determination unit 220, and a notification control unit 270.

[0037] The first determining unit 210 determines the position of an object located ahead of the vehicle 20 in the direction of travel. The time calculation unit 230 calculates the time until the vehicle 20 reaches the position of the object determined by the first determining unit 210. The second determining unit 220 determines the position of the object in a direction intersecting the direction of travel from an image captured by the camera 22. If the determining unit 240 determines that the time calculated by the time calculation unit 230 is shorter than a predetermined threshold and the position of the vehicle 20 in the direction of travel has reached the position determined by the first determining unit 210, it determines whether the difference between the position of the vehicle 20 in the direction intersecting the direction of travel and the position of the object is within a predetermined range. If the determining unit 240 determines that the difference between the position of the vehicle 20 in the direction intersecting the direction of travel and the position of the object is within a predetermined range, the notification control unit 270 performs notification control.

[0038] The notification control unit 270 performs notification control when the determination unit 240 determines that the difference between the position of the vehicle 20 in the direction intersecting the direction of travel and the position of the object is within a predetermined range, and the time until the vehicle 20 reaches the position of the object determined by the first determination unit 210 is less than a predetermined threshold.

[0039] The time calculation unit 230 can calculate the time until the vehicle 20 reaches the position of the object based on the time change rate of the image size of the object extracted from the image captured by the imaging unit. The angular velocity acquisition unit 250 acquires angular velocity information of the vehicle 20 from a sensor installed on the vehicle 20 that detects its rotational motion. The angular velocity acquisition unit 250 acquires the angular velocity information of the vehicle 20 based on information acquired from the yaw sensor 26. The second determination unit 220 calculates the position of the vehicle 20 in the direction intersecting the forward direction based on the angular velocity information of the vehicle 20 and the velocity information of the vehicle 20's forward direction.

[0040] The communication control unit 260 controls the location of the mobile terminal received from the mobile terminal, which is located at the position determined by the first determination unit 210. The communication control unit 260 controls the location of the mobile terminal received from the mobile terminal via the communication device 34. The determination unit 240 corrects a predetermined range based on the location of the mobile terminal received from the mobile terminal.

[0041] The notification control unit 270 can control calls to the call center 70, which can communicate with the passengers of the vehicle 20. The notification control unit 270 can notify the call center 70 of the location information of the vehicle 20. The notification control unit 270 can perform notification control even when the vehicle is stationary.

[0042] The notification control unit 270 can also perform notification control even when the airbags installed in the vehicle 20 have not deployed. After the AEB30 installed in the vehicle 20 starts working, the first determination unit 210, the time calculation unit 230, and the second determination unit 220 also continue to work, and the judgment unit 240 can determine whether the difference between the position of the vehicle in the direction intersecting the direction of travel and the position of the object is within a predetermined range.

[0043] Figure 3 This diagram is used to schematically illustrate an example of the processing flow implemented by the control device 40. In the vehicle 20, sensors 29 continuously perform processing to identify objects such as pedestrians 80. The first determination unit 210 determines the distance L from the vehicle 20 to the pedestrian 80. The time calculation unit 230 calculates the time until the vehicle 20 reaches the position of the pedestrian 80.

[0044] Based on information obtained from sensor 29 and considering factors such as the distance L from vehicle 20 to pedestrian 80 and vehicle speed, if AEB30 anticipates that vehicle 20 may be approaching pedestrian 80, it will issue an alert to the occupants of vehicle 20. Subsequently, if vehicle 20 approaches pedestrian 80 further, AEB30 will activate automatic braking. The occupants will then operate the foot brake to bring vehicle 20 to a stop.

[0045] When the determination unit 240 determines that the time until the vehicle 20 reaches the pedestrian 80 is less than a predetermined threshold and the vehicle 20 has reached the pedestrian 80 in the longitudinal direction, it performs an approach determination to the pedestrian 80. Specifically, the second determination unit 220 determines the lateral position of the vehicle 20 based on information obtained from the yaw sensor 26. Furthermore, the second determination unit 220 determines the lateral position of the pedestrian 80 from images continuously captured by the camera 22. The determination unit 240 determines the lateral position of the vehicle 20 from the information obtained from the yaw sensor 26 and determines whether the lateral position of the vehicle 20 and the lateral position of the pedestrian 80 are within a predetermined range. If the lateral position of the vehicle 20 is within the predetermined range relative to the lateral position of the pedestrian 80, the notification control unit 270 notifies the call center via the communication device 34. Furthermore, the notification control unit 270 can notify the occupants of the vehicle 20 via the display device 32 to notify the call center 70.

[0046] Figure 4 This diagram illustrates the processing when vehicle 20 approaches pedestrian 80. At time t1, the time calculation unit 230 extracts an image 412 of pedestrian 80 from image 410 captured by camera 22, and determines the size and position of image 412 within the image. At time t2, after time t1, it extracts an image 422 of pedestrian 80 from image 420 captured by camera 22, and determines the size and position of image 422 within the image. Based on the vehicle speed obtained by vehicle speed sensor 24, the time calculation unit 230 determines the travel distance D of vehicle 20 during the period from time t1 to time t2. Based on the ratio of the size of image 422 to the size of image 412, it calculates the time until vehicle 20 reaches the position of pedestrian 80 in the longitudinal direction. In addition, the time calculation unit 230 estimates the distance from vehicle 20 to pedestrian 80 based on the ratio of the size of image 422 to the size of image 412 and the travel distance D. Based on the distance from vehicle 20 to pedestrian 80 and the vehicle speed obtained by vehicle speed sensor 24, it calculates the time until vehicle 20 reaches the position of pedestrian 80 in the longitudinal direction.

[0047] Furthermore, the second determining unit 220 calculates the movement speed of the pedestrian 80 in the lateral direction based on the position difference Δx between the positions of image 412 and image 422 within the image, and times t2 and t1. The second determining unit 220 determines the position of the pedestrian 80 in the lateral direction based on the history of the pedestrian 80's movement speed. Additionally, the second determining unit 220 can calculate the relative movement speed of the pedestrian 80 with respect to the vehicle 20 in the lateral direction, and calculate the relative position of the pedestrian 80 with respect to the vehicle 20 in the lateral direction.

[0048] Figure 5The execution steps of the control method performed by the control device 40 are shown. In S502, the time calculation unit 230 determines whether a pedestrian is detected in the image captured by the camera 22. If no pedestrian is detected in the image, the determination in S502 is repeated. When a pedestrian is detected in the image captured by the camera 22, in S504, the time calculation unit 230 calculates the arrival time, which is the time until the vehicle 20 reaches the position of the pedestrian 80. (The last sentence appears to be incomplete and possibly refers to a different process.) Figure 4 As explained in connection with the above, the time calculation unit 230 can use the image captured by the camera 22 to calculate the arrival time.

[0049] In S506, it is determined whether the arrival time is shorter than a predetermined threshold 1. If the arrival time is higher than the predetermined threshold 1, the process proceeds to S504. If the arrival time is shorter than the predetermined threshold 1, in S508, the first determining unit 210 sets the distance to the pedestrian 80 detected in S502 as the target distance. In S510, the first determining unit 210 acquires information identified from the image captured by the camera 22. The information acquired in S510 includes the distance to the pedestrian 80 or the arrival time, etc. In S512, the first determining unit 210 acquires vehicle speed and angular velocity information from the vehicle speed sensor 24 and the angular velocity acquisition unit 250. In S514, the first determining unit 210 calculates the acceleration of the vehicle 20. In S516, the first determining unit 210 calculates the distance traveled by the vehicle 20 in the longitudinal direction. The first determining unit 210 calculates the travel distance based on the vehicle speed and time of the vehicle 20. Furthermore, the first determining unit 210 can correct the longitudinal movement distance of the vehicle 20 based on angular velocity information. In S518, the first determining unit 210 determines whether the target distance set in S508 has been reached. If the target distance has not been reached, the process is transferred to S504. In S518, if it is determined that the target distance has been reached, the process is transferred to S520.

[0050] In S520, the second determining unit 220 acquires information identified from the image captured by the camera 22. The information acquired in S520 is the lateral movement speed and position of the pedestrian 80. In S522, the second determining unit 220 acquires the angular velocity information acquired by the angular velocity acquisition unit 250. In S524, the second determining unit 220 calculates the angular velocity of the vehicle 20. In S526, the second determining unit 220 calculates the lateral positions of the vehicle 20 and the pedestrian 80. In S528, it is determined whether the difference between the lateral positions of the vehicle 20 and the pedestrian 80 calculated in S526 is within a predetermined range. If the difference between the lateral positions of the vehicle 20 and the pedestrian 80 is not within the predetermined range, the process proceeds to S504. If the difference between the lateral positions of the vehicle 20 and the pedestrian 80 is within the predetermined range, in S530, it is determined whether the arrival time is shorter than a predetermined threshold 2. Additionally, the arrival time can be information identified, for example, from an image captured by camera 22. If the arrival time is above a predetermined threshold 2, the process proceeds to S504. If the arrival time is below the predetermined threshold 2, in S532, the notification control unit 270 performs notification control. For example, the notification control unit 270 notifies the call center 70. Furthermore, the notification control unit 270 can display guidance information to the occupants of vehicle 20 via display device 32, which is used to notify the call center 70.

[0051] Furthermore, if pedestrian 80 possesses a mobile terminal capable of mobile communication or short-range wireless communication, the location information of the mobile terminal can be obtained from the mobile terminal held by pedestrian 80, and the obtained location information can be used to perform the aforementioned control. Communication between communication device 34 and the mobile terminal can be performed through direct communication. Communication device 34 can communicate directly with the mobile terminal via Cellular-V2X communication. Direct communication between communication device 34 and the mobile terminal can be achieved using Wi-Fi (registered trademark), DSRC (registered trademark) (Dedicated Short Range Communications), or any other direct communication method such as Bluetooth (registered trademark). Communication device 34 can also use the communication infrastructure provided by ITS (Intelligent Transport Systems) to communicate directly with the mobile terminal.

[0052] As an example, the communication control unit 260 obtains the location information of a mobile terminal located at a position determined by the first determination unit 210 via the communication device 34. For example, the communication control unit 260 sends a location request message via the communication device 34 using mobile communication or near-field communication, the location request message including the position determined by the first determination unit 210. When the mobile terminal receives the location request message from the communication device 34, if the distance between the mobile terminal's current position and the position included in the location request message is within a predetermined range, the mobile terminal sends a location request response to the communication device 34, the location request response including the mobile terminal's position. Upon receiving the response from the mobile terminal, the first determination unit 210 can correct the position of the pedestrian 80 determined by the first determination unit 210 based on the position of the mobile terminal included in the location request response. Furthermore, the determination unit 240 can correct the width of the range used in the determination in S528 based on the position of the mobile terminal included in the response received from the mobile terminal. For example, the greater the difference between the location of the mobile terminal included in the location request response and the location determined by the first determination unit 210, the wider the range used in the determination in S528 can be. Furthermore, when location request responses are received from multiple mobile terminals, the width of the range used in the determination in S528 can be increased. The notification control unit 270 can also correct the threshold 2 used in the determination in S530 based on the location of the mobile terminal included in the response received from the mobile terminal. For example, the greater the difference between the location of the mobile terminal included in the location request response and the location determined by the first determination unit 210, the larger the threshold 2 used in the determination in S530 can be. Furthermore, when location request responses are received from multiple mobile terminals, the threshold 2 used in the determination in S530 can be increased. Moreover, even if the determination unit 240 determines that the difference between the location of the vehicle 20 and the pedestrian 80 in the lateral direction is not within a predetermined range, the notification control unit 270 can perform notification control if the location included in the location request response obtained from the mobile terminal and the location of the vehicle 20 are within a predetermined range.

[0053] As explained above, the control device 40 can use information acquired from the camera 22 or yaw sensor 26 on the sensor 29 to consider the respective positions of the vehicle 20 and pedestrian 80 in the lateral direction to appropriately determine whether to notify the call center 70. Therefore, for example, if it is determined that the positions of the vehicle 20 and pedestrian 80 overlap in the lateral direction, a notification can be sent to the call center 70, thus improving safety. Furthermore, if it is determined that the positions of the vehicle 20 and pedestrian 80 do not overlap in the lateral direction, useless notifications to the call center 70 can be avoided.

[0054] Additionally, vehicle 20 is an example of a transportation device. A vehicle can be a car equipped with an internal combustion engine, an electric car, a fuel cell vehicle (FCV), etc. Cars include buses, trucks, two-wheeled motor vehicles, etc. A vehicle can be a saddle-type vehicle or a motorcycle. Transportation equipment can be any device used to transport people or goods. Transportation equipment is an example of a mobile body. A mobile body is not limited to transportation equipment and can be any movable device.

[0055] Figure 6 An example of a computer 2200 that can embody, in whole or in part, various embodiments of the present invention is shown. A program installed in the computer 2200 enables the computer 2000 to function as a device such as the control device 40 involved in the embodiments, or a component of that device, to perform operations related to that device or a component of that device, and / or to perform processes involved in the embodiments, or steps of that process. Such a program may be executed by the CPU 2012 for the purpose of causing the computer 2000 to perform the processing steps described in this specification and the defined operations related to some or all of the items in the blocks of the block diagram.

[0056] The computer 2000 of this embodiment includes a CPU 2012 and RAM 2014, which are interconnected by a main controller 2010. The computer 2000 also includes a ROM 2026, a flash memory 2024, a communication interface 2022, and an input / output chip 2040. The ROM 2026, flash memory 2024, communication interface 2022, and input / output chip 2040 are connected to the main controller 2010 via an input / output controller 2020.

[0057] CPU2012 operates according to the programs stored in ROM2026 and RAM2014, thereby controlling each unit.

[0058] The communication interface 2022 communicates with other electronic devices via a network. The flash memory 2024 stores programs and data usable by the CPU 2012 in the computer 2000. The ROM 2026 stores boot programs and / or programs dependent on the hardware of the computer 2000, which are executed by the computer 2000 upon activation. The input / output chip 2040 can also connect various input / output units such as keyboards, mice, and monitors to the input / output controller 2020 via input / output ports such as serial ports, parallel ports, keyboard ports, mouse ports, monitor ports, USB ports, and HDMI (registered trademark) ports.

[0059] The program is provided via a computer-readable storage medium or network, such as a CD-ROM, DVD-ROM, or memory card. RAM 2014, ROM 2026, or flash memory 2024 are examples of computer-readable storage media. The program is loaded into flash memory 2024, RAM 2014, or ROM 2026 and executed by CPU 2012. The information processing described in these programs is read into computer 2000, providing a link between the program and the aforementioned hardware resources of various types. The apparatus or method can be configured by a process that enables the manipulation or processing of information with the use of computer 2000.

[0060] For example, in the case of communication between computer 2000 and an external device, CPU 2012 may execute a communication program loaded in RAM 2014 and perform communication processing for communication interface 2022 based on the processing described in the communication program. Under the control of CPU 2012, communication interface 2022 reads transmission data stored in the transmission buffer processing area provided in a recording medium such as RAM 2014 and flash memory 2024, sends the read transmission data to the network, and writes received data received from the network into the receive buffer processing area provided on the recording medium, etc.

[0061] Alternatively, the CPU 2012 can read the entire contents or a required portion of a file or database stored in a recording medium such as flash memory 2024 into RAM 2014, and perform various processes on the data in RAM 2014. The CPU 2012 then writes the processed data back to the recording medium.

[0062] Information of various types, such as programs, data, data tables, and databases, can be stored in the recording medium and processed. The CPU 2012 can execute various operations described in this specification and specified by the program's instruction sequence on the data read from the RAM 2014, including information processing, conditional judgment, conditional branching, unconditional branching, information retrieval / replacement, etc., and write the results back to the RAM 2014. Furthermore, the CPU 2012 can retrieve information from files, databases, etc., in the recording medium. For example, when multiple entries are stored in the recording medium and each of these entries has an attribute value for a first attribute related to the attribute value of a second attribute, the CPU 2012 can retrieve from these multiple entries the entry that matches the condition specified by the attribute value of the first attribute, and read the attribute value of the second attribute stored in that entry, thereby obtaining the attribute value of the second attribute related to the first attribute that satisfies predetermined conditions.

[0063] The programs or software modules described above can be stored on or near the computer 2000 on a computer-readable storage medium. Recording media such as hard disks or RAM provided in server systems connected to a dedicated communication network or the Internet can be used as computer-readable storage media. The programs stored in the computer-readable storage medium can be provided to the computer 2000 via a network.

[0064] Programs installed on the computer 2000 and enabling the computer 2000 to function as a control device 40 can run in the CPU 2012 or similar, causing the computer 2000 to function as various parts of the control device 40. The information processing described in these programs is read into the computer 2000, thereby enabling each part of the control device 40 to function as a specific means for the software to work in conjunction with the various hardware resources described above. Then, by employing these specific means to perform the calculation or processing of information corresponding to the intended use of the computer 2000 in this embodiment, a control device 40 specific to that intended use can be constructed.

[0065] Various implementations have been described with reference to block diagrams, etc. In the block diagrams, each block may represent (1) a step in the process being executed or (2) a part of a device that performs the operation. Specific steps and parts may be implemented by dedicated circuitry, programmable circuitry supplied with computer-readable instructions stored on a computer-readable storage medium, and / or a processor supplied with computer-readable instructions stored on a computer-readable storage medium. Dedicated circuitry may include digital circuitry and / or analog hardware circuitry, and may include integrated circuits (ICs) and / or discrete circuitry. Programmable circuitry may include reconfigurable hardware circuitry including logic AND, logic OR, logic XOR, logic NAND, logic NOR, and other logic operations, and memory elements such as flip-flops, registers, field-programmable gate arrays (FPGAs), and programmable logic arrays (PLAs).

[0066] Computer-readable storage media can include any tangible device capable of storing instructions executable by a suitable device, such that a computer-readable storage medium having instructions stored in it constitutes at least a part of an product containing such instructions, which can be executed for the purpose of providing means for performing operations specified by a processing step or block diagram. Examples of computer-readable storage media include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media include floppy disks, optical disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), optical disc read-only memory (CD-ROM), digital versatile disk (DVD), Blu-ray (RTM) disk, memory stick, integrated circuit card, etc.

[0067] Computer-readable instructions may include any of the following: assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code described using any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, JAVA (registered trademark), C++, and existing procedural programming languages ​​such as the "C" programming language or similar programming languages.

[0068] This can be a computer-readable instruction provided to a processor or programmable circuit of a general-purpose computer, special-purpose computer, or other programmable data processing device via a local area network (LAN) or a wide area network (WAN) such as the Internet, and executed for the purpose of providing means for performing the operations specified by the described processing steps or block diagram. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0069] The present invention has been described above using embodiments, but the scope of the present invention is not limited to the scope described in the above embodiments. Those skilled in the art will recognize that various modifications or improvements can be made to the above embodiments. As can be seen from the description of the claims, embodiments made by applying such modifications or improvements can also be included within the scope of the present invention.

[0070] It should be noted that, regarding the execution order of the various processes such as operations, steps, stages, etc., in the apparatus, system, program, and method shown in the claims, specification, and drawings, they can be implemented in any order, unless specifically stated as "before," "prior to," etc., or it is stated that the output of the previous process is used in the subsequent process. For convenience, the workflow in the claims, specification, and drawings is described using terms such as "firstly," "secondly," etc., but even so, it does not mean that it must be implemented in this order.

[0071] Explanation of reference numerals in the attached figures

[0072] 22 cameras

[0073] 24 Vehicle speed sensor

[0074] 26 Yaw Sensor

[0075] 29 Sensors

[0076] 30 AEB

[0077] 32 Display devices

[0078] 34 Communication devices

[0079] 80 pedestrians

[0080] 210 First Determination Section

[0081] 220 Part 2

[0082] 230 Time Calculation Department

[0083] 240 Judgment Department

[0084] 250° angular velocity acquisition unit

[0085] 260 Communications Control Department

[0086] 270 Notification to Control Department

[0087] 280 Storage Department

[0088] 2000 Computer

[0089] 2010 Main Controller

[0090] 2012 CPU

[0091] 2014 RAM

[0092] 2020 Input / Output Controller

[0093] 2022 Communication Interface

[0094] 2024 Flash Memory

[0095] 2026 ROM

[0096] 2040 Input / Output Chip.

Claims

1. A control device, wherein, Possessing: a photographing section provided in a mobile body; a first determination section that determines a position of an object in front of a direction of advance of the mobile body from an image photographed by the photographing section; a time calculation section that calculates a time until the mobile body reaches the position of the object determined by the first determination section; a second determination section that determines a position of the object in a direction intersecting the direction of advance from the image photographed by the photographing section; a judgment section that judges whether or not a difference between the position of the mobile body in the direction intersecting the direction of advance and the position of the object is within a predetermined range in a case where it is judged that the time calculated by the time calculation section is shorter than a predetermined threshold value, at least one of a collision victim mitigation brake of the mobile body and a foot brake of an occupant of the mobile body starts to act, and the position of the mobile body reaches the position determined by the first determination section in the direction of advance; and a notification control section that judges that the object collides with the mobile body and performs notification control in a case where it is judged by the judgment section that the difference between the position of the mobile body in the direction intersecting the direction of advance and the position of the object is within the predetermined range.

2. The control device according to claim 1, wherein the notification control section performs the notification control in a case where it is judged by the judgment section that the difference between the position of the mobile body in the direction intersecting the direction of advance and the position of the object is within the predetermined range, and the time until the mobile body reaches the position of the object determined by the first determination section is shorter than the predetermined threshold value.

3. The control device according to claim 1 or 2, wherein the time calculation section calculates the time until the mobile body reaches the position of the object based on a time change rate of a size of an image of the object extracted from the image photographed by the photographing section.

4. The control device according to claim 1 or 2, wherein Further possessing: an angular velocity acquisition section that acquires angular velocity information of the mobile body from a sensor provided in the mobile body and detecting a rotational movement of the mobile body, the second determination section calculates the position of the mobile body in the direction intersecting the direction of advance based on the angular velocity information of the mobile body and speed information of the direction of advance of the mobile body.

5. The control device according to claim 1 or 2, wherein Further possessing: a communication control section that performs control to receive a position of a mobile terminal from the mobile terminal, the mobile terminal being in the position determined by the first determination section, the judgment section corrects the predetermined range based on the position of the mobile terminal received from the mobile terminal.

6. The control device according to claim 1, wherein the mobile body is a vehicle.

7. The control device according to claim 6, wherein the notification control section controls a call to a call center with which an occupant of the vehicle can talk.

8. The control device according to claim 6 or 7, wherein the notification control section performs the notification control in a case where the vehicle is stopped.

9. The control device according to claim 6 or 7, wherein the notification control section performs the notification control also in a case where deployment of an airbag provided in the vehicle is set.

10. The control device according to claim 6 or 7, wherein the first determination section, the time calculation section, and the second determination section continue to operate also after the collision victim mitigation brake provided in the vehicle starts to operate, and the judging section judges whether or not a difference between the position of the vehicle in a direction intersecting the advancing direction and the position of the object is within a predetermined range.

11. A mobile body, wherein provided with the control device according to any one of claims 1 to 10.

12. A notification control method in which, provided with: a first determination step of determining a position of an object from an image captured by a camera provided in a moving body, the object being located ahead of an advancing direction of the moving body; a calculation step of calculating a time until the moving body reaches the position of the object determined in the first determination step; a second determination step of determining a position of the object in a direction intersecting the advancing direction from the image captured by the camera; a judging step of judging whether or not a difference between the position of the moving body in the direction intersecting the advancing direction and the position of the object is within a predetermined range, in a case where it is judged that the time calculated in the calculation step is shorter than a predetermined threshold value, at least one of a collision victim mitigation brake of the moving body and a foot brake of an occupant of the moving body starts to act, and the position of the moving body reaches the position determined in the first determination step in the advancing direction; and a notification control step of judging that the object collides with the moving body, in a case where it is judged in the judging step that the difference between the position of the moving body in the direction intersecting the advancing direction and the position of the object is within the predetermined range, and performing notification control.

13. A computer readable storage medium storing a program, wherein, The program causes a computer to function as: a first determination section of determining a position of an object from an image captured by a camera provided in a moving body, the object being located ahead of an advancing direction of the moving body; a time calculation section of calculating a time until the moving body reaches the position of the object determined by the first determination section; a second determination section of determining a position of the object in a direction intersecting the advancing direction from the image captured by the camera; a judging section of judging whether or not a difference between the position of the moving body in the direction intersecting the advancing direction and the position of the object is within a predetermined range, in a case where it is judged that the time calculated by the time calculation section is shorter than a predetermined threshold value, at least one of a collision victim mitigation brake of the moving body and a foot brake of an occupant of the moving body starts to act, and the position of the moving body reaches the position determined by the first determination section in the advancing direction; and a notification control section of judging that the object collides with the moving body, in a case where it is judged by the judging section that the difference between the position of the moving body in the direction intersecting the advancing direction and the position of the object is within the predetermined range, and performing notification control.

Images