Electronic devices

Abstract

To provide an electronic device which controls a vehicle by predicting a region, in which a collision accident may occur, beforehand.SOLUTION: An electronic device (100) comprises: a position acquisition section (114) for acquiring an own position of an own vehicle (10); a map acquisition section (113) for acquiring map information; an attention object determination section (122) for determining an attention object based on the own position and the map information; a visual line information acquisition section (111) for acquiring visual line information of a driver of the vehicle; and a visual line determination section (121) for determining whether or not a visual line of the driver is directed to the attention object based on the visual line information.SELECTED DRAWING: Figure 1

Description

【Technical Field】 【0001】 The present invention relates to an electronic device. 【Background Art】 【0002】 In recent years, various automatic driving assistance technologies such as automatic driving in which a driver can intervene and driving assistance for vehicles (technologies that notify a driver of surrounding situations such as ADAS) have been proposed and realized. In the case of a vehicle equipped with an automatic driving assistance device (hereinafter, an autonomous vehicle), there is a limit to the response speed of emergency braking. 【0003】 Therefore, for example, on so-called residential roads where the road width is relatively narrow and the sidewalks are not well maintained, there is a high possibility of accidents due to sudden emergence of pedestrians or other people. In particular, in places where a large number of people gather, such as parks, schools, or elderly facilities, the flow of people (hereinafter also referred to as the human flow) is large, and there is a high possibility of accidents such as sudden emergence. 【0004】 Patent Document 1 describes a technology of a driver monitoring support device that evaluates a driver's monitoring level from peripheral information detecting the approaching direction and speed of a monitoring target in the surrounding traffic of the host vehicle, line-of-sight information detecting the line-of-sight direction of the driver of the host vehicle, and operation information detecting the operation of the host vehicle by the driver, and provides attention and control information to the driver according to the monitoring level. 【Prior Art Documents】 【Patent Documents】 【0005】 【Patent Document 1】 Japanese Patent Application Laid-Open No. 2019-012454 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0006】 In the method by which autonomous vehicles provide drivers with warnings and control information as described above, since they only provide warnings and control information to the driver, even with a high level of monitoring, they cannot respond to sudden emergences from blind spots in situations that were not anticipated in advance, which could lead to collisions. 【0007】 The present invention has been made in view of the above points, and its object is to provide an electronic device that predicts in advance areas where collision accidents may occur and controls the vehicle. [Means for solving the problem] 【0008】 To solve the above problems, the electronic device of the present invention is A position acquisition unit that acquires the vehicle's own position, Map acquisition unit that acquires map information, A unit for determining objects of concern based on its own position and the map information, A gaze information acquisition unit that acquires gaze information of the driver of the vehicle, The system is characterized by comprising a gaze determination unit that determines whether the driver is looking at the object of attention based on the gaze information. [Effects of the Invention] 【0009】 According to the present invention, it is possible to provide an electronic device that predicts in advance areas where collision accidents may occur and controls the vehicle accordingly. Further features related to the present invention will become apparent from the description herein and the accompanying drawings. In addition, problems, configurations, and effects other than those described above will become apparent from the following description of embodiments. [Brief explanation of the drawing] 【0010】 [Figure 1] A functional block diagram showing the schematic configuration of the electronic device of the first embodiment. [Figure 2] A flowchart of the driver monitoring and control process in the electronic device of the first embodiment. [Figure 3] An overview view showing an example of the object detection process and line-of-sight detection process in the electronic device shown in Figure 1. [Figure 4] An overview view showing an example of the hazardous area determination process in the electronic device shown in Figure 1. [Figure 5] A functional block diagram showing the schematic configuration of the electronic device of the second embodiment. [Modes for carrying out the invention] 【0011】 Embodiments of the present invention will be described below with reference to the drawings. In the following embodiments, the example of applying the electronic device to a left-hand traffic road will be used, but the electronic device can also be applied to a right-hand traffic road. 【0012】 <First Embodiment> Figure 1 is a functional block diagram showing the schematic configuration of the electronic device of the first embodiment, and Figure 2 is a flowchart of the driver monitoring and control process in the electronic device of the first embodiment. 【0013】 The electronic device 100 is a device installed in a vehicle (the vehicle itself) 10 that has an automated driving assistance system that provides driving assistance to the driver. The automated driving assistance system includes, for example, a device that notifies the driver of the vehicle of danger through projection or sound, and a device related to the deceleration of the vehicle, and the electronic device 100 provides these automated driving assistance systems with gaze determination results, which will be described later. 【0014】 The electronic device 100 is composed of a computer system that includes, for example, a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), input / output circuits, and communication circuits (all not shown). The electronic device 100 is realized when the CPU reads and executes computer programs stored in RAM, ROM, etc. 【0015】 The electronic device 100 is a device that predicts attention objects from the pedestrian flow information, map information, and position information described later, and notifies the driver or decelerates the host vehicle according to the driver's line of sight. As functions realized by the execution of a computer program, as shown in FIG. 1, the electronic device 100 includes an acquisition unit 110 and a determination unit 120. 【0016】 The acquisition unit 110 includes a line-of-sight information acquisition unit 111, a pedestrian flow information acquisition unit 112, a map acquisition unit 113, and a position acquisition unit 114. The determination unit 120 includes a line-of-sight determination unit 121, an attention object determination unit 122, a notification determination unit 123, and a deceleration determination unit 124. 【0017】 The line-of-sight information acquisition unit 111 acquires the driver's line-of-sight information from, for example, the driver monitoring camera 131 and provides it to the attention object determination unit 122. The driver's line of sight can be specified by monitoring the angle of the neck, the line of sight, etc. with the driver monitoring camera 131 installed in the vehicle. The method for acquiring the driver's line-of-sight information is not limited to, for example, the driver monitoring camera 131. The angle of the neck can be acquired with a gyro sensor worn by the driver to specify the direction in which the driver is facing, or the attention object degree described later can be specified by the movement of the driver's eyeballs by the driver monitoring camera 131. 【0018】 The pedestrian flow information acquisition unit 112 identifies people such as pedestrians and bicycles in front of the host vehicle 10 and stores them in the electronic device 100 as pedestrian flow information. The identification of people is performed using images, videos, etc. acquired from the vehicle front monitoring camera 132, drive recorder, etc. mounted on the host vehicle 10. Note that the method for identifying people may be to identify the attributes of people and evaluate the level of people who are likely to jump out in front of the vehicle. The pedestrian flow information acquisition unit 112 acquires the pedestrian flow information from the image captured by the vehicle front monitoring camera 132 and stores it in the electronic device 100. The pedestrian flow information acquisition unit 112 may acquire the pedestrian flow information acquired by other vehicles via the server. 【0019】 The map acquisition unit 113 acquires map information of a high-precision map including property information of attention properties such as parks, schools, and elderly facilities. The map information can be acquired from the map DB 133 of the car navigation mounted on the own vehicle 10. The high-precision map can associate property information with crowd flow information. The map acquisition unit 113 may receive map information from outside the electronic device 100 via a communication network. 【0020】 The position acquisition unit 114 acquires the current running position of the own vehicle 10 in coordinates. The coordinates of the current position of the own vehicle 10 can be acquired, for example, by GNSS (Global Navigation Satellite System) 134 or GPS (Global Positioning System). Then, by specifying the orientation of the own vehicle 10 with a gyro sensor such as an IMU (Inertial Measurement Unit), a more accurate determination of the current position can be made. Note that the position acquisition unit 114 may determine the current position of the vehicle itself, or may receive the coordinates of the current position of the vehicle from outside the electronic device 100. 【0021】 The line-of-sight determination unit 121 associates the line-of-sight information acquired by the line-of-sight information acquisition unit 111 with the determination result determined by the attention property determination unit 122, determines whether the driver is gazing in the direction of the attention property A, and provides the determination result to the notification determination unit 123 and the deceleration determination unit 124. The line-of-sight determination unit 121, for example, accumulates information on the driver's line-of-sight direction in the past α seconds, and determines that the driver is gazing in the direction of the attention property A when the number of times the line of sight in the direction of the attention property A is β seconds or more is γ times or more. The values of α, β, and γ may be determined from data accumulated with past driver line-of-sight information, or may be determined from data related to line-of-sight information acquired from the outside. 【0022】 The object of concern determination unit 122 determines whether or not object of concern A exists in front of the currently moving vehicle 10 based on the vehicle's current position (self position) and map information, and provides the determination result to the line of sight determination unit 121 and the deceleration determination unit 124. The map information includes object information, and the object of concern determination unit 122 can determine whether or not an object in front of the vehicle 10 is object of concern A based on this object information. Furthermore, the object of concern determination unit 122 can perform a more sophisticated determination of whether or not an object is object of concern A by adding and using pedestrian flow information. 【0023】 The notification determination unit 123 determines to notify the driver when the gaze determination unit 121 determines that the driver is not looking at object A. For example, the notification unit may display external information on an in-vehicle monitor, such as the notification device 141 installed in the vehicle 10, notify the driver with an alarm sound, or notify the driver by vibrating the steering wheel or other parts using a vibration device. 【0024】 The deceleration determination unit 124 determines whether or not to decelerate the vehicle 10 based on the determination results of the gaze determination unit 121, the object of concern determination unit 122, and the notification determination unit 123. When the gaze determination unit 121 determines that the driver is still not looking at object of concern A despite the notification device 141 having notified the driver, the deceleration determination unit 124 determines to decelerate the vehicle 10 with the braking device 142 before the vehicle 10 reaches the area of ​​the road facing object of concern A (hereinafter referred to as the dangerous area) B. 【0025】 The electronic device 100 is connected to, for example, a driver monitoring camera 131, a vehicle forward monitoring camera 132, a map database (map DB) 133, a GNSS 134, a notification device 141, and a braking device 142. The driver monitoring camera 131 detects the gaze of the driver operating the vehicle 10. The vehicle forward monitoring camera 132 is a device that captures images of the area in front of the vehicle 10, and may be a monocular camera, a stereo camera, or a LiDAR or millimeter-wave radar. 【0026】 Next, we will explain the driver monitoring and control process with reference to Figure 2. In S201, the position acquisition unit 114 acquires the coordinates of the current location of the vehicle 10. In S202, the map acquisition unit 113 acquires map information of the area around the vehicle 10, including property information. The map acquisition unit 113 acquires map information in map mesh units (e.g., 2km x 2km). In S203, the pedestrian flow information acquisition unit 112 acquires pedestrian flow information. For locations where pedestrian flow information is not yet available, the process in S203 is skipped and the process proceeds to S204. 【0027】 In S204, the object of concern determination unit 122 identifies the current position of the vehicle 10 and object information around the vehicle 10 from coordinate information from S201 and map information from S202. Next, based on the vehicle's position, map information (object information), and pedestrian flow information acquired in S201 to S203, it determines whether an object of concern A is one from which people may suddenly appear. For example, if pedestrian flow information indicating that a large number of people are gathered at a facility on the side of the road has been acquired in advance, that facility is determined to be an object of concern A. On the other hand, if pedestrian flow information indicating that the facility is unoccupied has been acquired in advance, that facility is determined not to be an object of concern A. S201 to S203 are performed, for example, at 100ms intervals. However, 100ms is just one example, and the interval time can be changed as appropriate. 【0028】 If there is no object of concern A in front of the currently moving vehicle 10, the process proceeds to S210 to determine whether to continue the driver monitoring and control process. On the other hand, if it is determined that object of concern A is in front of the vehicle 10, that is, in the direction of travel, the process proceeds to S205. In S205, the gaze information acquisition unit 111 acquires the driver's gaze information and provides the acquired information to the gaze determination unit 121. Then, in S206, the gaze determination unit 121 determines from the gaze information whether the driver's gaze is directed towards object of concern A, that is, whether the driver is fixated on the direction of object of concern A. 【0029】 Figure 3 is an overview diagram illustrating an example of the object detection process and line-of-sight detection process in the electronic device shown in Figure 1, and Figure 4 is an overview diagram illustrating an example of the hazardous area detection process in the electronic device shown in Figure 1. 【0030】 The road 20 shown in Figures 3 and 4 is a one-way road or a residential road with a relatively narrow road width W1 that allows two-way traffic. Along the side of the road 20, there is a facility, which has been determined by the attention object determination unit 122 to be an attention object A. Inside the vehicle 10, a driver monitoring camera 131 is installed to capture images of the driver. In this embodiment, the case where there is one driver monitoring camera 131 is shown, but there may be multiple cameras, and in that case the installation positions of the cameras can be determined arbitrarily. 【0031】 The gaze determination unit 121 identifies two boundary points between the object of concern A and the road 20: boundary point A1 on the near side in the direction of travel from the perspective of the vehicle 10, and boundary point A2 on the far side in the direction of travel. Then, it calculates the angular range (hereinafter referred to as the object of concern degree) θ that is enclosed between the straight lines L1 and L2, respectively, when the driver monitoring camera 131 connects boundary points A1 and A2. In the gaze determination process, if the driver of the vehicle 10 is looking within the object of concern degree θ, it is determined that the driver's gaze is directed towards object of concern A (Yes in S206). If the gaze determination unit 121 determines that the driver's gaze is directed towards object of concern A, it terminates the processing of this flow. 【0032】 If the gaze determination unit 121 determines that the driver's gaze is not directed towards object A (No in S206), the process proceeds to S207. 【0033】 In S207, the object of caution determination unit 122 determines whether the vehicle 10 has reached the hazardous area B, which is the road area in front of object of caution A. The object of caution determination unit 122 determines that the vehicle 10 has entered the hazardous area B if the vehicle 10 has crossed the boundary line B1 set on the road 20 (Yes in S207). 【0034】 The object detection unit 122 sets the boundary line B1 by drawing a virtual straight line along the width of the road 20 from the boundary point A1 of object A located on the near side in the direction of travel from the vehicle 10. Note that the method of setting the boundary line B1 by the object detection unit 122 is an example and can be changed as appropriate. 【0035】 The warning object determination unit 122 determines that when the vehicle 10 is traveling along the road 20 in the direction of travel from bottom to top as shown in Figure 4, the vehicle 10 has entered the danger zone B from the point where the front end of the vehicle 10 crosses the boundary line B1. The line of sight determination unit 121 determines that the vehicle 10 has already entered the danger zone B, and terminates the processing of this flow. On the other hand, if the vehicle 10 has not yet entered the danger zone B, that is, before entering the danger zone B, the process proceeds to S208 to notify the driver. In S208, the notification determination unit 123 sends a notification instruction to the notification device 141, and the notification device 141 notifies the driver that it is approaching the danger zone. After the notification by the notification device 141, the process proceeds to S209. 【0036】 In S209, the deceleration determination unit 124 slows down the vehicle 10 to a speed at which the possibility of collision with a person who suddenly jumps out onto the road 20 from object A is low. The method of deceleration is just one example; for example, it may be done by slowing down or coming to a complete stop, not just by decelerating. 【0037】 In S210, the attention object determination unit 122 determines the status of the vehicle 10. If the power to the vehicle 10 is ON (in a drivable state), the processes in S201, S202, and S203 are repeated. If the power to the vehicle is OFF (in an drivable state), this flow is terminated. 【0038】 The electronic device 100 of this embodiment predicts in advance a dangerous area B where there is a high probability of a person suddenly appearing, and notifies the driver if the driver is not looking in the direction of object A before reaching dangerous area B. Furthermore, if the driver is still not looking in the direction of object A even after being notified, the device may determine and control whether to decelerate. According to the electronic device 100 of this embodiment, it is possible to predict in advance a dangerous area where there is a high probability of a person suddenly appearing, based on self-position information, object information, and gaze information. Therefore, it can respond to sudden appearances of people from blind spots, enabling safer automated driving assistance. Moreover, in this embodiment, by narrowing the decision of whether or not to perform notification processing and deceleration processing to object A, traffic congestion caused by deceleration and the inconvenience experienced by the driver can be further reduced. 【0039】 The electronic device 100 of this embodiment predicts areas where collisions are likely to occur based on its own position and map information, and if the driver is not paying attention to the direction of object A, it provides notification and slows down, thereby reducing the likelihood of a collision. Furthermore, by focusing on the limited area of ​​object A, the accuracy of deceleration is improved, which can further reduce traffic congestion and the inconvenience experienced by the driver. 【0040】 <Second Embodiment> Next, a second embodiment of the present invention will be described with reference to Figure 5. Figure 5 is a functional block diagram showing the schematic configuration of an electronic device 100A according to a second embodiment of the present invention. Components and operations similar to those in the first embodiment are denoted by the same reference numerals and their descriptions are omitted. 【0041】 A distinctive feature of this embodiment is the presence of a server 500 that transmits and receives information with multiple vehicles. The server 500 acquires and aggregates pedestrian flow information from multiple vehicles, links it to map information to form an information group, and provides this information group to the electronic device 100A. The electronic device 100A according to this embodiment differs from the electronic device 100 according to the first embodiment in that it has a communication unit 200 that transmits and receives information with the server 500. 【0042】 The communication unit 200 is a communication device that enables the transmission and reception of information with the outside of the vehicle via wireless communication lines such as mobile phone networks (3G, 4G, 5G, etc.) or WiFi. The communication unit 200 connects to the electronic device 100A via the bus 400 to transmit and receive information. The communication unit 200 transmits pedestrian flow information acquired by the vehicle's forward-facing surveillance camera 132 to the server 500 and receives a group of information from the server 500. Communication 300A illustrates the communication between the communication unit 200 and the server 500. 【0043】 Server 500 is connected to multiple vehicles, including its own, via communication 300A. Server 500 holds pedestrian flow information 501 and map information 502. Server 500 aggregates the pedestrian flow information transmitted from multiple vehicles, links it to the map information to form an information group, and transmits it to each vehicle. 【0044】 Communication 300B illustrates the communication between server 500 and security camera 800. Server 500 has the function of identifying people from multiple cameras located throughout the city, such as fixed cameras and security cameras (security camera 800) that can communicate with the outside world, and aggregating the transmitted pedestrian flow information at server 500. 【0045】 According to the system of this embodiment, which has an electronic device 100A and a server 500, the server 500 acquires and aggregates pedestrian flow information from multiple vehicles, links it to map information to form a group of information, and the electronic device 100A can receive this group of information. Therefore, it is possible to use pedestrian flow information acquired by other vehicles, and for example, even in a place that the vehicle 10 is driving in for the first time, it is possible to perform notification processing and deceleration processing using pedestrian flow information, thereby enabling safer automated driving support. 【0046】 Although embodiments of the present invention have been described in detail above, the present invention is not limited to the embodiments described above, and various design modifications can be made without departing from the spirit of the invention as described in the claims. For example, the embodiments described above are described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Moreover, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations. [Explanation of symbols] 【0047】 10: Own vehicle, 20: Road, 100, 100A: Electronic device, 111: Eye-line information acquisition unit, 112: Pedestrian flow information acquisition unit, 113: Map acquisition unit, 114: Position acquisition unit, 121: Eye-line determination unit, 122: Caution object determination unit, 123: Notification determination unit, 124: Deceleration determination unit, 131: Driver monitoring camera, 132: Vehicle front monitoring camera, 133: Map DB, 134: GNSS, 141: Notification device, 142: Braking device, 200: Communication unit, 300A, 300B: Communication, 400: Bus, 501: Pedestrian flow information, 502: Map information, 800: Security camera, θ: Caution object frequency, A: Caution object, A1, A2: Boundary point, B: Hazardous area, B1: Boundary line

Claims

[Claim 1] A position acquisition unit that acquires the vehicle's own position, Map acquisition unit that acquires map information, A unit for determining objects of concern based on its own position and the map information, A gaze information acquisition unit that acquires gaze information of the driver of the vehicle from a driver monitoring camera installed in the vehicle, A gaze determination unit that determines whether the driver is looking at the object of attention based on the gaze information, Equipped with, The gaze determination unit determines that the driver is looking at the object of attention if the driver is looking at the angular range between the line connecting the driver monitoring camera and the boundary point on the near side in the direction of travel of the object of attention, and the line connecting the driver monitoring camera and the boundary point on the far side in the direction of travel of the object of attention. An electronic device characterized by the following features. [Claim 2] The electronic device according to claim 1, further comprising a notification determination unit that determines to notify the driver when the gaze determination unit determines that the driver is not looking at the object of attention. [Claim 3] The electronic device according to claim 2, further comprising a deceleration determination unit that determines, based on the determination result of the line of sight determination unit, whether or not to decelerate the vehicle before it reaches the dangerous area facing the object of attention. [Claim 4] The electronic device according to claim 3, characterized in that the deceleration determination unit determines to decelerate the vehicle if, after notification based on the determination of the notification determination unit, the gaze determination unit determines that the driver is not looking at the object of attention. [Claim 5] The aforementioned map information includes property information, The system includes a pedestrian flow information acquisition unit that links the aforementioned property information with surrounding pedestrian flow information. The electronic device according to claim 1, characterized in that the attention property determination unit determines whether an item is a attention property based on the property information and the pedestrian flow information.

Images