Obstacle information management device, obstacle information management method, and vehicle device

Abstract

When a plurality of vehicles pass near an obstacle registration point notified by a map server (2) as an obstacle, the plurality of vehicles upload data indicating the behavior of the vehicles as an obstacle location report to the map server (2). The map server (2) determines the appearance point of the obstacle based on the behavior data of the plurality of vehicles and registers it in an obstacle DB (251) as an obstacle location. In addition, the map server (2) determines that the obstacle has disappeared based on the fact that no action such as avoiding the obstacle is observed in the behavior of the vehicles passing near the obstacle registration point.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on patent application No. 2020-107961 filed in Japan on June 23, 2020, and the contents of the basic application are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates to an obstacle information management device and an obstacle information management method for determining the existence state of an obstacle that is an object that hinders the passage of a vehicle. Background Art

[0004] As a technology for detecting obstacles on the road, for example, Patent Document 1 discloses a configuration for detecting fallen objects on the road based on images from a vehicle-mounted camera. Specifically, Patent Document 1 discloses a configuration in which a vehicle uses a vehicle-mounted camera to confirm whether a fallen object notified from a server still remains, and sends the result back to the server, and the server updates the existence status of the fallen object based on the confirmation result from the vehicle. In addition, Patent Document 1 also mentions a configuration in which the server roughly predicts the time required to remove the fallen object based on the type of the fallen object, and distributes the predicted time to the vehicle.

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-40539

[0006] In the configuration disclosed in Patent Document 1, fallen objects are detected by analyzing the image of the vehicle-mounted camera. Therefore, in the configuration disclosed in Patent Document 1, the presence or absence of fallen objects can be confirmed only in the vehicle with the camera. In addition, the image recognition accuracy deteriorates in rainy days, at night, or in backlight, so there is a possibility of erroneously judging that there is no fallen object even though there is a fallen object. Summary of the invention

[0007] The present disclosure has been made based on this situation, and an object of the present disclosure is to provide an obstacle information management device, an obstacle information management method, and a vehicle device that can detect the disappearance of an obstacle without using an image from a vehicle-mounted camera.

[0008] The obstacle information management device for achieving this purpose comprises: a vehicle behavior acquisition unit, which acquires vehicle behavior data representing the behavior of multiple vehicles in correspondence with position information; an appearance determination unit, which determines the location where the obstacle appears based on the vehicle behavior data acquired by the vehicle behavior acquisition unit; and a disappearance determination unit, which determines whether the obstacle remains or disappears at the location determined by the appearance determination unit as the location where the obstacle exists, that is, the obstacle registration location, based on the vehicle behavior data acquired by the vehicle behavior acquisition unit.

[0009] According to the above configuration, the disappearance determination unit determines whether the obstacle remains or disappears based on the behavior of the vehicle passing through the obstacle registration location. In this configuration, it is not necessary to use the image of the vehicle-mounted camera. In other words, it is possible to detect the disappearance of the obstacle without using the image of the vehicle-mounted camera.

[0010] In addition, the obstacle information management method for achieving the above-mentioned purpose is a method for managing the location information of obstacles existing on the road, which is executed using at least one processor, and comprises: a vehicle behavior acquisition step, which establishes a correspondence to acquire vehicle behavior data representing the behavior of multiple vehicles at various locations; an appearance determination step, which determines the location where the obstacle appears based on the vehicle behavior data acquired in the vehicle behavior acquisition step; and a disappearance determination step, which determines whether the obstacle remains or disappears at the location determined as the location where the obstacle exists in the appearance determination step, that is, the obstacle registration location, based on the vehicle behavior data acquired in the vehicle behavior acquisition step.

[0011] According to the above configuration, the disappearance of an obstacle can be detected without using an image from a vehicle-mounted camera, by the same operation as the obstacle information management device.

[0012] The vehicle device for achieving the above-mentioned purpose is a vehicle device for sending information about the location where an obstacle exists on the road to a specified server, comprising: an obstacle location information acquisition unit, which acquires information about the obstacle registration location determined to be an obstacle by communicating with the server; and a vehicle behavior detection unit, which detects the behavior of the vehicle or at least any one of the other vehicles based on at least any one of the input signals from the vehicle state sensor that detects the physical state quantity representing the behavior of the vehicle, the input signals from the surrounding monitoring sensor, and the received data based on the vehicle-to-vehicle communication, and comprises: a report processing unit, which sends to the server vehicle behavior data representing the behavior of the vehicle and at least any one of the other vehicles when passing within a specified distance from the obstacle registration location.

[0013] The above-mentioned vehicle device sends to the server vehicle behavior data indicating the behavior of the vehicle or other vehicles when passing near the obstacle registration location notified from the server. Assuming that the obstacle remains and the vehicle / other vehicle is traveling on a lane with the obstacle, the vehicle behavior data received by the server becomes data indicating that the obstacle has been avoided. On the other hand, when the obstacle disappears, the vehicle behavior used to avoid the obstacle is not observed. In other words, the vehicle behavior data when passing near the obstacle registration location serves as an indicator of whether the obstacle remains. According to the above-mentioned vehicle device, information serving as a judgment material for whether the obstacle remains is concentrated on the server. The server can determine whether the obstacle still remains at the obstacle registration location or disappears based on the vehicle behavior data provided by multiple vehicles.

[0014] In addition, the reference numerals in parentheses in the claims indicate the correspondence with specific elements described in the embodiment described later as one mode, and do not limit the technical scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a diagram for explaining the configuration of the obstacle information distribution system 100 .

[0016] Figure 2 It is a block diagram showing the configuration of the vehicle-mounted system 1 .

[0017] Figure 3 It is a block diagram showing the structure of the positioner 14.

[0018] Figure 4 2 is a diagram showing an example of the obstacle notification image 80 .

[0019] Figure 5 It is a block diagram showing the structure of the map cooperation device 50.

[0020] Figure 6 This is a flowchart showing an example of upload processing.

[0021] Figure 7 This is a diagram for explaining the range of vehicle behavior data included in the obstacle location report.

[0022] Figure 8 This is a flowchart showing an example of upload processing.

[0023] Fig. 9 It is a flowchart showing an operation example of the map cooperation device 50.

[0024] Fig.10 This is a flowchart showing another example of the operation of the map cooperation device 50.

[0025] Fig.11 This is a flowchart corresponding to the lock processing of the report image.

[0026] Fig.12 This is a diagram schematically showing the operation of the lock processing of the report image.

[0027] Fig.13 It is a flowchart showing an operation example of the map cooperation device 50.

[0028] Fig.14 It is a block diagram showing the structure of the map server 2.

[0029] Fig.15 2 is a block diagram showing the functions of the map server 2 provided by the server processor 21 .

[0030] Fig.16 It is a flowchart for explaining the processing in the map server 2.

[0031] Fig.17 This is a diagram for explaining the operation of the appearance determination unit G31.

[0032] Fig.18 This is a diagram showing an example of how the verification area is set.

[0033] Fig.19 It is a diagram showing another example of the setting method of the verification area.

[0034] Fig. 20 This is a diagram showing an example of a criterion for the presence determination unit G31 to determine that an obstacle is present.

[0035] Fig.21 This is a diagram showing an example of a criterion for the disappearance determination unit G32 to determine that an obstacle has disappeared.

[0036] Fig. 22 This is a flowchart showing an example of vehicle control using obstacle information.

[0037] Fig.23 It is a diagram for explaining the effect of the obstacle information distribution system 100 .

[0038] Fig.24 This is a diagram for explaining a configuration in which the line of sight of a driver's seat passenger is used as a criterion for determining the presence or absence of an obstacle.

[0039] Fig.25 This is a diagram showing an example of a criterion used when the failure presence determination unit F51 calculates the detection reliability.

[0040] Fig.26 This is a diagram showing a modified example of the map server 2.

[0041] Fig. 27 This is a diagram schematically showing an example of a calculation rule of statistical reliability based on the map server 2.

[0042] Fig.28 This diagram shows the configuration of a system for dynamically setting and utilizing an area where automated driving is not possible based on obstacle information. DETAILED DESCRIPTION

[0043] Hereinafter, embodiments of the present disclosure will be described using drawings. Figure 1 FIG. 1 is a diagram showing an example of a schematic configuration of the obstacle information distribution system 100 of the present disclosure. Figure 1As shown, the obstacle information distribution system 100 includes a plurality of vehicle-mounted systems 1 and a map server 2 built in each of a plurality of vehicles Ma and Mb. Figure 1 For convenience, only two vehicles, vehicle Ma and vehicle Mb, are shown as vehicles equipped with the vehicle-mounted system 1, but there are actually more than three vehicles. The vehicle-mounted system 1 can be mounted on a vehicle that can travel on the road. In addition to four-wheeled vehicles, vehicles Ma and Mb can also be two-wheeled vehicles, three-wheeled vehicles, etc. Bicycles with prime movers can also be included in two-wheeled vehicles. From now on, from the perspective of the vehicle-mounted system 1, the vehicle equipped with the system (that is, itself) will also be recorded as the main vehicle.

[0044] <Overview of overall structure>

[0045] The vehicle-mounted system 1 mounted on each vehicle is configured to be wirelessly connectable to a wide area communication network 3. The wide area communication network 3 here refers to a public communication network provided by a telecommunications operator, such as a mobile phone network or the Internet. Figure 1 The base station 4 shown is a wireless base station for connecting the in-vehicle system 1 to the wide area communication network 3 .

[0046] Each vehicle-mounted system 1 sends a communication data packet indicating the status of the vehicle, i.e., a vehicle status report, to the map server 2 via the base station 4 and the wide area communication network 3 at a prescribed period. In the vehicle status report, in addition to the source information indicating the vehicle that sent the communication data packet (that is, the source vehicle), it also includes the time when the data was generated, the current position of the source vehicle, etc. The source information is identification information (the so-called vehicle ID) pre-assigned to the source vehicle to distinguish it from other vehicles. In addition to the above information, the vehicle status report may also include the direction of travel, driving lane ID, driving speed, acceleration, yaw rate, etc. of the vehicle. The driving lane ID indicates the lane in which the vehicle is traveling from the left or right end of the road. In addition, the vehicle status report may also include information such as the lighting status of the direction indicator and whether the vehicle is traveling across the lane dividing line.

[0047] In addition, each vehicle-mounted system 1 uploads a communication data packet (hereinafter referred to as an obstacle location report) indicating information related to the obstacle location notified from the map server 2 to the map server 2. The information related to the obstacle location refers to information used as a judgment material for the map server 2 to judge the existence status of obstacles on the road. The obstacle location report can be included in the vehicle status report. The obstacle location report and the vehicle status report can also be sent separately.

[0048] The map server 2 detects the location where an obstacle exists, the location where an obstacle disappears, etc. based on the obstacle location report uploaded from each vehicle. Then, the information related to the appearance / disappearance of the obstacle is multicast distributed to the vehicles to which the information should be distributed.

[0049] The map server 2 has a function of managing the current position of each vehicle as a sub-function for determining the distribution destination of information about the appearance / disappearance of obstacles. The management of the current position of each vehicle can be achieved using the vehicle position database described later. The current position of each vehicle is stored in the database in a manner that establishes a correspondence with the vehicle ID, etc. Whenever the map server 2 receives a vehicle status report, it refers to the content and updates the current position of the source vehicle registered in the database. In addition, in the structure of pull-type distribution of obstacle information, for example, the structure of the vehicle position, etc. for determining the distribution destination of the obstacle information is not necessarily required. The function of managing the position of each vehicle for determining the distribution destination is an arbitrary element. The transmission of the vehicle status report in the vehicle-mounted system 1 is also an arbitrary element.

[0050] <Overview of the in-vehicle system 1>

[0051] Figure 2 The vehicle-mounted system 1 shown has a front camera 11, a millimeter-wave radar 12, a vehicle status sensor 13, a locator 14, a V2X vehicle-mounted device 15, an HMI system 16, a map cooperation device 50, and a driving assistance ECU 60. In addition, ECU in the component name is an abbreviation of Electronic Control Unit, which refers to an electronic control device. In addition, HMI is an abbreviation of Human Machine Interface. V2X is an abbreviation of Vehicle to X (Everything), which refers to a communication technology that connects a vehicle to various objects.

[0052] The various devices or sensors constituting the vehicle-mounted system 1 are connected as nodes to the communication network constructed in the vehicle, namely the in-vehicle network Nw. The nodes connected to the in-vehicle network Nw can communicate with each other. In addition, specific devices can also be configured to communicate directly with each other without passing through the in-vehicle network Nw. For example, the map cooperation device 50 and the driving assistance ECU 60 can also be directly electrically connected via a dedicated line. In addition, although Figure 2 The in-vehicle network Nw is configured as a bus type, but is not limited to this. The network topology may also be a mesh, star, ring, etc. The network shape can be changed appropriately. As the standard of the in-vehicle network Nw, for example, a variety of standards such as Controller Area Network (hereinafter referred to as CAN: registered trademark), Ethernet (Ethernet is a registered trademark), FlexRay (registered trademark) can be adopted.

[0053] Hereinafter, the passenger sitting in the driver's seat of the vehicle, i.e., the driver's seat passenger, will also be recorded as a user. In addition, the front-back, left-right, and up-down directions in the following description are defined based on the vehicle. Specifically, the front-back direction is equivalent to the long side direction of the vehicle. The left-right direction is equivalent to the width direction of the vehicle. The up-down direction is equivalent to the height direction of the vehicle. According to other viewpoints, the up-down direction is equivalent to the direction perpendicular to the plane parallel to the front-back direction and the left-right direction.

[0054] <About the components of the vehicle-mounted system 1>

[0055] The front camera 11 is a camera that captures the front of the vehicle at a specified viewing angle. The front camera 11 is, for example, disposed at the upper end of the front glass on the inner side of the vehicle cabin, or at the front grille, roof, etc. The front camera 11 includes a camera body that generates image frames, and an ECU that detects a specified detection object by performing recognition processing on the image frames. The camera body is composed of at least an image sensor and a lens, and generates and outputs captured image data at a specified frame rate (for example, 60fps). The camera ECU is mainly composed of an image processing chip including a CPU or a GPU, and includes a recognizer as a functional module. The recognizer recognizes the type of an object based on, for example, a feature vector of an image.

[0056] The front camera 11 detects the specified detection object and determines the relative position of the detection object relative to the vehicle. The detection object here refers to, for example, pedestrians, other vehicles, landmark features, road ends, road surface markings, etc. Other vehicles also include bicycles, bicycles with prime movers, and motorcycles. Landmarks are three-dimensional structures set along the road. Structures set along the road are, for example, guardrails, curbs, trees, telephone poles, road signs, traffic lights, etc. Road signs include guide signs such as direction signs and road name signs. Landmark features are used for the localization processing described later. Road surface markings refer to spray patterns drawn on the road surface for traffic control and traffic restrictions. For example, lane dividing lines, crosswalks, stop lines, guide strips, safety zones, restriction arrows, etc. that indicate the boundaries of lanes are included in road surface markings. In lane dividing lines, in addition to spray patterns formed as dotted lines or continuous lines using yellow or white paint, lane dividing lines realized by road spikes such as Chatter bars or Botts' Dots are also included. Lane dividing lines are also called lane markings or lane markings.

[0057] In addition, the front camera 11 detects obstacles such as animal carcasses, fallen trees, and fallen objects. The obstacles here refer to three-dimensional objects that exist on the road and hinder the passage of vehicles. In addition to boxes, ladders, bags, and skis that are objects that fall from moving vehicles, obstacles also include tires, accident vehicles, and fragments of accident vehicles. In addition, restrictive materials such as arrow plates, road cones, and guide plates used for lane restrictions, or construction sites, parked vehicles, and the end of traffic jams can also be included in obstacles. In addition to stationary objects that hinder the passage of vehicles, obstacles can also include quasi-static map elements. For example, the front camera 11 identifies the type of obstacles such as fallen objects through image recognition and outputs them. The output data may also include a correct answer probability value indicating the likelihood of the recognition result. The correct answer probability value is equivalent to a score that represents the degree of consistency of the feature quantity. Preferably, the front camera 11 is configured to detect not only the lane where the vehicle is traveling, but also obstacles that exist in an area equivalent to an adjacent lane. Here, as an example, the front camera 11 is configured to be able to detect obstacles in the lane where the vehicle is traveling and in the adjacent lanes on the left and right.

[0058] The image processor of the front camera 11 separates and extracts the background and the detection object from the captured image based on image information including color, brightness, contrast related to color and brightness, etc. For example, the front camera 11 uses SfM (Structure from Motion) processing and the like to calculate the relative distance and direction (that is, relative position) and moving speed of the detection object such as lane dividing lines, road ends, obstacles, etc. from the vehicle. The relative position of the detection object relative to the vehicle can also be determined based on the size and inclination of the detection object in the image. In addition, the detection result data indicating the position and type of the detection object are sequentially provided to the map cooperation device 50 and the driving assistance ECU 60.

[0059] The millimeter wave radar 12 is a device that detects the relative position and relative speed of an object relative to the vehicle by sending millimeter waves or quasi-millimeter waves toward the front of the vehicle and analyzing the received data of the reflected wave reflected by the object and returned. The millimeter wave radar 12 is, for example, set in the front grille or front bumper. The millimeter wave radar 12 has a built-in radar ECU that identifies the type of the detected object based on the size, moving speed, and receiving intensity of the detected object. The radar ECU outputs data indicating the type, relative position (direction and distance), and receiving intensity of the detected object to the map cooperation device 50, etc. as a detection result. The millimeter wave radar 12 is also configured to detect part or all of the above-mentioned obstacles. For example, the millimeter wave radar 12 determines whether it is an obstacle based on the position, moving speed, size, and reflection intensity of the detected object. For example, it is possible to roughly determine whether it is a vehicle or a signboard based on the size of the detected object and the receiving intensity of the reflected wave.

[0060] The front camera 11 and the millimeter-wave radar 12 can also be configured to provide observation data used for object recognition, such as image data, to the driving assistance ECU 60, etc., via the in-vehicle network Nw, in addition to data indicating the recognition results. For example, the observation data for the front camera 11 refers to image frames. The observation data of the millimeter-wave radar refers to data indicating the receiving intensity and relative speed of each detection direction and distance, or data indicating the relative position and receiving intensity of the detected object. The observation data is equivalent to the original data observed by the sensor, or the data before the recognition process is performed. In addition, the front camera 11 and the millimeter-wave radar 12 are both equivalent to sensors that sense the outside world of the vehicle. Therefore, without distinguishing between the front camera 11 and the millimeter-wave radar 12, they are also recorded as peripheral monitoring sensors.

[0061] The object recognition process based on the observation data generated by the surrounding monitoring sensor may also be performed by an ECU other than the sensor, such as the driving assistance ECU 60. Part of the functions of the front camera 11 and the millimeter-wave radar 12 may also be provided in the driving assistance ECU 60. In this case, the camera or millimeter-wave radar as the front camera 11 only needs to provide the observation data such as the image data and the distance measurement data to the driving assistance ECU 60 as the detection result data.

[0062] The vehicle state sensor 13 is a sensor that detects physical state quantities related to the driving control of the vehicle. The vehicle state sensor 13 includes, for example, inertial sensors such as a three-axis gyroscope sensor and a three-axis acceleration sensor. The three-axis acceleration sensor is a sensor that detects the accelerations acting on the front and rear, left and right, and up and down directions of the vehicle. The gyroscope sensor is a sensor that detects the angular velocity of rotation around the detection axis, and the three-axis gyroscope sensor refers to a sensor with three mutually orthogonal detection axes. In addition, the vehicle state sensor 13 can also include a gear position sensor, a steering angle sensor, a vehicle speed sensor, etc. The gear position sensor is a sensor that detects the position of the gear lever. The steering angle sensor is a sensor that detects the rotation angle of the steering wheel (the so-called steering angle). The vehicle speed sensor is a sensor that detects the driving speed of the vehicle.

[0063] The vehicle state sensor 13 outputs data indicating the current value of the item to be detected (i.e., the detection result) to the in-vehicle network Nw. The output data of each vehicle state sensor 13 is obtained by the map cooperation device 50 or the like via the in-vehicle network Nw. In addition, the type of sensor used by the in-vehicle system 1 as the vehicle state sensor 13 can be appropriately designed, and it is not necessary to have all the above-mentioned sensors.

[0064] The locator 14 is a device that generates high-precision position information of the vehicle by combining multiple information for composite positioning. Figure 3 As shown, the positioning device 14 is implemented using a GNSS receiver 141 , an inertial sensor 142 , a map storage unit 143 , and a position calculation unit 144 .

[0065] The GNSS receiver 141 is a device that detects the current position of the GNSS receiver 141 in sequence by receiving navigation signals transmitted from positioning satellites constituting the GNSS (Global Navigation Satellite System). For example, when the GNSS receiver 141 is capable of receiving navigation signals from four or more positioning satellites, the positioning result is output every 100 milliseconds. As GNSS, GPS, GLONASS, Galileo, IRNSS, QZSS, Beidou, etc. can be used. The inertial sensor 142 is, for example, a three-axis gyro sensor and a three-axis acceleration sensor.

[0066] The map storage unit 143 is a non-volatile memory that stores high-precision map data. The high-precision map data here is equivalent to map data that shows the road structure and the position coordinates of the features arranged along the road with an accuracy that can be used for automatic driving. The high-precision map data, for example, has three-dimensional shape data of the road, lane data, feature data, etc. The three-dimensional shape data of the above-mentioned road contains node data related to the location (hereinafter referred to as a node) where multiple roads intersect, merge, and branch, and link data related to the road connecting the location (hereinafter referred to as a link). The link data may also include data indicating whether it is a dedicated road for cars or a general road. The dedicated road for cars here refers to a road that prohibits pedestrians and bicycles from entering, such as a toll road such as an expressway. The road type may also include attribute information indicating whether it is a road that allows autonomous driving. The lane data indicates the number of lanes, the setting position coordinates of the lane dividing line, the direction of travel of each lane, and the branch / merge location of the lane level. The feature data includes the location and type information of the road surface display such as a temporary stop line, and the location, shape, and type information of the landmark. Landmarks include three-dimensional structures installed along roads, such as traffic signs, signal lights, poles, and commercial signs.

[0067] The position calculation unit 144 measures the position of the vehicle in sequence by combining the positioning result of the GNSS receiver 141 with the measurement result of the inertial sensor 142. For example, when the GNSS receiver 141 cannot receive the GNSS signal in a tunnel or other place, the position calculation unit 144 uses the yaw rate and the vehicle speed to perform dead reckoning (Dead Reckoning / autonomous navigation). The yaw rate used for dead reckoning can be the yaw rate calculated by the front camera 11 using the SfM technology, or it can be the yaw rate detected by the yaw rate sensor. The measured vehicle position information is output to the in-vehicle network Nw and is used by the map cooperation device 50 and the like. In addition, the position calculation unit 144 determines the ID of the lane on the road where the vehicle is traveling (hereinafter referred to as the driving lane) based on the vehicle position coordinates determined by the above-mentioned configuration.

[0068] In addition, the locator 14 can also be configured to perform positioning processing. Positioning processing refers to the process of determining the detailed position of the vehicle by comparing the coordinates of the landmark determined based on the image captured by the front camera 11 with the coordinates of the landmark registered in the high-precision map data. Positioning processing can also be performed by comparing the three-dimensional detection point group data output by LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) with the three-dimensional map data. In addition, the locator 14 can also be configured to determine the driving lane based on the distance from the end of the road detected by the front camera 11 and the millimeter wave radar 12. The map cooperation device 50 or the driving assistance ECU 60 can also have some or all of the functions of the locator 14.

[0069] The V2X vehicle-mounted device 15 is a device for implementing wireless communication between the vehicle and other devices. In addition, the "V" of V2X refers to the car as the vehicle, and "X" refers to various existences other than the vehicle, such as pedestrians, other vehicles, road equipment, networks, servers, etc. The V2X vehicle-mounted device 15 has a wide area communication unit and a narrow area communication unit as communication modules. The wide area communication unit is a communication module for implementing wireless communication based on a prescribed wide area wireless communication standard. As the wide area wireless communication standard here, for example, LTE (Long Term Evolution), 4G, 5G and other standards can be adopted. In addition, the wide area communication unit can also be configured to implement wireless communication with other devices directly, that is, not via a base station, in addition to communication via a wireless base station in accordance with the wide area wireless communication standard. In other words, the wide area communication unit can also be configured to implement cellular V2X. This vehicle becomes an interconnected car that can be connected to the network by installing the V2X vehicle-mounted device 15. For example, the map cooperation device 50 can download the latest high-precision map data from the map server 2 and update the map data stored in the map storage unit 143 by cooperating with the V2X vehicle-mounted device 15.

[0070] The narrow-area communication unit of the V2X vehicle-mounted device 15 is a communication module for directly implementing wireless communication with other mobile bodies and roadside devices existing around the vehicle through a communication standard (hereinafter referred to as a narrow-area communication standard) whose communication distance is limited to several hundred meters. As other mobile bodies, it is not limited to vehicles, and pedestrians, bicycles, etc. can be included. As a narrow-area communication standard, any standard such as the WAVE (Wireless Access in Vehicular Environment) standard disclosed by IEEE1709 and the DSRC (Dedicated Short Range Communications) standard can be adopted. The narrow-area communication unit, for example, broadcasts the vehicle information of the vehicle to the surrounding vehicles at a prescribed transmission period, and receives the vehicle information sent from other vehicles. The vehicle information includes the vehicle ID, current position, direction of travel, moving speed, the operation status of the direction indicator, a timestamp, etc.

[0071] The HMI system 16 is a system that provides an input interface function for receiving user operations and an output interface function for presenting information to the user. The HMI system 16 includes a display 161 and an HCU (HMI Control Unit) 162. In addition, as a unit for presenting information to the user, in addition to the display 161, a speaker, a vibrator, a lighting device (e.g., an LED), etc. can be used.

[0072] The display 161 is a device for displaying images. The display 161 is, for example, a center display provided at the top of the center portion of the instrument panel in the vehicle width direction (hereinafter referred to as the center area). The display 161 is capable of full-color display, and can be implemented using a liquid crystal display, an OLED (Organic Light Emitting Diode) display, a plasma display, or the like. The HMI system 16 may also include a head-up display that projects a virtual image on a portion of the front glass in front of the driver's seat as the display 161. In addition, the display 161 may also be an instrument display.

[0073] The HCU 162 is a component that uniformly controls information presentation to the user. The HCU 162 is implemented using a processor such as a CPU or a GPU, a RAM, and a flash memory. The HCU 162 controls the display screen of the display 161 based on information provided from the map cooperation device 50 and a signal from an input device (not shown). For example, the HCU 162 displays information on the display 161 based on a request from the map cooperation device 50 or the driving assistance ECU 60. Figure 4 Obstacle notification image 80 is illustrated.

[0074] The obstacle notification image 80 is an image for notifying the user of information related to an obstacle. The obstacle notification image 80 includes information such as the positional relationship between the lane where the obstacle exists and the lane where the vehicle is traveling. Figure 4 , a case where an obstacle exists in the lane where the vehicle is traveling is illustrated. Figure 4 The image 81 in the figure represents the vehicle, the image 82 represents the lane boundary, the image 83 represents the obstacle, and the image 84 represents the road end. In addition, the obstacle notification image 80 may also include an image 85 indicating the remaining distance to the location where the obstacle exists. In addition, it may also include an image 86 indicating whether a lane change is required. Figure 4 For example, a scenario in which a lane change is performed due to an obstacle such as a parked vehicle in the lane where the vehicle is traveling is illustrated. The obstacle notification image 80 indicating the position of the obstacle, etc., may also be displayed on the head-up display so as to overlap with the real world observed by the driver's seat passenger. Preferably, the obstacle notification image 80 includes information indicating the type of obstacle.

[0075] The map cooperation device 50 is a device that obtains map data including obstacle information from the map server 2 and uploads information about obstacles detected by the vehicle to the map server 2. The functions of the map cooperation device 50 will be described in detail later. The map cooperation device 50 is mainly composed of a computer including a processing unit 51, a RAM 52, a storage device 53, a communication interface 54, and a bus connecting them. The processing unit 51 is hardware for calculation processing combined with the RAM 52. The processing unit 51 is a structure including at least one CPU (Central Processing Unit: Central Processing Unit) and other calculation cores. The processing unit 51 performs various processes for determining the existence / disappearance of obstacles by accessing the RAM 52. The storage device 53 is a structure including a non-volatile storage medium such as a flash memory. The program executed by the processing unit 51, that is, the obstacle reporting program, is stored in the storage device 53. The execution of the obstacle reporting program by the processing unit 51 is equivalent to executing a method corresponding to the obstacle reporting program. The communication interface 54 is a circuit for communicating with other devices via the in-vehicle network Nw. The communication interface 54 may be realized using analog circuit elements, ICs, and the like.

[0076] In addition, the map cooperation device 50 may be included in a navigation device, for example. The map cooperation device 50 may be included in the driving assistance ECU 60 or the automatic driving ECU. The map cooperation device 50 may be included in the V2X vehicle-mounted device 15. The functional configuration of the map cooperation device 50 can be changed appropriately. The map cooperation device 50 is equivalent to a vehicle device.

[0077] The driving assistance ECU 60 is an ECU that assists the driving operation of the driver's seat passenger based on the detection results of the surrounding monitoring sensors such as the front camera 11 and the millimeter wave radar 12, and the map information obtained by the map cooperation device 50. For example, the driving assistance ECU 60 prompts driving assistance information such as an obstacle notification image indicating the location of the obstacle. In addition, the driving assistance ECU 60 controls the driving actuator such as the actuator for driving based on the detection results of the surrounding monitoring sensors and the map information obtained by the map cooperation device 50, so as to replace the driver's seat passenger to perform part or all of the driving operation. The driving actuator includes, for example, a brake actuator (brake device), an electronic throttle valve, a steering actuator, etc.

[0078] The driving assistance ECU 60 provides a function of automatically implementing lane changes (hereinafter referred to as an automatic lane change function) as one of the vehicle control functions. For example, if the driving assistance ECU 60 reaches a predetermined lane change location on a driving plan generated separately, it will cooperate with the HMI system 16 to inquire whether the driver's seat passenger will implement the lane change. In addition, when it is determined that the driver's seat passenger has performed an operation on the input device to instruct the implementation of a lane change, a steering force in the direction of the target lane is generated in view of the traffic conditions of the target lane, so that the driving position of the vehicle is moved to the target lane. The predetermined lane change location can be defined as an interval with a certain length.

[0079] The driving assistance ECU 60 is similar to the map cooperation device 50 and is mainly composed of a computer including a processing unit, a RAM, a storage device, a communication interface, and a bus connecting them. The illustration of each element is omitted. The storage device of the driving assistance ECU 60 stores a program executed by the processing unit, that is, a driving assistance program. The processing unit executing the driving assistance program is equivalent to executing a method corresponding to the driving assistance program.

[0080] <Regarding the Configuration of the Map Cooperation Device 50>

[0081] Here we use Figure 5 The functions and operations of the map cooperation device 50 are described. The map cooperation device 50 provides information related to the obstacle report by executing the obstacle report program stored in the storage device 53. Figure 5 The functions of the various functional modules shown in the figure are as follows. That is, the map cooperation device 50 has a vehicle position acquisition unit F1, a map acquisition unit F2, a vehicle behavior acquisition unit F3, a detection object information acquisition unit F4, a report data generation unit F5, and a notification processing unit F6 as functional modules. The map acquisition unit F2 has an obstacle information acquisition unit F21. The report data generation unit F5 has an obstacle presence determination unit F51.

[0082] The vehicle position acquisition unit F1 acquires the position information of the vehicle from the locator 14. In addition, the vehicle position acquisition unit F1 acquires the driving lane ID from the locator 14. In addition, a part or all of the functions of the locator 14 may be included in the vehicle position acquisition unit F1.

[0083] The map acquisition unit F2 reads map data of a specified range determined based on the current position from the map storage unit 143. In addition, the map acquisition unit F2 acquires obstacle information existing within a specified distance in front of the vehicle from the map server 2 via the V2X vehicle-mounted device 15. As described later, the obstacle information is data about the location where the obstacle exists, including the lane where the obstacle exists, the type of the obstacle, etc. The configuration for acquiring the obstacle information is equivalent to the obstacle information acquisition unit F21 and the obstacle location information acquisition unit.

[0084] The obstacle information acquisition unit F21 can obtain the obstacle information corresponding to the current position of the vehicle by requesting the map server 2. Such a distribution method is also called pull distribution. In addition, the map server 2 can also automatically distribute obstacle information to vehicles near obstacles. Such a distribution method is also called push distribution. In other words, obstacle information can be obtained by using either pull distribution or push distribution. Here, as an example, the map server 2 selects a vehicle as a distribution object based on the position information of each vehicle, and performs push distribution on the distribution object.

[0085] The obstacle information acquired by the map acquisition unit F2 is temporarily stored in the memory M1 implemented by using the RAM 52 or the like. In addition, the obstacle information stored in the memory M1 can be deleted when the vehicle passes the location indicated by the data or when a certain time has passed. For convenience, the obstacle information acquired from the map server 2 is also recorded as the obstacle information on the map. In addition, the location where the obstacle exists shown by the obstacle information on the map is also recorded as the obstacle registration location, or simply recorded as the obstacle location.

[0086] The vehicle behavior acquisition unit F3 acquires data indicating the behavior of the vehicle from the vehicle state sensor 13. For example, the driving speed, yaw rate, lateral acceleration, longitudinal acceleration, etc. are acquired. In addition, the vehicle behavior acquisition unit F3 acquires information indicating whether the lane boundary is crossed and the offset of the driving position to the right or left relative to the center of the lane from the front camera 11. The longitudinal acceleration here is equivalent to the acceleration in the front-to-back direction, and the lateral acceleration is equivalent to the acceleration in the left-to-right direction. The vehicle behavior acquisition unit F3 is equivalent to the vehicle behavior detection unit.

[0087] The detected object information acquisition unit F4 acquires information about obstacles detected by the front camera 11 and the millimeter wave radar 12 (hereinafter referred to as detected obstacle information). The detected obstacle information includes, for example, the location of the obstacle, its type, size, etc. The location where the obstacle is detected by the surrounding monitoring sensor is also recorded as the obstacle detection location. For example, the obstacle detection location can be expressed in any absolute coordinate system such as WGS84 (World Geodetic System 1984: World Geodetic Coordinate System in 1984). The obstacle detection location can be calculated by combining the current position coordinates of the vehicle and the relative position information of the obstacles detected by the surrounding monitoring sensors relative to the vehicle. The detected object information acquisition unit F4 can not only acquire the recognition results of various surrounding monitoring sensors, but also acquire the observation data itself, such as the image data captured by the front camera 11. The detected object information acquisition unit F4 can be called an external information acquisition unit.

[0088] The obstacle detection position may also be information indicating in which lane the obstacle exists, for example. For example, the obstacle detection position may also be represented by a lane ID. In addition, it is preferred that the obstacle detection position includes the lateral position of the end of the obstacle in the lane. The lateral position information of the end of the obstacle in the lane can be used as information indicating how many lanes are blocked by the obstacle. The above-mentioned obstacle registration location indicates the obstacle position identified by the map server 2, while the obstacle detection position indicates the position actually observed by the vehicle.

[0089] The various data acquired in sequence by the vehicle position acquisition unit F1, the vehicle behavior acquisition unit F3, and the detection object information acquisition unit F4 are stored in a memory such as RAM52, and are referenced and utilized by the map acquisition unit F2, the report data generation unit F5, etc. In addition, various information is divided according to type and stored in the memory after, for example, a timestamp indicating the acquisition time of the data is attached. The timestamp plays the role of establishing associations between different types of information at the same time. By using the timestamp, the map cooperation device 50 can, for example, determine the vehicle behavior synchronized with the video outside the vehicle. In addition, the timestamp can also replace the acquisition time and be the output time, generation time, etc. of the data in the output source. When the output time or generation time is used as the timestamp, it is preferred to synchronize the time information of each vehicle-mounted device. The various information acquired by the map cooperation device 50 can be sorted and stored in a manner such that the latest data is at the beginning. Data that has passed a constant time since acquisition can be discarded.

[0090] The report data generation unit F5 is a component that generates a data set to be sent to the map server 2 and outputs it to the V2X vehicle-mounted device 15. The report data generation unit F5 can be called a report processing unit. The report data generation unit F5 generates the vehicle status report described at the beginning at a predetermined interval and uploads it to the map server 2 via the V2X vehicle-mounted device 15. In addition, the report data generation unit F5 generates an obstacle location report and uploads it to the map server 2 as an upload process described later.

[0091] The obstacle presence determination unit F51 determines whether there is an obstacle based on the detected obstacle information acquired by the detected object information acquisition unit F4 and the behavior data of the vehicle acquired by the vehicle behavior acquisition unit F3. For example, the obstacle presence determination unit F51 can also determine whether there is an obstacle by sensor fusion of the front camera 11 and the millimeter-wave radar 12. For example, it is also possible to determine that there is an obstacle when the front camera 11 detects an obstacle at a location where an obstacle or a three-dimensional stationary object of unknown type is detected by the millimeter-wave radar 12. In addition, it is also possible to determine that there is no obstacle when the millimeter-wave radar 12 does not detect an obstacle or a three-dimensional stationary object of unknown type at a location where the obstacle is detected by the front camera 11. In addition, the obstacle presence determination unit F51 can also determine whether there is an obstacle based on whether the vehicle has implemented a prescribed avoidance action when at least one of the front camera 11 and the millimeter-wave radar 12 detects an obstacle in the lane where the vehicle is traveling.

[0092] The avoidance action here is, for example, a vehicle behavior for avoiding an obstacle, such as a change in driving position. The change in driving position here refers to changing the lateral position of the vehicle on the road. The change in driving position includes not only lane changes, but also moving the driving position in the same lane closer to any left or right corner, and driving across the lane dividing line. In addition, in order to clarify the difference from the usual lane change, the avoidance action is preferably a change in driving position / steering operation accompanied by deceleration and subsequent acceleration. For example, a change in driving position accompanied by a deceleration operation, or a change in driving position accompanied by deceleration to a speed below a specified speed can be regarded as an avoidance action. In addition, the above description of the avoidance action shows the concept of the avoidance action assumed in the present disclosure. As described later, in addition to the driving trajectory, it is possible to detect whether a change in driving position as an avoidance action has been performed based on the change pattern of the lateral acceleration, the action history of the direction indicator, etc.

[0093] The obstacle presence determination unit F51 determines the direction in which the vehicle avoids and the avoidance direction based on the yaw rate and the displacement direction of the steering angle acting on the vehicle. For example, when the driving position of the vehicle moves to the right, in other words, when the steering is performed to the right, the avoidance direction is the right side. The avoidance direction is the direction in which there must be no obstacles. Conversely, the avoidance direction can be an indicator indicating the direction in which an obstacle exists. The obstacle presence determination unit F51 can also be called an avoidance action determination unit in one aspect.

[0094] The notification processing unit F6 is configured to notify the driver's seat passenger of information about an obstacle existing in front of the vehicle in cooperation with the HMI system 16 based on the obstacle information on the map. Figure 4The obstacle notification image shown as an example is displayed on the display 161. In addition, the notification of the obstacle may be performed by a voice message or the like. The driving support ECU 60 may be provided with a notification processing unit F6.

[0095] <About upload processing>

[0096] Here we use Figure 6 The flowchart shown in FIG. 1 illustrates the upload process performed by the map cooperation device 50. For example, the upload process is performed at a predetermined period (for example, every 100 milliseconds) while the vehicle is powered on. Figure 6 The driving power source is a power source that enables the vehicle to travel, and is, for example, an ignition power source in an engine vehicle. In an electric vehicle, the system main relay corresponds to the driving power source. The upload process includes steps S101 to S104 as an example.

[0097] In step S101, the notification data generator F5 reads the obstacle information on the map stored in the memory M1, and the process proceeds to step S102. Step S101 may be a process of acquiring obstacle information within a predetermined distance in front of the vehicle from the map server 2.

[0098] In step S102, based on the obstacle information on the map, it is determined whether there is an obstacle within a specified distance (hereinafter referred to as a reference distance) in front of the vehicle. Step S102 is equivalent to the process of determining whether there is an obstacle registration location within the reference distance. The reference distance is, for example, 200m or 300m. Preferably, the reference distance is longer than the limit value of the distance at which the front camera 11 can recognize an object. The reference distance can also be changed according to the driving speed of the vehicle. For example, the greater the driving speed of the vehicle, the longer the reference distance can be set. For example, the distance reached within a specified time such as 30 seconds can be calculated based on the speed of the vehicle, and this distance can be used as the reference distance.

[0099] If there is no obstacle recognized by the map server 2 within the reference distance in step S102, this flow ends. On the other hand, if there is an obstacle within the reference distance, that is, if there is an obstacle registration point, step S103 is executed.

[0100] In step S103, the vehicle behavior when traveling within the specified reporting target distance before and after the obstacle registration point on the map is obtained and the process moves to step S104. In step S104, a data set including the time series data of the vehicle behavior obtained in step S103, the transmission source information, and the reporting target point information is generated as an obstacle point report. The reporting target point information is information indicating which point the report is about. For example, the position coordinates of the obstacle registration point are set in the reporting target point information.

[0101] The reporting target distance is preferably set to a distance at which the driver's seat passenger and the surrounding monitoring sensor can recognize the status of the obstacle registration location. Figure 7 The setting shown is 100 meters before and after the obstacle registration point. In this case, the obstacle point report becomes, for example, a data set showing the vehicle behavior 100 meters before and after the obstacle registration point. The section before and after the obstacle registration point that is within the report target distance is also recorded as the report target section.

[0102] The vehicle behavior data included in the obstacle location report is data indicating whether the vehicle traveling in the lane where the obstacle exists has taken an action to avoid the obstacle (that is, an avoidance action). For example, as data indicating vehicle behavior, the vehicle position coordinates, travel direction, travel speed, longitudinal acceleration, lateral acceleration, yaw rate, etc. at each moment when passing near the obstacle registration location can be used. The vicinity of the obstacle registration location refers to, for example, within 20 meters of the obstacle registration location. In addition, the area within 50 meters or 100 meters before and after the obstacle registration location can also be regarded as the vicinity of the obstacle registration location. The range regarded as the vicinity of the obstacle registration location can also be changed according to the type of road and the statutory upper speed limit. The above-mentioned report object distance is determined based on where the obstacle registration location is regarded as being near. In addition, as data indicating vehicle behavior, it can include the steering angle, gear position, the action state of the direction indicator, the lighting state of the hazard warning light, whether the lane dividing line is crossed, whether a lane change is implemented, and the offset from the center of the lane.

[0103] Preferably, the obstacle location report includes the driving lane ID at each time when the vehicle passes near the obstacle registration location. This is because by including the driving lane ID, the map server 2 can determine whether the report is from a vehicle driving on the lane where the obstacle exists. Of course, the map server 2 can also determine whether the report is from a vehicle driving on the lane where the obstacle exists based on the time series data of the position coordinates included in the obstacle location report.

[0104] In addition, it is preferred that the obstacle location report includes not only the vehicle behavior until reaching the obstacle registration location, but also the vehicle behavior information after passing the obstacle registration location. This is because if the lane change and steering operation performed by a certain vehicle are actions for avoiding obstacles, there is a high possibility that the vehicle returns to the original lane after passing the obstacle. In other words, by including the vehicle behavior after passing the obstacle registration location in the obstacle location report, the accuracy of determining whether the action performed by the vehicle is an action for avoiding obstacles can be improved, and thus the accuracy of determining whether an obstacle actually exists can be improved.

[0105] The obstacle location report can be data indicating the vehicle state every 100 milliseconds during the period of traveling in the report target section. The sampling interval of vehicle behavior is not limited to 100 milliseconds, and can also be 200 milliseconds, etc. The shorter the sampling interval, the larger the data size. Therefore, from the perspective of traffic control, it is preferable to make the sampling interval long enough to analyze the vehicle's behavior.

[0106] If the reporting object distance is too short, for example, only data after the avoidance action is performed is collected in the map server 2, and it is unclear whether the avoidance action is performed. On the other hand, if the reporting object distance is set longer, the omission of data indicating the avoidance action is reduced, but the data size increases. It is preferred that the reporting object distance is set to include the location where the avoidance action is assumed to be performed on the obstacle. For example, it is preferred that the reporting object distance is set to be more than 25m.

[0107] In addition, the length of the reporting object distance may be changed depending on whether it is a general road or a dedicated road for automobiles. A dedicated road for automobiles refers to a road where pedestrians and bicycles are prohibited from entering, such as a toll road including a highway. For example, the reporting object distance on a general road may be set shorter than the reporting object distance on a dedicated road for automobiles. Specifically, the reporting object distance facing the dedicated road for automobiles may be set to more than 100m, while the reporting object distance facing the general road may be set to 30m or less, for example. This is because the visibility ahead is better on a dedicated road for automobiles than on a general road, and there is a possibility of starting avoidance action from a location far away from where an obstacle exists.

[0108] The sampling interval may also be changed according to the type of road, such as a dedicated road for automobiles or a general road. The sampling interval for dedicated roads for automobiles may be shorter than the sampling interval for general roads. By extending the sampling interval, the data size can be suppressed. In addition, the longer the distance to the reporting object is, the sparser the sampling interval may be. With such a configuration, the data size of the obstacle location report can be limited to a constant range.

[0109] Furthermore, the report target distance and the sampling interval may be dynamically determined based on the instruction signal from the map server 2. Furthermore, the type of information (ie, items) included in the obstacle point report may also be dynamically determined based on the instruction signal from the map server 2.

[0110] In addition, the reporting object distance, sampling interval, and items included in the obstacle location report can also be changed according to the type and size of the obstacle and the degree of lane congestion. For example, in a scenario where a lane change is necessary as an avoidance action, such as when an obstacle blocks more than half of the lane, the obstacle location report can also be limited to information used to determine whether the reporting vehicle has implemented a lane change. Whether a lane change has been implemented can be determined based on the presence or absence of changes in the driving trajectory and driving lane ID.

[0111] In addition, the obstacle location report may also include detection result information indicating whether an obstacle is detected by the surrounding monitoring sensor. The obstacle detection result may indicate the detection results of the front camera 11 and the millimeter wave radar 12, respectively, or the determination result of the obstacle presence determination unit F51. In the case where an obstacle is detected by the surrounding monitoring sensor, the obstacle location report may also include the detected obstacle information acquired by the detection object information acquisition unit F4. For example, the obstacle location report may also include image data of the front camera 11 captured at a specified distance from the obstacle registration location (for example, 10m).

[0112] In addition, the upload processing method is not limited to the above content. Figure 8 As shown, the process may also be configured to include steps S201 to S206. Figure 8 The steps S201 to S203 are the same as the above steps S101 to S101. If step S203 is completed, step S204 is executed.

[0113] In step S204, the detection object information acquisition unit F4 acquires the sensing information of at least one of the front camera 11 and the millimeter wave radar 12 when passing near the obstacle registration location. The sensing information here can also include the observation data itself in addition to the recognition result based on the observation data. Here, as an example, the recognition results related to the obstacle of the front camera 11 and the millimeter wave radar 12 (that is, the obstacle detection information) and the captured image of the front camera 11 are obtained. The collection period of the sensing information is the same as the vehicle behavior information, and can be from the point where the remaining distance to the obstacle registration location becomes less than the reporting object distance, until the obstacle registration location is located behind the reporting object distance. In addition, in the case of a peripheral monitoring sensor that does not have the rear of the vehicle as the detection range, the collection period of the sensing information can also be from the point where the remaining distance to the obstacle registration location becomes less than the reporting object distance, until the obstacle registration location is passed. If step S204 is completed, step S205 is executed.

[0114] In step S205, current status data indicating the current status of the obstacle registration location is generated based on the sensing information collected in step S204. For example, the current status data includes the recognition results of the surrounding monitoring sensor every 250 milliseconds during the period of collecting the sensing information. In addition, when an obstacle is detected by the front camera 11 during this period, at least one frame of image data used for detecting the obstacle is included. By making the current status data include at least one image frame showing the obstacle registration location, the resolution in the map server 2 can be improved.

[0115] In addition, the image frames included in the current situation data may be all frames captured during the period of collecting the sensing information, or may be image frames captured at intervals of 200 milliseconds. The more image frames included in the current situation data, the higher the resolution in the map server 2, but the more the communication volume increases. The amount of image frames included in the current situation data may also be selected so that the amount of data is below a prescribed upper limit. In addition, it may also be configured so that only the image area where the obstacle is reflected is extracted and included in the current situation data instead of the entire image frame.

[0116] If step S205 is completed, step S206 is executed. In step S206, a data set including the data representing the vehicle behavior acquired in step S203 and the current status data generated in step S205 is generated as an obstacle location report and uploaded to the map server 2.

[0117] According to the above-mentioned structure, not only the vehicle behavior but also the recognition results and image data of the surrounding monitoring sensors can be collected on the map server 2. As a result, it is possible to verify with better accuracy whether the obstacle still exists or disappears. In addition, although the vehicle traveling in the adjacent lane of the lane where the obstacle exists, that is, the vehicle traveling in the adjacent lane does not perform avoidance actions caused by the existence of the obstacle, the obstacle can be observed by the front camera 11 and millimeter wave radar 12 of the vehicle. In other words, the appearance of the lane where the obstacle exists (hereinafter referred to as the obstacle lane) can also be observed by the vehicle traveling in the adjacent lane. According to the above-mentioned structure, the map server 2 can also collect the sensor detection information of the surrounding monitoring sensors of the vehicles traveling in the adjacent lane, so it can verify with better accuracy whether there is an obstacle.

[0118] In addition, although the above discloses a method of uploading the situation when the vehicle is traveling in front of the vehicle near the obstacle registration location as an obstacle location report as an upload process, it is not limited to this. The map cooperation device 50 can also be configured to upload an obstacle location report even when there is no obstacle registration location, for example, when the vehicle behavior or sensing information indicating the existence of an obstacle is obtained.

[0119] like Fig. 9As shown, for example, the map cooperation device 50 may be configured to execute the processing including steps S301 to S303. For example, the processing may be executed independently of the upload processing at a predetermined execution interval. Fig. 9 In addition, for example, when it is determined that there is no obstacle registration point during the upload process (No in step S102 or step S202), the following processing can be performed: Fig. 9 The processing flow shown.

[0120] In step S301, the vehicle behavior within the most recent specified time (e.g., 10 seconds) is obtained and step S302 is executed. In step S302, by analyzing the vehicle behavior data obtained in step S301, it is determined whether avoidance action has been implemented. For example, in the case of a change in the driving position accompanied by deceleration or stopping, or a sudden steering operation, etc., it is determined that avoidance action has been implemented. It is also possible to determine whether the driving position has been changed based on the trajectory of the vehicle's position, and it can be determined based on the change in the yaw rate, steering angle, lateral acceleration relative to time, the lighting state of the direction indicator, etc. In addition, it is also possible to determine whether the driving position has been changed based on whether the lane dividing line has been crossed. In addition, it is also possible to determine that avoidance action has been implemented based on the yaw rate, steering angle, or lateral acceleration becoming above a specified value.

[0121] If it is determined in step S302 that an avoidance action has been taken, the process moves to step S303, and an obstacle location report is generated and sent, similar to the above-mentioned steps S103, S206, etc. The obstacle location report uploaded in step S303 may also include image frames captured within a specified time from the time the avoidance action is taken. The image data sent to the map server 2 as a trigger for the avoidance action is also referred to as a report image. The report image is equivalent to an image used by the map server 2 to determine the obstacle to be avoided by the vehicle, or to verify whether an obstacle actually exists. The obstacle location report sent in step S303 is equivalent to data that reveals the existence of an obstacle that has not yet been identified by the map server 2. The report location information of the obstacle location report generated in step S303 is set to the vehicle position immediately before the avoidance action is determined to be implemented. By setting the vehicle position before the avoidance action is implemented, the worry of mistakenly determining the lane where the obstacle exists can be reduced. In addition, a location at a specified distance (e.g., 20m) on the traveling direction side from the vehicle position before the avoidance action is implemented can also be set as the report location.

[0122] Regarding the uploading of the report image, the map cooperation device 50 may also cut out a partial image of a specified range including an area on the opposite side of the steering direction compared to a specified reference point set in the image from an image captured at the time of emergency steering or emergency braking or just after these actions, and send it as a report image. More specifically, when the avoidance direction is the right side, the report data generation unit F5 may also send a partial image located on the left side compared to the reference point as a report image. The reference point may be either a dynamically specified vanishing point or a pre-set center point of the image. In addition, the reference point may also be a point that is a specified amount above the center point of the image. In addition, the vanishing point can be calculated by optical flow and other techniques. The verification area described later can be applied as a cut-out range for the report image. According to the above configuration, even if the obstacle cannot be determined in real time in the map cooperation device 50, the obstacle that is the cause of the avoidance action can be determined by the map server 2. In addition, according to the configuration of sending only a part of the captured image data, the amount of data uploaded to the map server 2 can also be suppressed.

[0123] In connection with the above configuration, the report data generating unit F5 may also cut out a portion of the image captured at the time of emergency steering or emergency braking or immediately after these actions, which is located on the opposite side of the avoidance direction compared to the reference point and in which an object registered as an obstacle is reflected, and send it as a report image. In addition, instead of the portion in which an object registered as an obstacle is reflected, an image region in which a three-dimensional object is detected by the millimeter wave radar 12 may be extracted and sent as a report image.

[0124] In addition, for example, when an avoidance action such as emergency deceleration or emergency steering is performed due to a delay in recognizing the end of a traffic jam, an obstacle location report may be sent even though there is actually no stationary object that serves as an obstacle, depending on the setting of the transmission conditions of the obstacle location report. By including the image frame when the avoidance action is performed in the obstacle location report, it is possible to suppress erroneous detection of obstacles.

[0125] In addition, the map cooperation device 50 may be configured to lock and transmit an image frame showing an obstacle that is the cause of the avoidance action from among a plurality of image frames captured within a predetermined period of time based on the time when the avoidance action was performed. Fig.10 As shown, the process including steps S311 to S314 is executed. Fig.10 The processing flow shown can be used as Fig. 9 An alternative process to the one shown is executed.

[0126] Steps S311 to S312 are the same as steps S301 to S302. If the map cooperation device 50 detects that an avoidance action has been taken based on the vehicle behavior data, step S313 is executed. In step S313, the report data generation unit F5 performs a process of locking an image frame to be sent to the map server 2 as a report image from the image frames acquired within a predetermined time before and after the moment when the avoidance action is detected, i.e., a locking process. It is preferred to select an image frame that shows the obstacle as clearly as possible as the report image.

[0127] An example of a lock process is Fig.11 As shown. For example, the obstacle presence determination unit F51 determines the avoidance direction based on the yaw rate acting on the vehicle, etc., as a preparatory process for locking (step S321). In addition, the report data generation unit F5 extracts frames with a shooting time difference of one second from the image frames acquired within a prescribed period prescribed based on the time when the avoidance action is detected as a primary filtering process (step S322). In other words, the image frames are thinned out at one-second intervals.

[0128] Next, the report data generation unit F5 performs secondary filtering processing to extract frames with avoidance object candidates from the frames remaining after the primary filtering processing (step S323). In other words, frames without avoidance object candidates are discarded in step S323. The avoidance object candidates here refer to objects registered as obstacles in the dictionary data of object recognition, etc. Basically, all obstacles reflected in the image frame can become avoidance object candidates. For example, vehicles existing on the road, materials for road restrictions, etc. can become avoidance object candidates. Materials for road restrictions refer to road cones set up at construction sites, signs indicating no passage, etc., signs indicating arrows to the right or left (so-called arrow boards), etc.

[0129] If the processing in step S323 is completed, the process proceeds to step S324. In step S324, the report data generation unit F5 compares the frames in which the avoidance object candidates are mapped in sequence, and determines the avoidance object based on the relationship between the change pattern of the position and size of the avoidance object candidates in the image frame over time and the avoidance direction of the vehicle. The avoidance object refers to an obstacle that is estimated to be avoided by the vehicle, that is, the cause of the avoidance action. For example, an avoidance object candidate whose position in the image frame moves in the direction opposite to the avoidance direction as the shooting time passes is determined to be an avoidance object.

[0130] If the determination of the avoidance object is completed, the report data generation unit F5 selects the image frame that most appropriately reflects the avoidance object from multiple image frames, that is, the best frame (step S325). For example, the report data generation unit F5 selects the frame that most clearly reflects the avoidance object. The report data generation unit F5 can also select the frame that reflects the avoidance object as a whole as the best frame. The report data generation unit F5 can also select the frame with the highest probability of correct solution for the recognition result of the avoidance object, that is, the frame with the highest degree of conformity with the model data of the obstacle as the best frame. If the selection of the best frame is completed, the obstacle location report containing the image frame as the report image is sent to the map server 2 ( Fig.10 Step S314).

[0131] Furthermore, the report data generating unit F5 may further cut out a portion where the avoidance object is reflected in the optimal frame and transmit the portion as a report image. According to this configuration, an effect of reducing the amount of communication can be expected.

[0132] Fig.12 is a diagram schematically showing the operation of the above-mentioned locking process, Fig.12 (a) shows all image frames captured within a predetermined period after the avoidance action is performed. Fig.12 (b) shows a group of frames that are thinned out at predetermined time intervals through a single locking process. Fig.12 (c) shows a set of frames that are locked based on the condition that an object similar to the avoidance object is reflected, that is, a candidate avoidance object is reflected. Fig.12 (d) shows the final selected image frame. Fig.12 (e) shows a state where a portion of the image with the avoidance object is cut out. By making the lock processing of the report image include a primary filtering process, the processing load of the report data generating unit F5 can be reduced. In addition, by performing a secondary filtering process, the worry of mistakenly selecting an image frame without the avoidance object as the report image can be reduced.

[0133] In addition, as a primary filtering process, the interval for thinning out the image frames is not limited to one second, and can also be 500 milliseconds, etc. In addition, the primary filtering process is not an essential element and can be omitted. However, by performing the primary filtering process, the processing load of the processing unit 51 as the report data generating unit F5 can be reduced. In addition, in the case where there is no image frame that identifies the avoidance object candidate and the avoidance object cannot be determined, the image frame captured at the prescribed timing prescribed by taking the detection moment of the avoidance action as a reference can also be selected as the best frame.

[0134] And, if Fig.13 As shown, the map cooperation device 50 may also be configured to execute the processing including steps S401 to S403. For example, the map cooperation device 50 may be executed independently of the upload processing at a predetermined execution interval. Fig.13The processing flow shown may be executed when it is determined in the upload process that there is no obstacle registration point (No in step S102 or step S202).

[0135] In step S401, the sensor detection information of the most recent specified time (for example, 5 seconds) is obtained and step S402 is executed. In step S402, the obstacle presence determination unit F51 determines whether there is an obstacle by analyzing the sensor detection information obtained in step S401. When it is determined that there is an obstacle, an obstacle location report is prepared and uploaded in the same way as step S206. In addition, the sensor detection information contained in the obstacle location report uploaded in step S403 can be, for example, the recognition results and image frames of each surrounding monitoring sensor at the moment when it is determined that there is an obstacle. The obstacle location report sent in step S403 is also the same as the obstacle location report sent in step S303, which is equivalent to data indicating the existence of an obstacle that has not yet been recognized by the map server 2.

[0136] <About the structure of map server 2>

[0137] Next, the structure of the map server 2 is described. The map server 2 detects the generation and disappearance of obstacles based on obstacle location reports sent from multiple vehicles, and distributes them to the vehicles as obstacle information. The map server 2 is equivalent to an obstacle information management device. In addition, the record of the vehicle as the communication object of the map server 2 can be replaced by the vehicle-mounted system 1, and can be further replaced by the map cooperation device 50.

[0138] Map Server 2 Fig.14 As shown, it is equipped with a server processor 21, RAM 22, a storage device 23, a communication device 24, a map DB 25, and a vehicle position DB 26. The DB in the component name refers to a database. The server processor 21 is hardware for calculation processing combined with the RAM 52. The server processor 21 is a structure including at least one calculation core such as a CPU (Central Processing Unit). The server processor 21 performs various processes such as determining the existence status of obstacles by accessing the RAM 22. The storage device 23 is a structure including a non-volatile storage medium such as a flash memory. The program executed by the server processor 21, namely, the obstacle information management program, is stored in the storage device 23. The execution of the obstacle information generation program by the server processor 21 is equivalent to executing the method corresponding to the obstacle information management program, namely, the obstacle information management method. The communication device 24 is a device for communicating with other devices such as each vehicle-mounted system 1 via the wide area communication network 3.

[0139] The map DB 25 is, for example, a database storing high-precision map data. In addition, the map DB 25 has an obstacle DB 251 storing information related to the location where the obstacle was detected. The map DB 25 and the obstacle DB 251 are databases implemented using a rewritable non-volatile storage medium. The map DB 25 and the obstacle DB 251 are configured to be able to write, read, delete, etc. data based on the server processor 21.

[0140] The obstacle DB 251 stores data indicating locations where obstacles are detected (hereinafter referred to as obstacle location data). The obstacle location data indicates the location coordinates of each obstacle location, the lane where the obstacle exists, the type of obstacle, the size, the lateral position in the lane, the time of occurrence, the latest existence determination time, etc. The data on a certain obstacle location is updated periodically by the obstacle information management unit G3, for example, based on the obstacle location report from the vehicle for the location. The data of each obstacle location constituting the obstacle location data may be maintained in an arbitrary data structure such as a list form. For example, the data of each obstacle location may also be stored separately according to a prescribed division. The division unit may be a grid of a high-precision map, an administrative division unit, or another division unit. For example, it may be a road link unit. A map grid refers to a plurality of small areas formed by dividing a map according to a constant rule. The grid can also be replaced by a map tile.

[0141] The vehicle position DB26 is a database implemented using a rewritable non-volatile storage medium. The vehicle position DB26 is configured to be able to implement writing, reading, and deleting of data based on the server processor 21. In the vehicle position DB26, a correspondence is established with the vehicle ID to save data representing the current status of each vehicle including the position of the vehicle constituting the obstacle information distribution system 100 (hereinafter referred to as vehicle position data). The vehicle position data represents the position coordinates, driving lane, direction of travel, driving speed, etc. of each vehicle. The data about a certain vehicle is updated by the vehicle position management unit G2 described later whenever a vehicle status report is received from the vehicle. The data of each vehicle constituting the vehicle position data can be maintained in an arbitrary data structure such as a list format. For example, the data of each vehicle can also be saved separately according to the specified division. The division unit can be a grid of a high-precision map, an administrative division unit, or another division unit (such as a road link unit).

[0142] In addition, the storage medium storing information related to the location where the obstacle is detected may be a volatile memory such as RAM. The storage destination of the vehicle position data may also be a volatile memory. The map DB 25 and the vehicle position DB 26 may also be constructed using a plurality of types of storage media such as a non-volatile memory and a volatile memory.

[0143] The map server 2 provides information related to the obstacle information by executing the obstacle information management program stored in the storage device 23 through the server processor 21. Fig.15 The various functional modules shown in the figure correspond to functions. That is, the map server 2 includes a report data acquisition unit G1, a vehicle position management unit G2, an obstacle information management unit G3, and a distribution processing unit G4 as functional modules. The obstacle information management unit G3 includes an appearance determination unit G31 and a disappearance determination unit G32.

[0144] The report data acquisition unit G1 acquires the vehicle status report and obstacle location report uploaded from the vehicle-mounted system 1 via the communication device 24. The report data acquisition unit G1 provides the vehicle status report acquired from the communication device 24 to the vehicle position management unit G2. In addition, the report data acquisition unit G1 provides the obstacle location report acquired from the communication device 24 to the obstacle information management unit G3. The report data acquisition unit G1 is equivalent to the vehicle behavior acquisition unit.

[0145] The vehicle position management unit G2 updates the position information of each vehicle stored in the vehicle position DB 26 based on the vehicle status report sent from each vehicle. That is, whenever the report data acquisition unit G1 receives a vehicle status report, the position information of the source of the vehicle status report, the driving lane, the traveling direction, the driving speed and other prescribed management items stored in the vehicle position DB 26 are updated.

[0146] The obstacle information management unit G3 updates the data of each obstacle location stored in the obstacle DB 251 based on the obstacle location report sent from each vehicle. The appearance determination unit G31 and the disappearance determination unit G32 of the obstacle information management unit G3 are both elements for updating the data of each obstacle location. The appearance determination unit G31 is a structure for detecting the appearance of an obstacle. The presence or absence of an obstacle is determined by lane unit. In addition, as another method, the presence or absence of an obstacle can also be determined by road unit. The disappearance determination unit G32 is a structure for determining whether the obstacle detected by the appearance determination unit G31 still exists, that is, whether the detected obstacle disappears. The disappearance determination unit G32 determines the existence (disappearance) of an obstacle at a certain obstacle registration location based on the vehicle behavior data and sensing information received after setting the location as the obstacle registration location. The appearance determination unit G31 and the disappearance determination unit G32 will be described in detail later.

[0147] The distribution processing unit G4 is a component for distributing obstacle information. For example, the distribution processing unit G4 implements obstacle notification processing. Obstacle notification processing is a process of distributing a communication data packet indicating information about the location of an obstacle, that is, an obstacle notification data packet, to a vehicle scheduled to pass through the obstacle location. The obstacle notification data packet shows the location coordinates of the obstacle, the lane ID where the obstacle exists, the type of obstacle, etc. For example, the destination of the obstacle notification data packet can be, for example, a vehicle scheduled to pass through the obstacle location within a specified time (one minute or five minutes). For example, the scheduled driving path of each vehicle can be obtained to determine whether it is scheduled to travel at the obstacle location. In addition, a vehicle traveling on the same or connected road / lane as the road / lane where the obstacle exists can also be selected as a vehicle scheduled to pass through the obstacle location. The time required to reach the obstacle location can be calculated based on the distance from the current position of the vehicle to the obstacle location and the driving speed of the vehicle.

[0148] The distribution processing unit G4 selects the destination of the obstacle notification data packet using the road link and altitude information. This can reduce the worry of erroneous distribution to vehicles traveling on roads arranged parallel to the upper / lower side of the road where the obstacle exists. In other words, it is possible to suppress erroneous determination of distribution targets in elevated roads and road sections with double-layer structures. The distribution targets can be extracted based on the position information, travel speed, etc. of each vehicle registered in the vehicle position DB 26.

[0149] In addition, by adding the time condition until the obstacle is reached to the extraction condition of the distribution object, it is possible to suppress unnecessary distribution. This is because the existence state of the obstacle can change dynamically, so for example, even if the distribution is to a vehicle that has more than 30 minutes to arrive, the obstacle is likely to disappear when the vehicle arrives. In addition, the time condition until the obstacle is reached is an arbitrary element and may not be included in the extraction condition of the distribution object.

[0150] The distribution object may also be determined on a lane basis. For example, assuming that there is an obstacle in the third lane, the vehicle traveling in the third lane is set as the distribution object. The vehicle scheduled to travel in the first lane that is not adjacent to the obstacle lane may also be excluded from the distribution object. The vehicle traveling in the second lane that is an adjacent lane corresponding to the obstacle lane may also be included in the distribution object because it is necessary to be alert to the insertion from the third lane that is an obstacle lane. Of course, the distribution object may also be selected on a road basis instead of on a lane basis. The configuration of selecting the distribution object on a road basis can alleviate the processing load of the map server 2.

[0151] For example, the obstacle notification data packet can be distributed to multiple vehicles that meet the above distribution object conditions in a multicast manner. In addition, the obstacle notification data packet can also be distributed in a unicast manner. In the case of unicast distribution of the obstacle notification data packet, it can also be sent in sequence with priority starting from vehicles close to the obstacle location or vehicles that arrive earlier considering the vehicle speed. It is also possible to exclude from the distribution object vehicles that are so close that they do not respond to the control or report even if the location of the obstacle is notified.

[0152] In addition, the distribution processing unit G4 may also be configured to send an obstacle notification data packet via a roadside machine. In such a configuration, the roadside machine broadcasts the obstacle notification data packet received from the distribution processing unit G4 to vehicles within the communication area of ​​the roadside machine through narrow area communication. In addition, the obstacle notification data packet may also be distributed to vehicles within a specified distance from the obstacle registration location using a regional multicast method. Various methods can be used as a method of distributing information.

[0153] In addition, the distribution processing unit G4 implements disappearance notification processing. The disappearance notification processing is the processing of distributing a communication data packet indicating the disappearance of an obstacle (hereinafter referred to as a disappearance notification data packet). For example, the disappearance notification data packet can be distributed to vehicles to which the obstacle notification data packet has been sent by multicast. After the disappearance determination unit G32 determines that the obstacle has disappeared, the disappearance notification data packet is distributed as quickly as possible (that is, instantly). In addition, the disappearance notification data packet can be distributed in a unicast manner in the same way as the obstacle notification data packet. In the case of distributing the disappearance notification data packet in a unicast manner, it can also be sent in sequence starting from vehicles close to the obstacle location or from vehicles that arrive earlier considering the vehicle speed. It is also possible to remove from the distribution object vehicles that are close enough to only notify the disappearance of the obstacle but also to reflect and report on the control. In addition, the distribution object of the disappearance notification data packet is limited to vehicles that have been notified of the existence of the obstacle, so the distribution object is selected using road links and altitude information.

[0154] The distribution processing unit G4 may also manage the information of the vehicles to which the obstacle notification data packet has been sent in the obstacle DB251. By managing the vehicles to which the obstacle notification data packet has been sent, it is also possible to easily select the distribution object of the disappearance notification data packet. Similarly, the distribution processing unit G4 may also manage the information of the vehicles to which the disappearance notification data packet has been sent in the obstacle DB251. By managing whether the obstacle notification data packet / disappearance notification data packet has been notified in the map server 2, it is possible to suppress the repeated distribution of the same information. In addition, it is also possible to use a flag or the like on the vehicle side to manage whether the obstacle notification data packet / disappearance notification data packet has been obtained. The obstacle notification data packet and the disappearance notification data packet are equivalent to the obstacle information.

[0155] <Server-side processing>

[0156] use Fig.16 The flowchart shown in the figure explains the obstacle point registration process performed by the map server 2. For example, the map server 2 performs the registration process at a predetermined update cycle. Fig.16 The preferred update cycle is a relatively short time such as five minutes or ten minutes.

[0157] In the map server 2, the server processor 21 repeatedly receives the obstacle location report sent from the vehicle at a constant period (step S501). Step S501 is equivalent to the vehicle behavior acquisition step. If the server processor 21 receives the obstacle location report, it determines the location of the received obstacle location report as the reporting object (step S502), and saves the received obstacle location report according to the location (step S503). In addition, considering that the position information reported by the obstacle location report is biased, the obstacle location report can also be saved according to an interval with a specified length.

[0158] Then, the server processor 21 extracts a location that satisfies the specified update condition (step S504). For example, a location where the number of report receptions within a specified time is above a specified threshold and a specified standby time has passed since the last obstacle presence / absence determination process was performed is extracted as an update target location. The standby time can be a relatively short time such as three minutes or five minutes. In addition, the update condition can also be a location where the number of report receptions is above a specified threshold, or a location where a specified standby time has passed since the last update.

[0159] The implementation conditions of the appearance determination process described later may be different from the implementation conditions of the disappearance determination process. The number of report receptions for executing the appearance determination process may be less than the number of report receptions for executing the disappearance determination process. For example, the number of report receptions for executing the appearance determination process may be three times, while the number of report receptions for executing the disappearance determination process may be six times, which is twice the number. According to this configuration, the appearance of an obstacle can be detected quickly, and the accuracy of determining the disappearance of the obstacle can be improved.

[0160] If the extraction of the update object location is completed, any one of them is set as the processing object (step S505), and it is determined whether it is a place that has been registered as an obstacle location or an unregistered place. In the case where the processing object location is a place that has not been registered as an obstacle location, the appearance determination unit G31 implements the appearance determination processing (step S507). Step S507 is equivalent to the appearance determination step. On the other hand, in the case where the processing object location is a place that has been registered as an obstacle location, the disappearance determination unit G32 implements the disappearance determination processing (step S508). Step S508 is equivalent to the disappearance determination step. Then, based on the determination result of the appearance determination processing or the disappearance determination processing, the registration content of the obstacle DB251 is updated (step S509).

[0161] For example, for a location determined to be an obstacle, its information is additionally registered in the obstacle DB 251. For a location determined to be an obstacle that has disappeared, the location information is deleted from the obstacle DB 251, or a flag indicating disappearance, i.e., a disappearance flag, is set. For the data of the obstacle location with a disappearance flag set, it can also be deleted at a specified time (e.g., one hour) after the flag is set. In addition, changes to the registration content can be omitted for locations whose existence status has not changed. It is also possible to update only the time information of the time when the judgment was implemented to the latest information (i.e., the current time) for locations whose existence status has not changed.

[0162] If the appearance determination process or disappearance determination process is completed for all update target points extracted in step S504, this flow ends. On the other hand, if an unprocessed point remains, the unprocessed point is set as a target point and the appearance determination process or disappearance determination process is executed (step S510).

[0163] <About the occurrence determination process>

[0164] Here, the appearance determination processing performed by the appearance determination unit G31 is described. The appearance determination unit G31 determines whether an obstacle has appeared at a location as a determination target, using lane changes, acceleration and deceleration change patterns of passing vehicles, camera images, obstacle recognition results of the vehicle-mounted system 1, and change patterns of traffic volume in each lane. The expression of the location here includes the concept of an interval with a predetermined length.

[0165] The presence determination unit G31 determines, for example, that an obstacle exists at a location where lane changes are made more than a predetermined threshold number of times within a constant time. The presence or absence of lane changes can be determined using the judgment result or report in the vehicle, or the presence or absence of lane changes can be detected based on the vehicle's driving trajectory. In addition, the presence determination unit G31 can also determine that an obstacle exists at a location where lane changes are made more than a predetermined number of times (for example, three vehicles) in a row.

[0166] For example, Fig.17 As shown, the position of the obstacle based on the lane change can be determined based on the driving trajectory Tr1 with the latest lane change timing among the trajectories of multiple vehicles that have implemented the lane change. For example, it is determined that there is an obstacle Obs at a location that is a specified distance (for example, 5m) further to the traveling direction side from the separation point (hereinafter referred to as the deepest separation point) Pd1 located closest to the traveling direction in the corresponding lane. The separation point can be either a location where the steering angle is above a specified threshold, or a location where the offset from the center of the lane is above a specified threshold. Or it can be a location where the lane dividing line begins to be crossed. The obstacle location here has a specified width in the front-to-back direction in order to allow a certain degree of error. The front-to-back direction here is equivalent to the direction in which the road is extended.

[0167] In addition, the position of the obstacle can also be determined based on the position of the recovery point Pe1 (hereinafter referred to as the front recovery point) closest to the deepest separation point Pd1. For example, it can also be the middle point between the deepest separation point Pd1 and the front recovery point Pe1. The recovery point can be a location where the steering angle of a vehicle that uses a lane change to enter a lane estimated to have an obstacle becomes less than a specified threshold. In addition, the recovery point can also be a location where the offset from the center of the lane of a vehicle that uses a lane change to enter a lane estimated to have an obstacle becomes less than a specified threshold. Instead of the steering angle, the angle of the vehicle body relative to the direction in which the road is extended can be used. In addition, the position of the obstacle can also be determined based on the obstacle detection position information contained in the obstacle location report. In the case where the obstacle detection positions for the same obstacle location are obtained from multiple vehicles, the average position of these detection position information can also be used as the position of the obstacle.

[0168] However, as an avoidance action for avoiding obstacles, there are many cases where a lane change for leaving the adjacent lane (hereinafter referred to as a lane change for leaving) and a lane change for returning to the original lane (hereinafter referred to as a lane change for recovery) are implemented in groups. However, as shown in the driving trajectory Tr1, a vehicle that has changed lanes due to the presence of an obstacle does not necessarily return to the original lane. For example, in the case where there is a plan to turn right after passing the side of the obstacle, and there is no free space to return to the original lane due to other vehicles, the original lane is not restored. In addition, as illustrated as the driving trajectory Tr2, it is also considered that a vehicle traveling in the adjacent lane of the obstacle changes lanes to the obstacle lane after passing the side of the obstacle. The server processor 21 does not count the number of vehicles that have implemented both the lane change for leaving and the lane change for recovery, but extracts the location where each type of lane change is concentrated as the obstacle location, so that the appearance of the obstacle can be detected more quickly. Of course, as another method, the obstacle location can also be detected based on the number of vehicles that have implemented both the lane change for leaving and the lane change for recovery.

[0169] In addition, if Fig.17 As shown, the location where the obstacle exists appears on the map as an area where there is no vehicle's driving track temporarily (hereinafter referred to as a trackless area Sp). The appearance determination unit G31 may also determine the presence or absence of the trackless area Sp based on the driving tracks of multiple vehicles within a specified time. Moreover, the appearance determination unit G31 may also set the location of the trackless area Sp as the obstacle location. In other words, the appearance determination unit G31 may also detect the appearance of an obstacle based on the occurrence of the trackless area Sp. In addition, in the case where a fallen object is assumed to be an obstacle, in order to distinguish it from lane restrictions, etc., it is preferred to limit the trackless area Sp where an obstacle is considered to exist to an area less than a specified length (for example, 20m). In other words, the appearance determination unit G31 may also determine that the type of obstacle is not a fallen object, but road construction, lane restrictions, etc., when the length of the trackless area Sp is above a specified threshold.

[0170] In addition, the appearance determination unit G31 may also detect the appearance of an obstacle based on the image data included in the obstacle location report. For example, it may also be determined that an obstacle exists based on the fact that the presence of an obstacle on the lane is confirmed based on the camera images of multiple vehicles. In addition, the appearance determination unit G31 may also set the image area on the opposite side of the avoidance direction compared to the specified reference point in the camera image provided by the vehicle as the report image as a verification area, and perform image recognition processing for determining the obstacle only on the verification area. The avoidance direction of the vehicle can be determined based on the behavior data of the vehicle. The verification area can also be called an analysis area or a search area.

[0171] For example, it is possible to Fig.18 The verification area corresponding to the avoidance direction is set as shown. Fig.18 The Px shown is a reference point, for example, the center point of a fixed image frame. The reference point Px may also be a road end or a vanishing point where the regression line of the lane dividing line intersects. Fig.18 ZR1 and ZR2 are verification areas applied when the avoidance direction is right. ZR2 can be the range to search when no avoidance object candidate is found in ZR1. Fig.18 ZL1 and ZL2 are verification areas applied when the avoidance direction is left. ZL2 can be a range to search when no avoidance object candidate is found in ZL1. In addition, the verification area corresponding to the avoidance direction is not limited to Fig.18 The verification area can be set as shown in Fig.19 As shown in the example, various setting methods are adopted. The dotted lines in the figure schematically show the boundaries of the verification areas.

[0172] According to the above-mentioned structure, the range of image recognition for determining obstacles is limited, so the processing load in the map server 2 can be reduced. In addition, the determination of avoidance objects can be performed quickly. The introduction of the verification area can also be applied to the disappearance determination of obstacles by the disappearance determination unit G32 described later. According to the structure of determining whether there is an obstacle only within the verification area, the time and processing load spent on determining the disappearance / absence of the obstacle can also be reduced. In addition, the map cooperation device 50 can also use the concept of the verification area to implement the search and locking process of the avoidance object. Moreover, the map cooperation device 50 can also be configured to cut out the image area corresponding to the above-mentioned verification area corresponding to the avoidance direction and send it as a report image. For example, when the avoidance direction is to the right, the map cooperation device 50 can also send the image area containing the verification areas ZL1 and ZL2 as a report image.

[0173] In addition, the presence determination unit G31 may determine that an obstacle exists based on the obstacle detection results of the surrounding monitoring sensors included in the obstacle location reports from multiple vehicles. For example, if the number of reports indicating the presence of an obstacle within the most recent specified time is greater than a specified threshold, it may be determined that an obstacle exists at the location where the report was sent.

[0174] In addition, the occurrence determination unit G31 can also detect the location where the specified acceleration and deceleration pattern occurs as an obstacle location. Usually, the driver's seat occupant / automatic driving system recognizes that there is an obstacle in front of the vehicle and temporarily decelerates, changes the driving position, and then accelerates. In other words, it is assumed that an acceleration and deceleration pattern from deceleration to acceleration is observed near the obstacle location. Conversely, the area where the frequency of occurrence / continuous occurrence number of the above-mentioned acceleration and deceleration pattern within the recent specified time is greater than a specified threshold can also be extracted as an obstacle location. The change of driving position here includes not only lane changes, but also actions to bring the driving position in the same lane close to any left or right corner, and driving across lane dividing lines.

[0175] In addition, even when there are moving objects such as birds, pedestrians, and wild animals as momentary obstacles, an acceleration / deceleration pattern may be observed in which the vehicle decelerates temporarily and then accelerates. In such a case, it is preferred to perform obstacle location detection using the acceleration / deceleration pattern with the area accompanying the change in the driving position as the parent group. In other words, it is preferred that the occurrence determination unit G31 detects the area where the prescribed acceleration / deceleration pattern is observed together with the change in the driving position as the obstacle location.

[0176] The above discloses a method of using the acceleration in the front-rear direction for detecting the obstacle location, but it is assumed that when the driving position is changed to avoid the obstacle, the acceleration in the lateral direction also generates a predetermined pattern. For example, the area where the frequency / number of consecutive occurrences of the predetermined acceleration / deceleration pattern in the left-right direction within the recent predetermined time is greater than a predetermined threshold value can also be extracted as the obstacle location.

[0177] In addition, it can be expected that the traffic volume of the lane where the obstacle exists is reduced compared to the traffic volume of the adjacent lane. It is also possible to extract a lane whose traffic volume in the most recent specified time period is reduced by a specified value / prescribed ratio compared to the specified time period before, and when the traffic volume of its adjacent lane increases in the same time period, it is determined that there is an obstacle in the lane. In addition, it is sufficient to determine where the obstacle exists in the lane detected by the above method based on the driving trajectory of the vehicle traveling in the lane.

[0178] In addition, the presence determination unit G31 may also detect the location of an obstacle based on the transfer of authority from the automatic driving device to the passenger or the override by the driver's seat passenger. For example, the presence of an obstacle may be detected by acquiring and analyzing the image of the front camera 11 when the automatic driving device transfers authority to the passenger or detects that the driver's seat passenger has overrided the vehicle, and determining whether the cause is an obstacle.

[0179] In the above, a plurality of viewpoints for determining the presence of an obstacle are listed, but the presence determination unit G31 may use any of the above to determine the presence of an obstacle. In addition, a plurality of viewpoints may be used in combination to determine the presence of an obstacle. When a plurality of viewpoints are used in combination to determine the presence of an obstacle, a weight corresponding to the type of determination material may be added for determination. For example, when the weight for avoidance action is 1, the recognition result of the camera alone may be 1.2, and the recognition result of the fusion may be 1.5, and so on.

[0180] In addition, if Fig. 20 As shown, the threshold value of the number of vehicles that have taken avoidance action for determining that there is an obstacle may be changed based on whether the server processor 21 / operator has confirmed the existence of an obstacle as a result of analyzing the image provided by the vehicle. In addition, the number of vehicles that have taken avoidance action required for determining that there is an obstacle may be changed based on whether an obstacle is detected by the vehicle's surrounding monitoring sensor or the obstacle presence determination unit F51. Fig. 20 The column of the number of vehicles in can be replaced by the ratio of the number of vehicles that have taken avoidance action or the number of times obstacle location reports indicating that avoidance action has been taken have been received consecutively.

[0181] <About disappearance determination processing>

[0182] Here, the disappearance determination process performed by the disappearance determination unit G32 is described. The disappearance determination unit G32 is configured to periodically determine whether there is still an obstacle at the obstacle location detected by the appearance determination unit G31 based on the obstacle location report. As materials for determining whether the obstacle disappears, the presence or absence of lane changes, the vehicle's driving trajectory, the change pattern of acceleration and deceleration of passing vehicles, camera images, the recognition result of the obstacle by the vehicle-mounted system 1, the change pattern of the traffic volume in each lane, etc. can be used.

[0183] For example, the disappearance determination unit G32 can make a determination based on the decrease in the number of lane changes performed at the obstacle location. For example, when the number of lane changes near the obstacle location is less than a predetermined threshold, it can be determined that the obstacle has disappeared. In addition, the disappearance determination unit G32 can also compare the decrease in the number of lane changes near the obstacle location as a vehicle behavior with the time when the obstacle is detected, and when there is a statistically significant difference, it is determined that the obstacle has disappeared.

[0184] The disappearance determination unit G32 may determine that the obstacle has disappeared based on the decrease in the number of vehicles traveling across the lane dividing line near the obstacle location. In addition, the disappearance determination unit G32 may determine that the obstacle has disappeared based on the average value of the offset from the lane center on the obstacle lane becoming less than a predetermined threshold. In other words, the disappearance determination unit G32 may determine that the obstacle has disappeared when the lateral position change of the vehicle passing near the obstacle location becomes less than a predetermined threshold.

[0185] The disappearance determination unit G32 may also determine that an obstacle has disappeared based on the presence of a vehicle that passes directly through a lane containing an obstacle location (i.e., an obstacle lane) without performing an avoidance action such as a lane change. For example, the presence of a vehicle traveling at an obstacle location may be determined based on the driving trajectory. More specifically, the obstacle may be determined to have disappeared when the driving trajectory of a certain vehicle passes through the obstacle location. In addition, the obstacle may be determined to have disappeared when the number of such vehicles exceeds a specified threshold.

[0186] In addition, when the obstacle location report includes a camera image, the disappearance determination unit G32 may also determine whether there is still an obstacle by analyzing the camera image. The disappearance determination unit G32 may also perform statistical processing on the analysis results of the image data from multiple vehicles to determine whether the obstacle still exists. The statistical processing here includes majority decision and averaging.

[0187] When the obstacle location report includes information indicating whether the surrounding monitoring sensor has detected an obstacle (i.e., the obstacle detection result), the disappearance determination unit G32 may also determine whether the obstacle still exists or disappears by statistically processing the information. For example, when the number of times a report indicating that no obstacle has been detected is received becomes greater than a predetermined threshold, it may be determined that the obstacle has disappeared.

[0188] The disappearance determination unit G32 may also determine that the obstacle has disappeared when no predetermined acceleration / deceleration pattern is observed as the behavior of vehicles passing near the obstacle location. Alternatively, the obstacle may be determined to have disappeared based on the fact that there is no meaningful difference in traffic volume between the obstacle lane and its adjacent lanes on the left and right, or the difference is reduced and the traffic volume of the obstacle lane increases. The traffic volume can be, for example, the number of vehicles passing per unit time in the road section from the obstacle location to 400 meters before it.

[0189] In the above, a plurality of viewpoints for determining whether an obstacle has disappeared are listed, but the disappearance determination unit G32 may use any one of the above viewpoints to determine that an obstacle has disappeared, or may use a plurality of viewpoints in combination to determine that an obstacle has disappeared. In the case of using a plurality of viewpoints in combination to determine that an obstacle has disappeared, a weight corresponding to the type of determination material may be added for determination. In the case of setting the weight for vehicle behavior to 1, the recognition result of the camera alone may be 1.2, the recognition result of the fusion may be 1.5, and so on.

[0190] In addition, if Fig.21 As shown, the threshold value of the number of vehicles that have gone straight through the corresponding location for determining that an obstacle has disappeared may be changed based on whether the server processor 21 / operator has confirmed the disappearance of the obstacle as a result of analyzing the image provided by the vehicle. In addition, the threshold value of the number of vehicles that have gone straight through the corresponding location required for determining that an obstacle has disappeared may be changed based on whether an obstacle has been detected by the vehicle's surrounding monitoring sensor or the obstacle presence determination unit F51. In addition, Fig.21 The column for the number of vehicles can be replaced with the ratio of the number of vehicles that drive straight at the corresponding location, or the number of times that obstacle location reports indicating that there is no obstacle are continuously received. Driving straight here means driving along the road in the lane that has been driven so far without changing the driving position such as changing lanes. Driving straight here does not necessarily mean driving while maintaining the steering angle at 0°.

[0191] <Additional information on the method of determining the appearance / disappearance of obstacles>

[0192] Static map elements such as road structures are map elements that do not change over time, so the update of their map data can use many driving tracks accumulated over a week or a month, for example. By updating the map data using reports from many vehicles as a parent group, it is expected that the accuracy will be improved.

[0193] However, obstacles such as fallen objects are equivalent to dynamic map elements whose existence status changes in a relatively short period of time compared to road structures, etc. Therefore, the detection of the generation and disappearance of obstacles requires further real-time performance. In order to improve the accuracy of information such as the existence status and location of obstacles, it is preferred to use reports from many vehicles as a parent group, but if you want to collect more reports from vehicles, it will take time and impair real-time performance. In other words, in the structure of detecting the existence / disappearance of obstacles, in order to ensure real-time performance compared to the case of generating static maps, it is necessary to make judgments and distribute them as accurately as possible based on fewer vehicle reports.

[0194] According to such a situation, the above-mentioned appearance determination unit G31 detects the obstacle location based on the obstacle location report obtained within the specified first time from the current moment. In addition, the disappearance determination unit G32 determines the disappearance / continuity of the obstacle based on the obstacle location report obtained within the specified second time. In order to ensure real-time performance, the first time and the second time are preferably set to a time shorter than ninety minutes, for example. For example, the first time is set to ten minutes, twenty minutes or thirty minutes. The second time can also be ten minutes, twenty minutes or thirty minutes. The first time and the second time can be the same length or different lengths. The first time and the second time can also be five minutes or one hour.

[0195] In addition, it is considered that information about the presence of an obstacle is more useful for driving control than information about the absence of an obstacle. This is because if information about the lane where the obstacle is present can be acquired in advance as map data, sufficient avoidance actions can be planned and implemented. As a result, it is also assumed that there is a need to detect and distribute the presence of an obstacle earlier. Based on this, in order to be able to detect the presence of an obstacle and start distribution earlier, the first time can also be set to be shorter than the second time.

[0196] In addition, it is also assumed that there is a need to avoid erroneous determination that the obstacle has disappeared and erroneous distribution is avoided even though the obstacle still exists. In view of such a need, the second time can also be set to be longer than the first time. According to the configuration in which the second time is set to be longer than the first time, the occurrence of the obstacle can be notified quickly, and the worry of erroneous determination that the obstacle has disappeared can also be reduced.

[0197] The appearance determination unit G31 and the disappearance determination unit G32 may also be configured to determine the appearance / continuity of obstacles by, for example, increasing the weight, etc., by giving priority to using the information shown in the report with a newer acquisition time. For example, the weight of information acquired within ten minutes may be set to 1, the weight of information acquired within thirty minutes and more than ten minutes in the past may be set to 0.5, and the weight of information acquired in the past may be set to 0.25, etc., and a weighting coefficient corresponding to the newness of the information may be applied for statistical processing. According to such a configuration, the latest status can be more strongly reflected in the judgment result, and the real-time performance can be improved.

[0198] In addition, statistical processing can also be performed by weighting according to the characteristics of the reporting source. For example, the weight of the report from the autonomous driving vehicle can be set larger. It can be expected that the autonomous driving vehicle is relatively equipped with high-performance millimeter wave radar 12, front camera 11, LiDAR, etc. In addition, the possibility of the autonomous driving vehicle unnecessarily changing the driving position is low. The change of the driving position of the autonomous driving vehicle is relatively likely to be an action to avoid obstacles. Therefore, by giving priority to the use of reports from the autonomous driving vehicle, the accuracy of determining the presence or absence of obstacles can be improved.

[0199] In addition, the appearance determination unit G31 and the disappearance determination unit G32 may also be configured to treat reports from vehicles that frequently perform changes in driving positions such as lane changes, that is, vehicles with unstable driving positions as noise and not be used for determination processing. The vehicle position management unit G2 may determine an unstable driving position vehicle based on vehicle status reports uploaded in sequence, and manage the vehicle through signs, etc. According to such a configuration, the concern of misjudging the presence or absence of obstacles based on reports from vehicles driven by users who frequently perform lane changes can be reduced. A variety of conditions can be applied to the conditions for considering a vehicle with an unstable driving position. For example, a vehicle whose number of lane changes within a constant time is above a specified threshold can be extracted as an unstable driving position vehicle. The threshold here is preferably set to three or more times in order to exclude lane changes (two times of separation and recovery) for obstacle avoidance. For example, an unstable driving position vehicle may be a vehicle that performs four or more lane changes within a constant time such as ten minutes.

[0200] In addition, if Fig. 20 as well as Fig.21 As illustrated, the conditions for determining that an obstacle appears (such as threshold values, etc.) may be different from the conditions for determining that an obstacle disappears. For example, the conditions for determining that an obstacle disappears may be set more strictly than the conditions for determining that an obstacle appears. The judgment material for the appearance of an obstacle may be different from the judgment material for the disappearance of an obstacle. In addition, the weight of each type of information may be different when determining the appearance and when determining the disappearance. For example, when determining the appearance of an obstacle, the weight of the analysis result of the camera image may be greater than the vehicle behavior data, and on the other hand, when determining that an obstacle disappears, the weight of the vehicle behavior data may be greater than the analysis result of the camera image. This is because the camera image is oriented to verification of the presence of an object, and on the other hand, if the possibility of shooting other locations is considered, the reliability of verification without an object is poor.

[0201] Alternatively, at least one of the presence determination unit G31 and the disappearance determination unit G32 may determine whether the obstacle is a light object such as Styrofoam that may be moved by wind based on the deviation of obstacle detection positions reported from a plurality of vehicles.

[0202] <Application example to vehicle control>

[0203] Next use Fig. 22 An example of vehicle control using obstacle information is described below. Fig. 22 For example, when the automatic lane change function of the driving assistance ECU 60 is enabled based on a user operation, the automatic lane change function is executed at a predetermined period. Fig. 22 The vehicle control process shown in the figure is sufficient. In addition, the state in which the automatic lane change function is enabled also includes automatic driving in which the vehicle is autonomously driven according to a predetermined driving plan. Fig. 22 The vehicle control process shown includes steps S601 to S605 as an example. The driving support ECU 60 and the map cooperation device 50 cooperate to execute steps S601 to S605.

[0204] First, in step S601, the map cooperation device 50 reads the obstacle information on the map stored in the memory M1, provides it to the driving assistance ECU60 and moves to step S602. In step S602, the driving assistance ECU60 determines whether there is an obstacle within a specified distance ahead on the driving lane of the vehicle based on the obstacle information on the map. This process is equivalent to determining whether there is an obstacle identified by the map server 2 within the specified distance, that is, whether there is an obstacle registration location. If there is no obstacle on the driving lane of the vehicle, step S602 is negatively determined to end the process. In this case, the driving control based on the driving plan prepared separately is continued. On the other hand, if there is an obstacle on the driving lane of the vehicle, step S602 is affirmatively determined to execute step S603.

[0205] In step S603, the driving plan is revised. That is, a driving plan is prepared that includes a lane change from the current lane where the obstacle exists to the adjacent lane. The revised driving plan also includes the setting of the point of departure from the current lane (i.e., the lane change point). If step S603 is completed, step S604 is executed.

[0206] In step S604, the HMI system 16 cooperates to display information related to the revised driving plan. Figure 4 The image shown in the example is used to notify the passenger that a lane change is to be performed to avoid the obstacle. If step S604 is completed, the process proceeds to step S605. In step S605, the lane change is performed and the process ends.

[0207] In addition, although the above discloses a configuration in which no special processing is performed when the obstacle lane is not the driving lane of the vehicle, the present invention is not limited to this. When there is an obstacle in the adjacent lane of the vehicle's lane, other vehicles driving in the obstacle lane are more likely to change lanes to the vehicle's lane. In view of such a situation, it is preferred that when there is an obstacle in the adjacent lane, a predetermined insertion warning process such as setting a longer distance between the vehicle and the preceding vehicle or notifying the passengers to pay attention to the insertion from the adjacent lane is performed.

[0208] <An example of the operation and effect of the above system>

[0209] According to the above system configuration, first, the map cooperation device 50 uses the situation of taking avoidance action as a trigger to upload an obstacle location report. The map server 2 detects the location where the obstacle exists on the road based on the information uploaded from the vehicle. Then, the existence of the obstacle is notified to the vehicle scheduled to travel at the location where the obstacle exists. In addition, the map cooperation device 50 sends the vehicle behavior data representing the behavior of the vehicle when passing near the obstacle registration location notified from the map server 2 to the map server 2.

[0210] Here, assuming that an obstacle remains and the vehicle is traveling in a lane with an obstacle, the vehicle behavior data sent by the map cooperation device 50 to the map server 2 becomes data indicating that an avoidance action has been taken. In addition, even if the vehicle is traveling in a lane without an obstacle, it may decelerate in order to avoid a collision with other vehicles that change lanes to avoid an obstacle. In other words, specific behaviors that are not easy to occur when there is no obstacle may be observed, such as emergency deceleration to avoid a collision with an intervening vehicle. On the other hand, when the obstacle disappears, no vehicle behavior to avoid the obstacle or the intervening vehicle is observed. In this way, the vehicle behavior data when passing near the obstacle registration location serves as an indicator of whether the obstacle remains.

[0211] Therefore, the map server 2 can determine whether an obstacle still exists or disappears at the obstacle registration location based on the vehicle behavior data provided by multiple vehicles. In addition, when the disappearance of an obstacle is detected based on a report from a vehicle passing through the obstacle registration location, the vehicle to which the information of the obstacle has been distributed is notified of the disappearance of the obstacle.

[0212] Fig.23 is a diagram schematically showing the change in the behavior of a vehicle caused by the presence or absence of obstacle information on the map. Fig.23As shown in (A), starting from reaching the position where the front camera 11 can recognize the obstacle, avoidance actions such as lane changes are implemented. The map server 2 detects the existence / appearance of obstacles by collecting such vehicle behaviors and starts distributing them as obstacle information. The recognizable position may change depending on the performance of the front camera 11, millimeter wave radar 12, and the size of the obstacle. In a good environment such as on a sunny day, the recognizable position is a location close to the obstacle, for example, about 100m to 200m away.

[0213] Fig.23 (B) schematically shows the behavior of a vehicle that has obtained obstacle information on a map. Fig.23 As shown in (B), lane changes can be implemented before reaching a position that can be recognized. In other words, lane changes, handovers, etc. can be implemented in advance with sufficient time.

[0214] On the other hand, obstacles disappear as time passes, such as by removal. There is a predetermined time difference (i.e., delay) between the disappearance of an obstacle in the real world and the detection of this by the map server 2. Fig.23 As shown in (C), a scenario is created in which a vehicle passes by by changing lanes based on obstacle information on the map even though there is actually no obstacle.

[0215] However, the map server 2 of this embodiment is configured to obtain an obstacle location report from a vehicle passing near an obstacle registration location, so that the disappearance of an obstacle can be quickly recognized based on the obstacle location report. As a result, the disappearance of the obstacle can be quickly distributed to the vehicle, and the worry of unnecessary lane changes, handovers, etc. on the vehicle side can be reduced. Fig.23 (D) shows the state after the map server 2 confirms the disappearance of the obstacle.

[0216] In addition, the map server 2 of the present disclosure verifies whether the obstacle has truly disappeared based on reports from multiple vehicles and / or from multiple viewpoints. With such a configuration, the concern of misreporting that the obstacle has disappeared despite the fact that the obstacle actually exists can be reduced.

[0217] In addition, according to the configuration of the present disclosure, when a result of determination that an obstacle has disappeared is obtained as a result of analysis of an image uploaded from a vehicle, the threshold value for the number of vehicles that do not take avoidance action for determination that the obstacle has disappeared is lowered. For example, when the disappearance of an obstacle is also confirmed based on the result of image analysis in the server processor 21, it is also possible to determine that the obstacle has disappeared based on the behavior information of one to several vehicles. In addition, when a result of determination that an obstacle has disappeared is obtained by statistically processing the obstacle recognition results in a plurality of vehicles, the threshold value for the number of vehicles that do not take avoidance action for determination that the obstacle has disappeared is lowered. According to this configuration, the determination that an obstacle has disappeared can be made more quickly. As a result, for example, the time from Fig.23 The transition period from (C) to (D) is achieved by combining vehicle behavior and image analysis to determine the existence of obstacles, thereby achieving both real-time performance and information reliability.

[0218] In addition, the map server 2 determines the obstacle disappearance as a necessary condition that the vehicle does not take action to avoid the obstacle. The judgment is not made based solely on the image, so the worry of mistakenly judging that the obstacle disappears when the camera accidentally does not see the obstacle can be reduced.

[0219] In addition, in the above configuration, not only fallen objects are detected, but also parked vehicles on general roads (parked vehicles on the road) are detected as obstacles. There are cases where parked vehicles on the road block nearly half of the lane, which may become an obstacle to automatic driving / driving assistance functions on general roads. For example, due to the blocking of lanes by parked vehicles on the road, there is a possibility of interrupting services such as automatic driving. According to the configuration of distributing the positions of parked vehicles on the road as described above, it is possible to perform handovers with sufficient space, or adopt a path where there are no parked vehicles on the road.

[0220] In addition, in the above-mentioned configuration, the occurrence and disappearance of obstacles are detected based on the behavior of multiple vehicles. According to such a configuration, compared with the configuration disclosed in Patent Document 1, the worry of misjudging the presence or absence of falling objects in rainy days, at night, and in backlight can be reduced.

[0221] In addition, assuming that fallen objects are obstacles, it is difficult to determine whether the fallen objects are objects that hinder driving or objects that need not be avoided through image recognition alone. Therefore, in the configuration that detects obstacles such as fallen objects only through image recognition, there is a concern that objects that vehicles need not avoid are also detected as obstacles and distributed. Furthermore, there is a concern that the vehicle notified of the existence of an obstacle will perform unnecessary avoidance actions such as lane changes. In addition, objects that hinder driving are, for example, three-dimensional objects such as bricks and tires. Objects that need not be avoided are, for example, flat garbage such as folded corrugated paper.

[0222] In contrast, in the configuration of the present disclosure, the presence or absence of an obstacle is determined based on the behavior of multiple vehicles. If an obstacle-like object on the road is an object that the vehicle does not need to avoid, there is a high possibility that there is a vehicle among the multiple vehicles that does not take evasive action and passes over the object. Therefore, according to the configuration of the present disclosure, the concern that flat garbage is detected as an obstacle and distributed can be reduced.

[0223] The embodiments of the present disclosure are described above, but the present disclosure is not limited to the above-mentioned embodiments, and the various modifications described below are also included in the technical scope of the present disclosure, and in addition to the following, various changes can be made to implement without departing from the main purpose. For example, the various modifications described below can be appropriately combined and implemented within the scope that does not cause technical contradictions. In addition, the same figure marks are attached to the components having the same functions as the components described in the above-mentioned embodiments, and their descriptions are omitted. In addition, when only a part of the structure is mentioned, the structure of the previously described embodiment can be applied to the other parts.

[0224] <About the process of determining the disappearance of obstacles>

[0225] The disappearance determination unit G32 may also determine whether a vehicle has appeared to go straight through the obstacle registration location based on the vehicle status report, and determine that the obstacle has disappeared based on the occurrence of a vehicle going straight through the obstacle registration location. According to such a configuration, it is not necessary to send the obstacle location report separately from the vehicle status report. As a result, the processing on the vehicle side is simplified. In other words, in the configuration in which each vehicle sends a vehicle status report, the content of the vehicle status report can be cited as vehicle behavior data, so the obstacle location report becomes an arbitrary element.

[0226] <About the judgment material of the appearance / disappearance of obstacles>

[0227] The above discloses the configuration of the vehicle-mounted system 1 using the front camera 11 to detect obstacles, but is not limited thereto. It can also be configured to detect obstacles using a side camera that shoots the side of the vehicle or a rear camera that shoots the rear. Similarly, it can also be configured to detect obstacles using a side millimeter-wave radar that sends detection waves toward the side of the vehicle or a rear side millimeter-wave radar that uses the rear side (that is, obliquely rearward) as a detection range.

[0228] For example, the vehicle-mounted system 1 or the map server 2 may also use the image of the side camera to determine the presence or absence of an obstacle. In the case where an obstacle blocks the lane, the implementation of the lane change can be expected, but the vehicle does not drive in the obstacle lane after the lane change, so the obstacle is not easily reflected in the front camera 11. As a result, there is a concern that there is no obstacle after the lane change. According to the structure of using the image data of the side camera of the side where the obstacle exists for the judgment of the presence or absence of the obstacle, the concern of not seeing the obstacle when passing the side of the obstacle can be reduced. In addition, the side camera may also be a camera set in the rearview mirror to observe the rear side. In addition, the side camera and the front camera 11 may also be used complementary. For example, the report data generation unit F5 may also be configured to upload an obstacle location report containing an image of the front camera 11 captured during the approach of the obstacle registration location and an image of the side camera captured after the driving position is changed. The report image may also be selected from the images of the side camera and the rear camera. It is equivalent to the image outside the vehicle captured by the vehicle-mounted camera that captures the outside of the vehicle, such as the front camera 11, the side camera, and the rear camera.

[0229] In addition, when the vehicle is equipped with multiple cameras, the camera used for obstacle recognition and the uploaded camera image can be switched according to the surrounding environment of the vehicle. For example, when the distance between the front vehicle and the vehicle is less than a specified threshold and the distance between the rear vehicle and the vehicle is above a specified threshold, the image of the front camera 11 can be replaced by the image of the rear camera or the side camera as the judgment material for the presence or absence of obstacles of the vehicle-mounted system 1 or the map server 2. In addition, when the leading vehicle is a large vehicle such as a truck or a fire truck, and the following vehicle is a small vehicle such as a light car, the rear camera or the side camera can also be used as the camera used to determine the presence or absence of obstacles. In other words, the cameras used to determine the presence or absence of obstacles can also be used separately according to whether the field of view in front is wide. The same is true for millimeter-wave radars. When multiple millimeter-wave radars are provided, these multiple millimeter-wave radars can also be used separately according to the surrounding environment.

[0230] In addition, as a device for detecting obstacles, in addition to cameras and millimeter wave radars, LiDAR, sonar, etc. can also be used. Millimeter wave radars, LiDAR, sonar, etc. are equivalent to distance measuring sensors. The map cooperation device 50 can also be configured to detect obstacles and the like using multiple types of devices. That is, the map cooperation device 50 can also detect obstacles through sensor fusion.

[0231] The obstacle presence determination unit F51 or the obstacle information management unit G3 may also be as follows: Fig.24As shown, the presence or absence of an obstacle is determined based on the intended action of the driver's seat passenger detected by the DSM (Driver Status Monitor) 17. The DSM 17 is a device that uses a near-infrared camera to capture the face of the driver's seat passenger, and performs image recognition processing on the captured image to sequentially detect the direction of the driver's seat passenger's face, the direction of sight, the degree of eyelid opening, etc. The DSM 17 is, for example, arranged on the upper surface of the steering column cover, or the upper surface of the instrument panel, the rearview mirror, etc., with the near-infrared camera facing the headrest of the driver's seat, so that the face of the driver's seat passenger can be captured.

[0232] For example, the obstacle presence determination unit F51 or the obstacle information management unit G3 may also determine that there is an obstacle based on the fact that the line of sight of the driver's seat passenger is toward the direction determined to be the obstacle when passing on the side of the obstacle. In addition, it may also be determined that the obstacle has disappeared based on the fact that the passengers of the vehicle traveling on the adjacent lane of the obstacle do not observe the direction in which the obstacle exists. In other words, the movement of the eyes of the driver's seat passenger when passing on the side of the obstacle can also serve as material for determining the presence or absence of an obstacle. The vehicle-mounted system 1 may also upload time series data of the line of sight direction of the driver's seat passenger when passing on the side of the obstacle as an obstacle location report. In addition, the vehicle-mounted system 1 may also upload the determination result of whether the line of sight of the driver's seat passenger is toward the obstacle registration location when passing on the side of the obstacle. The obstacle information management unit G3 may also determine whether there is an obstacle based on the passenger's line of sight information.

[0233] Furthermore, the more difficult it is to understand the type of obstacle, the more likely it is to attract people's attention. Therefore, according to the above method, there is an advantage that objects that are not easily identified as obstacles by image recognition or the like are easily detected as obstacles. In addition, in the case where the obstacle is a large parked vehicle, in order to confirm whether a person does not fly out from the back of the parked vehicle, it is also possible to expect the driver's seat passenger to turn his or her sight toward the parked vehicle. In other words, according to the above configuration, the detection accuracy of the parked vehicle as an obstacle can also be improved.

[0234] In addition, the map cooperation device 50 may also change the content and format of the uploaded obstacle location report according to the type of the detected obstacle. For example, when the obstacle is a point-like object such as a fallen object, the location information, type, size, color tone, etc. are uploaded. On the other hand, when the obstacle is a regional event with a specified length in the direction of the road extension such as lane restriction or construction, the start and end positions of the obstacle section and the type of obstacle may also be uploaded.

[0235] In addition, the map cooperation device 50 may also upload the behavior of surrounding vehicles as a material for determining whether there is an obstacle to the map server 2. For example, when the leading vehicle also changes lanes, the behavior data of the leading vehicle may also be uploaded to the map server 2 together with the behavior of the vehicle. Specifically, the offset of the leading vehicle relative to the lane center may be digitized in the front camera 11 to determine whether a lane change has been made in the vicinity of the obstacle registration location, and an obstacle location report including the determination result may be sent.

[0236] The behavior of surrounding vehicles can be determined based on input signals from surrounding monitoring sensors. More specifically, the behavior of surrounding vehicles can be determined by referring to technologies such as SLAM (Simultaneous Localization and Mapping). The behavior of surrounding vehicles can also be determined based on the received data of inter-vehicle communication. In addition, it is not limited to the preceding vehicle mentioned above, and it is also possible to upload whether the subsequent vehicle has changed lanes. The behavior of the uploaded surrounding vehicles is not limited to lane changes, but can also be changes in the driving position in the same lane. In addition, insertions from adjacent lanes can also be uploaded as indicators of obstacles in the adjacent lanes. The structure that obtains the behavior of other vehicles traveling around the vehicle based on inter-vehicle communication or signals from surrounding monitoring sensors is also equivalent to the vehicle behavior detection unit. The data representing the behavior of surrounding vehicles is equivalent to other vehicle behavior data. In order to distinguish it from other vehicle behavior data, the vehicle behavior data of this vehicle will also be referred to as this vehicle behavior data.

[0237] However, it is assumed that the vehicle equipped with the map cooperation device 50, that is, the carrying vehicle, changes lanes in advance to a lane without obstacles, etc., after obtaining obstacle information from the map server 2, and then passes the side of the obstacle. Therefore, when the map server 2 recognizes that there is an obstacle at a certain location, it is not easy for the carrying vehicle to perform avoidance action near the obstacle registration location. The vehicle that performs avoidance action just before reaching the obstacle registration location may at best be a vehicle that is not equipped with the map cooperation device 50, that is, a non-carrying vehicle. Of course, after the distribution of obstacle information begins, the carrying vehicle may also perform actions that indicate the existence of an obstacle, such as deceleration caused by the insertion of a non-carrying vehicle. However, the inserted vehicle is not always decelerated. After the distribution of information that an obstacle exists at a certain location begins, the usefulness of the vehicle behavior data of the carrying vehicle at the corresponding location is relatively reduced compared to before the distribution begins.

[0238] Based on such a situation, the map cooperation device 50 may also be configured to send the behavior data of surrounding vehicles (that is, the behavior data of other vehicles) and the detection results of surrounding monitoring sensors as an obstacle location report in preference to the behavior data of the vehicle itself when passing through an obstacle registration location. For example, the map cooperation device 50 may also be configured to send at least one of the behavior data of other vehicles and the detection results of surrounding monitoring sensors instead of the behavior data of the vehicle itself when passing through an obstacle registration location. The surrounding vehicles that are preferably the objects of the report here are other vehicles traveling on the obstacle lane. This is because the obstacle lane is most susceptible to the influence of obstacles and is more useful as an indicator of whether an obstacle remains. According to the above-mentioned configuration, the uploading of less useful information regarding the detection of the disappearance of an obstacle can be suppressed. In addition, information that is more useful when detecting the disappearance of an obstacle can be preferentially concentrated on the map server 2.

[0239] Furthermore, even when approaching an obstacle registration location, there may be a situation where an obstacle lane is used as the driving lane of the vehicle based on the driver's instructions. The map cooperation device 50 may also be configured to upload the behavior data of the vehicle in priority over the behavior data of surrounding vehicles when the driving lane of the vehicle is an obstacle lane at a judgment point closer to the specified distance than the obstacle registration location. The map cooperation device 50 may also send a data set containing the behavior data of the vehicle as an obstacle location report when the driving lane of the vehicle at the judgment point is an obstacle lane, and send a data set not containing the behavior data of the vehicle when the driving lane of the vehicle is not an obstacle lane. The judgment point can be set, for example, to a location where the distance to the object is reported from the obstacle registration location toward the vehicle side.

[0240] The map cooperation device 50 may also be configured to reduce the amount of information of the vehicle behavior data included in the obstacle location report sent when passing near the obstacle registration location when the vehicle driving lane at the determination point is not an obstacle lane, compared to the case where it is an obstacle lane. For example, the amount of information of the behavior data can be reduced by extending the sampling interval or reducing the number of items sent as the vehicle behavior data. The method of reducing the amount of information of the vehicle behavior data included in the obstacle location report also includes the case where the obstacle location report does not include the vehicle behavior data at all.

[0241] Furthermore, the map cooperation device 50 may also be configured to change the content of the data set sent to the map server 2 in the case of an obstacle not notified from the map server 2 being found and in the case of an obstacle registration location that has been received. For convenience, the data set sent as an obstacle location report when an obstacle not notified from the map server 2 is found is also recorded as an unregistered location report. In addition, the data set sent as an obstacle location report when passing near an obstacle notified from the map server 2 is also recorded as a registered location report. The unregistered location report can be, for example, a data set containing the behavior data of the vehicle and input data from the surrounding monitoring sensor, and on the other hand, the registered location report can be, for example, a data set containing the behavior data of other vehicles and input data from the surrounding monitoring sensor. The registered location report can be a data set that suppresses the size of the behavior data of the vehicle to less than half, for example, compared to the unregistered location report. According to this configuration, information corresponding to the characteristics of the appearance judgment and disappearance judgment of the obstacle can be efficiently concentrated on the map server 2.

[0242] In addition, in the configuration of uploading the behavior of surrounding vehicles, there is a possibility that the behavior of the same vehicle may be reported in an overlapping manner on the map server 2. In order to prevent the map server 2 from counting the behavior of the same vehicle in an overlapping manner, it is preferred to upload the behavior of the own vehicle and surrounding vehicles in a corresponding relationship with their respective vehicle IDs. The vehicle IDs of surrounding vehicles can be obtained by using inter-vehicle communication or by image recognition of license plate numbers.

[0243] <Calculation of detection reliability of the map cooperation device 50>

[0244] The obstacle presence determination unit F51 may also calculate the possibility of the actual existence of an obstacle as the detection reliability based on the combination of whether the obstacle is detected by the front camera 11, whether the obstacle is detected by the millimeter wave radar 12, and whether the avoidance action is taken. Fig.25 The above can also be configured so that the more points (sensors, actions, etc.) that suggest the existence of obstacles, the higher the detection reliability is calculated. Fig.25 The method of determining the detection reliability shown is an example and can be changed appropriately.

[0245] also, Fig.25The vehicle behavior refers to the avoidance action of the vehicle when an obstacle exists in front of the vehicle's driving lane. When an obstacle exists in an adjacent lane, the behavior of surrounding vehicles traveling on the obstacle lane can be used instead for the calculation of detection reliability. For example, the presence or absence of insertion from the obstacle lane into the vehicle's driving lane can be used as a perspective for calculating detection reliability. In addition, when there is an insertion from the obstacle lane into the vehicle's driving lane, it can be expected that the traffic flow in the vehicle's driving lane will also slow down. Therefore, when traveling on the adjacent lane of the obstacle lane, when a decrease in the driving speed of the vehicle is observed in front of the obstacle registration point, it can also be determined that the surrounding vehicles have taken avoidance actions.

[0246] The obstacle location report may include the detection reliability calculated by the obstacle presence determination unit F51. The map server 2 may also perform statistical processing on the detection reliability included in the reports from multiple vehicles to determine whether there is an obstacle. The detection reliability may also be evaluated using the passenger's line of sight information detected by DSM, etc. For example, when passing on the side of an obstacle, the detection reliability may be set higher if the line of sight of the driver's seat passenger is in the direction determined to have an obstacle.

[0247] The above-mentioned detection reliability indicates the reliability of reports such as the existence of obstacles. Therefore, the above-mentioned detection reliability can also be called existence report reliability. In addition, the obstacle presence determination unit F51 can also calculate the possibility of the absence of obstacles as non-detection reliability based on a combination of whether it is detected by the front camera 11, whether it is detected by the millimeter-wave radar 12, and the presence or absence of avoidance action. The non-detection reliability is equivalent to the inverse of the above-mentioned detection reliability. The higher the detection reliability, the lower the non-detection reliability can be set. The non-detection reliability indicates the reliability of reports such as the existence of obstacles. Therefore, the above-mentioned non-detection reliability can also be called non-existence report reliability.

[0248] <Calculation of the actual presence accuracy of the map server 2>

[0249] The map server 2 may also be configured to calculate the possibility of the existence of an obstacle as the actual existence accuracy and distribute it. The actual existence accuracy is equivalent to the determination result that an obstacle exists and the reliability of the distributed information. For example, the obstacle information management unit G3 may be configured to calculate the possibility of the existence of an obstacle as the actual existence accuracy and distribute it. Fig.26 As shown, the accuracy calculation unit G33 may be provided for calculating the reliability of the obstacle existence determination result as the actual existence accuracy.

[0250] The accuracy calculation unit G33 calculates the actual presence accuracy based on the behavior data of the plurality of vehicles and the ratio of vehicles that have taken avoidance actions. Fig. 27As shown, the more vehicles that report the existence of an obstacle, the higher the actual existence accuracy is set. In addition to vehicles that have implemented avoidance actions, vehicles that report the existence of an obstacle also include vehicles that are traveling in lanes adjacent to the obstacle lane and have uploaded information on detected obstacles. In addition, the accuracy calculation unit G33 may also set the situation where the existence of an obstacle is confirmed by image analysis of the server processor 21 or visual observation of the operator as 100, and calculate the actual existence accuracy based on the number and type of reports indicating the existence of the obstacle. For example, the more the number of vehicles that have implemented avoidance actions or the number of vehicles that have detected obstacles through surrounding monitoring sensors, the higher the actual existence accuracy is set.

[0251] The accuracy calculation unit G33 may also perform calculations based on the difference between the number of reports of obstacle presence and the number of reports of obstacle absence. For example, the number of reports of obstacle presence and the number of reports of obstacle absence may be the same as the actual existence accuracy of 50%. The accuracy calculation unit G33 may also perform statistical processing on the detection reliability contained in the reports from multiple vehicles to calculate the actual existence accuracy. The accuracy calculation unit G33 may also calculate the actual existence accuracy periodically.

[0252] The distribution processing unit G4 may also distribute the obstacle notification data packet including the above-mentioned actual existence accuracy. The distribution processing unit G4 may also distribute the obstacle notification data packet including the updated actual existence accuracy to the vehicle to which the obstacle notification data packet of the location has been distributed, when the actual existence accuracy of the obstacle at a certain location has changed. The distribution processing unit G4 may also distribute the obstacle notification data packet periodically together with the information including the probability of the existence of the obstacle. For example, the obstacle notification data packet indicating the actual existence accuracy in three stages such as "still exists", "high probability of still existing", and "high probability of disappearance" may be distributed at a constant interval.

[0253] However, the value obtained by subtracting the actual existence accuracy from 100% is equivalent to the disappearance accuracy indicating the probability of the obstacle disappearing. The distribution processing unit G4 may also send a disappearance notification data packet including the disappearance accuracy of the obstacle. In addition, it may be configured so that a person (e.g., an operator) or a vehicle that has removed the obstacle can send a report to the map server 2 to the effect that the obstacle has been removed. The map server 2 may also immediately distribute a disappearance notification data packet with a higher disappearance accuracy set when receiving a report to the effect that the obstacle has been removed from an operator or the like.

[0254] <About the distribution method of obstacle information>

[0255] Preferably, the obstacle notification data packet includes the position, type, and size of the obstacle. In addition, the obstacle position information may include not only the position coordinates but also the lateral position of the end of the obstacle as the detailed position in the lane. The obstacle notification data packet may also include width information of the area in the obstacle lane where the vehicle can travel, excluding the part blocked by the obstacle.

[0256] According to the lateral position information of the obstacle end and the drivable width of the obstacle lane included in the obstacle notification data packet, the vehicle receiving the obstacle notification data packet can determine whether a lane change is required or whether the obstacle can be avoided by adjusting the lateral position. In addition, when driving across the lane dividing line, the protrusion amount to the adjacent lane can also be calculated. If the protrusion amount of the adjacent lane used to avoid the obstacle can be calculated, the protrusion amount of the vehicle can be notified to the vehicle driving in the adjacent lane by using inter-vehicle communication, and the driving position can be coordinated with the surrounding vehicles.

[0257] In addition, the obstacle notification data packet may also include information on the time when an obstacle is determined to have occurred, and the latest (i.e., final) time when the obstacle is determined to still exist. By including these determination times, the vehicle as the recipient of the information can estimate the reliability of the received information. For example, the shorter the elapsed time from the final determination time, the higher the reliability. The obstacle notification data packet may also include information such as the number of vehicles that have confirmed the existence of the obstacle. The more vehicles that have confirmed the existence of the obstacle, the higher the reliability of the obstacle information can be estimated. In addition, depending on the reliability of the obstacle information, the control method in the vehicle may be changed, such as whether it is used for vehicle control or limited to notification to passengers.

[0258] The obstacle notification data packet may also include the color and characteristics of the obstacle. In addition, it may also include an image of the obstacle photographed by a certain vehicle. According to this configuration, the vehicle-mounted system 1 or the passenger who is scheduled to pass through the obstacle registration location can easily establish a correspondence between the obstacle notified from the map server 2 and the obstacle in the real world. In addition, as a result, the accuracy of determining whether the obstacle notified from the map server 2 still exists or disappears is improved.

[0259] The distribution processing unit G4 may also set and distribute a lane change recommendation POI (Point of Interest) at a location on the obstacle lane that is a predetermined distance before the obstacle registration location. The lane change recommendation POI refers to a location where a lane change is recommended. By setting and distributing the lane change recommendation POI in the map server 2 in this manner, the process of calculating the lane change point can be omitted on the vehicle side, and the processing load of the processing unit 51 and the driving assistance ECU 60 can be reduced. In the configuration of suggesting a lane change to the user, the lane change recommendation POI can also be used to determine the timing of displaying the obstacle notification image.

[0260] The obstacle notification data packet may include information indicating whether the risk at the location remains, such as whether the obstacle has disappeared or moved. The above-mentioned actual existence accuracy may also be used to indicate whether the risk remains at the location. The obstacle disappearance data packet is also similar to the obstacle notification data packet, and preferably includes the characteristics of the obstacle, the time when the disappearance determination is made, etc.

[0261] In addition, the distribution processing unit G4 may be configured to distribute obstacle notification packets only to vehicles executing specified applications such as autonomous driving applications. In addition to autonomous driving applications, specified applications such as ACC (Adaptive Cruise Control), LTC (Lane Trace Control), and navigation applications may also be used. In addition, in the configuration for pull-type distribution of obstacle information, the map cooperation device 50 may be configured to request obstacle information from the map server 2 under the condition that a specific application is executed. According to the above configuration, excessive information distribution can be suppressed and the control stability of the driving assistance ECU 60 can be improved. In addition, the distribution processing unit G4 may be configured to push-distribute obstacle notification packets only to vehicles that are set to automatically receive obstacle information based on user settings. According to such a configuration, the concern that the map server 2 and the map cooperation device 50 conduct wireless communication in violation of the user's intention can be reduced.

[0262] Furthermore, the distribution processing unit G4 may also distribute obstacle information in units of grids / map tiles. For example, obstacle information on a map tile may also be distributed to vehicles existing in the map tile or vehicles requesting a map of the map tile. This configuration is equivalent to a configuration for distributing obstacle notification data packets in units of map tiles in one viewpoint. According to such a configuration, the selection of distribution objects can be simplified, and information on multiple obstacle registration locations can be distributed at the same time. As a result, the processing load of the map server 2 can be reduced. In addition, how the received obstacle information is used depends on which application is started in the vehicle-mounted system 1. According to the above configuration, the diversity and flexibility of the use of obstacle information in the vehicle-mounted system 1 can be improved.

[0263] <About the Upload Processing of the Map Cooperation Device 50>

[0264] The map cooperation device 50 may also be configured to send an obstacle location report only when the content registered in the map as obstacle information is different from the content observed by the vehicle. In other words, it may also be configured not to send an obstacle location report when the content of the map is consistent with the actual situation. For example, an obstacle is observed at a location where the existence of an obstacle is not registered on the map, or an obstacle location report is sent when there is no obstacle at a location where an obstacle is registered on the map. According to the above-mentioned configuration, the communication volume can be suppressed. In addition, the server processor 21 may not perform determination processing related to the presence or absence of obstacles on the part of the real world that is consistent with the map registration content. In other words, the processing load of the server processor 21 can also be reduced.

[0265] In addition, although the above discloses a configuration in which the map cooperation device 50 spontaneously uploads vehicle behavior data to the map server 2 when passing near an obstacle, the configuration of the map cooperation device 50 is not limited to this. As another method, it is also possible to consider a configuration in which the map cooperation device 50 uploads vehicle behavior data to the map server 2 only when a specified action such as lane change or sudden deceleration is performed. In a configuration in which vehicle behavior data is uploaded only when each vehicle performs a specific action, there is a concern that it is not easy for the map server 2 to collect information for determining whether an obstacle has disappeared. This is because if the obstacle disappears, the vehicle does not perform any special action.

[0266] Based on the above concerns, the server processor 21 may also send a control signal, i.e., an upload instruction signal, to a vehicle that is passing / scheduled to pass through the obstacle registration location to instruct it to upload the obstacle location report. In other words, the map cooperation device 50 may also be configured to decide whether to upload the obstacle location report based on the instruction from the map server 2. According to such a configuration, the upload status of the obstacle location report of each vehicle can be controlled through the judgment of the map server 2, and unnecessary communication can be suppressed. For example, in the case where it is possible to fully collect information related to the appearance and disappearance of obstacles, it is also possible to adopt a response such as suppressing the upload from the vehicle.

[0267] In addition, the server processor 21 can also set the location where the vehicle behavior indicating the existence of an obstacle is observed based on the vehicle status report as the verification location, and send an upload instruction signal to the vehicle scheduled to pass the verification location. The location where the vehicle behavior indicating the existence of an obstacle is observed is, for example, a location where two or three vehicles have changed lanes in succession. According to this configuration, information on locations suspected of having obstacles can be collected centrally and quickly, and the existence status of obstacles can be detected in real time.

[0268] In addition, it can also be configured to be able to set whether to upload the obstacle location report on the vehicle side. For example, it can also be configured so that the user can set whether to upload the obstacle location report via an input device. And it can also be configured so that the user can also set and change the information items uploaded as the obstacle location report. According to such a configuration, it is possible to reduce the user's concerns about accidentally uploading the vehicle behavior data to the map server 2 or the increase in communication volume. In addition, from the perspective of privacy protection, it can also be configured to use a specified encryption code to rewrite the sending source information into a number different from the actual vehicle ID and upload it to the map server 2.

[0269] In addition, the obstacle information distribution system 100 may also be configured to reward users who actively upload information related to obstacles. By providing rewards for sending obstacle location reports, it is easy to collect information related to obstacles, and the effectiveness of the obstacle information distribution system 100 can be improved. As rewards, it can be a reduction in taxes related to automobiles, a reduction in map service usage fees, or the granting of points that can be used for the purchase of goods or the use of services. Points that can be used for the purchase of specified goods or the use of services also include the concept of electronic currency.

[0270] <Application example to autonomous driving>

[0271] The obstacle information generated by the map server 2 can also be used, for example, to determine whether the automatic driving can be performed. As a road condition for automatic driving, there can also be a configuration in which the number of specified lanes is greater than a specified amount n. The specified amount n is an integer greater than "2", for example, "2", "3", "4", etc. In such a configuration, an interval in which the number of effective lanes is less than n due to road obstacles such as fallen objects, construction sections, and parked vehicles on the road can become an interval in which automatic driving is not possible. The number of effective lanes refers to the number of lanes on which the vehicle can actually travel. For example, if one lane of a two-lane road on one side is blocked by an obstacle on the road, the number of effective lanes on the road is "1".

[0272] It may also be configured so that, for example, the vehicle-mounted device such as the driving assistance ECU 60 or the automatic driving ECU determines whether or not it corresponds to a non-automatic driving section. In addition, the map server 2 may set a non-automatic driving section based on the obstacle information and distribute the non-automatic driving section. For example, in the map server 2, a section where the number of effective lanes is insufficient due to obstacles on the road is set as a non-automatic driving section and distributed, and when it is confirmed that the obstacles on the road have disappeared, the non-automatic driving setting is released and distributed. In addition, the server that distributes the setting of the non-automatic driving section may also be used as the automatic driving management server 7, such as Fig.28 As shown, it is set up independently from the map server 2. The automatic driving management server is equivalent to a server that manages the sections where automatic driving is possible or not. As mentioned above, the obstacle information can be used to determine whether the operational design domain (ODD) set according to the vehicle is satisfied. Fig.28 As shown, a system that distributes information related to whether autonomous driving is possible to vehicles based on road obstacle information is called a non-autonomous driving section distribution system.

[0273] <Postscript (1)>

[0274] The control unit and method described in the present disclosure can also be implemented by a dedicated computer that is composed of a processor programmed to execute one or more functions embodied by a computer program. In addition, the device and method described in the present disclosure can also be implemented by a dedicated hardware logic circuit. In addition, the device and method described in the present disclosure can also be implemented by one or more dedicated computers composed of a combination of a processor that executes a computer program and one or more hardware logic circuits. In addition, the computer program can also be stored in a non-migrating tangible recording medium that can be read by a computer as an instruction executed by a computer. For example, the units and / or functions provided by the map cooperation device 50 and the map server 2 can be provided only by software, only by hardware, or by a combination of them through software recorded in a physical memory device and a computer that executes the software. It is also possible to implement part or all of the functions possessed by the map cooperation device 50 and the map server 2 as hardware. The way to implement a certain function as hardware includes the way of implementing it using one or more ICs, etc. For example, the server processor 21 can also be implemented using an MPU or GPU instead of a CPU. In addition, the server processor 21 can also be implemented by combining multiple types of computing processing devices such as CPU, MPU, GPU, etc. Furthermore, the ECU may be implemented using an FPGA (field-programmable gate array) or an ASIC (application specific integrated circuit). The same applies to the processing unit 51. Various programs can be stored in a non-transitory tangible storage medium. As a storage medium for the program, a variety of storage media such as HDD (Hard-disk Drive), SSD (Solid State Drive), EPROM (Erasable Programmable ROM), flash memory, USB memory, SD (Secure Digital) memory card, etc. can be used.

[0275] <Postscript (2)>

[0276] The present disclosure also includes the following configurations.

[0277] The map server is configured such that at least one of the appearance determination unit and the disappearance determination unit determines whether an obstacle exists using camera images captured by a vehicle in addition to vehicle behavior data of a plurality of vehicles.

[0278] The map server is configured to change a combination of information types used to determine the presence of an obstacle when determining the presence of the obstacle and when determining the disappearance of the obstacle.

[0279] The map server is configured to use the analysis result of the image captured by the vehicle-mounted camera in conjunction with the appearance determination, while not using the analysis result of the image captured by the vehicle-mounted camera in the disappearance determination.

[0280] The map server is configured to change the weight of each type of information used to determine the presence of an obstacle when determining the presence of the obstacle and when determining the disappearance of the obstacle.

[0281] The map server is configured to use the analysis result of an image captured by the vehicle-mounted camera as a material for determining whether an obstacle exists, and to reduce the weight of the image analysis result in disappearance determination compared to appearance determination.

[0282] The map server is configured to determine the presence or absence of an obstacle by comparing the traffic volume of each lane.

[0283] The map server is configured to adopt a lane change performed after deceleration as an avoidance action. In addition, according to this configuration, lane changes for overtaking can be eliminated.

[0284] Even if an obstacle is detected by the camera, the obstacle presence determination device or the map server does not determine that an obstacle exists if no three-dimensional object is detected by the distance measuring sensor.

[0285] The map server does not distribute information about the obstacle to a vehicle traveling / scheduled to travel in a lane that is not adjacent to the lane where the obstacle exists, that is, in a lane that is away by one or more lanes.

[0286] The map cooperation device as a vehicle device is configured to upload an obstacle point report including vehicle behavior to the map server based on an instruction from the map server or voluntarily when the vehicle is traveling within a certain range from an obstacle registration point notified from the map server.

[0287] The map cooperation device as a vehicle device transmits an obstacle point report indicating that there is no obstacle when a point where an obstacle exists is notified by a map server and the notified obstacle is not detected based on an input signal from a surrounding monitoring sensor.

[0288] The map cooperation device outputs the obstacle information obtained from the map server to the navigation device or the automatic driving device.

[0289] Displaying an obstacle notification image generated based on obstacle information acquired from a map server on a display.

[0290] When the vehicle is traveling / scheduled to travel in a lane that is not adjacent to the lane where the obstacle is located, that is, in a lane that is more than one lane away, the HMI system does not notify the passengers of the information about the obstacle.

[0291] The driving assistance device is configured to switch, based on the accuracy of the actual existence of an obstacle notified from the map server, whether to execute vehicle control based on the information or to simply provide information.

Claims

1. An obstacle information management device, wherein: have: A vehicle behavior acquisition unit that acquires vehicle behavior data representing the behavior of a plurality of vehicles in correspondence with the position information; an appearance determination unit that determines a location where an obstacle appears based on the vehicle behavior data acquired by the vehicle behavior acquisition unit; as well as a disappearance determination unit that determines whether the obstacle remains or disappears at the location where the obstacle is determined to exist by the appearance determination unit, that is, the obstacle registration location, based on the vehicle behavior data acquired by the vehicle behavior acquisition unit; A distribution processing unit distributes the information of the obstacle to the vehicle; as well as an accuracy calculation unit for calculating an actual existence accuracy indicating a probability that the obstacle exists at the obstacle registration point; The distribution processing unit distributes an obstacle notification packet to the vehicle scheduled to pass through the obstacle registration point, wherein the obstacle notification packet is a communication packet indicating information on the obstacle including the actual existence accuracy calculated by the accuracy calculation unit.

2. The obstacle information management device according to claim 1, wherein: The disappearance determination unit and the appearance determination unit are configured to determine whether the vehicle is performing a predetermined avoidance action based on the vehicle behavior data. The presence determination unit determines that the obstacle exists at a location as a processing target based on the fact that at least one of the vehicles is performing the avoidance action near the location, The disappearance determination unit detects the disappearance of the obstacle based on the fact that the vehicle passing the vicinity of the obstacle registration point does not perform the avoidance action.

3. The obstacle information management device according to claim 2, wherein: The appearance determination unit and the disappearance determination unit have different conditions for determining that the obstacle exists.

4. The obstacle information management device according to claim 2 or 3, wherein: The obstacle information management device is configured to acquire, from the vehicle, an image of a location where the obstacle exists. The disappearance determination unit determines that the obstacle has disappeared based on a situation in which the number or ratio of the vehicles that do not perform the avoidance action at the obstacle registration point exceeds a threshold value, The disappearance determination unit changes a threshold value for determining that the obstacle has disappeared based on whether the obstacle is reflected in the image.

5. The obstacle information management device according to any one of claims 1 to 3, wherein: The vehicle behavior acquisition unit is configured to acquire, as the vehicle behavior data, behavior data of surrounding vehicles of the transmission source in addition to the behavior data of the vehicle as the transmission source. The disappearance determination unit determines the disappearance of the obstacle based on the behavior of the vehicle as the transmission source and the behavior of surrounding vehicles of the vehicle as the transmission source.

6. The obstacle information management device according to any one of claims 1 to 3, wherein: The obstacle information management device is configured to instruct the vehicle scheduled to pass through the obstacle registration point to transmit a predetermined type of information including vehicle behavior data for determining a presence status of the obstacle.

7. The obstacle information management device according to any one of claims 1 to 3, wherein: The distribution processing unit is configured to distribute a disappearance notification packet to the vehicle to which the obstacle notification packet regarding the disappeared obstacle has been distributed when the disappearance determination unit determines that the obstacle has disappeared, wherein the disappearance notification packet is a communication packet indicating that the obstacle has disappeared.

8. The obstacle information management device according to claim 7, wherein: The obstacle notification data packet includes information indicating the location of the obstacle and information indicating the characteristics of the obstacle. The information indicating the characteristics of the obstacle includes at least one of the type, size, and color tone of the obstacle.

9. The obstacle information management device according to claim 7, wherein: The obstacle notification data packet includes, as information indicating the position of the obstacle, at least one of a number of a lane where the obstacle exists and a lateral position of an end of the obstacle on the lane.

10. The obstacle information management device according to any one of claims 1 to 3, wherein: The accuracy calculation unit is configured to periodically calculate the actual existence accuracy for each of the obstacle registration points. The distribution processing unit is configured to redistribute the obstacle notification data packet based on a change in the actual existence accuracy calculated by the accuracy calculation unit over time.

11. The obstacle information management device according to any one of claims 1 to 3, wherein: The obstacle information management device is configured as follows: The occurrence determination unit makes a determination based on the information acquired within a specified first time, and The disappearance determination unit makes a determination based on the information acquired within a predetermined second time period. The first time and the second time are both set to a length of less than 90 minutes.

12. The obstacle information management device according to any one of claims 1 to 3, wherein: The disappearance determination unit determines whether the obstacle has disappeared by adding a weight corresponding to a newness or oldness of the vehicle behavior data to the vehicle behavior data received from each of the plurality of vehicles.

13. The obstacle information management device according to any one of claims 1 to 3, wherein: The above-mentioned appearance determination unit is configured as follows: acquiring, from the vehicle via wireless communication, an image outside the vehicle captured within a predetermined time period from the moment when the vehicle performs a prescribed avoidance action for avoiding the obstacle on the road, and The presence or absence of the obstacle and the type of the obstacle are determined by analyzing a portion of the vehicle exterior image that is determined according to an avoidance direction, wherein the avoidance direction is a direction that the vehicle avoids as the avoidance action.

14. An obstacle information management method is a method for managing the location information of obstacles existing on the road, which is executed by using at least one processor, wherein: have: A vehicle behavior acquisition step of acquiring vehicle behavior data representing the behavior of a plurality of vehicles at various locations in a corresponding relationship; an occurrence determination step of determining a location where the obstacle appears based on the vehicle behavior data acquired in the vehicle behavior acquisition step; a disappearance determination step of determining whether the obstacle remains or disappears at the location where the obstacle is determined to exist in the appearance determination step, i.e., the obstacle registration location, based on the vehicle behavior data acquired in the vehicle behavior acquisition step; a step of calculating actual existence accuracy indicating the degree of possibility that the obstacle exists at the obstacle registration point; as well as The step of distributing an obstacle notification data packet to the vehicle scheduled to pass through the obstacle registration point, wherein the obstacle notification data packet is a communication data packet indicating information about the obstacle including the actual existence accuracy.

15. A vehicle device for transmitting information about a location where an obstacle exists on a road to a predetermined server, wherein: have: an obstacle location information acquisition unit that acquires information on an obstacle registration location determined to be where the obstacle exists by communicating with the server; a vehicle behavior detection unit that detects the behavior of at least one of the own vehicle and the other vehicle based on at least any one of an input signal from a vehicle state sensor that detects a physical state quantity indicating the behavior of the own vehicle, an input signal from a surrounding monitoring sensor, and received data based on inter-vehicle communication; and a report processing unit, which transmits vehicle behavior data indicating the behavior of at least one of the own vehicle and other vehicles to the server, When the vehicle is traveling within a predetermined distance from the obstacle registration point, the notification processing unit automatically transmits the vehicle behavior data even if the vehicle does not take evasive action. On the other hand, when driving in an area separated from the obstacle registration location by a specified distance or more, the report processing unit sends the vehicle behavior data based on the condition that the own vehicle has taken avoidance action, other vehicles have taken avoidance action, or the obstacle has been detected by the surrounding monitoring sensor.

16. The vehicle device according to claim 15, wherein: The vehicle device receives input of output signals of the vehicle-mounted camera and the distance measuring sensor as the surrounding monitoring sensor. The vehicle behavior data is a data set representing the behavior of the vehicle or other vehicles at multiple times. The notification processing unit is configured to change a sampling interval of the information indicating the behavior included in the vehicle behavior data, according to a type of a road on which the host vehicle is traveling.

17. The vehicle device according to claim 15 or 16, wherein: have: an avoidance action determination unit that determines whether the host vehicle has performed a predetermined avoidance action for avoiding an obstacle on the road based on an input signal from a vehicle state sensor that detects a physical state quantity indicating the behavior of the host vehicle; as well as The external information acquisition unit acquires image data captured by a vehicle-mounted camera that captures images outside the vehicle. The avoidance action determination unit determines an avoidance direction, the avoidance direction being a direction that the vehicle avoids as the avoidance action. The report processing unit is configured to establish a correspondence between the image data of the object registered as the obstacle in the part determined according to the avoidance direction, and the position information of the image data among the plurality of image data acquired within a specified time from the performance of the avoidance action, and send them to the server based on the situation where the avoidance action determination unit determines that the avoidance action has been performed.

18. The vehicle device according to claim 15 or 16, wherein: have: an avoidance action determination unit that determines whether the host vehicle has performed a predetermined avoidance action for avoiding an obstacle on the road based on an input signal from a vehicle state sensor that detects a physical state quantity indicating the behavior of the host vehicle; as well as The external information acquisition unit acquires image data captured by a vehicle-mounted camera that captures images outside the vehicle. The avoidance action determination unit determines an avoidance direction, the avoidance direction being a direction that the vehicle avoids as the avoidance action. The report processing unit is configured to implement: By analyzing the image data acquired within a predetermined time from the time when the avoidance action is performed, an object existing in front of the vehicle when the avoidance action is performed is extracted as an avoidance object candidate; When at least one of the avoidable object candidates is obtained, determining the object to be avoided by the vehicle, that is, the avoidable object, from the at least one avoidable object candidate based on the avoidance direction; as well as The image data on which the avoidance object is reflected is sent to the server in correspondence with the position information.

19. The vehicle device according to claim 15 or 16, wherein: have: an avoidance action determination unit that determines whether the host vehicle has performed a predetermined avoidance action for avoiding an obstacle on the road based on an input signal from a vehicle state sensor that detects a physical state quantity indicating the behavior of the host vehicle; as well as The external information acquisition unit acquires image data captured by a vehicle-mounted camera that captures images outside the vehicle. The avoidance action determination unit determines an avoidance direction, the avoidance direction being a direction that the vehicle avoids as the avoidance action. The report processing unit is configured to send a partial image obtained by cutting out a portion determined by the avoidance direction or a portion reflecting the object registered as the obstacle from the image data acquired within a specified time from the avoidance action to the server as a report image.

Images