Vehicle control device and vehicle control system

Abstract

This vehicle control device comprises: an internal road surface state acquisition unit that acquires information on a road surface state observed by a sensor mounted on a vehicle; an external road surface state acquisition unit that acquires information on a road surface state transmitted from the outside; and an external communication unit that transmits information on the road surface state acquired by the internal road surface state acquisition unit to the outside. The external road surface state acquisition unit acquires information on the road surface state observed before the acquisition time of the information on the road surface state by the internal road surface state acquisition unit, and the external communication unit transmits information on the road surface state acquired by the internal road surface state acquisition unit to the external device regardless of whether or not the travel plan unit creates a travel plan.

Description

Vehicle control device and vehicle control system

[0001] The present invention relates to a vehicle control device, and more particularly to a technology for linking external sensing results with devices external to the vehicle, and a vehicle control technology using the linked information.

[0002] In recent years, development has been underway for vehicle driving assistance technologies and autonomous driving technologies. These technologies control the vehicle based on external information acquired via communications and road surface information obtained through sensing, and assist the driver in driving operations by providing steering control assistance and acceleration / deceleration control assistance.

[0003] Background art in this field includes Patent Document 1 (JP 2021-190038 A) and Patent Document 2 (JP 2022-139515 A). Patent Document 1 describes a vehicle control device in which, when a first vehicle included in a plurality of vehicles slips, a slip road surface avoidance plan creation unit transmits information identifying the slip occurrence road surface from the first vehicle to other vehicles in the vicinity via vehicle-to-vehicle communication, and in the other vehicles, the slip road surface avoidance plan creation unit creates a slip road surface avoidance plan for the host vehicle based on the information identifying the slip occurrence road surface received from the first vehicle, and a control unit controls the traveling of the host vehicle based on the slip road surface avoidance plan for the host vehicle.

[0004] Patent Document 2 describes a vehicle control system that includes a water splash threshold map that sets a water splash threshold to reduce water splashing that occurs when a vehicle passes through a puddle based on surrounding environment information, puddle information, road map information, and vehicle information, and that sets a puddle-avoiding driving route based on information acquired by an external information receiving device.If a puddle-avoiding driving route cannot be set, a puddle-passing driving route is set.If a puddle is recognized while driving on the puddle-passing driving route, a water splash threshold is set according to the presence and positional relationship of pedestrians and the like around the vehicle, and a new driving route is set that includes driving control involving at least one of deceleration and steering control according to the set water splash threshold.

[0005] JP 2021-190038 A JP 2022-139515 A

[0006] In order to improve the accuracy of avoiding slippery roads and puddles, suppressing slippage, and reducing water splashing, it is necessary to accurately detect the location and type of road surface in question and control driving according to the situation. Furthermore, to prevent passengers from experiencing a worsening ride and reduce driver stress during driving assistance, it is necessary to identify road surface obstacles at an early stage and to moderate changes in vehicle behavior. In other words, it is necessary to accurately grasp road surface conditions, including not only their location but also their type, at an early stage and control driving accordingly. However, it is difficult to accurately recognize distant road surface conditions using external sensors, making it difficult to appropriately control driving with ample time to respond to the type of situation.

[0007] A representative example of the invention disclosed in this application is as follows. That is, the vehicle control device includes a calculation unit that executes calculation processing and a memory unit accessible by the calculation unit, wherein the calculation unit includes an internal road surface condition acquisition unit that acquires information on road surface conditions on a driving path observed by a sensor mounted on the vehicle, an external road surface condition acquisition unit that acquires information on road surface conditions on the driving path transmitted from an external device, a driving planning unit that, in accordance with the information on the road surface conditions, creates a driving plan that reduces at least one of the impact of the road surface condition on surrounding traffic participants and unstable behavior of the vehicle when it is likely that the vehicle will pass through a location with the road surface condition, and an external communication unit that the calculation unit transmits the information on the road surface condition acquired by the internal road surface condition acquisition unit to the external device, wherein the external road surface condition acquisition unit acquires information on road surface conditions observed before the time when the information on the road surface condition was acquired by the internal road surface condition acquisition unit, and regardless of whether the driving planning unit creates a driving plan, the external communication unit transmits the information on the road surface condition acquired by the internal road surface condition acquisition unit to the external device.

[0008] According to one aspect of the present invention, it is possible to create a driving plan appropriate for passing through or avoiding a location with a road surface condition. Problems, configurations, and effects other than those described above will become apparent from the following description of the preferred embodiment of the present invention.

[0009] 1 is a diagram illustrating the operation of a system according to an embodiment of the present invention; FIG. 2 is a diagram illustrating an example of an increase in the recognition reliability of a road surface condition according to an embodiment of the present invention; FIG. 3 is a diagram illustrating updating of road surface condition information according to an embodiment of the present invention; FIG. 4 is a diagram illustrating updating of road surface condition information according to an embodiment of the present invention; FIG. 5 is a block diagram illustrating an example of a configuration of an electronic control unit according to an embodiment of the present invention; FIG. 6 is a flowchart of road surface condition response processing executed by an electronic control unit according to an embodiment of the present invention; FIG. 7 is a diagram illustrating an example of transmitting and receiving road surface condition information via an information sharing center according to an embodiment of the present invention; FIG. 8 is a diagram illustrating an example of transmitting and receiving road surface condition information via an information sharing center according to an embodiment of the present invention; FIG. 9 is a diagram illustrating an example of a data format of all information transmitted from a vehicle according to an embodiment of the present invention; FIG. 10 is a diagram illustrating an example of the content of common information according to an embodiment of the present invention; FIG. 11 is a diagram illustrating an example of road surface condition information according to an embodiment of the present invention; FIG. 12 is a diagram illustrating an example of road surface condition location E and example of position and shape information of road surface condition additional information according to an embodiment of the present invention; FIG. 13 is a diagram illustrating an example of a road surface condition where a pedestrian is present next to a small puddle A according to an embodiment of the present invention; FIG. 14 is a diagram illustrating an example of a road surface condition where a pedestrian is present next to a large puddle B according to an embodiment of the present invention; FIG. 15 is a diagram illustrating an example of a road surface condition where a small frozen spot A exists on a curved road according to an embodiment of the present invention; FIG. 16 is a diagram illustrating an example of a road surface condition where a large frozen spot B exists on a curved road according to an embodiment of the present invention.

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] In an embodiment of the present invention, regardless of the impact of sensing road surface conditions on the vehicle's driving plan, the type and location of the road surface conditions are sensed when approaching a location with the road surface conditions and, if necessary, even after passing through the location, thereby increasing the reliability of the sensing of road surface conditions. The sensing results are shared with other vehicles via a server or directly via communication.

[0012] During autonomous driving, the vehicle uses road surface condition information, which indicates the type and location of road surface conditions shared with other vehicles, to recognize road surface conditions outside the range detected by the vehicle's external sensing function, and changes the driving plan to avoid or mitigate road surface conditions that may affect driving. During driving assistance, the vehicle assists with steering and deceleration operations to avoid or mitigate road surface conditions.

[0013] If the vehicle's external sensing function determines that there is an error in the road surface condition information shared with other vehicles, the driving plan will be gradually changed to one based on correct information, as long as safety is ensured.

[0014] First, the operation of the system according to the embodiment of the present invention will be described with reference to Fig. 1. In the system shown in Fig. 1, second vehicle 200 transmits information about road surface condition location A510 that it has recognized to the outside, and first vehicle 100 receives the transmitted information and modifies its driving plan to follow route A410 to avoid road surface condition location A510, and then drives the vehicle.

[0015] In this specification, the term "road surface condition" refers to a partial condition of the road surface that may affect driving, and may also be referred to as a "road obstacle." The term "road surface condition" also includes, for example, unevenness on the road surface and small fallen objects, including those that are difficult to detect from a distance using external sensors. For convenience of explanation, the vehicle that transmits the road surface condition to the outside will be referred to as the second vehicle 200, and the vehicle that receives and uses the transmitted information will be referred to as the first vehicle 100. However, the first vehicle 100 also has the functions of the second vehicle 200, and the second vehicle 200 also has the functions of the first vehicle 100.

[0016] The first vehicle 100 has a camera AA110 as an external sensor that mainly monitors the front, a camera AB120 as an external sensor that mainly monitors the rear, a camera AC130 as an external sensor that mainly monitors the left side, and a camera AD140 as an external sensor that mainly monitors the right side. Area AA115 indicates an area where the road surface condition can be recognized from the image captured by camera AA110, and area AC135 indicates an area where the road surface condition can be recognized from the image captured by camera AC130.

[0017] The second vehicle 200 has a camera BA210 as an external sensor that mainly monitors the front, a camera BB220 as an external sensor that mainly monitors the rear, a camera BC230 as an external sensor that mainly monitors the left side, and a camera BD240 as an external sensor that mainly monitors the right side. Area BC235 indicates an area in which road surface conditions can be recognized from the image captured by camera BC230.

[0018] Area AA115, area AC135, and area BC135 merely indicate areas in which road surface conditions can be recognized from the images captured by each camera, and areas in which traffic participants such as vehicles and pedestrians can be recognized may differ from areas in which road surface conditions can be recognized. Normally, traffic participants such as vehicles and pedestrians that are at a sufficient height above the road surface can be recognized farther away than the road surface conditions, due in part to the influence of the orientation of the surface of the recognition target captured by an external sensor such as a camera.

[0019] In the situation shown in FIG. 1 , the external sensors of the first vehicle 100 cannot recognize the road surface condition location A510, and the road surface condition location A510 is not included in the area in which the first vehicle 100 can recognize road surface conditions. Therefore, without information provided from the outside, the first vehicle 100 cannot modify its driving plan to avoid the road surface condition location A510 until it approaches the road surface condition location A510 until it is within the area AA115. However, in the situation shown in FIG. 1 , the second vehicle 200 recognizes the road surface condition location A510 and transmits information about the existence, type, and location of the road surface condition location A510. The first vehicle 100 receives the information transmitted by the second vehicle 200, and based on the received information, modifies the route plan to avoid the road surface condition location A510 when the first vehicle 100 is sufficiently far from the road surface condition location A510. The first vehicle 100 then revises the driving plan well in advance so that it can travel according to the modified route plan. The first vehicle 100 controls the vehicle according to the revised driving plan and travels along the driving route EA410.

[0020] In this way, the driving plan can be changed at an early stage, thereby reducing at least one of the impact on surrounding traffic participants (e.g., the occurrence of damage from water splashes or mud splashes) and unstable vehicle behavior. In other words, by changing the driving plan at an early stage, the steering angle of the vehicle to avoid road surface condition areas can be reduced, and the required deceleration and deceleration rate can be reduced, thereby suppressing a decrease in occupant comfort due to avoidance of road surface condition areas. Furthermore, when intervening in the occupant's driving operation during driving assistance, the amount of feedback such as steering associated with the vehicle's evasive action and the difference between the occupant's driving operation and the actual vehicle movement can be reduced, thereby reducing the stress felt by the occupant.

[0021] When the second vehicle 200 transmits the information on the road surface condition location A510 to the outside, the second vehicle 200 continues the recognition process using the external sensors attached to the second vehicle 200 and transmits the results of the recognition process to the outside as appropriate, regardless of whether the information is reflected in the driving plan of the second vehicle 200. In other words, the second vehicle 200 transmits to the outside updated information on the road surface condition location A510 using information acquired while passing over or near the road surface condition location A510 and information acquired after passing over the road surface condition location A510.

[0022] Information on road surface condition location A510 is transmitted to the outside when there is a change in the recognition result, such as type, position, or shape information, or when there is a change in the recognition reliability. Even if the recognition reliability for a certain road surface condition location has decreased, if a certain level of recognition reliability or higher is ensured, the recognition result may be updated and the updated recognition result may be transmitted. Newly recognized road surface conditions and new shape information that serves as additional information on the recognized road surface conditions (e.g., information on ruts left after a vehicle has passed when the road surface condition location A510 is snow-covered) may be added as road surface condition information. Since the recognition process for road surface conditions continues during and after the vehicle has passed, if a certain level of recognition reliability or higher can be ensured for the road surface conditions that have changed due to the vehicle's passage, the recognition result may be updated to one that corresponds to the changed conditions, and the updated recognition result may be transmitted.

[0023] In the example shown in FIG. 1 , the second vehicle 200 has passed through road surface condition location A510, and although it is difficult for the second vehicle 200 to take driving measures that take into account the impact of road surface condition location A510, the second vehicle 200 is continuously executing a recognition process to recognize road surface condition location A510. If the recognition accuracy of the type information of road surface condition location A510 improves as a result of passing over road surface condition location A510, the updated recognition result is transmitted to the outside. For example, if the road surface condition location A510 is a puddle, the passage of the second vehicle 200 causes disturbances on the water surface and splashes, and if this situation is photographed and recognized by camera BC230, the recognition accuracy of the type being "puddle" can be improved.

[0024] Another example of increasing the reliability of road surface condition recognition will be described with reference to FIG.

[0025] FIG. 2 shows an example in which the second vehicle 200 is approaching a road surface condition location B520. The second vehicle 200 continuously recognizes the road surface condition location B520 using the camera BC230. At this time, if the road surface condition location B520 is a highly reflective object such as a puddle or a slightly frozen area, the camera BC230 captures an image of the road surface condition location B520 that is affected by light reflected from a direction B610. The direction B610 depends on the angle θ, and the angle θ changes as the second vehicle 200 approaches the road surface condition location B520. In other words, by utilizing this change in the recognition process, the reflectivity of the road surface condition location B520 can be used to improve the accuracy of recognizing the type of the road surface condition location B520, or the difference in reflectivity between the road surface condition location B520 and its surroundings can be used to improve the accuracy of detecting the road surface condition.

[0026] 3A and 3B, the updating of road surface condition information by continuing the recognition process of the road surface condition even after passing through a road surface condition location will be described.

[0027] 3A shows the second vehicle 200 before it passes through the road surface condition location C530, and FIG. 3B shows the second vehicle 200 after it passes through the road surface condition location C530. Dashed line BA216 indicates the horizontal angle of view of camera BA210, and dashed line BB226 indicates the horizontal angle of view of camera BB220.

[0028] As shown in FIG. 3A , before the second vehicle 200 passes the road surface condition location C530, the second vehicle 200 recognizes the road surface condition location C530 using images of the road surface condition location C530 captured by the camera BA210 and the camera BC230. Even after the second vehicle 200 passes the road surface condition location C530, the second vehicle 200 continues to recognize the road surface condition location C530 using images captured by the camera BB220, as shown in FIG. 3B . As a result, the second vehicle 200 can detect ruts C535 that have formed in the road surface condition location C530 as a result of its passage, and can provide information on the road surface condition location C530 with the ruts C535 added to it to an external device of the second vehicle 200. Specific examples of the road surface condition location C530 include accumulated snow and earth and sand.

[0029] Examples of obstacles whose shape changes as a vehicle passes include small objects such as empty cans, plastic bottles, and instant noodle containers. These objects are small and can be difficult to recognize until the object is approached, so recognition processing using images captured by approaching the object, as shown in this embodiment, is effective. Furthermore, empty cans and plastic bottles that are not crushed tend to roll and move easily, so in such a state, only rough information about the obstacle's location can be provided to the outside. However, if it can be detected that the object has been crushed by the passage of a vehicle, the information provided to the outside can be updated, and specific location information can be provided.

[0030] FIG. 4 is a block diagram showing an example of the configuration of an electronic control unit that realizes the functions of the first vehicle 100 and the second vehicle 200.

[0031] The electronic control device of this embodiment has an external sensor input unit 310, an environmental information acquisition unit 315, a map information management unit 325, a self-position estimation unit 320, a map information management unit 325, an object information acquisition unit 330, a road surface condition acquisition unit 335, an external communication unit 340, an external communication antenna 345, an obstacle information management unit 350, a driving planning unit 360, a vehicle behavior integrated control unit 370, a vehicle behavior acquisition unit 375, a driving operation input / output unit 380, a brake control unit 391, a drive output / transmission control unit 392, and a steering control unit 393.

[0032] The external sensor input unit 310 receives external information observed by an external sensor (a device capable of sensing the external environment around the vehicle, such as a camera or Lidar) mounted on the vehicle. The external sensor input unit 310 sends the external information around the vehicle received from the external sensor to the object information acquisition unit 330 and the road surface condition acquisition unit 335. In this embodiment, an example will be described in which a camera is used as the external sensor, but a sensor other than a camera may be used as long as it is an external sensor capable of recognizing road surface conditions, and the external sensor input unit 310 has a function corresponding to the external sensor used.

[0033] The environmental information acquisition unit 315 calculates temperature, humidity, and air pressure using sensor output signals received from environmental sensors such as a temperature sensor, humidity sensor, and air pressure sensor mounted on the vehicle. The environmental information acquisition unit 315 sends the calculated environmental information to the road surface condition acquisition unit 335, the external communication unit 340, and the drive output / transmission control unit 392. Note that it is not necessary to implement all of the temperature sensor, humidity sensor, and air pressure sensor, and some of the environmental sensors (for example, the humidity sensor and the air pressure sensor) may be omitted.

[0034] The object information acquisition unit 330 performs object recognition processing using external information received from the external sensors to recognize nearby traffic participants (various vehicles, pedestrians, animals, etc.) and objects fixed to the ground (buildings, roadside trees, signs, etc.). The object information acquisition unit 330 sends the recognition results to the obstacle information management unit 350 and the self-position estimation unit 320. The obstacle information management unit 350 determines whether the recognition results of traffic participants and objects will obstruct passage, and manages the recognition results of traffic participants and objects that will obstruct passage in an integrated manner. The self-position estimation unit 320 estimates the position and orientation of the vehicle on a map.

[0035] The road surface condition acquisition unit 335 executes a recognition process using external information received from the external sensors to recognize road surface conditions. At this time, the road surface condition recognition process may be executed by also referring to obstacle information (including road surface condition information) acquired by the external communication unit 340 from outside the vehicle as reference information. The obstacle information acquired by the external communication unit 340 from outside the vehicle is managed in an integrated manner by the obstacle information management unit 350. Road surface conditions include flat areas on the road surface, such as puddles and icy roads, depressions in the road surface, such as potholes, and small fallen objects. For example, the road surface condition information received from the outside may be used to narrow down the range in which the recognition process should be executed in detail, and the road surface condition may be detected early and accurately by the external sensors of the vehicle.

[0036] When performing the road surface condition recognition process, the road surface condition acquisition unit 335 may refer to the environmental information received by the environmental information acquisition unit 315. For example, frozen areas recognized when the temperature is high may be treated as false positives and not present.

[0037] The road surface condition acquisition unit 335 may detect the road surface condition and a sunny area in the vicinity where sunlight is shining. The sunny area may be treated as additional information on the nearby road surface condition.

[0038] The road surface condition acquisition unit 335 may execute a recognition process for recognizing ray information using external information acquired by an external sensor. The road surface condition may affect the recognition of lanes. For example, it may be necessary to take into account ruts in the driving path. Therefore, it is preferable for the road surface condition acquisition unit 335 to recognize lanes.

[0039] The road surface condition acquisition unit 335 may acquire information on wheel spin and vehicle slippage from the vehicle behavior integrated control unit 370 and recognize slip risk areas that cannot be detected by external sensors. The vehicle behavior integrated control unit 370 manages vehicle slippage detected from acceleration observed by the inertial sensor. After detecting slippage, the external sensor for rear monitoring may be used to recognize in detail the area after the wheels of the vehicle have passed, thereby improving the detection accuracy of the road surface conditions by the external sensor, and the result may be transmitted to an external device.

[0040] The self-position estimation unit 320 estimates the position and orientation of the vehicle from the position information received from the GNSS receiver, the vehicle behavior information received from the vehicle behavior integrated control unit 370, information on objects fixed to the ground on the map managed by the map information management unit 325, and information obtained by object matching by the object information acquisition unit 330. The information on the position and orientation of the vehicle estimated by the self-position estimation unit 320 is sent to the external communication unit 340, the obstacle information management unit 350, and the driving plan unit 360.

[0041] The external communication unit 340 transmits and receives information to and from external parties such as other vehicles and the information sharing center 700 using the external communication antenna 345. For example, the external communication unit 340 acquires self-position estimation information, which serves as a position reference for road surface conditions, from the self-position estimation unit 320 and transmits it to the outside, and also transmits road surface condition information acquired from the road surface condition acquisition unit 335 to the outside. The external communication unit 340 also transmits environmental information acquired from the environmental information acquisition unit 315 to the outside. In addition, the external communication unit 340 sends road surface condition information received from the outside to the obstacle information management unit 350, which enables the obstacle information management unit 350 to collectively manage obstacle information (including road surface conditions) including information received from the outside. The obstacle information management unit 350 integrates and manages information including obstacle information (including road surface conditions), road surface information, and lane information acquired from the road surface condition acquisition unit 335 and the object information acquisition unit 330, as well as road surface condition information received from the outside via the external communication unit 340. The obstacle information management unit 350 uses the information on the position and orientation of the vehicle from the self-position estimation unit 320 to calculate the relationship between the information received from the outside and the position and orientation of the vehicle.

[0042] The information managed by the obstacle information management unit 350 is sent to the driving plan unit 360 and the vehicle behavior integrated control unit 370 .

[0043] The driving plan unit 360 creates a driving plan so that the vehicle can travel in accordance with the map and route information to the destination provided by the map information management unit 325. The driving plan unit 360 may acquire information on lanes, obstacles, and road surface conditions from the obstacle information management unit 350, and create a detailed driving route plan and driving speed plan for the road.

[0044] The vehicle behavior acquisition unit 375 acquires the outputs of the inertial sensors and wheel speed pulses necessary for detecting the vehicle behavior, and detects changes in the direction of the vehicle and the occurrence of slippage at each wheel. The vehicle behavior acquisition unit 375 sends the detected information to the vehicle behavior integrated control unit 370.

[0045] The driving operation input / output unit 380 receives signals from devices related to the driver's driving operation, such as the operation of the steering, accelerator, brake, selector lever, etc., and outputs feedback instruction signals to devices that generate feedback on the driving operation, such as instructions to generate reaction forces to the steering, brake, etc. The driving operation input / output unit 380 also outputs display requests related to the driving operation to a display panel as necessary.

[0046] The vehicle behavior integrated control unit 370 outputs drive instructions to the brake control unit 391, drive output / transmission control unit 392, and steering control unit 393 so that the vehicle behavior conforms to the signals indicating the driving operations of the occupant obtained from the driving operation input / output unit 380 and the content planned by the driving planner 360. At this time, if passing through an area where road surface conditions change (specific road surface conditions) is predicted based on information obtained from the obstacle information management unit 350, the drive instructions are adjusted so that the vehicle can travel in a manner appropriate for passing through the road surface conditions. Furthermore, if contact with an obstacle that must be avoided is predicted, emergency braking processing is performed. If the occurrence of abnormal vehicle behavior such as slippage is detected from the information obtained from the vehicle behavior acquisition unit 375, the drive torque distribution to each wheel is changed and drive instructions adjusted to restore the vehicle behavior may be output.

[0047] The brake control unit 391 issues instructions to the actuators for controlling the brake devices so as to generate braking forces for each wheel as instructed by the vehicle behavior integrated control unit 370. At this time, in the case of a vehicle with an energy regeneration function such as an electric vehicle or a hybrid vehicle, the brake control unit 391 cooperates with the drive output / transmission control unit 392 to issue instructions to the actuators for controlling the brake devices so as to generate a desired braking force by combining the braking force resulting from energy regeneration with the braking force produced by the actuators for brake control.

[0048] The drive output / transmission control unit 392 issues instructions to the drive system consisting of the vehicle's drive force generation source and drive force transmission mechanism, and controls it so that the drive force of each wheel instructed by the vehicle behavior integrated control unit 370 is obtained.

[0049] The steering control unit 393 controls the actuator of the steering device in accordance with instructions from the vehicle behavior integrated control unit 370.

[0050] The electronic control device of this embodiment is a control device having an arithmetic unit and a storage device. The arithmetic unit is a processor (e.g., a microcomputer) that executes programs stored in the storage device. The arithmetic unit operates as a functional unit that provides various functions by executing a predetermined program. The storage device includes a non-volatile storage area and a volatile storage area. The non-volatile storage area includes a program area that stores programs executed by the arithmetic unit and a data area that stores data used by the arithmetic unit when executing the program. The volatile storage area stores data used by the arithmetic unit when executing the program. In order to improve the execution speed of the arithmetic unit, the program area may be located in the volatile storage area. In this case, the program is transferred from the non-volatile storage area to the volatile area when it is needed for program execution. A program compressed and stored in the non-volatile storage area may be expanded during transfer.

[0051] FIG. 5 is a flowchart of the road surface condition response process executed by the electronic control unit configured as shown in FIG.

[0052] First, the self-position estimation unit 320 acquires the position and orientation of the vehicle (S810). The position and orientation of the vehicle are necessary to convert road surface condition information obtained by communication from outside the vehicle into the coordinate system of the vehicle, and to convert road surface condition information obtained by the external sensor of the vehicle into a position on a map so that it can be shared with outside the vehicle.

[0053] Next, the road surface condition acquisition unit 335 acquires road surface condition information outside the host vehicle received via communication from a source outside the host vehicle, such as another vehicle (S820). When acquiring road surface condition information from a source outside the host vehicle, it is preferable to discard information that does not affect the host vehicle's future driving plan and information that can recognize the road surface condition using the host vehicle's external sensors, thereby reducing the subsequent processing load and the amount of road surface condition information. On the other hand, information that can recognize the road surface condition using the host vehicle's external sensors, even if it does not affect the host vehicle's driving plan, is retained for provision to a source outside the host vehicle, such as another vehicle.

[0054] The obstacle information management unit 350 recognizes road surface conditions using information acquired by external sensors of the host vehicle, and then integrates the road surface condition information acquired by the multiple external sensors of the host vehicle for each road surface condition location (S830).

[0055] The obstacle information management unit 350 then integrates road surface condition information acquired from outside the vehicle (external road surface condition information) with road surface condition information recognized from the sensing results of the vehicle's external sensors (internal road surface condition information) (S840). Note that the external road surface condition information also includes road surface condition information from multiple information sources, such as multiple other vehicles and the information sharing center 700, which is integrated for each road surface condition location. The information sharing center 700 is a management server that manages road surface condition information transmitted from vehicles.

[0056] In step S840, the obstacle information management unit 350 refers to external road surface condition information, determines whether a road surface condition is correct based on the recognition reliability and consistency between multiple information sources, even if the road surface condition is not observed by the external sensors of the host vehicle, and treats the road surface condition location as existing if the external road surface condition information is determined to be correct. This allows the driving plan unit 360 to update the driving plan taking into account the existence of the road surface condition location even before it is recognized by the external sensors of the host vehicle.

[0057] Thereafter, the obstacle information management unit 350 determines whether a valid road surface condition location has been detected (S850). If a valid road surface condition location has not been detected (No in S850), the road surface condition response process ends. A valid road surface condition location is a road surface condition location whose recognition reliability is equal to or greater than a predetermined threshold.

[0058] On the other hand, if a valid road surface condition location is detected (Yes in S850), the obstacle information management unit 350 determines whether the host vehicle needs to take some kind of driving action for the road surface condition location (S860). At this time, it is advisable to also take into consideration the presence of traffic participants in the vicinity of the road surface condition.

[0059] If the road surface condition location requires a driving response from the host vehicle, the driving planner 360 updates the driving plan to ensure appropriate driving, taking into consideration the location, shape, size, and type of the road surface condition location, as well as surrounding traffic participants (vehicles, pedestrians, animals, etc.) that may affect the driving of the host vehicle (S870). On the other hand, if the road surface condition location does not require a driving response from the host vehicle, the driving planner 360 omits the driving plan update step (S870) and proceeds to step S880.

[0060] Next, the road surface condition acquisition unit 335 converts the data required for transmitting the road surface condition information to an externally transmittable format (S880), and the external communication unit 340 transmits the converted road surface condition information to an externally transmittable format (S890).

[0061] If it is determined in step S850 that a valid road surface condition location has been detected through the above processing, then in step S890, regardless of whether any action is required in terms of vehicle driving, road surface condition information is transmitted outside the vehicle. This makes it possible to provide road surface condition recognition information recognized when approaching a road surface condition location to an outside of the vehicle, and to provide road surface condition information obtained by the external environment recognition function of the vehicle to an outside of the vehicle.

[0062] 6A and 6B are diagrams showing an example of transmitting and receiving road surface condition information via the information sharing center 700. FIG. 6B shows the situation after a predetermined time (for example, several minutes to several hours) from FIG. 6A.

[0063] 6A , immediately after the second vehicle 200 passes the road surface condition spot D540, the second vehicle 200 continuously recognizes the road surface condition spot D540 from before passing the road surface condition spot D540, thereby increasing the recognition reliability of the road surface condition spot D540. Furthermore, the second vehicle 200 recognizes the state of the road surface condition spot D540 after passing by using the camera BB220, and in the situation shown in FIG. 6A , transmits the final information obtained by recognizing the road surface condition spot D540 to the information sharing center 700.

[0064] Even while the second vehicle 200 is continuously recognizing the road surface condition spot D540, each time the recognition reliability improves or the recognition content is corrected, the second vehicle 200 appropriately transmits road surface condition information for the road surface condition spot D540 to the information sharing center 700. However, once the second vehicle 200 passes the road surface condition spot D540 and moves far away from the road surface condition spot D540, and the recognition reliability for the road surface condition spot D540 decreases, the recognition information for the road surface condition spot D540 thereafter becomes unavailable, and therefore the road surface condition information transmitted to the outside of the second vehicle 200 no longer includes information related to the road surface condition spot D540.

[0065] 6B , the situation changes to one in which the first vehicle 100, another vehicle, approaches road surface condition location D540. In the situation in FIG. 6B , time has passed since the second vehicle 200 passed road surface condition location D540, and the first vehicle 100 is now positioned far away from the second vehicle 200. Therefore, the second vehicle 200 does not directly transmit road surface condition information for road surface condition location D540 to the first vehicle 100 via vehicle-to-vehicle communication, and the first vehicle 100 is unable to obtain the information for road surface condition location D540 from the second vehicle 200. Therefore, the first vehicle 100 refers to the road surface condition information for road surface condition location D540 provided by the information sharing center 700, obtains the road surface condition information for road surface condition location D540, and uses the information in the travel plan for the first vehicle 100.

[0066] When passing through road surface condition location D540, the first vehicle 100 recognizes the road surface condition location D540 and transmits the recognized road surface condition location D540 to the information sharing center 700, so that the road surface condition information for the road surface condition location D540 held by the information sharing center 700 is updated appropriately.

[0067] The information sharing center 700 refers to the time elapsed since the last time the road surface condition information was received, environmental information and time of the location where the road surface condition exists, and the type and size of the road surface condition, and further determines the validity of the road surface condition information based on the history of changes in the road surface condition at the location where the road surface condition exists, and transmits the road surface condition information determined to be valid externally. If there is a possibility that the road surface condition has changed at the time of determining the validity, the road surface condition information may be transmitted with an indication of the possibility of a change in the road surface condition. For this reason, the information sharing center 700 refers to environmental information added to the road surface condition information transmitted from the vehicle. In addition, the information sharing center 700 may refer to information obtained from sources other than the vehicle, such as weather information.

[0068] For example, even if the road surface is frozen, if it is spring or the temperature is sufficiently high (e.g., above 10 degrees Celsius), or if the temperature is around 5 degrees Celsius but the location is in sunny weather and the weather is clear, the ice may melt and turn into a puddle in a relatively short time, so a parameter indicating the possibility of the puddle turning into a puddle is added as a large value to the road surface condition information. Conversely, if it is winter or the temperature is below 3 degrees Celsius, the possibility of the puddle freezing increases over time, so a parameter indicating the possibility of the puddle turning into a frozen location is added as a large value to the road surface condition information. Furthermore, because puddles grow in size depending on the product of the amount of rainfall and the elapsed time, a parameter indicating the possibility of the puddle growing may be added to the road surface condition information as a value corresponding to the amount of rainfall.

[0069] In determining the validity of road surface condition information, the information sharing center 700 should invalidate the road surface condition information after a certain period of time (several hours) has passed since the last time the information on the road surface condition was received, because the reliability of the road surface condition information is poor. However, this period of time may be changed based on past records of condition changes at the location where the road surface condition exists, taking into account factors such as the time of year and traffic volume. For example, if traffic volume is heavy, snow, which is a road surface condition, is likely to disappear quickly due to passing vehicles, so the validity period of the road surface condition information should be shortened.

[0070] The information sharing center 700 may receive information on the same road surface condition location from multiple vehicles. In principle, the same road surface condition location is updated with newly received road surface condition information. However, information that can be determined to be the same road surface condition location received from several to several dozen vehicles immediately before receiving road surface condition information is retained for a predetermined period of time (e.g., approximately 10 minutes). If the recognition reliability of the newly received information is lower than the recognition reliability of the retained road surface condition information or if the information is significantly different from road surface condition information previously obtained from other vehicles, the information sharing center 700 determines that the reliability of the information is low and temporarily suspends use of the newly received information. The suspended road surface condition information is compared with information with reliability above a certain threshold transmitted from a vehicle other than the vehicle that provided the road surface condition information. When it is determined that the road surface condition is the same, the information sharing center 700 considers the information valid and includes it in the information to be transmitted from the information sharing center 700.

[0071] 7, 8 and 9, examples of data formats when road surface condition information is transmitted from a vehicle will be described.

[0072] FIG. 7 is a diagram showing an example of the data format of all the information transmitted from the vehicle, FIG. 8 is a diagram showing an example of the contents of the common information 910, and FIG. 9 is a diagram showing an example of road surface condition information.

[0073] As shown in FIG. 7, the information transmitted from the vehicle includes common information 910, number of pieces of information 920, and information 1 (921) to information n (92n) in this order.

[0074] The common information 910 stores data that is commonly used in subsequent information. The number of pieces of information 920 indicates the number of subsequent sets of information. Information 1 (921) to information n (92n) are data strings that make up one set of information.

[0075] The example shown in FIG. 7 does not include an error detection code such as a checksum or CRC, but in practice, an error detection code should be added to each piece of information or each series of data to detect and correct errors in communication.

[0076] An example of the content included in the common information 910 will be described with reference to FIG. 8 . V1 (transmitter identification code) is unique identification information identifying the information transmitter. V2 (transmission time) is International Atomic Time based on the reference clock of the information transmitter. V3 (transmitter location information) is information on the position (three-dimensional, including the height direction) and orientation (yaw angle, roll angle, pitch angle) on a map obtained by the vehicle transmitting the information through self-location estimation. V4 (surrounding environment information) is environmental information about the vehicle's surroundings acquired by the environmental information acquisition unit 315, such as the surrounding temperature, humidity, and atmospheric pressure. V4 (surrounding environment information) may be information obtained by the transmitting vehicle alone (e.g., temperature). V5 (recognition reliability) is the reliability of recognition processing for each recognition target group (groups such as traffic participants, signs, lanes, and traffic lights). A vehicle providing road surface condition information to an external party also outputs a single recognition reliability for the recognition processing performance for the road surface condition group.

[0077] V5 (recognition reliability) may be calculated by averaging the degree of agreement between the recognition results recognized from information acquired by the external sensors of the vehicle transmitting the information and the information obtained through communication. That is, if there is a low degree of agreement between highly reliable information obtained through communication (information provided by the information sharing center 700 that is highly reliable and recently acquired) and the recognition results recognized from information acquired by the external sensors of the vehicle transmitting the information, V5 (recognition reliability) will be low; if this degree of agreement is high, V5 (recognition reliability) will be high. A vehicle receiving information from outside checks the received V5 (recognition reliability), and if this value is lower than a predetermined threshold, it determines that the recognition process of the vehicle transmitting the information is inappropriate, discards the information transmitted from the vehicle, and invalidates any information suspected of being erroneously recognized.

[0078] Referring to FIG. 9, an example of the content when a series of pieces of information included in each of information 1 (921) to information n (929) are information relating to a single road surface condition location will be described.

[0079] X1 (information type code) indicates the type of information. If the information is about a road surface condition location, X1 (information type code) is a code indicating "road surface condition." In other words, X1 (information type code) determines the format of the data that follows in a set of information. If X1 (information type code) is not a code indicating "road surface condition," the data format following X1 (information type code) may be different from that shown in FIG. 9.

[0080] X2 (road surface condition recognition time) indicates the time when the road surface condition included in the set of information was recognized. This time is not the time when the road surface condition was first recognized, but the time when the external sensor acquired information on the latest recognition result to be transmitted to the outside in a situation where the external sensor is continuously recognizing the road surface condition.

[0081] X3 (road surface condition identification code) is unique identification information assigned to each information sender and each road surface condition. X3 (road surface condition identification code) may use, for example, the date and time when the road surface condition was first recognized and a serial number assigned at the same date and time. The road surface condition identification code assigned to a certain road surface condition continues to use the same identification information as long as the road surface condition is tracked. A vehicle receiving information can track the same road surface condition location for each information sender by using V1 (sender code) and X3 (road surface condition identification code).

[0082] X4 (road surface condition type code) is identification information corresponding to the type of the road surface condition location. For example, road surface conditions include puddles, ice, snow, holes (such as potholes), small objects (hard objects), small objects (soft objects), small objects (unknown hardness), undefined obstacles, and unknown obstacle types. The difference between undefined obstacles and unknown obstacle types is that the undefined obstacle is used when it is known that the object is not of a defined type (such as a puddle), while the unknown obstacle is used when it is unclear whether the object is of a defined type. The distinction between hard and soft small objects can be made by classifying them based on whether they are significantly deformed by the passage of a vehicle. The classification of hard and soft objects is performed by estimating the detailed type of the small object during the recognition process. If the type of a small object is unknown, it is treated as a small object (unknown hardness). The classification of the hardness of a small object may be updated based on whether the road surface condition is deformed or not, as detected by an external sensor after the vehicle has passed.

[0083] Note that an object that is higher than the minimum ground clearance of a vehicle permitted by law should be treated as a normal object, not as a small object or road surface condition.

[0084] X4 (road surface condition type code) may represent multiple types at the same time, for example, "puddle" and "frozen." When representing multiple types, it is preferable to represent them in order of likelihood.

[0085] X5 (position / shape information) is information about the position and shape of the road surface condition. The position is expressed in the form of a position on a map. The shape is expressed by approximating the road surface condition to a polygon and information about the number of vertices and the position of each vertex. The shape may also include the height from the road surface. The height is the height of the part of the road surface that is recognized as having the greatest deviation from the road surface. For road surface conditions that are recessed from the road surface, such as potholes, the height is a negative number. Also, if the height of the road surface condition is unknown, it may be expressed as "height unknown."

[0086] X6 (type reliability) is the recognition reliability of X4 (road surface condition identification code) and may be expressed, for example, as a percentage. The recognition reliability is a value output when recognizing road surface conditions from information acquired by an external sensor that transmits the information. The recognition reliability may be calculated taking into account road surface condition information acquired externally. For example, if the type of road surface condition recognized at a certain location matches many road surface condition information acquired externally at the same location, the reliability is increased; if it does not match many road surface condition information acquired externally at the same location, the reliability is decreased. When X4 (road surface condition type code) represents multiple types, the reliability may be indicated for each type. When the reliability is expressed as a percentage, the reliability for each type does not exceed 100, but the total reliability may exceed 100. For example, in a situation where ice and water are mixed, both the ice (frozen) type and the water (puddle) type may have high recognition reliability.

[0087] X7 (detected position / shape reliability) is the recognition reliability of X5 (position / shape information) and is a value output when recognizing road surface conditions from information acquired by an external sensor of the vehicle that transmitted the information. X7 (detected position / shape reliability) includes reliability values ​​for both horizontal position and vertical height. The recognition reliability of the detected position / shape may be calculated taking into account road surface condition information acquired from an external source. For example, if the position of the road surface condition recognized at a certain location matches many pieces of road surface condition information acquired from an external source at the same location, the reliability is increased, and if it does not match the road surface condition information acquired from an external source at the same location, the reliability is decreased. Similarly, the height of the road surface condition may be calculated based on whether it matches the height of road surface condition information acquired from an external source at the same location.

[0088] X8 (number of external references) is the number of information transmission sources of received information referenced in the recognition process. Information transmitted from multiple other vehicles around the vehicle may be received, or multiple pieces of information transmitted from the information sharing center 700 may be received, and information about the road surface condition at one location may be provided from multiple information transmission sources. Of these transmission sources, the number of transmission sources referenced in the recognition process for the road surface condition is X8 (number of external references). Note that the road surface condition information transmitted from the information sharing center 700 is usually based on information about the road surface condition transmitted from multiple vehicles, but it is advisable to count the number of information transmission sources separately for the other vehicles and the information sharing center 700. In addition, transmission sources of information that were received externally but not referenced in the recognition process for the road surface condition are excluded.

[0089] X9 (external reference recognition statistical information) is statistical information on the recognition reliability of road surface condition information received from an external source and used as a reference in the recognition process. X9 (external reference recognition statistical information) includes the reliability of the information on the road surface condition received from an external source and used in the recognition process of the vehicle (e.g., the type in X6 (type reliability), and the minimum, average, and maximum of the planar position and height in X7 (detection position / shape reliability)). The receiver of the road surface condition information may use X8 (external reference number) and X9 (external reference recognition statistical information) to determine whether to use the received information. For example, if the average reliability is below a predetermined threshold or the minimum value is below a predetermined threshold, the information may be ignored as unusable information.

[0090] X10 (road surface condition estimated change tendency) is a trend of temporal changes in road surface condition information, and represents changes in position, shape (including height), and type. For example, if a small object has a shape that is easy to roll, such as a sphere or a cylinder, the road surface condition may move by rolling, so X10 (road surface condition estimated change tendency) may represent "high possibility of position changing in a short period of time." For objects whose shape or height changes due to the passage of a vehicle (e.g., small objects or soft snow), X10 (road surface condition estimated change tendency) may represent "high possibility of shape or height changing in a short period of time." Information on the direction of change, such as whether the shape becomes larger or smaller or whether the height becomes larger or smaller, may be added to X10 (road surface condition estimated change tendency). Furthermore, information on the temporal speed of change may be added to X10 (road surface condition estimated change tendency). If a change in type can be estimated, such as snow or ice turning into puddles, or puddles turning into ice, information on the change in type may be added to X10 (road surface condition estimated change tendency).If the sender of the information does not have the function to estimate the road surface condition estimated change tendency, or if it has the estimation function but it is difficult to estimate, X10 (road surface condition estimated change tendency) may be set to "no information."

[0091] A vehicle that receives road surface condition information uses X10 (estimated change tendency of road surface condition) to determine whether to use the received road surface condition information or whether it can change its driving plan with sufficient positional leeway. For example, if it is predicted that a puddle will become large, it is advisable to create a driving plan that avoids the puddle with sufficient positional leeway.

[0092] X11 (number of additional road surface condition information) indicates the number of pieces of additional road surface condition information included after X12. If there is no additional road surface condition information after X12, X11 (number of additional road surface condition information) is 0.

[0093] X12 (additional road condition information 1) is the first additional road condition information. The additional road condition information is information on additional objects added to the road surface condition, such as information on ruts when the road surface is snow-covered. There are as many additional road condition information as indicated by X11 (number of additional road condition information).

[0094] Each piece of road surface condition additional information includes the same information as X4 (road surface condition type code) to X9 (external reference recognition reliability). However, X4 (road surface condition type code) is the type code of the addition, X5 (position / shape information) is information on the position / shape of the addition, X6 (type reliability) is the reliability of the type of the addition, X7 (detection position / shape reliability) is the reliability of the position / shape of the addition, X8 (number of external references) is the number of sources of information received from outside and used as reference for information on the addition, and X9 (external reference recognition reliability) is statistical information on the recognition reliability of the received information used as reference in the recognition process of the addition. The reliability of each piece of road surface condition additional information and the number of sources of reference information may be different from the reliability of each piece of road surface condition to which the addition is added and the number of sources of reference information.

[0095] The types of additional objects include, for example, ruts, partial protrusions or depressions, small obstacles, etc. Additional road surface condition information allows vehicles to share information about additional shapes and small obstacles in the road surface condition.

[0096] 7, 8, and 9 may be partially modified and used as road surface condition information transmitted from the information sharing center 700. In the road surface condition information transmitted from the information sharing center 700, V1 (transmission source code) is used to distinguish that the information was transmitted from the information sharing center 700, and the receiving vehicle changes how it handles the received road surface condition information.

[0097] In the information transmitted from the information sharing center 700, V3 (transmitter location information) is the location information of the information destination, and the direction is "no information." Since the information sharing center 700 collects information from vehicles traveling at various locations, it divides the area on the map into predetermined sections and transmits a series of information summarizing the road surface conditions present within each section. The location information of the information destination indicates the section on the map.

[0098] When receiving information from the information sharing center 700, it is advisable to determine the necessity of reception based on the location information of the information provider, discard information relating to sections on the map that do not affect the running of the vehicle, and reduce the processing load of the received information. Furthermore, when transmitting road surface condition information wirelessly, the amount of data transferred wirelessly can be reduced by not transmitting road surface condition information outside the area covered by wireless and its surrounding areas.

[0099] The information sharing center 700 may provide road surface condition information at predetermined time intervals, or the vehicle may transmit its location information and request information on the new section from the information sharing center 700 before passing through the section indicated by the location information of the information provider that the vehicle is receiving from the information sharing center 700, and the information sharing center 700 may transmit the road surface condition information in response to the request. In order to reduce the effect of the time lag until the vehicle obtains road surface condition information on the section on the map from the information sharing center 700 after passing through the section, the information sharing center 700 may transmit road surface condition information on the section adjacent to the vehicle's position to the vehicle, and the vehicle may receive information on the next section that it is likely to enter from the information sharing center 700.

[0100] V4 (surrounding environment information) is environmental information (temperature, humidity, air pressure, wind speed, wind direction, weather, etc.) of the section corresponding to the information destination location information. If the section is large, the section may be divided into multiple sections, and environmental information for each divided section may be used. The environmental information transmitted by the information sharing center 700 is generated from the environmental information transmitted from the vehicle, but environmental information obtained from a weather information provider or environmental information observed by a pre-installed environmental observation device may also be used.

[0101] V5 (recognition reliability) is the prediction reliability, and is a value indicating the reliability of the predicted time change in the section corresponding to the location information of the information destination from the information sharing center 700. The prediction content corresponds to X10 (road surface condition estimated change tendency). Since the information transmitted from the information sharing center 700 may include information that is older than the time elapsed, if the information receiving side has low prediction reliability, it is recommended that it discard information that has elapsed a long time since X2 (road surface condition recognition time). The prediction reliability may be calculated from environmental information, etc. For example, in weather conditions where the temperature changes suddenly and the prediction is likely to deviate from the prediction, it is recommended that the prediction reliability be lowered.

[0102] The information sharing center 700 discards road surface condition information that is judged to be unusable based on the prediction reliability and the elapsed time since the road surface condition was recognized, and thereafter does not transmit unusable road surface condition information.

[0103] X3 (road surface condition identification code) is unique identification information assigned to each road surface condition by the information sharing center 700. The information sharing center 700 integrates road surface condition information obtained from multiple vehicles, and uses the same identification information for the same road surface condition as long as that road surface condition exists. When valid information related to that road surface condition disappears, the identification information for that road surface condition is invalidated.

[0104] 10 is a diagram showing an example of a road surface condition portion E550 and position and shape information of the road surface condition additional information. The road surface condition portion E550 shows partial snow accumulation on the road, and ruts E556 exist in the snow accumulation.

[0105] X5 (position / shape information) of the road surface condition portion E550 is represented by the positions of the vertices of a polygon surrounding the road surface condition portion E550. However, to prevent an increase in processing load due to an increase in the number of vertices, it is preferable to set an upper limit on the number of vertices, and the "polygon" does not have to be a minimum, as long as it is as small as possible. Therefore, the polygon representing the road surface condition portion E550 is represented, for example, by polygon E555, and X5 (position / shape information) can be represented by a data string in which each vertex is arranged in the order that constitutes the side of the polygon.

[0106] When representing the positions of vertices, in order to reduce the amount of data, the position on the map of any one vertex of the polygon may be represented, and other vertices may be represented by their relative positions from that vertex or their relative positions from the position of the previous vertex in the data string.

[0107] The rut E556 is represented as one piece of road surface condition additional information for the road surface condition location E550. Each piece of road surface condition additional information includes X5 (position and shape information), and like the road surface condition, the X5 (position and shape information) of the road surface condition additional information is represented by the vertex positions of a polygon surrounding the object. However, the shape of the additional information is valid only within the shape of the road surface condition location to which the road surface condition additional information is added. When a road surface condition location is completely crossed, such as a rut, the crossing direction is treated as an expanded shape that completely crosses the shape of X5 (position and shape information) indicating the road surface condition. The same applies to crossings that are not crossed vertically but crossed diagonally. As a result, the X5 (position and shape information) of the rut E556 is represented, for example, by polygon E557. By treating it in this manner, a vehicle that has received road surface condition information can determine whether it can pass through the road surface condition location by riding on a rut when creating a driving plan.

[0108] The positions of the vertices of a polygon indicating the road surface condition additional information may be represented in terms of their relative positional relationship with the road surface condition location to which the road surface condition additional information is added. For example, the position information of polygon E557 surrounding rut E556 may be represented as a relative position from the position of the first vertex of X5 (position / shape information) of road surface condition location E550. Alternatively, the position of the first vertex of polygon EX557 may be represented as a relative position from the position of the first vertex of X5 (position / shape information) of road surface condition location E550, and the other vertices may be represented as relative positions from that vertex or from the position of the previous vertex in the data string. By representing the position information of the road surface condition additional information as a relative position to the road surface condition location to which the road surface condition additional information is added, the accuracy of representing the relative position of the addition to the road surface condition location can be improved compared to using coordinates on a map, which requires the handling of a large range.

[0109] The consistency of road surface condition information for the same road surface condition location obtained from multiple transmission sources can be determined by the ratio of the area sizes. For example, the road surface condition areas represented by X5 (position and shape information) can be compared, and the ratio of the area of ​​the overlapping parts and the non-overlapping parts (areas indicated by only one of the road surface condition information) of the polygons of the two road surface condition information can be calculated as the degree of consistency, which can be used as an index of consistency. In this method, the degree of consistency is calculated for each combination of road surface condition information.

[0110] An example of improving a vehicle travel plan based on receiving road surface conditions from an external source will be described with reference to FIGS. 11 to 14. FIG.

[0111] 11 is a diagram showing an example of a road surface condition in which there is a small puddle A560 and a pedestrian 650 is present near the puddle A560. Because the pedestrian 650 is present near the puddle A560, the first vehicle 100 should drive in a way that prevents water and mud from splashing on the pedestrian 650.

[0112] Travel route AC422 is an example of a route of the left front wheel when first vehicle 100 does not consider puddle A560 or cannot find puddle A560. In this case, puddle A560 is not considered and the travel speed is not reduced, so there is a high possibility that pedestrian 650 will be splashed with water or mud.

[0113] Travel route AB421 is an example of a route of the left front wheel of the first vehicle 100 according to a travel plan created so that the first vehicle 100 discovers puddle A560 and pedestrian 650 using its external sensors and reduces the possibility of water or mud splashing on the pedestrian 650. The range in which the first vehicle 100 can detect road surface conditions ahead using its external sensors is within area AA115. Therefore, after the first vehicle 100 approaches puddle A560, the first vehicle 100 recognizes the possibility of water or mud splashing and changes the travel plan, so the left front wheel of the first vehicle 100 travels along travel route AB421. Travel route AB421 does not completely avoid puddle A560, and since the distance to pedestrian 650 is close, it is desirable to slow down before passing through puddle A560. At this time, in order to ensure that water and mud are not splashed onto the pedestrian 650, the first vehicle 100 may unavoidably slow down, even if this reduces the ride comfort of the occupants, while also taking into consideration the safety of following vehicles.

[0114] Travel route AA420 is an example of a route for the left front wheel of first vehicle 110 according to a travel plan that is generated so that first vehicle 100 detects the presence of puddle A560 from road surface condition information received from the outside, detects pedestrian 650 using the vehicle's external sensors, and reduces the possibility of water or mud splashing on pedestrian 650. Because objects such as pedestrians are taller than the road surface conditions, they can be detected relatively easily through recognition processing using the external sensors, and therefore their presence can be recognized by the vehicle's external sensors even outside area AA115, which is the range in which road surface conditions can be detected.

[0115] In this way, by generating a driving plan using road surface condition information obtained from an external source, the driving route AA420 becomes a driving route that reduces the possibility of damage caused by splashes of water or mud due to puddle A560 compared to driving route AB421. By generating a driving plan that avoids puddle A560 early on, puddle A560 can be completely avoided, making deceleration ineffective.

[0116] Even if it is determined early on that the possibility of damage caused by puddle A560 needs to be reduced, there may be cases where it is difficult to avoid puddle A560 due to the presence of an oncoming vehicle, etc. Even in such cases, by changing the driving plan early, the steering angle required to reduce the possibility of damage occurring can be reduced and deceleration can be performed early, thereby reducing the lateral G caused by steering and the longitudinal G caused by deceleration. In addition, the amount of change in lateral G and longitudinal G per unit time can also be reduced.

[0117] In other words, by creating a driving plan based on road surface condition information obtained from an external source, unnecessary deceleration can be suppressed, and the magnitude of lateral and vertical G forces and the amount of change per unit time can be reduced, improving the ride comfort for passengers and reducing energy consumption associated with re-acceleration.

[0118] FIG. 12 is a diagram showing an example of a road surface condition in which there is a large puddle B570 and a pedestrian 650 is present near the puddle B570.

[0119] In the example shown in Fig. 12, pedestrian 650 is near a large puddle B570, so first vehicle 100 should reduce the possibility of water or mud splashing on pedestrian 650. In the example shown in Fig. 12, puddle B570, which is a road surface condition, is larger than in the example shown in Fig. 11, making it difficult to develop a driving plan that avoids puddle B570, and it is necessary to pass through puddle B570 to reduce the possibility of water or mud splashing on pedestrian 650.

[0120] Travel route BB431 is an example of a route for the left front wheel of first vehicle 110 in a travel plan created so that first vehicle 100 can discover puddle B570 and pedestrian 650 using only its own external sensors and reduce the possibility of water or mud splash damage to pedestrian 650. Because the range in which first vehicle 100 can detect road surface conditions ahead using its own external sensors is within area AA115, it is not until first vehicle 100 approaches puddle B570 that it recognizes the possibility of water or mud splash damage and changes its travel plan, and the left front wheel of first vehicle 110 travels along travel route BB431.

[0121] On travel route BB431, when passing through puddle B570, it is necessary to sufficiently decelerate because it is not possible to maintain a sufficient distance from pedestrian 650. In this case, considering that the size of puddle B570 means that the amount of water and mud splashing is likely to be large, and that the left front wheel will completely enter the puddle, it is necessary to travel through the puddle at a slower speed, which requires sudden deceleration, even if the distance to pedestrian 650 is the same when passing through the puddle. Alternatively, it is necessary to increase the amount of steering to maintain a sufficient distance from pedestrian 650 when passing through puddle B570. In some cases, it may be necessary to increase the amount of steering while suddenly decelerating.

[0122] Driving route BA430 is an example of a route for the left front wheel of first vehicle 110 in a driving plan generated so that first vehicle 100 becomes aware of the presence of puddle B570 based on road surface condition information received from the outside, detects pedestrian 650 using the vehicle's external sensors, and reduces the possibility of pedestrian 650 being splashed with water or mud.

[0123] By generating a driving plan using road surface condition information obtained from an external source, it is possible to create a driving route BA430 that reduces the possibility of damage caused by splashes of water or mud due to puddle B570 earlier than driving route BB431. By reflecting this information in the driving plan early, it is possible to ensure a sufficient distance from pedestrian 650 when passing through puddle B570, and it is not necessary to decelerate as much as with driving route BB431, and the longer distance over which deceleration is required allows for gradual deceleration.

[0124] Even if a driving plan is created early to reduce the occurrence of damage caused by splashes of water or mud from puddle B570, the presence of an oncoming vehicle or the like may cause the vehicle to come close to pedestrian 650 when passing through puddle B570. Even in such cases, by changing the driving plan early, the deceleration distance can be increased, allowing for gradual deceleration.

[0125] As shown in FIG. 12 , even when the road surface condition is large and passing over it is unavoidable, by creating a travel plan based on information on the road surface condition obtained from an external source, it is possible to suppress a decrease in speed or deceleration and to make steering gentler, thereby improving the ride comfort for passengers and reducing energy consumption associated with re-acceleration.

[0126] 13 is a diagram showing an example of a road surface condition on a curved road where there is a small frozen spot A580. When there is an frozen spot, it is necessary to reduce the possibility of slipping.

[0127] Travel route CB441 is an example of the path of the left front wheel of first vehicle 110 when first vehicle 100 does not consider frozen spot A580, cannot find frozen spot A580, or finds frozen spot A580 late. If frozen spot A580 is not considered or cannot be found, the first vehicle 100 will not slow down when passing frozen spot A580 and will not avoid frozen spot A580, which could result in a slip. Since there is a high possibility that undiscovered frozen spots exist around frozen spot A580, and if frozen spot A580 is discovered late, sudden deceleration could cause a slip. Therefore, the first vehicle 100 must continue steering and gradually decelerate, following a travel plan that reduces the possibility of a slip. While the steering angle may be slightly reduced to reduce lateral G, reducing the steering angle will result in a route on the outside of the curve compared to travel route CB441, limiting the margin to the road edge and the route after passing frozen spot A580.

[0128] Driving route CA440 is an example of the route of the left front wheel of the first vehicle 110 in a driving plan generated so that the first vehicle 100 receives road surface condition information from the outside, recognizes the presence of frozen areas A580, and reduces the possibility of slipping.

[0129] By generating a driving plan using road surface condition information obtained from an external source, it is possible to generate a driving route CA440 that can reduce the possibility of slipping due to icy spot A580 earlier than driving route CB441. As a result, by gradually decelerating from a distance from icy spot A580 and slightly increasing the steering angle, the vehicle can travel along driving route CA440, which is on the inside of the curve than driving route CB441, avoiding icy spot A580 and reducing the possibility of slipping.

[0130] Even when a driving plan is generated using road surface condition information obtained from an external source, or when the vehicle is forced to travel along route CB441 due to the presence of an oncoming vehicle or the like, the vehicle can be decelerated gradually by slowing down from a point far from frozen area A580.

[0131] In other words, by generating a driving plan early using road surface condition information obtained from an external source, it is possible to create a driving plan that allows for sufficient deceleration, gentle deceleration, and avoids frozen areas, thereby reducing the possibility of slipping and improving passenger comfort.

[0132] FIG. 14 is a diagram showing an example of a road surface condition on a curved road where a frozen spot B590 exists, which is larger than the frozen spot in the example shown in FIG.

[0133] The driving route DB451 is an example of the route of the left front wheel of the first vehicle 110 when the first vehicle 100 does not consider the frozen spot B590, cannot find the frozen spot B590, or finds the frozen spot B590 late. If the frozen spot B590 is not considered or cannot be found, no driving measures are taken to deal with the frozen spot B590, and therefore there is a high possibility of slipping. If the frozen spot B590 is found late, sudden steering or sudden deceleration will cause a slip, and since it is difficult to avoid the frozen spot B590 due to its size, the only driving plan that can reduce the possibility of slipping is to slow down slightly.

[0134] Driving route DA450 is an example of the route of the left front wheel of the first vehicle 110 in a driving plan generated by the first vehicle 100 to receive road surface condition information from the outside, identify the presence of frozen area B590, and reduce the possibility of slipping.

[0135] By generating a driving plan using road surface condition information acquired from an external source, a driving plan can be generated for the driving route DA450 that reduces the possibility of slipping due to the frozen spot B590 earlier than the driving route DB451, and the route shown in the driving route DB451 can be adopted. When generating a driving plan for the driving route DA450 that passes through the frozen spot B590, since the frozen spot B590 is a distance away, it is possible to travel along a route that reduces the steering angle when passing through the frozen spot B590, as shown in the driving route DB451. By reducing the steering angle when passing through the frozen spot B590, the lateral acceleration when passing through the frozen spot B590 can be reduced, thereby reducing the possibility of slipping. Furthermore, since the vehicle can gradually decelerate from a distance from the frozen spot B590, the rate of change in deceleration can also be reduced. Therefore, the possibility of slipping can be reduced while the longitudinal acceleration and changes in longitudinal acceleration can be reduced, preventing a deterioration in the ride comfort of the occupants.

[0136] 11 to 14, if the external sensors of the host vehicle detect the need for risk avoidance, priority is given to revising the driving plan necessary for risk avoidance (including emergency braking and emergency steering avoidance). Furthermore, if the external sensors of the host vehicle determine that there is an error in the information received from the outside, and if the error increases the risk to driving safety, the driving plan should be revised again to reduce the risk. Because responding to road surface conditions based on the recognition results of the external sensors of the host vehicle can easily lead to sudden changes in behavior, the driving plan revised based on information received from the outside should be revised again only when it is unavoidable to avoid risk. Even if revision is unavoidable, it should be kept to a minimum.

[0137] 11 to 14 show examples in which a driving plan corresponding to road surface conditions is created and the vehicle drives according to the driving plan, but this example is useful not only for automated driving but also for driving assistance that intervenes in the occupant's driving to respond to road surface conditions. When intervening in the occupant's driving as driving assistance to avoid danger due to road surface conditions, the driving plan can be changed at a point far from the road surface conditions and a gradual driving intervention operation can be performed, thereby reducing the occurrence of situations that surprise or cause discomfort to the occupant and reducing the degree of discomfort.

[0138] To avoid dangers caused by road surface conditions, information on the type of obstacle is important, along with information on the location and shape of the road surface. If the obstacle is a puddle, consideration must be given to its impact on surrounding traffic participants (for example, damage caused by water or mud splashes), and if it is frozen, the possibility of slipping must be reduced. If there is a hole in the road surface, such as a pothole, it must be avoided. In other words, the driving plan must be modified appropriately depending on the type of obstacle.

[0139] Furthermore, from the viewpoint of avoiding dangers due to road surface conditions and ensuring passenger comfort, it is important to acquire information on the position, shape, and type of road surface conditions at points far from the road surface.However, the difference in height between the road surface and the vehicle is small or almost the same, and it is difficult to acquire information on road surface conditions quickly and with high accuracy only by recognition using external sensors of the vehicle.

[0140] By continuing the recognition process until the road surface condition is close to the target condition, obtaining road surface condition information with high recognition accuracy, and sharing the obtained road surface condition information with other vehicles, an appropriate driving plan can be created that avoids the road surface condition by using the highly accurate road surface condition information when the target road surface condition is still some distance away. This reduces the impact on surrounding traffic participants and / or ensures comfort for occupants by avoiding unstable vehicle behavior when responding to road surface conditions.

[0141] As described above, according to the embodiment of the present invention, external sensing is performed while approaching a road surface condition location requiring driving countermeasures, thereby enabling high-resolution road surface condition information to be acquired and improving the accuracy of road surface condition recognition. Furthermore, even on road surfaces with high reflectivity, such as water or ice, by using road surface condition information after approaching or passing through the road, changes in light reflected by the road surface can be recognized and the accuracy of determining the type of road surface condition can be improved. Furthermore, changes in road surface conditions after passing through (such as the generation of waves in the case of water or changes in shape in the case of snow) can be used to accurately identify the type and location of road surface conditions. Furthermore, in cases where the shape of the road surface changes as a vehicle passes through, such as snow, by continuing to detect road surface conditions even after the vehicle has passed through, information on the road surface condition that subsequent vehicles should consider can be obtained. Sharing this information with other vehicles and using high-precision road surface condition information allows for the creation of appropriate driving plans according to the location and type of road surface condition, enabling appropriate automated driving and driving assistance.

[0142] Furthermore, by incorporating the shared road surface condition information into the driving plan, even if road conditions requiring driving measures cannot be detected until the vehicle approaches, a driving plan that avoids the road surface condition can be created in advance, reducing the impact on surrounding traffic participants (for example, the occurrence of damage caused by splashes of water or mud).In addition, sudden changes in the vehicle's behavior can be suppressed, reducing unstable vehicle behavior, preventing a decrease in passenger comfort, and reducing passenger stress caused by sudden intervention in driving assistance.

[0143] Although the present specification shows an example in which a camera is used as an external sensor, other sensors such as LiDAR or ultrasonic sensors may be used as long as they can be used to detect road surface conditions. Also, a combination of multiple types of external sensors may be used.

[0144] Furthermore, the configuration of the electronic control device shown in this specification is an example, and devices with other configurations may be used as long as they are capable of controlling a vehicle based on the techniques shown in this specification. The processing in the electronic control device may be implemented using various methods, such as software implementation, electronic circuitry such as logic circuits, or a combination of software and electronic circuits. Furthermore, the electronic control device does not have to be housed in a single housing, and may be in any form that can realize the function, such as a SoC (System On Chip) or a board on which electronic circuits are mounted.

[0145] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with other configurations.

[0146] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by a processor interpreting and executing a program that realizes each function.

[0147] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, hard disk, or SSD (Solid State Drive), or in a recording medium such as an IC card, SD card, or DVD.

[0148] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines that are necessary for implementation. In reality, it can be considered that almost all components are interconnected.

Claims

1. A vehicle control device, comprising: an arithmetic unit that executes arithmetic processing; a storage unit accessible by the arithmetic unit; an internal road surface state acquisition unit that acquires information on the road surface state on the travel road observed by a sensor mounted on the vehicle; an external road surface state acquisition unit that acquires information on the road surface state on the travel road transmitted from an external device; a travel plan unit that creates a travel plan for reducing at least one of the influence on traffic participants around the road surface state and the unstable behavior of the vehicle when the vehicle is likely to pass through the location of the road surface state according to the information on the road surface state; and an external communication unit that transmits the information on the road surface state acquired by the internal road surface state acquisition unit to the external device. The external road surface state acquisition unit acquires information on the road surface state observed before the acquisition time of the information on the road surface state by the internal road surface state acquisition unit. The external communication unit transmits the information on the road surface state acquired by the internal road surface state acquisition unit to the external device regardless of whether the travel plan unit creates a travel plan.

2. The vehicle control device according to claim 1, wherein the external communication unit transmits the information on the road surface state acquired by the internal road surface state acquisition unit to the external device with information on the environment around the observed road surface state added thereto.

3. The vehicle control device according to claim 2, wherein the information on the road surface state is managed according to an expiration date that is changed based on the information on the environment.

4. The vehicle control device according to claim 1, wherein the external device is at least one of a control device provided in another vehicle and a management server provided outside the vehicle in which the vehicle control device is provided.

5. The vehicle control device according to claim 1, wherein the information on the road surface state acquired by the external road surface state acquisition unit is information observed when another vehicle approaches the location of the road surface state.

6. The vehicle control device according to any one of claims 1 to 4, wherein the internal road surface condition acquisition unit has a function of acquiring information on the road surface condition observed behind the vehicle, and the external communication unit has a function of transmitting the information on the road surface condition observed behind to the external device.

7. A vehicle control system comprising a vehicle control device mounted on a vehicle and a management server capable of communicating with the vehicle control device, wherein the vehicle control device includes an internal road surface condition acquisition unit that acquires information on the road surface condition on the traveling road of the vehicle observed by a sensor mounted on the vehicle, an external road surface condition acquisition unit that acquires information on the road surface condition on the traveling road of the vehicle transmitted from the management server, a travel plan unit that creates a travel plan for reducing at least one of the influence on traffic participants in the vicinity of the road surface condition and the unstable behavior of the vehicle when the vehicle is likely to pass through the road surface condition according to the information on the road surface condition, and an external communication unit that transmits the information on the road surface condition acquired by the internal road surface condition acquisition unit to the management server, the external road surface condition acquisition unit acquires information on the road surface condition observed before the acquisition time of the information on the road surface condition by the internal road surface condition acquisition unit, and the external communication unit transmits the information on the road surface condition acquired by the internal road surface condition acquisition unit to the management server regardless of whether the travel plan unit creates a travel plan.

8. The vehicle control system according to claim 7, wherein the management server manages the expiration date of the information on the road surface condition based on the information on the surrounding environment including the road surface condition acquired from the vehicle control device.

9. The vehicle control system according to claim 7, wherein the management server manages the expiration date of the information on the road surface condition using at least one of the time period and the traffic volume.

Images