Driving Support Devices

Abstract

To provide a driving support device capable of performing driving support in the case that a vehicle travels through a place similar to a place needing driving support.SOLUTION: A memory stores a support pattern including a first peripheral sensor map showing a map around a vehicle and support contents of driving support of the vehicle corresponding to the first peripheral sensor map. A processor (a sensor information acquisition part 101) acquires sensor information showing of objects around the vehicle sensed by the sensor. The processor (a peripheral sensor map acquisition part 102) acquires a second peripheral sensor map on the basis of the sensor information. The processor (a map similarity search part 103, and a support instruction part 104) performs driving support of support contents corresponding to the first peripheral sensor map in the case that the second peripheral sensor map is similar to the first peripheral sensor map.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a driving assistance device. [Background technology]

[0002] As a technology for supporting automobile safety, there is a system that provides driving support such as issuing warnings based on registered positions or under specified road conditions, based on map information from a car navigation system or the like.

[0003] Patent Document 1 discloses a system that detects near misses by a driver, records the detected positions on a map, and provides driving assistance. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-047914 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, driving assistance in a scene where a driver experiences a near miss should be provided not only at that location, but also in similar locations.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a driving assistance device that can provide driving assistance when a vehicle is traveling in a place similar to a place where driving assistance is required. [Means for solving the problem]

[0007] In order to achieve the above object, the present invention An exampleThe driving assistance device includes: a memory that stores an assistance pattern including a first surrounding sensor map showing a map of the surroundings of a vehicle and assistance content of driving assistance for the vehicle corresponding to the first surrounding sensor map; and a processor that acquires sensor information showing information of objects around the vehicle sensed by a sensor, acquires a second surrounding sensor map based on the sensor information, and performs driving assistance of the assistance content corresponding to the first surrounding sensor map when the second surrounding sensor map is similar to the first surrounding sensor map. The memory stores a list including operation information indicating an operation state requiring driving assistance and the assistance content corresponding to the operation information, and the processor acquires environmental information indicating the environment around the vehicle including direction information indicating the direction of the vehicle and time, and when the operation state indicated by the operation information is detected, registers / updates the assistance pattern with a set of the environmental information, the second surrounding sensor map, and the assistance content corresponding to the operation information, and when the second surrounding sensor map is similar to the first surrounding sensor map and the environment indicated by the environmental information is detected, performs driving assistance of the assistance content corresponding to the first surrounding sensor map. . Effect of the Invention

[0008] According to the present invention, driving assistance can be performed when a vehicle travels through a location similar to a location where driving assistance is required. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0009] [Figure 1] 1 is a functional block diagram of a driving assistance device according to a first embodiment. [Figure 2A] 1 is a schematic diagram showing the sensing ranges of five sensors mounted on a vehicle. [Figure 2B] FIG. 2 is a schematic diagram showing the sensing ranges of seven sensors mounted on the vehicle. [Figure 3A] FIG. 2 is a diagram illustrating an example of an environment around a vehicle. [Figure 3B] 3B is a diagram showing a sensing result of a sensor mounted on the host vehicle of FIG. 3A. FIG. [Figure 3C] FIG. 4 is a diagram showing a surrounding sensor map acquired by a surrounding sensor map acquisition unit; [Figure 4] 10 is a flowchart illustrating an operation procedure of a map similarity search unit. [Figure 5A] 13 is a flowchart illustrating an operation procedure of a support determining unit. [Figure 5B] FIG. 13 is a diagram showing a support target operation list. [Figure 6]13 is a flowchart illustrating an operation procedure of a support pattern update unit. [Figure 7A] 13 is a flowchart showing an updating method of the support pattern updating unit. [Figure 7B] 13 is a flowchart showing another updating method of the support pattern updating unit. [Figure 8A] FIG. 13 is a diagram showing an example in which a pedestrian suddenly appears and the driver has to suddenly brake. [Figure 8B] FIG. 8B shows a surrounding sensor map for the example of FIG. 8A. [Figure 8C] FIG. 13 is a diagram showing an example of a sudden decrease in speed when traveling along a narrow road. [Figure 8D] FIG. 8D shows a surrounding sensor map for the example of FIG. 8C. [Figure 8E] FIG. 13 is a diagram showing an example of driving close to a wall when turning at a narrow intersection. [Figure 8F] FIG. 8F shows a surrounding sensor map for the example of FIG. 8E. [Figure 9] FIG. 11 is a functional block diagram of a driving assistance device according to a second embodiment. [Figure 10A] FIG. 2 is a diagram illustrating an example of an environment around a vehicle. [Figure 10B] 10B is a diagram showing a sensing range of a camera mounted on the host vehicle in FIG. 10A. FIG. [Figure 10C] 10B is a diagram showing pattern recognition by a camera mounted on the host vehicle of FIG. 10A. FIG. [Figure 10D] FIG. 4 is a diagram showing a target recognized by a target recognition unit; [Figure 11A] FIG. 1 is a diagram showing an example in which a pedestrian suddenly approaches the road while the vehicle is traveling on a narrow road, causing the driver to suddenly apply the brakes. [Figure 11B] 11B is a diagram showing the positional relationship between the surrounding sensor map and a target in the example of FIG. 11A. FIG. [Figure 11C] FIG. 13 is a diagram showing an example in which the terrain is similar but there is no target ahead. [Figure 12] FIG. 11 is a functional block diagram of a driving assistance device according to a third embodiment. [Figure 13A]11 is a conceptual diagram illustrating an example of the contents stored in a table of a pre-pattern storage unit. FIG. [Figure 13B] 13 is a conceptual diagram illustrating an example of the contents stored in another table in the pre-pattern storage unit. FIG. [Figure 14] 13 is a flowchart illustrating another operation procedure of the support pattern update unit. [Figure 15] FIG. 11 is a functional block diagram of a driving assistance device according to a fourth embodiment. [Figure 16A] FIG. 1 is a diagram showing an example in which a motorcycle and a vehicle are present around a host vehicle; [Figure 16B] FIG. 13 is a diagram showing an example in which a sensing range is limited by a target. [Figure 16C] 11 is a diagram showing surrounding topographical information acquired by a surrounding sensor map acquisition unit; FIG. [Figure 17A] FIG. 1 is a diagram showing an intersection with poor visibility. [Figure 17B] 17B is a diagram showing a surrounding sensor map in an example in which the driver suddenly applies the brakes when the host vehicle is about to turn left at the intersection in FIG. 17A. FIG. [Figure 17C] FIG. 13 is a diagram showing an example of driving straight on a narrow road at a speed sufficiently lower than the speed limit. [Figure 17D] FIG. 17D shows a surrounding sensor map for the example of FIG. 17C. [Figure 18A] 13 is a diagram showing an example in which the vehicle passes close to a wall when turning on a narrow road, or in which the vehicle determines that the vehicle cannot continue turning in the current state and shifts into reverse gear. FIG. [Figure 18B] FIG. 18B shows a surrounding sensor map for the example of FIG. 18A. [Figure 18C] This is a diagram showing an example in which, when turning right at an intersection, there is an oncoming vehicle and visibility in the oncoming lane is poor, and a motorcycle or the like suddenly appears from behind the oncoming vehicle and applies the brakes. [Figure 18D] 18D is a diagram showing the positional relationship between the surrounding sensor map and the target in the example of FIG. 18C. FIG. [Figure 19] FIG. 11 is a functional block diagram of a driving assistance device according to a fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Embodiment 1> DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described in detail below with reference to the drawings. Fig. 1 is a block diagram of a driving assistance device 100 in the first embodiment.

[0011] The driving assistance device 100 is incorporated in an integrated controller mounted on a vehicle (FIG. 2A, host vehicle 10) to assist driving, and in this embodiment, is configured to assist driving based on the results of sensing by cameras attached around the vehicle.

[0012] The driving support device 100 is configured by a computer having a CPU (Central Processing Unit), memory, I / O (Input / Output: Input / Output device), etc., in which a predetermined process is programmed and the process is repeatedly executed at a predetermined period T. In this embodiment, the CPU and the memory cooperate to realize the following functions.

[0013] As shown in FIG. 1 , the driving assistance device 100 includes a sensor information acquisition unit 101, a surrounding sensor map acquisition unit 102, a map similarity search unit 103, an assistance instruction unit 104, an operation information acquisition unit 105, an assistance determination unit 106, an assistance pattern update unit 107, and an assistance pattern storage unit 108.

[0014] The sensor information acquisition unit 101 acquires distances to objects around the vehicle. For example, as shown in Fig. 2A and 2B, the sensor information acquisition unit 101 acquires distances to objects around the vehicle and point clouds from sensors such as cameras, radars, LiDARs, and sonars attached to positions where objects around the vehicle 10 can be detected. In this embodiment, the sensor information acquisition unit 101 acquires distances to surrounding obstacles and point clouds from cameras attached around the vehicle.

[0015] FIG. 2A shows an example in which the vehicle 10 is equipped with five cameras, and FIG. 2B shows an example in which the vehicle 10 is equipped with seven cameras. In the case of a stereo camera, the distance and point cloud can be obtained by calculating the parallax. In the case of a monocular camera, it is possible to obtain the distance and point cloud by the principle of motion stereo as the vehicle moves. Hereinafter, the point cloud will be represented as PT[c][p]=(X, Y, Z). Here, c is the camera ID, p is the point cloud ID, and X, Y, and Z are the coordinate values ​​in the relative coordinate system from the sensor obtained as a result of measurement.

[0016] The surrounding sensor map acquisition unit 102 integrates the point clouds PT[c][p] obtained from the sensors to generate a map MP[x][y] that represents the surrounding terrain. In this embodiment, the map MP[x][y] is two-dimensional data. First, the distance point clouds PT[c][p] obtained from each sensor are converted into relative coordinates from the center of the vehicle based on the mounting information (external parameters) of each sensor on the vehicle. The mounting information on the vehicle is assumed to have been measured in advance.

[0017] Next, from the converted distance point cloud PTR[c][p], point cloud information that exists at a position higher than the road surface is extracted, and the relative coordinates (X, Y) excluding the height of the extracted point cloud are converted at a certain scale to (x, y) and plotted as a binary image pattern MP[x][y]. Figures 3A, 3B, and 3C show this process. In the environment of Figure 3A, if the vehicle 10 has the sensor configuration of Figure 2A, a sensing result such as that of Figure 3B is obtained. Figure 3C is a plot of all the point clouds obtained from these sensors. Furthermore, a map that accumulates time-series information, such as an occupancy probability map, may be used.

[0018] 1 is a RAM (Random Access Memory) area in which support patterns PTN[n] are stored. Here, n is an ID for appropriately managing multiple support patterns PTN when they are stored. The support pattern PTN[n] includes a validity flag V, a map MP[x][y], a support function ASS, a validity criterion CND, and a priority PR as a data structure.

[0019] The map similarity search unit 103 compares the similarity between the map MP[x][y] and the map MP[x][y] in the support pattern PTN[n]. If the similarity between the map MP and any map MP in the support pattern PTN[n] is high, the support instruction unit 104 is notified to implement the corresponding support content. The details of the process will be described later.

[0020] The assistance instruction unit 104 instructs another controller or actuator in the vehicle to implement driving assistance corresponding to the assistance function ASS in the assistance pattern PTN[n] determined by the map similarity search unit 103 to have a high similarity.

[0021] The operation information acquisition unit 105 acquires information regarding the operation of the steering wheel, accelerator, and brake by the driver. This information may be acquired by directly inputting a sensor signal to the driving support device 100, or by performing communication using a LAN (Local Area Network).

[0022] The assistance determination unit 106 determines whether driving assistance is necessary from the driving operation acquired from the operation information acquisition unit 105. Whether driving assistance is necessary is determined according to a list that is stored in advance, which indicates what assistance is necessary in response to what operation, such as the need for warning assistance in the case of a sudden braking operation. If it is determined that driving assistance is necessary, the map MP generated by the surrounding sensor map acquisition unit 102 is acquired, and the map MP is notified to the assistance pattern update unit together with the corresponding assistance content. The details of the process will be described later.

[0023] When notified by the support determination unit 106, the support pattern update unit 107 registers the map and the support contents in the support pattern storage unit. In addition, the support pattern update unit 107 determines the effectiveness of the support contents, and for support contents that are determined to be ineffective, it also performs processing to invalidate the support pattern in the support pattern storage unit. The details of the processing will be described later.

[0024] [Explanation of the process of the map similarity search unit 103] The contents of the processing in the map similarity search unit 103 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of processing in the map similarity search unit 103.

[0025] First, in step S401, the orientation of the map MP is corrected. This process is for the purpose of unifying the orientation of the vehicle in a predetermined direction so that the similarity between maps can be calculated even if the orientations are different, in the case where the map MP is generated in a coordinate system in which the orientation of the vehicle is fixed in order to shorten the processing time when accumulating time-series information such as an occupancy probability map. In this embodiment, the map MP is corrected so that the vehicle faces upward.

[0026] Next, steps S402 to S405 are performed for all support patterns PTN[n], n = 1 to N. First, in step S402, a support pattern PTN[n] is obtained from the support pattern storage unit 108. As described above, the support pattern PTN[n] has, as data, a validity flag V, a map MP, a support function ASS, a validity determination criterion CND, and a priority PR.

[0027] Furthermore, in step S403, it is determined whether the support pattern PTN[n] is valid using a valid flag V of the support pattern PTN[n]. The valid flag V is set by the support pattern update unit 107 described later, and if the support pattern is valid, the process proceeds to step S404, and if the support pattern is invalid, the process returns to step S402 to read the next support pattern.

[0028] If the support pattern is valid, the similarity SML[n] between the map MP in the support pattern PTN[n] and the surrounding sensor map MP is calculated in step S404. Because the map MP in the support pattern PTN[n] and the surrounding sensor map MP are two-dimensional image patterns, the similarity calculation is performed using a method for calculating the similarity of simple images such as SAD or SSD.

[0029] The similarity SML[n] obtained as a result of the calculation is calculated such that the more similar the two patterns are, the closer to 1 they are, and the more dissimilar they are, the closer to 0 they are. Note that the similarity calculation may also use a method of detecting feature points such as SIFT to handle subtle differences in shape, and calculating the similarity between the feature points.

[0030] In step S405, it is determined whether the similarity SML[n] is equal to or greater than the threshold value TH_SML. If the similarity SML[n] is equal to or greater than the threshold value, the process proceeds to step S406, where the support content is registered in a table TBL for notifying the support determination unit 106 to execute the support function ASS in the support pattern PTN[n].

[0031] After steps S402 to S406 are performed for all n=1 to N, in step S407, the registered contents of table TBL for notifying the support determination unit 106 are checked. If multiple support contents are registered, the support determination unit 106 is notified to implement the support content with the highest priority based on the priority PR of the support contents. The priority PR is set according to the support content, with a high priority being set for intervention in vehicle control such as deceleration and a low priority being set for only notifications such as warnings.

[0032] [Explanation of the processing by the support determination unit 106] 5A and 5B, the contents of the processing in the support determining unit 106 will be described. FIG 5A is a flowchart showing the flow of processing in the support determining unit 106.

[0033] First, in step S501, a support target operation list LST to be the target of support judgment is obtained. The support target operation list LST describes operation criteria TRG (operation information), support functions ASS (support contents), validity judgment criteria CND, and priority PR. In this embodiment, as shown in Fig. 5B, it is assumed that sudden braking, driving at a speed lower than the speed limit, and a certain value or more of response frequency to an obstacle approach warning are registered as operation criteria in the support target operation list LST.

[0034] Next, in step S502, it is determined whether an operation that matches the operation criteria TRG of the support target operation list LST is detected. If it is detected, the process proceeds to step S503. If it is not detected, the process waits until the next processing cycle.

[0035] In step S503, the surrounding sensor map MP generated by the surrounding sensor map acquisition unit 102 is acquired. Next, in step S504, a shape determination of the surrounding sensor map MP is performed. The shape determination determines the complexity of the map MP. For example, if there are no objects or only a few objects in the map, the map MP will have less information, and erroneous detection by the map similarity search unit 103 will be more likely to occur.

[0036] For this reason, the map MP is quantified based on the ratio of pixels where objects exist, the complexity used in texture analysis, etc., and if this value is equal to or greater than a certain value in step S505, the process proceeds to step S506, where a support pattern PTN consisting of a set of the map MP, the support content ASS written in the support target operation list LST, and the effectiveness judgment criterion CND is notified to the support pattern update unit 107.

[0037] [Explanation of the processing of the support pattern update unit 107] The contents of the processing in the support pattern update unit 107 will be described with reference to Fig. 6, Fig. 7A, and Fig. 7B. Fig. 6 is a flowchart showing the processing flow of the support pattern update unit 107. Fig. 7A and Fig. 7B are flowcharts showing the processing flow related to the update method of the support pattern update unit 107.

[0038] First, in step S601, it is confirmed whether or not there has been a notification from step S506 of the support determination unit 106, and if there has been a notification, the process proceeds to step S602. If there has not been a notification, the process proceeds to step S604.

[0039] In step S602, the map MP is acquired and the orientation is corrected. This is the same as the process in step S401 described above. When accumulating time-series information such as an occupancy probability map, if the map MP is generated in a coordinate system in which the orientation of the vehicle is fixed in order to shorten the processing time, this process unifies the orientation of the vehicle in a predetermined direction so that the similarity between maps can be calculated even if the orientations are different. In this embodiment, the map MP is corrected so that the vehicle faces upward.

[0040] Next, in step S603, the map MP, the support content ASS, and the validity determination criterion CND are registered as a support pattern PTN[a] in an addition table TBL_A to be added to the support pattern storage unit .

[0041] Step S604 and subsequent steps are a process for invalidating the registered assistance pattern. First, in step S604, driver operation information is acquired from the operation information acquisition unit 105. Next, in step S605, a surrounding sensor map MP is acquired.

[0042] In step S606, it is determined whether or not a support instruction has been given by the support instruction unit 104. If a support instruction has been given, the process proceeds to step S607. If a support instruction has not been given, the process waits until the next processing cycle.

[0043] In step S607, the implemented support content ASS and the effectiveness determination criterion CND are obtained from the support pattern storage unit 108. Then, in step S608, the effectiveness of the support instruction is determined based on the effectiveness determination criterion CND.

[0044] The validity of the assistance instruction differs depending on the assistance content, but in the case of an assistance instruction that issues a warning, if the brake is applied after the warning timing, the assistance instruction is determined to be valid, and if the brake is applied before the warning timing, the assistance instruction is determined to be invalid. This may be a binary value of valid / invalid, or may be a continuous value from 0.0 to 1.0, for example, where the closer to 1.0 the value is, the more valid the instruction is.

[0045] In step S609, if the assistance instruction effectiveness is used and it is determined that the effectiveness is low, the process proceeds to step S610, otherwise the process waits until the next processing cycle.

[0046] In step S610, the map MP, the support content ASS, and the validity determination criterion CND of the corresponding support pattern in the support pattern storage unit 108 are registered in the invalidation table TBL_I as the invalidated support pattern PTN[b].

[0047] After the above processing is performed, the results are reflected in the support pattern storage unit 108 according to the method shown in Figures 7A and 7B. Figures 7A and B each show a different reflection method.

[0048] The first registration method will be described with reference to Fig. 7A. First, in step S701, it is determined whether the number of registrations in the additional table TBL_A and the invalidation table TBL_I is equal to or greater than a certain number, and if it is equal to or greater than the certain number, step S702 is performed. If not, the process waits until the next processing cycle.

[0049] In step S702, the content of the additional table TBL_A is registered as a new support pattern in the support pattern storage unit 108. Also, the content of the invalidation table TBL_I is extracted from the support pattern storage unit 108, and an invalidation flag is added.

[0050] Next, a different registration method will be described with reference to Fig. 7B. First, in step S703, it is determined whether the number of registrations in the additional table TBL_A and the invalidation table TBL_I is equal to or greater than a certain number, and if it is equal to or greater than the certain number, step S704 is performed. If not, the process waits until the next processing cycle.

[0051] In step S704, the content of the additional table TBL_A is registered as a new support pattern in the support pattern storage unit 108. Next, in step S705, the content of the invalidation table TBL_I is stored in the invalidation pattern storage unit 109. The invalidation pattern storage unit 109 is a storage area separate from the support pattern storage unit 108.

[0052] Then, in step S706, clustering is performed using the shape of the map MP in the support pattern storage unit 108 and in the invalidation pattern storage unit 109. As a result of the clustering, a group PTG[j] of similar patterns is generated in the support pattern storage unit 108. Similarly, a group PTIG[k] of similar patterns is generated in the invalidation pattern storage unit 109. The similarity between the group PTG[j] in the support pattern storage unit 108 and the group PTIG[k] in the invalidation pattern storage unit 109 is also calculated.

[0053] In step S707, it is first determined whether the number of patterns of group PTG[j] in the support pattern storage unit 108 is equal to or greater than a certain number. If the number is equal to or greater than the certain number, the process proceeds to step S708. If the number is less than the certain number, the process proceeds to step S710.

[0054] In step S708, the group PTIG[k] in the invalidation pattern storage unit 109 that has a high similarity to the group PTG[j] in the support pattern storage unit 108 calculated in step S706 is referenced, and it is determined whether the number of patterns is equal to or greater than a certain number. If the similarity to the group PTG[j] in the support pattern storage unit 108 is less than a certain number, the number of patterns is determined to be 0. As a result, if the number is equal to or greater than the certain number, the process proceeds to step S710. If the number is less than the certain number, the process proceeds to step S709.

[0055] Step S709 is performed when the number of patterns of group PTG[j] in support pattern storage unit 108 is equal to or greater than a certain number, and the number of patterns of group PTIG[k] in the corresponding invalidation pattern storage unit 109 is less than a certain number. The validity flag V of support pattern PTN[n] belonging to group PTG[j] in support pattern storage unit 108 is set to "valid".

[0056] Step S710 is performed when the number of patterns of group PTG[j] in support pattern storage unit 108 is less than a certain number, or the number of patterns of group PTIG[k] in the corresponding invalidation pattern storage unit 109 is equal to or greater than a certain number. The validity flag V of support pattern PTN[n] belonging to group PTG[j] in support pattern storage unit 108 is set to "invalid."

[0057] A specific example of the first embodiment described above will be described with reference to Figures 8A to 8F. Figure 8A shows an example in which a pedestrian 20 suddenly jumps out and the driver suddenly brakes when the vehicle 10 enters an intersection with poor visibility. The map MP at the time when the driver suddenly brakes is stored as an assistance pattern PTN[n] as shown in Figure 8B. Then, when the vehicle approaches a similar intersection, an alarm can be issued as the assistance function ASS, and the driver can be alerted.

[0058] Also, Fig. 8C is an example of a sudden drop in speed when the driver is driving on a narrow road. By acquiring the speed of the road from the navigation system or the sign recognition function of the external recognition sensor, it is compared with the vehicle speed, and it is detected that the speed is below a certain rate, and the map MP is stored as shown in Fig. 8D. Then, when entering a similar road, an alarm can be issued as an assistance function ASS, and the driver can be alerted.

[0059] Also, Fig. 8E shows an example where the driver drives close to the wall when turning at a narrow intersection. The system detects the distance to an obstacle, and if the distance is less than a certain value, issues an obstacle approach warning, and the map MP is stored as shown in Fig. 8F by acquiring the information of the system as the driver's operation information. Then, when entering a similar road, the support function ASS issues an obstacle approach warning earlier than usual, thereby encouraging the driver to turn earlier.

[0060] The features of this embodiment can be summarized as follows.

[0061] The driving assistance device includes at least a memory (FIG. 1, assistance pattern storage unit 108) and a processor (CPU). The memory stores an assistance pattern PTN including a first surrounding sensor map (map MP) showing a map of the surroundings of a vehicle (host vehicle 10) and assistance contents (assistance function ASS) of the vehicle driving assistance corresponding to the first surrounding sensor map (map MP).

[0062] The processor (sensor information acquisition unit 101) acquires sensor information indicating information of objects around the vehicle sensed by a sensor. The processor (surrounding sensor map acquisition unit 102) acquires a second surrounding sensor map (map MP) based on the sensor information. The processor (map similarity search unit 103, assistance instruction unit 104) performs driving assistance with assistance content (assistance function ASS) corresponding to the first surrounding sensor map (map MP) when the second surrounding sensor map (map MP) is similar to the first surrounding sensor map (map MP). This makes it possible to perform driving assistance when the vehicle travels in a place similar to a place requiring driving assistance.

[0063] The memory stores operation information (FIG. 5B, operation criterion TRG) indicating an operation state requiring driving assistance, and a list (assistance target operation list LST) including assistance contents (assistance function ASS) corresponding to the operation information (operation criterion TRG). When an operation state indicated by the operation information (operation criterion TRG) is detected, the processor (assistance determination unit 106, assistance pattern update unit 107) registers / updates an assistance pattern PTN with a set of the second surrounding sensor map (map MP) and the assistance contents (assistance function ASS) corresponding to the operation information (operation criterion TRG). As a result, the assistance contents are associated with the first surrounding sensor map in response to the detection of an operation state requiring driving assistance.

[0064] The operation state indicated by the operation information (operation criterion TRG) is, for example, a state in which a sudden braking operation has been performed (row of priority 1 in FIG. 5B). The assistance content (assistance function ASS) corresponding to the operation information (operation criterion TRG) is, for example, an alarm or control to suppress deceleration or acceleration. As a result, when a sudden braking operation is detected, the assistance content is associated with the first surrounding sensor map. Then, when the vehicle travels in a place similar to a place where a sudden braking operation was detected in the past, an alarm or control to suppress deceleration or acceleration can be performed as driving assistance.

[0065] Moreover, the operation state indicated by the operation information (operation criterion TRG) is, for example, a state in which the traveling speed of the vehicle (host vehicle 10) is slower than the speed limit at the traveling position of the vehicle (row of priority 2 in FIG. 5B). The assistance content (assistance function ASS) corresponding to the operation information (operation criterion TRG) is, for example, an alarm or control of deceleration or acceleration suppression. As a result, when a state in which the speed is slower than the speed limit is detected, the assistance content is associated with the first surrounding sensor map. Then, when the vehicle travels in a place similar to a place where a state in which the speed is slower than the speed limit was detected in the past (difficult scene), an alarm (suggestion to reduce speed) or control of deceleration or acceleration suppression can be performed as driving assistance.

[0066] Furthermore, the operation state indicated by the operation information (operation criterion TRG) is, for example, a state in which the frequency of driving assistance is higher than a predetermined value (row of priority 3 in FIG. 5B). The assistance content (assistance function ASS) corresponding to the operation information (operation criterion TRG) is, for example, a change in a parameter that adjusts the driving assistance function. As a result, when a state in which the frequency of driving assistance is higher than a predetermined value is detected, the assistance content is associated with the first surrounding sensor map. Then, when the vehicle travels in a place similar to a place where a state in which driving assistance (for example, an obstacle approach warning due to driving close to an obstacle) was frequently detected in the past, a change in a parameter that adjusts the driving assistance function (for example, a change in a parameter to issue an obstacle approach warning earlier) can be performed as driving assistance.

[0067] In this embodiment, the processor (FIG. 1, surrounding sensor map acquisition unit 102) generates a second surrounding sensor map (map MP) based on the sensor information. This allows necessary driving assistance to be performed even if there is a change in the road shape, obstacles, etc.

[0068] The processor (FIG. 1, assistance pattern update unit 107, map similarity search unit 103, assistance instruction unit 104) clusters the first surrounding sensor map (map MP) (FIG. 7B, S706), and if the number of first surrounding sensor maps (map MP) in the cluster is equal to or greater than a certain number (FIG. 7B, S707: YES), performs driving assistance with assistance content (assistance function ASS) corresponding to the first surrounding sensor map (map MP) in the cluster. This improves the accuracy of, for example, locations requiring driving assistance.

[0069] In this embodiment, the list (FIG. 5B, support target operation list LST) includes a criterion (validity determination criterion CND) for determining whether the driving support corresponding to the support content (support function ASS) is valid or invalid. The processor (support pattern update unit 107) determines whether the driving support is valid or invalid based on the criterion (validity determination criterion CND) corresponding to the support content (support function ASS) that performed the driving support (FIG. 6, S608).

[0070] If the number of first surrounding sensor maps (map MP) corresponding to the support content determined to be invalid within the cluster is less than a certain number (FIG. 7B, S708: NO), the processor (support pattern update unit 107, map similarity search unit 103, support instruction unit 104) performs driving support of the support content (support function ASS) corresponding to the first surrounding sensor map (map MP).

[0071] On the other hand, if the number of first surrounding sensor maps corresponding to the support contents determined to be invalid in the cluster is equal to or greater than a certain number (FIG. 7B, S708: YES), the processor (support pattern update unit 107, map similarity search unit 103, support instruction unit 104) does not provide driving support of the support contents corresponding to the first surrounding sensor map. This allows the driver to receive only driving support that is valid.

[0072] As described above, this is a system that generates a map MP using distance information of objects detected around the vehicle, and provides driving assistance based on the similarity between that map and a similar assistance pattern PTN[n].The need for driving assistance is determined based on the driver's driving operation, and the map MP when a driving operation that satisfies the conditions is performed and the assistance content corresponding to the driving operation are registered as an assistance pattern PTN[n], making it possible to provide a system that can provide appropriate driving assistance based on the driver's past experience.

[0073] <Embodiment 2> Next, a second embodiment of the in-vehicle driving assistance device of the present invention will be described below with reference to the drawings.

[0074] 9 is a block diagram showing the configuration of a driving support device 200 in embodiment 2. In the following description, only the parts different from the driving support device 100 in embodiment 1 described above will be described in detail, and the same parts will be assigned the same numbers and detailed description thereof will be omitted.

[0075] This embodiment is characterized in that it also takes into consideration information about surrounding obstacles obtained from a target recognition unit 201, and information about the time, date, and direction obtained from an environmental information acquisition unit 202.

[0076] The target recognition unit 201 recognizes targets OBJ[m] such as surrounding vehicles, motorcycles, and pedestrians from the point cloud PT[c][p] obtained from the sensor. The target OBJ[m] has a data structure including relative coordinates (X, Y) from the vehicle itself and a type of target (vehicle, pedestrian, motorcycle).

[0077] In this embodiment, a camera is used as a sensor, and the result of the pattern recognition of the camera is acquired, and a point cloud identified as a vehicle, a two-wheeled vehicle, or a pedestrian is recognized as a target OBJ[m]. FIGS. 10A to 10D are examples of the processing contents (detection results) of the target recognition unit 201. In an environment such as that of FIG. 10A, if a camera that recognizes the surroundings of the vehicle 10 as shown in FIG. 10B is mounted, the camera performs pattern recognition as shown in FIG. 10C, and acquires a target OBJ[m] as shown in FIG. 10D. As a result, when the similarity SML is calculated in step S404 of the map similarity search unit 103, processing that takes into account not only the information of the surrounding sensor map but also the information of the target is possible.

[0078] The environmental information acquisition unit 202 acquires information such as date, time, and direction. This information is obtained through communication with other in-vehicle devices such as a navigation system, or wireless communication with the outside of the vehicle. As a result, when the map similarity search unit 103 calculates the similarity SML in step S404, it is possible to perform processing that takes this information into account in addition to the information from the surrounding sensor map.

[0079] Regarding this effect, first, the effect of using a target will be described. In this embodiment, the support pattern PTN[n] includes a positional relationship POS with the target closest to the vehicle among the target OBJ[m]. The positional relationship POS has three values, namely, the front, the side, and the rear, as seen from the vehicle, and the map similarity search unit 103 compares the positional relationship POS in the support pattern PTN[n] with the positional relationship POS with the closest target among the detected target OBJ[l], and calculates a target similarity SML_O that is 0 if they do not match and 1 if they match.

[0080] The similarity SML with the surrounding sensor map is multiplied by the target similarity SML_O to obtain the similarity SML. This allows assistance to be performed when the positional relationship with the target is similar.

[0081] The effect will be described with reference to Figures 11A, 11B, and 11C. Figure 11A shows a scene in which a pedestrian is present ahead while driving on a narrow road, and the pedestrian suddenly approaches the road, causing the driver to apply the brakes suddenly. In this case, as shown in Figure 11B, by storing the positional relationship POS with the target object in the support pattern PTN[n] together with the surrounding terrain information, as shown in Figure 11C, when the terrain is similar but there is no target object ahead, the similarity becomes low, making it possible to provide more appropriate support.

[0082] Next, the effect of using the environmental information will be described. In this embodiment, the support pattern PTN[n] includes the time TM when the support pattern is stored. The map similarity search unit 103 calculates a time similarity SML_T such that the closer the difference between the time TM in the support pattern PTN[n] and the current time TM is, the closer it is to 1.0, and the closer it is to 0.0. Then, the similarity SML with the surrounding sensor map is multiplied by the time similarity SML_T, and the result is used as the similarity SML. This allows support to be performed in a time zone close to the timing when the support pattern was registered.

[0083] Furthermore, by using the date, it is possible to determine the times of sunrise and sunset, which makes it possible to realize functions such as support during times when visibility for drivers is said to be reduced, such as dawn or dusk, or support only at night, rather than simply determining whether the times are close.

[0084] When storing a support pattern, by further registering the date in the date DT, it is possible to classify the pattern into four patterns, for example, daytime, dawn, dusk, and nighttime, together with the time TM. A similarity SML_S is determined which is 1.0 if it matches the classification results of the current date DT and time TM into daytime, dawn, dusk, and nighttime, and 0.0 if it does not match.

[0085] The similarity SML with the surrounding sensor map is multiplied by the date and time similarity SML_S to obtain the similarity SML. This allows assistance to be performed in a time period close to the timing when the assistance pattern is registered.

[0086] In addition, by using the vehicle's direction, the vehicle's orientation relative to the position of the sun estimated from the date and time is taken into consideration, making it possible to realize support in a backlit situation, for example. When storing a support pattern, a direction DR is further registered with north as 0, and then, in combination with the date DT and time TM, a direction similarity SML_D is calculated, which is 1.0 when the absolute difference between the direction DR in the support pattern and the current direction DR is smaller, and 0.0 when the absolute difference is larger.

[0087] In addition, the distance between the height and direction of the sun estimated from the date DT and time TM in the support pattern and the height and direction of the sun estimated from the current date DT and time TM is calculated, and a solar position similarity SML_N is calculated, with the smaller the absolute difference, the larger the difference, being 1.0, and the larger the difference, the larger the solar position similarity, being 0.0.

[0088] The similarity SML with the surrounding sensor map is multiplied by the orientation similarity SML_D and the solar position similarity SML_N described above, and the result is used as the similarity SML. This allows assistance to be performed when the solar position and the vehicle direction are close to the timing when the assistance pattern was registered.

[0089] Furthermore, by acquiring weather information, it becomes possible to realize support that takes the weather into consideration. Similarly, the weather WT is registered in the support pattern. The weather WT may be, for example, four patterns: sunny, cloudy, rainy, and snowy. Then, the weather similarity SML_W is calculated, which is 1.0 if the weather WT in the support pattern matches the current weather WT, and 0.0 if not.

[0090] The similarity SML with the surrounding sensor map is multiplied by the weather similarity SML_W to obtain the similarity SML. This allows assistance to be performed when the weather is close to that at the time when the assistance pattern was registered.

[0091] The features of this embodiment can be summarized as follows.

[0092] The processor (FIG. 9, environment information acquisition unit 202) acquires environment information indicating the environment around the vehicle (host vehicle 10), including at least time. When registering / updating an assistance pattern PTN, the processor (assistance pattern update unit 107) includes the environment information in the assistance pattern. When the second surrounding sensor map (map MP) is similar to the first surrounding sensor map (map MP) and the environment indicated by the environment information is detected, the processor (map similarity search unit 103, assistance instruction unit 104) performs driving assistance of the assistance content (assistance function ASS) corresponding to the first surrounding sensor map (map MP). This makes it possible to determine whether to perform driving assistance depending on the environment when the vehicle travels through a place similar to a place requiring driving assistance.

[0093] The environmental information includes, for example, direction information indicating the direction of the vehicle (host vehicle 10) (FIG. 9). This makes it possible to determine whether or not to perform driving assistance depending on, for example, the time when the sun is dazzling (backlit) and the direction.

[0094] The processor (FIG. 9, target recognition unit 201) recognizes targets OBJ around the vehicle (host vehicle 10) based on sensor information. When registering / updating a support pattern, the processor (support pattern update unit 107) includes positional relationship information indicating a positional relationship between the vehicle and the target in the support pattern. When the second surrounding sensor map (map MP) is similar to the first surrounding sensor map (map MP) and the positional relationship indicated by the positional relationship information is detected, the processor (map similarity search unit 103, support instruction unit 104) performs driving support of the support content (assistance function ASS) corresponding to the first surrounding sensor map (map MP). This makes it possible to determine whether to perform driving support depending on the positional relationship between the vehicle and the target when the vehicle travels in a place similar to a place requiring driving support.

[0095] <Embodiment 3> Next, a third embodiment of an in-vehicle driving assistance device according to the present invention will be described below with reference to the drawings.

[0096] Fig. 12 is a block diagram showing the configuration of a driving support device 300 in embodiment 3. In the following description, only the parts different from the driving support device 100 in embodiment 1 and the driving support device 200 in embodiment 2 will be described in detail, and the same parts will be assigned the same numbers and detailed descriptions thereof will be omitted.

[0097] A feature of this embodiment is that the driver's selection obtained from the driver input unit 301 is taken into consideration.

[0098] 13A and 13B explain the process of registering the result of the selection made by the driver input unit 301 in the assistance pattern storage unit 108. A pre-pattern storage unit 302 is prepared in the driving assistance device 300, and tables such as those shown in Figs. 13A and 13B are stored therein.

[0099] The table shown in Fig. 13B includes information required for registering the support pattern information such as the map MP, the positional relationship with the target POS, the time TM, the date DT, the direction DR, etc., and the table shown in Fig. 13A includes the name of the difficult scene that verbalizes this information and whether support is required. Note that in the examples of Figs. 13A and 13B, the records of both tables are linked by the value of the "No" field.

[0100] The name of the difficult scene and whether or not support is required are displayed at the first startup or on the setting screen (display) of the car, and further, whether or not support is required can be switched by input from the driver. As a result of this selection, the contents of the advance pattern storage unit 302 are registered in the support pattern storage unit 108.

[0101] The assistance pattern update unit 107 also cooperates with the driver input unit 301. Fig. 14 shows a process flow when the assistance pattern update unit 107 uses the input from the driver input unit 301.

[0102] First, in step S1401, it is determined whether driving of the vehicle has been completed. If driving has been completed, the process proceeds to step S1402. If driving is in progress, the process waits until the next processing cycle.

[0103] Next, in step S1402, it is determined whether the number of registrations in the additional table TBL_A and the invalidation table TBL_I is equal to or greater than a certain number, and if it is equal to or greater than the certain number, step S1403 is performed. If not, the process waits until the next processing cycle.

[0104] In step S1403, the support pattern PTN registered in the addition table TBL_A and the invalidation table TBL_I is verbalized or visualized. For verbalization, for example, a machine learning method that pre-learns combinations of patterns and languages ​​is used. In this embodiment, a map MP within the support pattern PTN is displayed as visualization.

[0105] In step S1404, the driver uses the driver input unit 301 to input whether or not the driver desires the assistance for the assistance pattern PTN on the additional table TBL_A visualized in step S1403.

[0106] If the input result indicates that support is desired, proceed to step S1405, and the added pattern is registered in the support pattern storage unit 108. If not, perform the process of step S1404 for the next support pattern PTN on the addition table TBL_A. When the process is completed for all support patterns PTN in the addition table TBL_A, proceed to step S1406.

[0107] In step S1406, the driver uses the driver input unit 301 to input whether or not the driver wishes to disable the assistance pattern PTN on the disablement table TBL_I visualized in step S1403.

[0108] If the input result indicates that the user wishes to disable the support, the process proceeds to step S1407, where the validity flag V of the corresponding support pattern in the support pattern storage unit 108 is set to invalid, or the pattern is registered in the invalidation pattern storage unit 109. If not, the process of step S1406 is performed for the next support pattern PTN on the invalidation table TBL_I. When the process has been completed for all support patterns PTN in the invalidation table TBL_I, the process ends.

[0109] The features of this embodiment can be summarized as follows.

[0110] The driving assistance device 300 (FIG. 12) includes an input unit (driver input unit 301) that receives the driver's selection as an input. A memory (pre-arrangement pattern storage unit 302) stores a pre-arrangement assistance pattern including a first surrounding sensor map (map MP) and assistance contents (assistance function ASS) of driving assistance of a vehicle (host vehicle 10) corresponding to the first surrounding sensor map (map MP). A processor (assistance pattern update unit 107) registers / updates the assistance pattern with the pre-arrangement assistance pattern selected by the driver. This allows the driver to register necessary assistance patterns in advance.

[0111] The processor (assistance pattern update unit 107) also presents a set of a second surrounding sensor map (map MP) and an assistance content (assistance function ASS) corresponding to the operation information to the driver (FIG. 14, S1402), and registers / updates the assistance pattern with the set selected by the driver. This allows the driver to select a place requiring driving assistance (difficult scene) from the proposed set.

[0112] As a result, the driver can select an assistance pattern detected at the time of initial startup or automatically, or an assistance pattern to be disabled, so that assistance more appropriate for the driver can be provided.

[0113] <Embodiment 4> Next, a fourth embodiment of the in-vehicle driving assistance device of the present invention will be described below with reference to the drawings.

[0114] Fig. 15 is a block diagram showing the configuration of a driving support device 400 in embodiment 4. In the following description, only the points different from the driving support device 100 in embodiment 1, the driving support device 200 in embodiment 2, and the driving support device 300 in embodiment 3 will be described in detail, and the same points will be denoted by the same numbers and detailed description thereof will be omitted.

[0115] A characteristic feature of this embodiment is that it has a sensor map storage unit 403, an in-map position estimation unit 401 that estimates a position on a sensor map in the sensor map storage unit, and a surrounding sensor map acquisition unit 402 that extracts the surrounding sensor map at the estimated position on the sensor map.

[0116] The sensor map storage unit 403 is an area for storing a sensor map MP_G that is a combination of the surrounding sensor map MP generated at various locations and stitched together. The stitching is performed by combining a method such as a Global Navigation Satellite System (GNSS) that acquires the absolute position of the vehicle and a dead reckoning that calculates the relative position of the vehicle. The sensor information acquired by the sensor information acquisition unit 101 may be stitched together using a technology such as SLAM.

[0117] As a result, the sensor map MP_G is stored in association with the position information. In this embodiment, the sensor map MP_G is stored by piecing together surrounding sensor maps MP generated when the vehicle has traveled in the past. In this case, maps are generated in advance for roads that are usually used, so driving assistance can be performed in situations where the vehicle is usually used. Also, sensor maps of the main environment may be stored at an initial stage, or may be obtained from an external source via communication. In this case, driving assistance is possible even in places where the vehicle is traveling for the first time.

[0118] The intra-map position estimation unit 401 acquires the position information of the vehicle and estimates the position LCL of the vehicle within the sensor map MP_G. The position information of the vehicle is acquired by combining the GNSS that acquires the absolute position of the vehicle with a method such as dead reckoning that calculates the relative position of the vehicle.

[0119] GNSS can obtain absolute positions within the sensor map MP_G, but the processing cycle is slow, and when the radio wave conditions are poor, the position cannot be obtained with sufficient accuracy. Therefore, dead reckoning, which can process at a relatively high cycle, is used in combination, and between GNSS reception cycles or when the radio wave conditions are poor, the relative position is calculated by dead reckoning from the last received GNSS position to estimate the position on the sensor map MP_G. Alternatively, it is also possible to generate a provisional surrounding sensor map MP_T using the sensor information obtained from the sensor information acquisition unit 101, and estimate the position by map matching between the provisional surrounding sensor map MP_T and the sensor map MP_G.

[0120] The surrounding sensor map acquisition unit 402 extracts a sensor map MP_G around the vehicle position LCL from the vehicle position LCL and the sensor map MP_G, and sets the sensor map MP_G as the surrounding sensor map MP. The sensor map MP_G cut out from the vicinity of the vehicle position LCL may be used as the surrounding sensor map MP as it is, but it is preferable to integrate a provisional surrounding sensor map MP_T and the sensor map MP_G cut out from the vicinity of the vehicle position LCL, use the surrounding sensor map MP_T for the vicinity, and use information cut out from the sensor map MP_G for the outside of the sensing range of the surrounding sensor map MP_T.

[0121] 16A, 16B, and 16C are schematic diagrams for explaining the effect of the surrounding sensor map acquisition unit 402. As shown in Fig. 16A, when a two-wheeled vehicle 22 or a vehicle 21 is present around the vehicle, the provisional surrounding sensor map MP_T generated using the sensor information acquired from the sensor information acquisition unit 101 has a sensing range limited by targets as shown in Fig. 16B, and topographical information behind the two-wheeled vehicle 22 or the vehicle 21 cannot be obtained. Note that the dotted circle or ellipse in Fig. 16B indicates a blind spot caused by targets.

[0122] However, by performing processing by the surrounding sensor map acquisition unit 402 shown in this embodiment, even if the sensing range is limited by the two-wheeled vehicle 22 or the vehicle 21, surrounding topographical information can be acquired as shown in FIG. 16C.

[0123] As described above, by using the sensor map storage unit 403, a map with a wider range than the provisional surrounding sensor map MP_T generated using the sensor information acquired from the sensor information acquisition unit 101 can be used.

[0124] In addition, the sensor map storage unit 403 can store a driving route PTH of the vehicle if it is a route that has been traveled in the past, and more appropriate driving support can be achieved by using the driving route PTH. Figures 17A to 17D are schematic diagrams for explaining the contents. The driving route PTH may be acquired from a navigation system or the like, and may be superimposed on the surrounding sensor map MP_G.

[0125] Fig. 17A shows a scene in which the vehicle passes through an intersection with poor visibility. When the driver suddenly brakes the vehicle while trying to turn left from such an intersection, the surrounding sensor map MP and the like are stored in the assistance pattern storage unit 108 as shown in Fig. 17B.

[0126] Furthermore, when driving in a similar environment, by having the vehicle's driving route PTH on the sensor map MP_G, in a scene where the vehicle passes through an intersection with poor visibility, not only can a warning be issued as shown in embodiments 1 and 2, but also assistance can be provided such as stopping the vehicle just before the intersection and suppressing acceleration when starting off in the vicinity of the intersection.

[0127] 17C shows a scene in which the vehicle passes through a narrow road. When the driver drives straight through such a narrow road at a speed sufficiently lower than the speed limit, the surrounding sensor map MP and the like are stored in the support pattern storage unit 108 as shown in FIG. 17D.

[0128] Furthermore, when driving in a similar environment, by having the vehicle's driving route PTH on the sensor map MP_G, on such narrow roads, not only can an alarm be issued as shown in embodiments 1 and 2, but also assistance (vehicle control) can be provided such as generating a steering reaction force if the vehicle deviates from a straight driving route and suppressing acceleration when starting in that vicinity.

[0129] Also, Fig. 18A shows a scene of turning on a narrow road. When the driver passes close to a wall while turning on such a narrow road, or when the driver judges that the vehicle cannot turn as it is and switches to reverse gear during turning, the surrounding sensor map MP and the like are stored in the assistance pattern storage unit 108 as shown in Fig. 18B. When driving in a similar environment, by having the vehicle's driving route PTH on the sensor map MP_G, assistance is possible when turning on such a narrow road, such as moving the steering wheel from the position where turning should start and suppressing acceleration when starting in the vicinity of the position, in addition to warnings as shown in the first and second embodiments.

[0130] 18C shows a scene in which the vehicle turns right at an intersection. When an oncoming vehicle is present and visibility in the oncoming lane is poor, if a motorcycle or the like suddenly appears from behind the oncoming vehicle and the vehicle suddenly brakes, the surrounding sensor map MP and the positional relationship POS with respect to the target object and the like are stored in the support pattern storage unit 108 as shown in FIG. 18D.

[0131] Furthermore, when driving in a similar environment, by having the vehicle's driving route PTH on the sensor map MP_G, when turning right at an intersection, sensor information can be utilized that allows the driver to see oncoming vehicles and motorcycles earlier than the driver's point of view, and warnings, acceleration suppression, braking, etc. can be implemented according to the approaching conditions.

[0132] The features of this embodiment can be summarized as follows.

[0133] The memory (FIG. 15, sensor map storage unit 403) stores a sensor map MP_G that is a combination of multiple second surrounding sensor maps (map MP). The processor (intra-map position estimation unit 401) estimates a position LCL of the vehicle (host vehicle 10) in the sensor map MP_G based on the sensor information. The processor (surrounding sensor map acquisition unit 402) acquires the second surrounding sensor map (map MP) from the sensor map MP_G at the estimated vehicle position LCL. This makes it possible to utilize, for example, the second surrounding sensor map (map MP) of a place that has been traveled in the past. In addition, the traveling route PTH of the host vehicle can be acquired from the position LCL of the vehicle (host vehicle 10) in the sensor map MP_G.

[0134] The operation state indicated by the operation information (operation standard TRG) is, for example, a state in which a reverse operation is performed while the vehicle (host vehicle 10) is turning (FIG. 18A). The assistance content (assistance function ASS) corresponding to the operation information (operation standard TRG) is, for example, control of the turning of the vehicle. As a result, when a state in which a reverse operation is performed while the vehicle is turning is detected, the assistance content is associated with the first surrounding sensor map. Then, when the vehicle travels through a place similar to the place where a reverse operation was performed while turning, the turning of the vehicle can be controlled as driving assistance.

[0135] As described above, by having the sensor map storage unit 403 and using the sensor map MP_G, the driving route PTH of the vehicle can be referred to, making it possible to provide assistance such as stopping assistance and steering assistance using the driving route PTH.

[0136] <Embodiment 5> Next, a fifth embodiment of the in-vehicle driving assistance device of the present invention will be described below with reference to the drawings.

[0137] Fig. 19 is a block diagram showing the configuration of a driving support device 500 in embodiment 5. In the following description, only the points different from the driving support device 100 in embodiment 1, the driving support device 200 in embodiment 2, the driving support device 300 in embodiment 3, and the driving support device 400 in embodiment 4 will be described in detail, and the same points will be denoted by the same numbers and detailed description thereof will be omitted.

[0138] A distinctive feature of this embodiment is that the similarity with the patterns in the assistance pattern storage unit 108 is calculated and registered in advance in the sensor map MP_G in the sensor map storage unit 403, and the assistance content acquisition unit 501 directly references the assistance content from the sensor map MP_G based on the host vehicle position LCL and notifies the assistance instruction unit 104.

[0139] As a result, by registering locations corresponding to the support pattern PTN in advance in the sensor map MP_G in the sensor map storage unit 403, the processing of the map similarity search unit while the vehicle is traveling can be omitted, thereby reducing the processing load.

[0140] In the above-described support, the alarm is not limited to a simple sound alarm, but may include an alarm using LED lighting, a display on a monitor inside the vehicle, and even, if the sensor installed in the vehicle is a camera, displaying camera images on a monitor.

[0141] The present invention is not limited to the above-described embodiment, and includes various modified examples. For example, the above-described embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those including all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0142] In addition, the above configurations, functions, etc. may be realized in hardware, partly or in whole, by designing them as integrated circuits, etc. In addition, the above configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0143] The embodiment of the present invention may have the following aspects.

[0144] (1) A driving assistance device comprising: a sensor information acquisition unit that acquires information from an external recognition sensor mounted around the vehicle; a surrounding sensor map acquisition unit that acquires a sensor map around the vehicle based on the sensor information; an assistance pattern storage unit that stores an assistance pattern including a shape of the sensor map for which driving assistance control is to be implemented and assistance content corresponding to the sensor map shape; and a map similarity search unit that calculates a similarity between the acquired sensor map and the registered assistance pattern, and when the map similarity search unit determines that the sensor map has a high similarity to the sensor map shape for which driving assistance control is to be implemented and that the location is one where the driving assistance control should be implemented, an assistance instruction unit that instructs the execution of driving assistance of assistance content corresponding to the sensor map shape that has a high similarity to the sensor map.

[0145] (2) The driving assistance device described in (1) further includes a driving operation information acquisition unit that acquires driving operation information of a driver, and an assistance determination unit that determines whether driving assistance is necessary based on the driving operation information and the sensor map, and when the assistance determination unit determines that assistance is necessary, the driving assistance device further includes an assistance pattern update unit that registers / updates the sensor map shape and assistance content as a set in the assistance pattern storage unit, assuming that the corresponding sensor map shape indicates a situation in which driving assistance should be provided.

[0146] (3) A driving assistance device as described in (2), characterized in that the surrounding sensor map acquisition unit generates a sensor map of the surroundings of the vehicle based on sensor information acquired by the external environment recognition sensor.

[0147] (4) The driving assistance device described in (2) has a sensor map memory unit that stores the sensor map, and an in-map position estimation unit that estimates the vehicle's position within the sensor map based on the sensor information, and a surrounding sensor map acquisition unit that acquires a sensor map of the surroundings of the vehicle from the map stored in the sensor map memory unit based on the estimated position of the vehicle within the map.

[0148] (5) A driving assistance device as described in (2), further comprising an environmental information acquisition unit that acquires environmental information including at least time, and adds the environmental information to the assistance pattern and registers it in the assistance pattern storage unit.

[0149] (6) A driving assistance device as described in (2), further comprising an orientation information acquisition unit that acquires orientation information of the vehicle, and adds the orientation information to the assistance pattern and registers it in the assistance pattern storage unit.

[0150] (7) A driving assistance device as described in (2), characterized in that it has a target recognition unit that recognizes the mutual positional relationships of targets composed of dynamic object information around the vehicle based on the sensor information, and the positional relationship between the targets and the vehicle, and adds the positional relationship information of the targets to the assistance pattern and registers it in the assistance pattern storage unit.

[0151] (8) A driving assistance device as described in (3), characterized in that when the assistance determination unit detects a sudden braking operation based on the input of the operation information acquisition unit, the assistance pattern update unit registers and updates an alarm as driving assistance in a manner corresponding to the corresponding sensor map shape.

[0152] (9) A driving assistance device as described in (3), characterized in that when the assistance judgment unit acquires speed limit information at the vehicle's driving position and detects that the vehicle's driving speed acquired from the operation information acquisition unit is slower than the speed limit, the assistance pattern update unit registers and updates an alarm as driving assistance in a manner that corresponds to the corresponding sensor map shape.

[0153] (10) A driving assistance device as described in (3), wherein the assistance pattern update unit, when the assistance determination unit detects that the driving assistance function equipped in the vehicle is operated frequently based on the information acquired from the operation information acquisition unit, registers and updates, as driving assistance, the adjustment of parameters of the driving assistance function equipped in the vehicle in a manner that corresponds to the corresponding sensor map shape.

[0154] (11) A driving assistance device as described in (4), characterized in that when the assistance determination unit detects a sudden braking operation based on the input of the operation information acquisition unit, the assistance pattern update unit registers and updates an alarm, deceleration assistance, or acceleration suppression control as driving assistance in a manner corresponding to the corresponding sensor map shape.

[0155] (12) A driving assistance device as described in (4), characterized in that the assistance pattern update unit, when the assistance judgment unit acquires speed limit information at the vehicle driving position, detects that the speed acquired from the operation information acquisition unit is slower than the speed limit, and registers and updates an alarm, deceleration assistance, or acceleration suppression control as driving assistance in a manner corresponding to the corresponding sensor map shape.

[0156] (13) A driving assistance device as described in (4), characterized in that the assistance pattern assistance judgment unit detects a reverse operation during turning from the operation information acquisition unit, and registers and updates turning assistance control as driving assistance in a manner that corresponds to the corresponding sensor map shape.

[0157] (14) A driving assistance device as described in (2), wherein the assistance pattern update unit classifies the sensor map shapes in the assistance pattern based on pattern similarity, and as a result of the classification, only assistance patterns in which a certain number or more of similar sensor map shapes are collected are set as search targets for the map similarity search unit.

[0158] (15). A driving assistance device as described in (14), characterized in that an assistance pattern update unit observes the effectiveness of the driving assistance instructed by the assistance instruction unit, and, among assistance patterns having a certain number or more of similar sensor map shapes as a result of the classification, an assistance pattern having a higher than a certain ratio of invalid assistance patterns is not used as a search target of the map similarity search unit.

[0159] (16).A driving assistance device as described in (2), further comprising an input unit through which the driver inputs, and an assistance pattern memory unit registers a pre-stored sensor map shape and assistance content in the assistance pattern memory unit based on the input from the driver.

[0160] (17).A driving assistance device as described in (2), comprising an input unit through which the driver inputs, the sensor map shape and the assistance content for which the system has determined that assistance is necessary are presented to the driver, and when the driver inputs a request for assistance into the input unit, the corresponding sensor map shape is made the search target of the map similarity search unit.

[0161] According to (1) to (17), by automatically detecting a situation in which the driver requires driving assistance and memorizing information about the surrounding terrain and obstacles, it becomes possible to provide appropriate driving assistance in similar situations. [Explanation of symbols]

[0162] 100...Driving assistance device 101...sensor information acquisition unit 102...Surrounding sensor map acquisition unit 103…Map Similarity Search Unit 104…Support instruction department 105...Operation information acquisition unit 106…Support Judgment Department 107…Support Pattern Update Section 108…Support pattern memory unit 109…Invalidation pattern storage unit 200...Driving assistance device 201…Target recognition unit 202…Environmental information acquisition department 300…Driving assistance device 301…Driver input section 302...Preliminary pattern storage unit 400…Driving assistance device 401…Map position estimation unit 402: Surrounding sensor map acquisition unit 403…Sensor map memory unit 406…Sensor map memory unit 500…Driving assistance device 501…Support content acquisition department

Claims

1. a memory that stores an assistance pattern including a first surrounding sensor map showing a map of the surroundings of a vehicle and assistance content of driving assistance for the vehicle corresponding to the first surrounding sensor map; acquiring sensor information indicating information of objects around the vehicle sensed by a sensor; Obtaining a second surrounding sensor map based on the sensor information; a processor that performs driving assistance of the assistance content corresponding to the first surrounding sensor map when the second surrounding sensor map is similar to the first surrounding sensor map; The memory stores a list including operation information indicating an operation state requiring driving assistance and the assistance content corresponding to the operation information; The processor, Acquire direction information indicating a direction of the vehicle and environmental information indicating an environment around the vehicle, the environmental information including time, When the operation state indicated by the operation information is detected, the assistance pattern is registered / updated by a set of the environmental information, the second surrounding sensor map, and the assistance content corresponding to the operation information; A driving assistance device that performs driving assistance with the assistance content corresponding to the first surrounding sensor map when the second surrounding sensor map is similar to the first surrounding sensor map and the environment indicated by the environmental information is detected.

2. The driving assistance device according to claim 1, The processor, generating the second surrounding sensor map based on the sensor information; A driving assistance device comprising:

3. The driving assistance device according to claim 1, The memory includes: storing a sensor map obtained by stitching together a plurality of the second surrounding sensor maps; The processor, estimating a position of the vehicle on the sensor map based on the sensor information; Obtaining the second surrounding sensor map from the sensor map at the estimated position of the vehicle. A driving assistance device comprising:

4. The driving assistance device according to claim 1, The processor, Recognizing targets around the vehicle based on the sensor information; When registering / updating the support pattern, positional relationship information indicating a positional relationship between the vehicle and the target is included in the support pattern; When the second surrounding sensor map is similar to the first surrounding sensor map and the positional relationship indicated by the positional relationship information is detected, driving assistance is performed with the assistance content corresponding to the first surrounding sensor map. A driving assistance device comprising:

5. The driving assistance device according to claim 1, The operation state indicated by the operation information is a state in which a sudden braking operation has been performed, The support content corresponding to the operation information is a warning or a control for suppressing deceleration or acceleration. A driving assistance device comprising:

6. The driving assistance device according to claim 1, The operation state indicated by the operation information is a state in which the traveling speed of the vehicle is slower than a speed limit at a traveling position of the vehicle in a difficult scene for the driver, The support content corresponding to the operation information is a warning or a control for suppressing deceleration or acceleration. A driving assistance device comprising:

7. The driving assistance device according to claim 1, The operation state indicated by the operation information is a state in which the frequency of the driving assistance is higher than a predetermined value, the assistance content corresponding to the operation information is a change in a parameter for adjusting a function of the driving assistance, The driving assistance is generated earlier by changing the parameter than before the parameter was changed. A driving assistance device comprising:

8. The driving assistance device according to claim 1, The operation state indicated by the operation information is a state in which a reverse operation is performed while the vehicle is turning forward, The assistance content corresponding to the operation information is control of turning of the vehicle. A driving assistance device comprising:

9. The driving assistance device according to claim 1, The processor, Clustering the first surrounding sensor map; When the number of the first surrounding sensor maps in the cluster is equal to or greater than a certain number, driving assistance is performed according to the assistance content corresponding to the first surrounding sensor map in the cluster. A driving assistance device comprising:

10. A memory that stores an assistance pattern including a first surrounding sensor map showing a map of the surroundings of a vehicle and assistance content of the driving assistance of the vehicle corresponding to the first surrounding sensor map; acquiring sensor information indicating information of objects around the vehicle sensed by a sensor; Obtaining a second surrounding sensor map based on the sensor information; a processor that performs driving assistance of the assistance content corresponding to the first surrounding sensor map when the second surrounding sensor map is similar to the first surrounding sensor map; The memory stores a list including operation information indicating an operation state requiring driving assistance and the assistance content corresponding to the operation information; The processor, When the operation state indicated by the operation information is detected, the assistance pattern is registered / updated by using a set of the second surrounding sensor map and the assistance content corresponding to the operation information; A driving assistance device that performs clustering of the first surrounding sensor map, and when a number of the first surrounding sensor maps in a cluster is equal to or greater than a certain number, performs driving assistance of the assistance content corresponding to the first surrounding sensor map in the cluster, The list includes a determination criterion for determining whether the driving assistance corresponding to the assistance content is enabled or disabled, The processor, determining whether the driving assistance is valid or invalid based on a determination criterion corresponding to the assistance content in which the driving assistance is performed; When the number of the first surrounding sensor maps corresponding to the assistance content determined to be invalid in the cluster is less than a certain number, driving assistance is performed for the assistance content corresponding to the first surrounding sensor map; When the number of the first surrounding sensor maps corresponding to the assistance contents determined to be invalid in the cluster is equal to or greater than a certain number, driving assistance of the assistance contents corresponding to the first surrounding sensor map is not performed. A driving assistance device comprising:

11. The driving assistance device according to claim 1, An input unit that accepts a driver's selection as an input, The memory includes: storing a pre-assistance pattern including the first surrounding sensor map and an assistance content of the driving assistance of the vehicle corresponding to the first surrounding sensor map; The processor, The assistance pattern is registered / updated with the advance assistance pattern selected by the driver. A driving assistance device comprising:

12. The driving assistance device according to claim 1, An input unit that accepts a driver's selection as an input, The processor, presenting the set of the second surrounding sensor map and the assistance content corresponding to the operation information to a driver; The assistance pattern is registered / updated with the set selected by the driver. A driving assistance device comprising:

Images