Driver Assistance Systems

Abstract

To obtain a driving support system that can reduce performance of hardware which is mounted on a mobile object such as a vehicle, and reduce cost.SOLUTION: A driving support system that recognizes a peripheral environment of a mobile object Ve using a recognition model supplied from a server 200 is provided. In the driving support system, the mobile object Ve comprises: a data integration unit 154-1 that acquires environmental data on the peripheral environment of the mobile object Ve; and a priority setting unit 154-2 that sets priority which indicates magnitude of influence affected to recognition accuracy of the peripheral environment by the environmental data. The server 200 comprises a data processing device 220 that selects a recognition model corresponding to the peripheral environment of the mobile object Ve among a plurality of recognition models for recognizing the peripheral environment of the mobile object Ve on the basis of the environmental data and the priority.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a driving assistance system that assists in automatic driving control of, for example, an automatic driving vehicle. [Background technology]

[0002] US Pat. No. 6,399,433 discloses a system and method for building, using, and updating a sparse map for autonomous vehicle navigation. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-60221 A Summary of the Invention [Problem to be solved by the invention]

[0004] In advanced driving systems (ADAS) and autonomous driving (AD), it is necessary to select the optimal recognition model according to the driving environment to prevent the performance of the currently applied recognition model from deteriorating due to changes in the driving environment. However, installing all recognition models that can handle various environments in a vehicle requires hardware to store a huge amount of information, which leads to an increase in the cost of the vehicle system.

[0005] The present invention has been made in consideration of the above points, and an object of the present invention is to provide a driving assistance system that can reduce the performance of hardware installed in a moving object such as a vehicle, thereby reducing costs. [Means for solving the problem]

[0006] The driving assistance system of the present invention that solves the above problems includes: A driving assistance system that recognizes a surrounding environment of a moving object using a recognition model provided from a cloud server, the moving body includes an environmental data integration unit that acquires environmental data of a surrounding environment of the moving body, and a priority setting unit that sets a priority of a recognition condition that indicates a degree of influence of the environmental data on recognition accuracy of the surrounding environment, the cloud server comprises a data processing unit that selects a recognition model corresponding to the surrounding environment of the mobile object from among a plurality of recognition models for recognizing the surrounding environment of the mobile object based on the environmental data and the priority of the recognition conditions; The present invention is characterized by having the following. Effect of the Invention

[0007] According to the present invention, it is possible to lower the performance of hardware mounted on a moving object such as a vehicle, thereby reducing costs.

[0008] Further features related to the present invention will become apparent from the description of the present specification and the accompanying drawings. In addition, problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0009] [Figure 1] 1 is an overall diagram conceptually showing a driving assistance system according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating the internal configuration of a server according to the first embodiment. [Diagram 3] FIG. 2 is a block diagram illustrating the internal configuration of a data processing device of the server according to the first embodiment. [Figure 4] FIG. 2 is a block diagram illustrating the internal configuration of the vehicle control device according to the first embodiment. [Diagram 5] FIG. 2 is a block diagram illustrating the internal configuration of the perception and judgment device according to the first embodiment. [Figure 6] 4 is a table showing an example of objects to be recognized in a travel area in the first embodiment. [Figure 7] 4 is a table showing an example of a relationship between driving environment data, an influence on recognition performance, and a priority level in the first embodiment. [Figure 8A]4 is a flowchart illustrating data processing by a driving environment data processing unit included in the recognition determination device of the vehicle control device in the first embodiment. [Figure 8B] 6 is a flowchart for explaining data processing by the data processing device of the server in the first embodiment. [Figure 9A] FIG. 4 is a diagram for explaining a recognition model used in the first embodiment. [Figure 9B] FIG. 4 is a diagram for explaining a recognition model used in the first embodiment. [Figure 10A] 10 is a flowchart illustrating data processing by a driving environment data processing unit included in the recognition determination device of the vehicle control device in the second embodiment. [Figure 10B] 10 is a flowchart for explaining data processing by a data processing device of a server in the second embodiment. [Figure 11] 1 is a table showing the robust characteristics of various in-vehicle sensors. [Figure 12A] 11 is a table showing an example of priority settings when a camera is used. [Figure 12B] A table showing an example of priority setting when using LiDAR. [Figure 13A] FIG. 1 is a diagram for explaining a case where a stereo camera is used for object recognition. [Figure 13B] A diagram explaining the use of LiDAR for object recognition. [Figure 14] FIG. 11 is a block diagram illustrating the internal configuration of a perception and judgment device according to a third embodiment. [Figure 15] 11 is a flowchart illustrating data processing by a driving environment data processing unit included in the recognition determination device of the vehicle control device in the third embodiment. [Figure 16] 13 is a flowchart for explaining data processing by a data processing device of a server in the fourth embodiment. [Figure 17] FIG. 13 is a diagram for explaining the contents of a fourth embodiment in which all recognition models corresponding to each section of a route are stored in a memory and used. [Figure 18] 13 is a flowchart illustrating data processing by a driving environment data processing unit included in the recognition determination device of the vehicle control device in the fifth embodiment. [Figure 19] FIG. 13 is a diagram for explaining the contents of a fifth embodiment in which recognition models corresponding to each section of a route are sequentially stored in a memory and used. [Figure 20] FIG. 13 shows Example 6 in which the driving assistance device is used to control a multicopter. [Figure 21A] 13 is a flowchart for explaining data processing by a driving environment data processing unit included in the recognition and judgment device of the multicopter in the sixth embodiment. [Figure 21B] 20 is a flowchart for explaining data processing by a data processing device of a server in the sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] [Example 1] Next, a first embodiment of the present invention will be described. FIG. 1 is an overall view conceptually showing a driving assistance system in the first embodiment.

[0011] The driving assistance system includes a cloud 201, a server 200 which is a cloud server, and a mobile body. The cloud 201 is a so-called cloud service that provides data and software as a service via a network in response to a request from a terminal, and in this embodiment, stores a plurality of recognition models for recognizing the surrounding environment of the mobile body. The server 200 is a cloud server for providing a cloud service to a terminal, and in this embodiment, selects a recognition model suitable for recognizing the surrounding environment of the mobile body Ve from the plurality of recognition models stored in the cloud 201 and supplies it to the mobile body Ve. The mobile body Ve uses the recognition model supplied from the server 200 to recognize objects in the surrounding environment and performs ADAS and AD.

[0012] FIG. 1 shows a state in which a vehicle Ve, which is a moving object, is traveling on a highway during the daytime when the sun W is out. Ahead of the vehicle Ve, a preceding vehicle Ob-VF is traveling, and an oncoming vehicle Ob-Vn is traveling in the opposite lane. Further ahead of the preceding vehicles Ob-VF, a traffic light Signal, a toll booth Ob-TG, and a pylon Ob indicating the center lane are located. The vehicle Ve is provided with an environment recognition sensor for recognizing the surrounding environment of the vehicle, and is capable of detecting three-dimensional objects such as the preceding vehicle Ob-VF, the oncoming vehicle Ob-Vn, and the pylon Ob. The vehicle Ve can receive positioning radio waves from multiple GNSS satellites 101-a floating in the sky to determine its own position. The vehicle Ve can also perform two-way communication with a base station Base by wireless communication, and exchange information with a cloud 201 via a server 200.

[0013] FIG. 2 is a block diagram illustrating an internal configuration of the server according to the first embodiment. The cloud 201 is connected to a server 200 that transmits and receives information to and from vehicles. The server 200 can obtain information in the cloud 201 via a communication network such as the Internet. The server 200 includes a data processing device 220 and a wireless communication device 210.

[0014] The wireless communication device 210 performs two-way communication of information with the base station Base. The wireless communication device 210 provides the vehicle Ve with model information of the recognition model, weather and climate information, and time information via the base station Base, and receives data on the driving environment and priority setting information from the vehicle Ve. The priority setting information here is information on the priority of the recognition conditions that indicates the magnitude of the influence that the environmental data has on the recognition accuracy of the surrounding environment, and will be described in detail later.

[0015] The data processing device 220 acquires data on the driving environment of the vehicle Ve (information on the surrounding environment) and priority setting information from the wireless communication device 210. Then, the data processing device 220 selects a recognition model corresponding to the surrounding environment of the vehicle Ve from the information of the group of recognition models in the cloud 201 based on the data on the driving environment and the priority setting information, and performs processing to provide the recognition model to the wireless communication device 210.

[0016] FIG. 3 is a block diagram illustrating the internal configuration of the data processing device of the server in the first embodiment. The cloud 201 stores a plurality of recognition models 1, 2, . . . , n-1, n-2 for the vehicle Ve to recognize the surrounding environment around the vehicle. The data processing device 220 has a model extraction unit 221 and a model evaluation unit 222. The model extraction unit 221 acquires information on the recognition model group from the cloud 201. Then, based on the driving environment data and priority setting information acquired from the vehicle Ve via the wireless communication device 210, the model extraction unit 221 extracts a model group that meets the recognition condition from the recognition model group. The model evaluation unit 222 evaluates each model of the model group extracted by the model extraction unit 221, and selects the most appropriate recognition model for the surrounding environment of the vehicle Ve from the model group. Then, the model evaluation unit 222 provides the model to the wireless communication device 210 as information on the model to be distributed. The wireless communication device 210 transmits the model to be distributed provided by the model evaluation unit 222 from the base station Base to the vehicle Ve as model information of the recognition model.

[0017] FIG. 4 is a block diagram illustrating an internal configuration of the vehicle control device according to the first embodiment. The vehicle control device 100 is mounted on a host vehicle Ve. The vehicle control device 100 includes an external sensor 110, an internal sensor 120, a navigation system 130, an HMI device 140, an in-vehicle wireless communication device 101, a perception and judgment device 150, a vehicle motion control device 160, a steering control device 170, an accelerator control device 180, and a brake control device 190.

[0018] The external sensor 110 is an environment recognition sensor for recognizing the surrounding environment of the host vehicle Ve, and includes a stereo camera 110-b and a LiDAR 110-a. In addition to the above, the environment recognition sensor may be a combination of a monocular camera and a laser radar, or may use an ultrasonic sensor. In this embodiment, the external sensor 110 is configured with the stereo camera 110-b. The internal sensor 120 is a sensor for detecting the operating state of the host vehicle Ve, and includes at least one sensor such as an acceleration sensor, a vehicle speed sensor, an engine rotation sensor, and a throttle valve opening sensor.

[0019] The navigation system 130 is a system for showing a route to a destination on a map based on map information and vehicle position information, and guiding the vehicle Ve to the destination. The HMI device 140 is a device operated by a passenger of the vehicle Ve, and is operated, for example, when inputting a destination into the navigation system 130.

[0020] The in-vehicle wireless communication device 101 is a communication device for performing wireless communication between the base station Base or between other vehicles, and performs processing to acquire information on an updated model from the base station Base and provide it to the perception judgment device 150, and transmit the driving environment data and priority setting information acquired from the perception judgment device 150 to the server 200 via the base station Base.

[0021] The recognition judgment device 150 performs a process of recognizing the surrounding environment around the vehicle using a recognition model. For example, when a recognition model for an expressway is set, a recognition process of a target object to be recognized while traveling on the expressway is performed. The recognition judgment device 150 transmits information on a target trajectory and a target speed to the vehicle motion control device 160 based on the result of the recognition process.

[0022] The vehicle motion control device 160 outputs each command value for controlling the steering, accelerator opening, and braking of the host vehicle Ve to the steering control device 170, the accelerator control device 180, and the brake control device 190. The steering control device 170, the accelerator control device 180, and the brake control device 190 control the steering, accelerator opening, and braking of the host vehicle Ve based on the command values ​​from the vehicle motion control device 160.

[0023] The recognition and judgment device 150, the vehicle motion control device 160, the steering control device 170, the accelerator control device 180, and the brake control device 190 are configured as electronic control devices (hereinafter referred to as ECUs) having microcomputers. An ECU may be provided exclusively for each device, or two or more devices may be integrated into one ECU.

[0024] FIG. 5 is a block diagram illustrating an internal configuration of the cognition and judgment device according to the first embodiment. The recognition judgment device 150 has a recognition unit 151, a driving action planning unit 152, a trajectory planning unit 153, and a driving environment data processing unit 154. The recognition unit 151 acquires environmental data of the surrounding environment of the vehicle Ve from the external sensor 110, and also acquires information of a recognition model for updating from the server 200 via the in-vehicle wireless communication device 101. The recognition unit 151 performs a process of recognizing a target object using detection information detected by the external sensor 110 and a recognition model selected by the server 200.

[0025] The driving behavior planning unit 152 performs processing to plan driving behavior of the host vehicle Ve based on the recognition result of the recognition unit 151, the detection result by the internal sensor 120, and map information from the navigation system 130. The trajectory planning unit 153 plans a trajectory of the host vehicle Ve based on external information, the driving behavior plan, the detection result by the internal sensor 120, and the map information from the navigation system 130.

[0026] The driving environment data processing unit 154 has a data integration unit 154-1 and a priority setting unit 154-2. The data integration unit 154-1 integrates driving area information, weather and climate information, time information, and road surface condition information to create driving environment data. The priority setting unit 154-2 sets a priority indicating the magnitude of influence that environmental data has on the recognition accuracy of the surrounding environment based on the driving environment data.

[0027] FIG. 6 is a table illustrating an example of objects to be recognized in a travel area in the first embodiment. For example, when the driving area is an urban area, examples of objects recognized by the recognition model include buildings, pedestrians, bicycles, motorbikes, passenger cars, emergency vehicles, pylons, pets, etc. On the other hand, when the driving area is an expressway, examples of objects recognized include passenger cars, emergency vehicles, trucks, expressway toll payment machines, pylons, etc. As such, the objects recognized differ depending on the driving area, and it is necessary to adopt a recognition model that can appropriately recognize these objects for each driving area and perform recognition.

[0028] FIG. 7 is a table illustrating an example of a relationship between the driving environment data, the influence on the recognition performance, and the priority in the first embodiment. Fig. 7 shows an example of recognizing a recognition target by capturing an image of the surrounding environment with a camera. For example, the weather has a very large effect on the recognition accuracy of the surrounding environment when captured by a camera, and the priority is set to 100 points. In addition, the road surface condition has a small effect on the recognition accuracy when captured by a camera, and the priority is set to 30 points.

[0029] FIG. 8A is a flowchart illustrating data processing by a driving environment data processing unit included in the recognition determination device of the vehicle control device in the first embodiment. The driving environment data processing unit 154 of the recognition judgment device 150 acquires weather, time zone, and route information from driving area information, weather and climate information, time information, and road surface condition information (S100), and acquires environmental information around the vehicle (S101). Then, the data integration unit 154-1 integrates the environmental information around the vehicle to generate data on the driving environment (S102), and the priority setting unit 154-2 sets the priority of the recognition conditions based on the data on the driving environment (S103). Here, for example, the priority is set as shown in FIG. 7. Then, when it is determined that the cloud 201 can be connected (YES in S104), the data on the driving environment integrated by the data integration unit 154-1 and the priority of the recognition conditions set by the priority setting unit 154-2 are transmitted to the server 200 (S105).

[0030] FIG. 8B is a flowchart illustrating data processing by the data processing device of the server in the first embodiment. The data processing device 220 of the server 200 receives the integrated driving environment data and the information on the priority of the recognition conditions (S106). Then, the model extraction unit 221 of the data processing device 220 extracts at least one recognition model that meets the recognition conditions from the group of recognition models in the cloud 201 based on this information (S107).

[0031] Then, the model evaluation unit 222 evaluates at least one recognition model that meets the recognition conditions (S108). The evaluation is performed based on the total priority score from among the extracted recognition models, and one recognition model with the highest total priority score is selected (S109). Then, if it is determined that a connection to the cloud has been established (YES in S110), the recognition model selected by the model evaluation unit 222 is transmitted to the server 200 (S111).

[0032] 9A and 9B are diagrams for explaining a recognition model used in the first embodiment. 9A and 9B show that the recognition model to be used is one that has been trained using image data captured at the time of use. For example, when driving at night, the model evaluation unit 222 selects a recognition model that has been trained using image data captured at night, and when driving during the day, the model evaluation unit 222 selects a recognition model that has been trained using image data captured during the day.

[0033] In the driving assistance system of the present embodiment described above, the vehicle Ve acquires environmental data of the surrounding environment of the vehicle Ve, and sets a priority of the recognition condition indicating the degree of influence of the environmental data on the recognition accuracy of the surrounding environment. Then, the server 200 selects a recognition model corresponding to the surrounding environment of the moving body from among a plurality of recognition models for recognizing the surrounding environment of the moving body based on the environmental data and the priority of the recognition condition. Therefore, an appropriate recognition model is supplied from the server 200 to the vehicle Ve according to a change in the traveling environment, and the surrounding environment can be recognized by the recognition model suitable for the traveling environment in which the vehicle Ve is traveling. Therefore, it is not necessary to mount all recognition models compatible with various environments on the vehicle, and hardware for storing a huge amount of information is not required, and the cost of the vehicle system can be reduced.

[0034] [Example 2] Second Embodiment Next, a second embodiment of the present invention will be described with reference to FIGS. A feature of this embodiment is that the priority for selecting the external sensor 110 is changed according to the weather and the time of day.

[0035] The external sensor 110 has different detection characteristics depending on the type, and the detection accuracy may vary depending on the weather and the time of day. Therefore, by mounting multiple types of sensors on the vehicle and increasing the priority of a recognition model that uses a sensor suitable for the environment, it is possible to adopt a recognition model that uses a sensor with high detection accuracy. The model extraction unit 221 of the server 200 extracts at least one recognition model that meets the recognition conditions from among the multiple recognition models and the external sensor to be used in the recognition model, based on the environmental data, the priority of the recognition conditions, and information on the type of the external sensor used by the host vehicle Ve for detection.

[0036] Fig. 10A is a flowchart for explaining data processing of a driving environment data processing unit included in a recognition judgment device of a vehicle control device in Example 2. The difference is that S201 is added compared to the flowchart of Example 1 shown in Fig. 8A. The driving environment data processing unit 154 transmits the driving environment data and the priority of the recognition conditions integrated in S105 to the server 200, and then acquires environmental information around the vehicle (S201).

[0037] Fig. 10B is a flowchart for explaining data processing of the data processing device of the server in the embodiment 2. The processes from S108 to S111 are the same as those in the embodiment 1 shown in Fig. 8B, and therefore detailed explanations thereof will be omitted.

[0038] The traveling environment data processing unit 154 of the server 200 receives the integrated traveling environment data, the priority information of the recognition conditions, and information on the external sensor 110 from the recognition judgment device 150 (S202). Then, it extracts a recognition model that corresponds to the information on the external sensor 110 and the recognition conditions from the model group (S203).

[0039] FIG. 11 is a table showing the robust characteristics of various on-board sensors. As shown in Fig. 11, for example, with respect to weather, ultrasonic sensors and radars have high robustness, cameras have low robustness, and LiDAR has medium robustness. Also, with respect to the time of day (day / night), cameras have low robustness, while ultrasonic sensors, radar, and LiDAR have high robustness.

[0040] FIG. 12A is a table showing an example of priority setting when a camera is used, and FIG. 12B is a table showing an example of priority setting when LiDAR is used. When a camera is used as the external sensor, the degree of influence of day / night and weather on the recognition model is large, so the priority is set to 100 for each, as shown in Fig. 12A. On the other hand, the degree of influence of road surface conditions on the recognition model is small, so the priority is set to 30.

[0041] When LiDAR is used as the external sensor, the weather has a large effect on the recognition model, and as shown in Fig. 12B, the priority is set to 100. On the other hand, the day / night and road conditions have a small effect on the recognition model, and the priority is set to 30.

[0042] 13A and 13B are diagrams for explaining a recognition model used in the second embodiment. In Fig. 13A and Fig. 13B, when a predetermined sensor is used for object recognition, a recognition model trained with image data captured by the sensor is used. When a stereo camera is used for object recognition, the model evaluation unit 222 uses a recognition model trained with image data captured by the stereo camera 110-SC, as shown in Fig. 13A. When a LiDAR is used for object recognition, a recognition model trained with image data captured by the LiDAR 110-L, as shown in Fig. 13B, is used.

[0043] The driving assistance system of the present embodiment described above has a configuration in which the priority for selecting the multiple types of external sensors 110 is changed according to the weather and time of day. Therefore, a recognition model using a sensor that is most suitable for the situation is selected, and the recognition accuracy of the surrounding environment can be improved.

[0044] [Example 3] Next, a third embodiment of the present invention will be described below with reference to FIGS. FIG. 14 is a block diagram illustrating an internal configuration of the recognition and judgment device in the third embodiment, and FIG. 15 is a flowchart illustrating data processing of a driving environment data processing unit included in the recognition and judgment device of the vehicle control device in the third embodiment.

[0045] A characteristic feature of this embodiment is that, as shown in Fig. 14, a response as to whether or not to update can be input from the HMI device 140 to the recognition unit 151. The driving assistance system of this embodiment has a configuration in which both the recognition result based on the currently set recognition model and the recognition result based on the recognition model for update are displayed on the HMI device 140, and a user such as a passenger can select either one.

[0046] The recognition unit 151 acquires information from the external sensor 110 (S301), and performs object recognition using an existing recognition model (S302). Then, it determines whether there is an update recognition model in response to a change in the driving environment, and if there is an update recognition model (YES in S303), it performs object recognition using the update recognition model (S304). Then, it transmits the object recognition results of the existing recognition model and the object recognition results of the update recognition model to the HMI device 140 (S305), and displays them so that the user can select either the existing recognition model or the update recognition model.

[0047] If the user's selection input to the HMI device 140 is to select an update recognition model (YES in S306), the existing recognition model is switched to the update recognition model (S307), and the object recognition result of the updated recognition model is output (S308). On the other hand, if there is no update recognition model (NO in S303) or if the user's selection is to select an existing recognition model (NO in S306), the object recognition result of the existing recognition model is output (S309).

[0048] According to the driving assistance system of this embodiment, object recognition is performed using a recognition model whose recognition results have been confirmed by the user, so that driving assistance control of the vehicle can be performed in accordance with the needs of the user.

[0049] [Example 4] Next, a fourth embodiment of the present invention will be described below with reference to FIGS. A feature of this embodiment is that, when a travel route is determined in advance by the navigation system 130, recognition models corresponding to each section in the travel route are extracted in advance and all of them are stored in the memory of the vehicle Ve. When a travel route for the vehicle Ve is set, the server 200 selects multiple recognition models to be used in each section when traveling along the travel route when setting the travel route and transmits them to the vehicle Ve. The vehicle Ve stores multiple recognition models in its memory and performs object recognition using the recognition models corresponding to each section when traveling along the travel route.

[0050] FIG. 16 is a flowchart illustrating data processing by the data processing device of the server in the fourth embodiment. The data processing device 220 of the server 200 receives the integrated driving environment data and the priority of the recognition conditions from the host vehicle Ve (S106), and further receives information on the driving route that the host vehicle Ve passes through. Then, from the driving route information, all patterns of the recognition conditions of the driving route are determined (S401), and a recognition model is selected from the recognition conditions of each section of the driving route (S402 to S403). Then, if it is determined that the connection to the cloud has been established (YES in S110), the recognition model selected by the model evaluation unit 222 is extracted from the cloud 201 and transmitted to the vehicle (host vehicle Ve) (S112). By using the driving route information provided by the navigation system 130, the host vehicle Ve can obtain information on the road surface condition, weather, time period, and driving area of ​​the section that the host vehicle Ve passes through.

[0051] FIG. 17 is a diagram for explaining the contents of the fourth embodiment in which all recognition models corresponding to each section of a route are stored in a memory and used. The memory of the vehicle Ve stores recognition models for sections 1 to n. For example, section 1 has the following conditions: road surface condition: dry, weather: sunny, time period: daytime, and driving area: mountain road. Section 2 has the following conditions: road surface condition: dry, weather: sunny, time period: daytime, and driving area: tunnel. Section k has the following conditions: road surface condition: wet, weather: rain, time period: daytime, and driving area: expressway. Therefore, the vehicle Ve performs object recognition in each section using the recognition model corresponding to that section.

[0052] According to this embodiment, the vehicle Ve extracts and acquires in advance a recognition model corresponding to each section in the travel route, and therefore it is possible to prevent a situation in which the recognition model cannot be acquired due to a deterioration in communication conditions, etc., compared to a case in which the corresponding recognition model is acquired from the server 200 each time according to a change in the travel environment. For example, even if the data communication connection environment in a tunnel in section 2 is poor and the recognition model corresponding to section 2 cannot be received inside the tunnel, according to this embodiment, the recognition model for section 2 is received from the server 200 and stored in memory, and therefore the recognition model can be used to appropriately recognize objects inside the tunnel.

[0053] Furthermore, according to this embodiment, only the recognition models required for each section along the travel route are received from the server and stored in the memory of the host vehicle Ve, which has the advantage that the memory capacity of the memory installed in the host vehicle Ve can be significantly reduced compared to the case in which all recognition models are stored.

[0054] [Example 5] Next, a fifth embodiment of the present invention will be described below with reference to FIGS. A feature of this embodiment is that, when a driving route is determined in advance by the navigation system 130, recognition models corresponding to each section in the driving route are sequentially retrieved from the server according to the driving position, and the recognition models of sections that have been passed are deleted from the memory of the vehicle Ve. When the capacity of multiple recognition models is larger than the capacity of the memory, the vehicle Ve receives and stores in memory only the recognition model that will be used first when traveling the driving route.

[0055] FIG. 18 is a flowchart for explaining data processing of the driving environment data processing unit included in the recognition determination device of the vehicle control device in the fifth embodiment, and is a flow that is executed only the first time.

[0056] The data processing device 220 of the server 200 receives the integrated driving environment data and the priority of the recognition conditions from the vehicle Ve (S106), and further receives information on the driving route that the vehicle Ve will pass through. Then, from the driving route information, it determines all patterns of the recognition conditions of the driving route (S401), and extracts the recognition conditions from section 1 to section k of the driving route (S501 to S403). Then, if it is determined that the cloud has been connected (YES in S110), the recognition model selected by the model evaluation unit 222 is transmitted to the server 200 (S111).

[0057] FIG. 19 is a diagram for explaining the content of the fifth embodiment in which recognition models corresponding to each section of a route are sequentially stored in a memory and used. In this embodiment, only the recognition models for two sections out of all sections are stored in the memory of the host vehicle Ve. For example, when traveling in section 1, only the recognition models for section 1 and section 2 are stored. Then, in section 1, object recognition is performed using the recognition model corresponding to section 1, and in section 2, object recognition is performed using the recognition model corresponding to section 2. While traveling in section 2, the recognition model corresponding to section 1 is deleted from the memory of the host vehicle Ve, and the recognition model corresponding to section 3, which is the next section, is stored in the memory of the host vehicle Ve. That is, while traveling in section 2, the recognition model corresponding to section 2 and the recognition model corresponding to section 3 are stored in the memory of the host vehicle Ve. Similarly, in section 3, object recognition is performed using the recognition model corresponding to section 3, and while traveling in section 3, the recognition model corresponding to section 2 is deleted from the memory of the host vehicle Ve, and the recognition model corresponding to section 4, which is the next section, is stored in the memory of the host vehicle Ve.

[0058] The driving assistance system of this embodiment deletes the recognition model corresponding to the section already passed from the memory of the vehicle Ve, and stores the recognition model corresponding to the section to be passed after the currently traveling section from the server 200 in the memory of the vehicle Ve. Therefore, the recognition models stored in the memory of the vehicle Ve can be limited to only two, that is, the recognition model corresponding to the currently traveling section and the recognition model corresponding to the next section to be passed, and the memory capacity can be reduced. As a result, even if the traveling route is long, for example, there is no need to increase the memory capacity, and costs can be reduced.

[0059] [Example 6] Sixth Embodiment Next, a sixth embodiment of the present invention will be described with reference to FIGS. 20, 21A, and 21B. The object of driving assistance by the driving assistance system of the present invention is not limited to a vehicle, but may be another moving object. For example, it may be a multicopter such as a drone. When flying the multicopter to a destination, the recognition model can be changed according to the surrounding environment.

[0060] Fig. 20 is a diagram showing an example in which the driving support device is used to control a multicopter. Fig. 20 shows an example in which a pilot 2012 pilots a multicopter 2011 to approach a bridge pier 2001 and uses the multicopter 2011 for inspection.

[0061] FIG. 21A is a flowchart for explaining data processing of a driving environment data processing unit of a multicopter recognition judgment device in the sixth embodiment, and FIG. 21B is a flowchart for explaining data processing of a server data processing device in the sixth embodiment.

[0062] After the processes of S100 to S105, the driving environment data processing unit 154 of the perception judgment device 150 transmits the target recognition task to the cloud (S601). The data processing device 220 of the server 200 receives the integrated driving environment data and the priority of the recognition conditions from the multicopter 2011 (S106). Then, the data processing device 220 receives the target recognition task from the multicopter 2011, which is the recognition task of recognizing the crack 2002 in this embodiment (S602).

[0063] Then, a recognition model that meets the recognition conditions and the target recognition task is extracted from a group of multiple recognition models (S603).Then, the extracted recognition model is evaluated based on the received data (S108), and a recognition model is selected based on the evaluation result (S109).If it is determined that a connection to the cloud has been established (YES in S110), the recognition model selected by the model evaluation unit 222 is transmitted to the server 200 (S111).

[0064] According to the driving assistance system of this embodiment, the multicopter 2011 can fly using a recognition model for object recognition until it approaches the pier 2001. Therefore, for example, if an obstacle is recognized during flight and it is determined that the multicopter will collide with the obstacle if the situation continues, driving assistance such as evasive maneuvering can be performed. Then, when the multicopter approaches the crack 2002 that is the inspection target of the pier 2001, the recognition model can be updated from the obstacle recognition model to a recognition model for recognizing the crack 2002 in the pier 2001, and the crack 2002 can be recognized.

[0065] Although the embodiments of the present invention have been described above in detail, the present invention is not limited to the above-described embodiments, and various design changes can be made without departing from the spirit of the present invention described in the claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]

[0066] 110: external sensor, 130: navigation system, 140: HMI device, 150: perception and judgment device, 151: recognition unit, 152: driving action planning unit, 154: driving environment data processing unit, 154-1: data integration unit, 154-2: priority setting unit, 200: server, 201: cloud, 220: data processing device, 221: model extraction unit, 222: model evaluation unit, Ve: vehicle, Base: base station

Claims

1. A driving assistance system that recognizes a surrounding environment of a moving object using a recognition model provided from a cloud server, the moving body includes an environmental data integration unit that acquires environmental data of a surrounding environment of the moving body, and a priority setting unit that sets a priority of a recognition condition that indicates a degree of influence of the environmental data on recognition accuracy of the surrounding environment, The cloud server includes a data processing unit that selects a recognition model corresponding to the surrounding environment of the mobile body from among a plurality of recognition models for recognizing the surrounding environment of the mobile body based on the environmental data and a priority of the recognition condition, a model extraction unit that extracts at least one recognition model corresponding to the recognition condition from among the plurality of recognition models based on the environmental data and the priority of the recognition condition, and a model evaluation unit that determines one recognition model from among the extracted recognition models based on a total score of the priority. A driving assistance system comprising:

2. the moving object has a plurality of types of external sensors, The driving assistance system of claim 1, characterized in that the model extraction unit of the cloud server extracts at least one recognition model that meets the recognition conditions from among the multiple recognition models and the external sensors to be used in the recognition models based on the environmental data, the priority of the recognition conditions, and information on the type of the external sensor used by the moving body for detection.

3. A driving assistance system that recognizes a surrounding environment of a moving object using a recognition model provided from a cloud server, The mobile body includes an environmental data integration unit that acquires environmental data of a surrounding environment of the mobile body, a priority setting unit that sets a priority of a recognition condition indicating a degree of influence of the environmental data on recognition accuracy of the surrounding environment, and a recognition unit that recognizes a target object using a recognition model selected by the cloud server from detection information detected by an external sensor, The recognition unit is Acquire detection information from the external sensor; Perform object recognition using existing recognition models, When an updated recognition model is set, object recognition is performed using the updated recognition model; displaying, on a display device, an object recognition result based on the existing recognition model and an object recognition result based on the update recognition model; The object recognition result selected by the input of the selection signal is output. the cloud server comprises a data processing unit that selects a recognition model corresponding to the surrounding environment of the mobile object from among a plurality of recognition models for recognizing the surrounding environment of the mobile object based on the environmental data and a priority of the recognition condition; A driving assistance system comprising:

4. A driving assistance system that recognizes a surrounding environment of a moving object using a recognition model provided from a cloud server, the moving body includes an environmental data integration unit that acquires environmental data of a surrounding environment of the moving body, and a priority setting unit that sets a priority of a recognition condition that indicates a degree of influence of the environmental data on recognition accuracy of the surrounding environment, the cloud server includes a data processing unit that selects a recognition model corresponding to the surrounding environment of the mobile object from among a plurality of recognition models for recognizing the surrounding environment of the mobile object based on the environmental data and a priority of the recognition condition; when a travel route of the moving body is set, the cloud server selects a plurality of recognition models to be used in each section when traveling along the travel route when setting the travel route, and transmits the selected recognition model to the moving body; the moving body stores the plurality of recognition models in a memory, and uses a recognition model corresponding to each section when traveling along the travel route; A driving assistance system characterized by:

5. The driving assistance system according to claim 4, characterized in that, when the capacity of the plurality of recognition models is larger than the capacity of the memory, the moving body receives and stores in the memory only the recognition model that will be used first when traveling along the traveling route.

Images