Model update system

Abstract

To automatically re-learn a machine learning model which is not used in control of a vehicle.SOLUTION: A vehicle (1) is mounted with a first machine learning model A and a second machine learning model B, and an output signal of a sensor (40) is input to both of the first machine learning model A and the second machine learning model B. A vehicle control device (40) is controlled using output information from the first machine learning model A. The first machine learning model A and the second machine learning model B are compared and evaluated whether either of the machine learning models is dominant, on the basis of recognition contents in the first machine learning models A and B, when it is evaluated that the first machine learning model A is dominant, the second machine learning model B is re-learned. Update information of the re-learned second machine learning model B is transmitted to the vehicle (1), and the second machine learning model B mounted on the vehicle (1) is updated according to the received update information.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a model update system.

Background Art

[0002] During vehicle travel, an image recognition module using machine learning technology is often used to recognize moving objects and obstacles in front of the vehicle. This image recognition module needs to learn an image identification function based on image data acquired in various driving scenes of the vehicle. Therefore, a simulation system that learns the image identification function of the image recognition module using simulation technology is known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, when using machine learning, it is necessary to be able to explain safety. At this time, actual data when actually driving on the market is the most powerful as evidence for proving safety. An object of the present invention is to provide a system capable of obtaining actual data when actually driving on the market.

Means for Solving the Problems

[0005] According to the present invention, a vehicle is equipped with a processor, and the vehicle is equipped with a plurality of machine learning models that output specific information used for control of the vehicle control device when an output signal of a sensor mounted on the vehicle is input. The machine learning models include a first machine learning model and a second machine learning model that are different from each other. The processor The output signal of a sensor mounted on a vehicle is input to both a first machine learning model and a second machine learning model to output the above-described specific information from the first machine learning model and the second machine learning model, Among the above-described specific information output from the first machine learning model and the above-described specific information output from the second machine learning model, the vehicle control device is controlled using the above-described specific information output from the first machine learning model, It is configured to transmit the recognition content in the first machine learning model and the recognition content in the second machine learning model to a data server outside the vehicle, The processor of the data server, Based on the recognition content in the first machine learning model and the recognition content in the second machine learning model, compare and evaluate which of the first machine learning model and the second machine learning model is superior, When the first machine learning model is evaluated to be superior to the second machine learning model, the second machine learning model is re-learned, It is configured to transmit the update information of the re-learned second machine learning model to the vehicle, The processor mounted on the vehicle, A model update system is provided that is configured to update the second machine learning model mounted on the vehicle according to the received update information.

[0006] Further, according to the present invention, a processor mounted on a vehicle and a memory mounted on the vehicle are provided, The memory stores a plurality of machine learning models that output specific information used for controlling the vehicle control device when the output signal of a sensor mounted on the vehicle is input. The machine learning models include a first machine learning model and a second machine learning model that are different from each other, The processor, The output signal of a sensor mounted on the vehicle is input to both a first machine learning model and a second machine learning model to output the above-described specific information from the first machine learning model and the second machine learning model, Control the vehicle control device using either the above-mentioned specific information output from the first machine learning model or the above-mentioned specific information output from the second machine learning model. It is configured to transmit the recognition content in the first machine learning model and the recognition content in the second machine learning model to a data server outside the vehicle. The processor of the data server Based on the recognition content in the first machine learning model and the recognition content in the second machine learning model, compare and evaluate which machine learning model of the first machine learning model and the second machine learning model is superior. When it is evaluated that the first machine learning model is superior to the second machine learning model, use the above-mentioned specific information output from the first machine learning model for the control of the vehicle control device. When it is evaluated that the first machine learning model is superior to the second machine learning model, re-learn the second machine learning model. It is configured to transmit the update information of the re-learned second machine learning model to the vehicle. The processor mounted on the vehicle Is configured to update the second machine learning model stored in the memory mounted on the vehicle according to the received update information. The processor of the data server When it is evaluated that the second machine learning model is superior to the first machine learning model, use the control signal output from the second machine learning model for the control of the vehicle control device. When it is evaluated that the second machine learning model is superior to the first machine learning model, re-learn the first machine learning model. Is configured to transmit the update information of the re-learned first machine learning model to the vehicle. The processor mounted on the vehicle A model update system is provided that is configured to update the first machine learning model stored in the memory mounted on the vehicle according to the received update information.

Effect of the Invention

[0007] In the first invention, the second machine learning model is automatically updated, and in the second invention, the first machine learning model and the second machine learning model are automatically updated.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Modes for Carrying Out the Invention

[0009] First, with reference to FIG. 1, the configuration of vehicle 1 will be described. Referring to FIG. 1, 10 is a vehicle drive unit for applying driving force to the drive wheels of vehicle 1, 11 is a braking device for braking vehicle 1, 12 is a steering device for steering vehicle 1, and 13 is an electronic control unit mounted in vehicle 1, respectively. As shown in FIG. 1, the electronic control unit 13 consists of a digital computer and includes a CPU (processor) 15, a memory 16 composed of a ROM and a RAM, and an input / output port 17, which are connected to each other by a bidirectional bus 14.

[0010] On the one hand, as shown in FIG. 1, various sensors 18 are installed in the vehicle 1, that is, sensors for detecting the state of the vehicle 1 and sensors for detecting the surroundings of the vehicle 1. In this case, as sensors for detecting the state of the vehicle 1, an acceleration sensor, a speed sensor, an azimuth sensor, and a geomagnetic sensor are used. As sensors for detecting the surroundings of the vehicle 1, cameras for photographing the front, side, and rear of the vehicle 1, lidar (LIDAR), radar, etc. are used. In addition, a GNSS (Global Navigation Satellite System) receiver 19, a map data storage device 20, a navigation device 21, and an HMI 22 are provided in the vehicle 1. The GNSS receiver 19 can detect the current position of the vehicle 1 (for example, the latitude and longitude of the vehicle 1) based on information obtained from a plurality of artificial satellites. Therefore, the current position of the vehicle 1 can be acquired by this GNSS receiver 19. As this GNSS receiver 19, for example, a GPS receiver is used.

[0011] Map data and the like are stored in the map data storage device 20. In addition, the HMI 22 is composed of a display device, an audio generating device, etc. arranged in the vehicle interior. These various sensors 18, GNSS receiver 19, map data storage device 20, navigation device 21, and HMI 22 are connected to the electronic control unit 13. A communication device 23 connected to the electronic control unit 13 is mounted on the vehicle 1, and the electronic control unit 13 can communicate with the data server 30 through the communication device 23. Note that this data server 30 is equipped with a processor and a memory.

[0012] In the example shown in FIG. 1, the vehicle 1 can be manually driven, and when there is a request for driving control by automatic driving for the vehicle 1, it is possible to cause the vehicle 1 to perform driving control by automatic driving. In the example shown in FIG. 1, the vehicle drive unit 10 of the vehicle 1 is composed of an electric motor driven by a secondary battery or an electric motor driven by a fuel cell, and the drive wheels are driven and controlled by these electric motors according to the output signal of the electronic control unit 13. Also, in the example shown in FIG. 1, it is possible to perform braking control of the vehicle 1 by the braking device 11 based on the output signal of the electronic control unit 13, and it is also possible to perform steering control of the vehicle 1 by the steering device 12 based on the output signal of the electronic control unit 13.

[0013] FIG. 2 shows a functional configuration diagram of a first embodiment of a control system 41 that controls the vehicle control device 40 based on the output signal of the sensor 18 shown in FIG. 1. In this case, as the sensor 18, it is possible to adopt various sensors described above, and as the vehicle control device 40, any of the vehicle drive unit 10, the braking device 11, and the steering device 12 shown in FIG. 1 can be adopted. Hereinafter, taking as an example the case where a camera that captures the front of the vehicle 1 is used as the sensor 18 and the braking device 11 is used as the vehicle control device 40, and when the presence of a moving object such as a vehicle or a pedestrian or the presence of a stationary obstacle is detected in front of the vehicle 1 by the front camera, the vehicle 1 is urgently stopped by the braking device 11, the present invention will be described.

[0014] In the first embodiment according to the present invention, as shown in FIG. 2, the control system 41 includes a system unit 43 that holds a first machine learning model A indicated by reference numeral 42, and a system unit 45 that holds a second machine learning model B indicated by reference numeral 44. In the first embodiment, these first machine learning model A and second machine learning model B are respectively stored in the memories of the system units 43 and 45. In addition to the system unit 43 that holds the first machine learning model A and the system unit 45 that holds the second machine learning model B, the control system 41 may include three or more system units such as a system unit that holds a third machine learning model C and a system unit that holds a fourth machine learning model D. In the example shown in FIG. 2, the first machine learning model A and the second machine learning model B are respectively formed from different machine learning models.

[0015] As shown in FIG. 2, the output signal of the sensor 18, that is, the front camera, is input to both the first machine learning model A and the second machine learning model B. At this time, in both the first machine learning model A and the second machine learning model B, a moving object such as a vehicle or a pedestrian, or a stationary obstacle in front of the vehicle 1 is detected by an object detection method using image recognition technology. Here, this object detection method will be briefly described by taking, for example, the case of using a well-known R-CNN (regions with CNN features) using a convolutional neural network. FIGS. 3A and 3B schematically show the screen captured by the front camera. As shown in FIGS. 3A and 3B, bounding boxes 50 and 51, which are object region candidates, are shown on the screen captured by the front camera.

[0016] In the example shown in FIG. 3A, a bounding box 50 surrounding the vehicle is shown, and in the example shown in FIG. 3B, a bounding box 51 surrounding the pedestrian is shown. The images within these object region candidates, the bounding boxes 50 and 51, are respectively input into a pre-trained convolutional neural network CNN, and vectors called CNN features for each image are generated. The CNN features generated in the convolutional neural network CNN are input into a classifier consisting of a support vector machine, and the class of the object (e.g., vehicle, pedestrian) within the bounding box that is the object region candidate is identified. That is, it is identified what the object is. Of course, as an object detection method, it is also possible to use Fast R-CNN, Faster R-CNN, YOLO, or SSD, which are evolved forms of this R-CNN.

[0017] On the other hand, the position of the underlined part (road surface) of the bounding boxes 50 and 51 on the screen is uniquely determined from the mounting height and focal direction of the front camera and the distance to the vehicle or pedestrian. Therefore, the distance to the vehicle or pedestrian can be calculated based on, for example, the position of the underlined part (road surface) of the bounding boxes 50 and 51. In the example shown in FIG. 2, the distance to the vehicle or pedestrian is obtained based on the position of the underlined part (road surface) of the bounding boxes 50 and 51 on the screen.

[0018] In this case, in the example shown in FIG. 2, when the presence of a moving object such as a vehicle or pedestrian or the presence of a stationary obstacle is detected in front of vehicle 1 by the first machine learning model A, and when it is determined that the distance to the moving object or the like is less than or equal to the set distance, an output signal for urgently stopping vehicle 1 by the braking device 11 is output from the system unit 43. On the other hand, when the presence of a moving object such as a vehicle or pedestrian or the presence of a stationary obstacle is detected in front of vehicle 1 by the second machine learning model B, and when it is determined that the distance to the moving object or the like is less than or equal to the set distance, an output signal for urgently stopping vehicle 1 by the braking device 11 is output from the system unit 45.

[0019] By the way, in the example shown in FIG. 2, only the output signal from the system unit 43 is sent to the vehicle control device 40, that is, the braking device 11, and the output signal from the system unit 45 is not sent to the vehicle control device 40, that is, the braking device 11. Therefore, in the example shown in FIG. 2, only when an output signal for emergency stopping the vehicle 1 by the braking device 11 is output from the system unit 43 including the first machine learning model A, the vehicle 1 is emergency stopped by the braking device 11. Even if an output signal for emergency stopping the vehicle 1 by the braking device 11 is output from the system unit 45 including the second machine learning model B, the vehicle 1 is not emergency stopped by the braking device 11.

[0020] Note that both the first machine learning model A and the second machine learning model B are composed of learned models that have been pre-learned using a large number of learning data. In this case, for example, a moving object such as a vehicle or a pedestrian, or a photograph showing a stationary obstacle with a bounding box surrounding the moving object or the stationary obstacle added is used as the learning data, and using these large numbers of learning data, the first machine learning model A, which is a learned model, and the second machine learning model B, which is a learned model, are created. On the other hand, in this case, the first machine learning model A and the second machine learning model B are each learned using different large numbers of learning data. Therefore, as described above, the first machine learning model A and the second machine learning model B are different from each other.

[0021] By the way, when vehicle control is performed using an object detection method based on a machine learning model, as time passes, the form of the object to be detected changes and the types of objects to be detected increase. Therefore, in order to maintain the discrimination ability, it is sometimes necessary to update the machine learning model. For this purpose, in the first embodiment, two machine learning models, a first machine learning model A and a second machine learning model B, are used. While one machine learning model, in the example shown in FIG. 2, controls the braking device 11 based on the discrimination result by the first machine learning model A, the discrimination ability of the other machine learning model, that is, the second machine learning model B, is enhanced. In this case, in the first embodiment, from the viewpoint that an emergency stop is performed when an emergency stop is truly necessary, that is, from the viewpoint of safety, it is evaluated which of the first machine learning model A and the second machine learning model B is superior. When it is evaluated that the second machine learning model B is superior, the first machine learning model A and the second machine learning model B are replaced. In this case, there are cases where the first machine learning model A and the second machine learning model B are replaced by replacing the system unit 43 and the system unit 45, and on the one hand, the second machine learning model B is overwritten and stored in the memory of the system unit 43, and on the other hand, the first machine learning model A is overwritten and stored in the memory of the system unit 45 to replace the first machine learning model A and the second machine learning model B.

[0022] On the other hand, when replacing the first machine learning model A and the second machine learning model B, it is necessary to explain the safety when using the second machine learning model B. At this time, actual data when actually driving on the market is the most powerful as evidence to prove safety. Therefore, in the first embodiment, when the vehicle 1 is actually driving on a public road, the recognition content of the first machine learning model A and the recognition content of the second machine learning model, that is, the actual data when actually driving on the market, are accumulated. When it is evaluated that the second machine learning model is superior from these accumulated recognition contents, and safety is proven from the accumulated actual data when actually driving on the market, the first machine learning model A and the second machine learning model B are replaced.

[0023] Thus, in the embodiment according to the present invention, an evaluation is made as to which of the first machine learning model and the second machine learning model is superior. In the first embodiment, this evaluation is performed in the data server 30, and the data necessary for performing this evaluation is transmitted from the vehicle 1 to the data server 30. In order to transmit the data necessary for performing this evaluation from the vehicle 1 to the data server 30, as shown in FIG. 2, the control system 41 includes a trigger processing system 46 and a trigger processing system 47. The output signal of the system unit 43 is input to the trigger processing system 46, and the output signal of the system unit 44 is input to the trigger processing system 47.

[0024] When an output signal for requesting an emergency stop of the vehicle 1 is output from the system unit 43, the trigger processing system 46 acquires, from the system unit 43, captured image data by the front camera within a certain period before and after the generation of this emergency stop output signal, and image data having bounding boxes 50, 51 as shown in FIGS. 3A and 3B within this certain period, and transmits the acquired image data to the data server 30 by the communication device 23. Similarly, when an output signal for requesting an emergency stop of the vehicle 1 is output from the system unit 45, the trigger processing system 47 acquires, from the system unit 45, captured image data by the front camera within a certain period before and after the generation of this emergency stop output signal, and image data having bounding boxes 50, 51 as shown in FIGS. 3A and 3B within this certain period, and transmits the acquired image data to the data server 30 by the communication device 23.

[0025] These image data transmitted from a number of vehicles to the data server 30 are stored and accumulated in the memory of the data server 30. When the accumulated amount of these image data exceeds a certain amount, in the data server 30, comparison data of the evaluation of the machine learning model A and the evaluation of the second machine learning model B is created. FIG. 4 shows an example of a comparison data creation routine for the evaluation of the machine learning models A and B executed by a processor in the data server 30, and FIG. 5 shows a comparison data table of the evaluation of the machine learning models A and B created in the data server 30 and stored in the memory of the data server 30.

[0026] In the example shown in FIG. 4, using a plurality of verification machine learning models created in different forms such as R-CNN, Fast R-CNN, Faster R-CNN, YOLO, and SSD, the evaluation of the recognition content by the first machine learning model A and the evaluation of the recognition content by the second machine learning model B are performed. In this case, for example, when an object surrounded by the bounding box 50 (FIG. 3A) on the image is recognized as a vehicle in the first machine learning model A, when the object surrounded by the bounding box 50 (FIG. 3A) on this image is also recognized as a vehicle in the verification machine learning model, the recognition content by the first machine learning model A is determined to be correct.

[0027] On the other hand, in FIG. 3C, 53 represents soft grass growing on the road. FIG. 3C shows a case where, for example, in the first machine learning model A, this grass 53 is recognized as a pedestrian, and as a result, this grass 53 is surrounded by the bounding box 52. Thus, when an object surrounded by the bounding box 53 on the image is recognized as a pedestrian in the first machine learning model A, when the object surrounded by the bounding box 52 on this image cannot be recognized in the verification machine learning model, the recognition content by the first machine learning model A is determined to be a misrecognition.

[0028] In the first embodiment, such a discrimination operation is performed using a plurality of verification machine learning models. That is, in the first embodiment, for all the verification machine learning models, the recognition content by the machine learning model A is compared with the recognition content by the verification machine learning model. Among all the verification machine learning models, when the ratio of the recognition content by the first machine learning model A to the recognition content by the verification machine learning model exceeds a certain ratio, the recognition content by the first machine learning model A is determined to be correct. On the contrary, among all the verification machine learning models, when the ratio of the recognition content by the first machine learning model A to the recognition content by the verification machine learning model is equal to or less than a certain ratio, the recognition content by the first machine learning model A is determined to be a misrecognition.

[0029] Next, while referring to FIG. 4, a comparison data creation routine for evaluating the machine learning models A and B executed in the data server 30 will be described. As described above, the captured image data by the front camera within a certain period before and after the generation of the emergency stop output signal, and the image data having the bounding box within this certain period are transmitted from a large number of vehicles to the data server 30, and these transmitted image data are stored and accumulated in the memory of the data server 30. The routine shown in FIG. 4 is repeatedly executed when the accumulated amount of these image data exceeds a certain amount.

[0030] Referring to FIG. 4, first, in step 60, after the previous comparison data is created, one piece of image data having a bounding box is acquired from the image data first stored for the first machine learning model A. Next, in step 61, the verification machine learning model to be first used is determined from among the plurality of verification machine learning models. Next, in step 62, a verification operation using the determined verification machine learning model is executed. That is, a verification operation is executed to determine whether the recognition content by the first machine learning model A matches the recognition content by the verification machine learning model for the image data acquired in step 60. Next, in step 63, it is determined whether the verification is completed.

[0031] In step 63, when it is determined that the verification is completed, the process proceeds to step 64 to determine the verification machine learning model to be used next. Then, the process proceeds to step 62, and a verification operation using the verification machine learning model to be used next is executed. When the verification operation using the verification machine learning model is completed for all the verification machine learning models, the process proceeds to step 65 to verify whether the ratio of the recognition content by the first machine learning model A to the recognition content by the verification machine learning model among all the verification machine learning models exceeds a certain ratio, that is, whether the recognition content by the first machine learning model A is correct or misrecognition. The verification result is stored in the memory of the data server 30 in the form of a comparison data table of the evaluations of the machine learning models A and B shown in FIG. 5. In FIG. 5, No. indicates the number of the image data used for the verification.

[0032] When the verification for the image data initially stored for the first machine learning model A is completed and the verification result is stored in the memory of the data server 30 in the form of a comparison data table of the machine learning models A and B shown in FIG. 5, the verification operation for the next stored image data for the first machine learning model A is started. When the verification for all the image data stored for the first machine learning model A is completed, this time, the verification starts from the image data initially stored for the second machine learning model B. In this way, as shown in FIG. 5, a comparison data table for all the image data stored for the first machine learning model A and the second machine learning model B is created.

[0033] FIG. 6 shows a model update processing routine executed by the processor in the data server 30 to implement the first embodiment according to the present invention. This routine is executed when the comparison data table shown in FIG. 5 is newly created.

[0034] Referring to FIG. 6, first, in step 70, a routine for creating comparison data of the evaluations of the machine learning models A and B shown in FIG. 4 is executed, and a comparison data table of the evaluations of the machine learning models A and B shown in FIG. 5 is created. Next, in step 71, based on the comparison data table of the evaluations of the machine learning models A and B shown in FIG. 5, a comparative evaluation is performed to determine which of the first machine learning model A and the second machine learning model B is superior. In this case, for example, in the comparison data table of the evaluations of the machine learning models A and B shown in FIG. 5, the machine learning model with the larger number of correct answers is considered superior. When it is evaluated in step 71 that the first machine learning model A is superior, the process proceeds to step 72 where the learning data is adjusted, and then to step 73 where the second machine learning model B is re-learned using the adjusted learning data.

[0035] The adjustment of the learning data in step 72 is performed, for example, by replacing a part of these image data with new image data suitable for re-learning while maintaining the total number of image data used when the second machine learning model B was learned last time. In this case, for example, when a comparative evaluation is performed to determine which of the first machine learning model A and the second machine learning model B is superior, the image data with the bounding box of the image misrecognized by the second machine learning model B removed is used as the new image data. For example, in the second machine learning model B, if the grass 53 (FIG. 3C) is recognized as a pedestrian and thus the grass 53 is surrounded by the bounding box 52, the image data with the bounding box 52 removed from the image data where the grass 53 is surrounded by the bounding box 52 is used as the new image data.

[0036] On the one hand, when the second machine learning model B was previously learned, for example, the same number of image data as the new image data is randomly removed from the used image data. Using the training data adjusted in this way, the second machine learning model B is relearned. When the relearning is performed in this way, when the relearned second machine learning model B is used, the grass 53 will be recognized as not being a pedestrian or a vehicle. When the relearning of the second machine learning model B is completed, the process proceeds to step 74, and the update information of the relearned second machine learning model B is transmitted to each vehicle. That is, the update command of the second machine learning model B held in the system unit 45 shown in FIG. 2 is transmitted from the data server 30 to each vehicle. When each vehicle receives the update command of the second machine learning model B, the second machine learning model B included in the system unit 45 shown in FIG. 2 is updated according to the update information. At this time, the output signal of the second machine learning model B is not used to control the vehicle control device 40, that is, the braking device 11.

[0037] As long as it continues to be evaluated that the first machine learning model A is superior in step 71, the second machine learning model B continues to be relearned in step 73, and the second machine learning model B included in the system unit 45 shown in FIG. 2 continues to be updated according to the update information. Therefore, as time passes, the discrimination ability of the second machine learning model B increases.

[0038] On the one hand, when it is evaluated that the updated second machine learning model B is superior in step 71, the process proceeds to step 75, and data indicating that the updated second machine learning model B is superior to the first machine learning model A is collected. This data is the comparison data table itself of the evaluations of the machine learning models A and B shown in FIG. 5. Therefore, the data indicating that the updated second machine learning model B is superior can be collected from the data server 30. The data indicating that the updated second machine learning model B is superior is actual data when actually driving in the market. Therefore, as evidence for proving safety, it is the most powerful. In actual society, when adopting the updated second machine learning model B, it may be necessary to present evidence for proving safety. In the first embodiment according to the present invention, in such a case, it is possible to present actual data that is the most powerful evidence. Step 76 represents a waiting state until it is confirmed whether the first machine learning model A can be updated to the updated second machine learning model B.

[0039] If it is confirmed that the first machine learning model A can be updated to the updated second machine learning model B, the process proceeds from step 77 to step 78, and an update command for replacing the first machine learning model A and the updated second machine learning model B is transmitted from the data server 30 to each vehicle. When each vehicle receives this update command, the first machine learning model A held in the system unit 43 shown in FIG. 2 is replaced with the updated second machine learning model B, and the updated second machine learning model B becomes the new first machine learning model A. Therefore, thereafter, the output signal of the new first machine learning model A is used to control the vehicle control device 40, that is, the braking device 11.

[0040] Next, in step 79, a new third machine learning model C is obtained. That is, as time passes, the shape of the vehicle changes, the fashion of pedestrians changes, and new vehicles such as electric kick scooters appear. Therefore, in order to accurately identify them, it is necessary to use a machine learning model trained with new learning data including these. The new third machine learning model C represents a machine learning model trained with such new learning data, and this new third machine learning model C is stored in the memory of the data server 30.

[0041] Next, in step 80, an update command for replacing the second machine learning model B held in the system unit 45 shown in FIG. 2 with this new third machine learning model C is transmitted from the data server 30 to each vehicle. When each vehicle receives this update command, the second machine learning model B held in the system unit 45 shown in FIG. 2 is replaced with this new third machine learning model C. After that, when actual data during the running of the vehicle necessary for learning is accumulated, the new third machine learning model C is relearned. When the new third machine learning model C becomes superior to the machine learning model used to control the braking device 11, the machine learning model used to control the braking device 11 is replaced with this new third machine learning model C. In this way, the vehicle control device 40, that is, the machine learning model used to control the braking device 11, will continue to be updated as time passes.

[0042] Thus, in the first embodiment according to the present invention, the vehicle 1 is equipped with the processor 15. The vehicle 1 is equipped with a plurality of machine learning models that output specific information used for the control of the vehicle control device 40 when the output signal of the sensor 18 mounted on the vehicle 1 is input. The machine learning models include a first machine learning model A and a second machine learning model B that are different from each other. In this case, in the embodiment according to the present invention, this specific information represents, for example, detection information of moving objects such as vehicles and pedestrians recognized by a front camera, stationary obstacles, or detection information indicating that the distance to a moving object such as a vehicle or a pedestrian or a stationary obstacle has become equal to or less than a set distance.

[0043] Furthermore, in the first embodiment according to the present invention, the processor 15 inputs the output signal of the sensor 18 mounted on the vehicle 1 to both the first machine learning model A and the second machine learning model B to output the above-mentioned specific information from the first machine learning model A and the second machine learning model B, and controls the vehicle control device 40 using the above-mentioned specific information output from the first machine learning model A among the above-mentioned specific information output from the first machine learning model A and the above-mentioned specific information output from the second machine learning model B, and is configured to transmit the recognition content in the first machine learning model A and the recognition content in the second machine learning model B to a data server outside the vehicle. In this case, in the embodiment according to the present invention, this recognition content indicates a vehicle, a pedestrian, or the like.

[0044] Furthermore, in the first embodiment according to the present invention, the processor of the data server 30 compares and evaluates which of the first machine learning model A and the second machine learning model B is superior based on the recognition content in the first machine learning model A and the recognition content in the second machine learning model B. When it is evaluated that the first machine learning model A is superior to the second machine learning model B, the second machine learning model B is re-learned, and update information of the re-learned second machine learning model B is transmitted to the vehicle 1. In addition, the processor 15 mounted on the vehicle 1 is configured to update the second machine learning model B mounted on the vehicle 1 according to the received update information.

[0045] In the first embodiment according to the present invention, the processor of the data server 30 is configured to store and accumulate in the memory of the data server 30 a comparison evaluation result regarding which of the first machine learning model A and the updated second machine learning model B is superior. Therefore, when the updated second machine learning model B is evaluated to be superior to the first machine learning model A, a comparison evaluation indicating that the updated second machine learning model B is superior to the first machine learning model A can be collected from the stored comparison evaluation result.

[0046] Also, in this case, when the processor of the data server 30 can replace the first machine learning model A with the updated second machine learning model B based on the collected comparison evaluation, a third machine learning model C different from both the first machine learning model A and the updated second machine learning model B is obtained, and an update command to replace the first machine learning model A and the updated second machine learning model B and the third machine learning model C are transmitted to the vehicle 1. Further, when the processor 15 mounted on the vehicle 1 receives an update command to replace the first machine learning model A and the updated second machine learning model B, the first machine learning model A and the updated second machine learning model B are replaced, the updated second machine learning model B is set as a new first machine learning model, and further, the first machine learning model A is replaced with the third machine learning model C to set the third machine learning model C as a new second machine learning model B.

[0047] Next, a second embodiment according to the present invention will be described with reference to FIGS. 7 and 8. In this second embodiment, it is evaluated which of the first machine learning model A and the second machine learning model B is superior. When it is evaluated that the first machine learning model A is superior, the vehicle control device is controlled using the first machine learning model A and the second machine learning model B is relearned. When it is evaluated that the second machine learning model B is superior, the vehicle control device is controlled using the second machine learning model B and the first machine learning model A is relearned. Note that also in this second embodiment, the comparison data creation routine for evaluating the machine learning models A and B shown in FIG. 4 is used, and the comparison data table for evaluating the machine learning models A and B shown in FIG. 5 is used.

[0048] FIG. 7 shows a control system 48 used in the second embodiment according to the present invention. The difference between this control system 48 and the control system 41 shown in FIG. 2 is that in the control system 48, the output of the system unit 43 having the first machine learning model A indicated by reference numeral 42 and the output of the system unit 45 having the second machine learning model B indicated by reference numeral 44 are connected to the vehicle control device 40 via the switching device 49, and the first machine learning model A and the second machine learning model B are only stored in the memory 16 of the electronic control unit 13. In other respects, it is the same as the control system 41 shown in FIG. 2, and the description of this identical part will be omitted. Note that in this second embodiment, either the output of the system unit 43 or the output of the system unit 45 is connected to the vehicle control device 40 by the switching device 49, and the vehicle control device 40 is controlled by the output of either the system unit 43 or the system unit 45.

[0049] FIG. 8 shows a model update processing routine executed by the processor in the data server 30 to implement the second embodiment according to the present invention. This routine is executed when the comparison data table shown in FIG. 5 is newly created.

[0050] Referring to FIG. 8, first, in step 90, a comparison data creation routine for evaluating the machine learning models A and B shown in FIG. 4 is executed, and a comparison data table for evaluating the machine learning models A and B shown in FIG. 5 is created. Next, in step 91, based on the comparison data table for evaluating the machine learning models A and B shown in FIG. 5, a comparative evaluation is performed to determine which of the first machine learning model A and the second machine learning model B is superior. In this case, for example, in the comparison data table for evaluating the machine learning models A and B shown in FIG. 5, the machine learning model with a larger number of correct answers is considered superior. When it is evaluated in step 91 that the first machine learning model A is superior, the process proceeds to step 92 to determine whether the first machine learning model A is being used for the control of the vehicle control device 40.

[0051] In step 92, when it is determined that the first machine learning model A is being used for the control of the vehicle control device 40, the process proceeds to step 94. On the other hand, in step 92, when it is determined that the first machine learning model A is not being used for the control of the vehicle control device 40, the process proceeds to step 93, and the output of the system unit 43 is connected to the vehicle control device 40 by the switching device 49 so that the vehicle control device 40 is controlled by the first machine learning model A. Then, the process proceeds to step 94. In step 94, adjustment of the learning data required to relearn the second machine learning model B is performed in the same manner as described in the first embodiment.

[0052] Next, in step 95, the second machine learning model B is relearned using the adjusted learning data. When the relearning of the second machine learning model B is completed, the process proceeds to step 96, and the update information of the relearned second machine learning model B is transmitted to each vehicle. That is, the update command of the second machine learning model B held in the system unit 45 shown in FIG. 7 is transmitted from the data server 30 to each vehicle. When each vehicle receives the update command of the second machine learning model B, the second machine learning model B stored in the memory 16 of the electronic control unit 13 is updated according to the update information. At this time, the output signal of the second machine learning model B is not used to control the vehicle control device 40, that is, the braking device 11.

[0053] As long as it continues to be evaluated that the first machine learning model A is superior in step 91, the second machine learning model B continues to be relearned in step 95, and the second machine learning model B held in the system unit 45 shown in FIG. 7 continues to be updated according to the update information. Therefore, as time passes, the discrimination ability of the second machine learning model B increases.

[0054] When it is evaluated in step 91 that the second machine learning model B is superior, the process proceeds to step 97 to determine whether the second machine learning model B is used for the control of the vehicle control device 40. In step 92, when it is determined that the second machine learning model B is used for the control of the vehicle control device 40, the process proceeds to step 99. On the other hand, in step 97, when it is determined that the second machine learning model B is not used for the control of the vehicle control device 40, the process proceeds to step 98, and the output of the system unit 45 is connected to the vehicle control device 40 by the switching device 49 so that the vehicle control device 40 is controlled by the second machine learning model B. Next, the process proceeds to step 99. In step 99, the learning data necessary for relearning the first machine learning model A is adjusted in the same manner as described in the first embodiment.

[0055] Next, in step 100, the first machine learning model A is relearned using the adjusted learning data. When the relearning of the first machine learning model A is completed, the process proceeds to step 101, and the update information of the relearned first machine learning model A is transmitted to each vehicle. That is, the update command of the first machine learning model A held in the system unit 43 shown in FIG. 7 is transmitted from the data server 30 to each vehicle. When each vehicle receives the update command of the first machine learning model A, the first machine learning model A stored in the memory 16 of the electronic control unit 13 is updated according to the update information. At this time, the output signal of the first machine learning model A is not used to control the vehicle control device 40, that is, the braking device 11.

[0056] As long as the second machine learning model B continues to be evaluated as superior in step 91, the first machine learning model A continues to be relearned in step 100, and the first machine learning model A held in the system unit 43 shown in FIG. 7 continues to be updated according to the update information. Therefore, as time passes, the discrimination ability of the first machine learning model A increases.

[0057] As described above, in the second embodiment according to the present invention, the vehicle 1 is provided with the processor 15 mounted on the vehicle 1 and the memory 16 mounted on the vehicle 1. The memory 16 stores a plurality of machine learning models that output specific information used for controlling the vehicle control device 40 when receiving the output signal of the sensor 18 mounted on the vehicle 1, and the machine learning models include a first machine learning model A and a second machine learning model B that are different from each other. In this case, as described above, in the embodiment according to the present invention, this specific information represents, for example, detection information of moving objects such as vehicles and pedestrians recognized by a front camera, stationary obstacles, or detection information indicating that the distance to a moving object such as a vehicle or a pedestrian or a stationary obstacle has become equal to or less than a set distance.

[0058] Furthermore, in the second embodiment according to the present invention, the processor 15 causes the output signal of the sensor 40 mounted on the vehicle 1 to be input to both the first machine learning model A and the second machine learning model B, and causes the first machine learning model A and the second machine learning model B to output the above-described specific information. The vehicle control device 40 is controlled using either the above-described specific information output from the first machine learning model A or the above-described specific information output from the second machine learning model B, and the recognition content in the first machine learning model A and the recognition content in the second machine learning model B are transmitted to the data server 30 outside the vehicle. In this case, as described above, in the embodiment according to the present invention, this recognition content indicates a vehicle, a pedestrian, or the like.

[0059] Furthermore, in the second embodiment according to the present invention, the processor of the data server 30 compares and evaluates which of the first machine learning model A and the second machine learning model B is superior based on the recognition content in the first machine learning model A and the recognition content in the second machine learning model B. When the first machine learning model A is evaluated to be superior to the second machine learning model B, the above-described specific information output from the first machine learning model A is used for the control of the vehicle control device 40. When the first machine learning model A is evaluated to be superior to the second machine learning model B, the second machine learning model B is re-learned, and the updated information of the re-learned second machine learning model B is transmitted to the vehicle 1. Further, the processor 15 mounted on the vehicle 1 is configured to update the second machine learning model B stored in the memory 16 mounted on the vehicle 1 according to the received updated information.

[0060] Furthermore, in the second embodiment according to the present invention, when the processor of the data server 30 evaluates that the second machine learning model B is superior to the first machine learning model A, it uses the control signal output from the second machine learning model B for the control of the vehicle control device 40. When the second machine learning model B is evaluated to be superior to the first machine learning model A, the first machine learning model A is re-learned, and the update information of the re-learned first machine learning model A is transmitted to the vehicle 1. Further, the processor 15 mounted on the vehicle 1 is configured to update the first machine learning model A stored in the memory 16 mounted on the vehicle 1 according to the received update information.

Explanation of Reference Numerals

[0061] 1 Vehicle 15 Processor 18 Sensor 30 Data Server 40 Vehicle Control Device 42 First Machine Learning Model A 43 System Unit 44 Second Machine Learning Model B 45 System Unit

Claims

1. A vehicle is equipped with a processor, and the vehicle is equipped with a plurality of machine learning models that output specific information for use in controlling a vehicle control device when an output signal of a sensor mounted on the vehicle is input. The machine learning models include a first machine learning model and a second machine learning model that are different from each other. The processor inputs the output signal of the sensor mounted on the vehicle to both the first machine learning model and the second machine learning model to output the specific information from the first machine learning model and the second machine learning model, controls the vehicle control device using the specific information output from the first machine learning model among the specific information output from the first machine learning model and the specific information output from the second machine learning model, is configured to transmit the recognition content in the first machine learning model and the recognition content in the second machine learning model to a data server outside the vehicle, The processor of the data server compares and evaluates which of the first machine learning model and the second machine learning model is superior based on the recognition content in the first machine learning model and the recognition content in the second machine learning model, when the first machine learning model is evaluated to be superior to the second machine learning model, the second machine learning model is re-learned, is configured to transmit update information of the re-learned second machine learning model to the vehicle, The processor mounted on the vehicle A model update system configured to update the second machine learning model mounted on the vehicle according to the received update information.

2. The processor of the data server is configured to store and accumulate in the memory of the data server a comparison evaluation result regarding which of the first machine learning model and the updated second machine learning model is superior, The model update system according to claim 1, wherein when the updated second machine learning model is evaluated to be superior to the first machine learning model, a comparison evaluation indicating that the updated second machine learning model is superior to the first machine learning model can be collected from the stored comparison evaluation result.

3. The processor of the data server If the first machine learning model can be replaced with an updated second machine learning model based on the collected comparative evaluations, a third machine learning model different from both the first machine learning model and the updated second machine learning model is obtained, and an update command to replace the first machine learning model and the updated second machine learning model and the third machine learning model are configured to be transmitted to the vehicle. The model update system according to claim 2, wherein when a processor mounted on the vehicle receives an update command to replace the first machine learning model and the updated second machine learning model, the first machine learning model and the updated second machine learning model are replaced, the updated second machine learning model is set as a new first machine learning model, and further, the first machine learning model is replaced with the third machine learning model to set the third machine learning model as a new second machine learning model.

4. Comprising a processor mounted on the vehicle and a memory mounted on the vehicle. The memory stores a plurality of machine learning models that output specific information used for controlling the vehicle control device when receiving the output signal of a sensor mounted on the vehicle. The machine learning models include a first machine learning model and a second machine learning model that are different from each other. The processor is configured to input the output signal of a sensor mounted on the vehicle to both the first machine learning model and the second machine learning model to output the specific information from the first machine learning model and the second machine learning model. The vehicle control device is controlled using either the specific information output from the first machine learning model or the specific information output from the second machine learning model. configured to transmit the recognition content in the first machine learning model and the recognition content in the second machine learning model to a data server outside the vehicle. The processor of the data server is configured to compare and evaluate which of the first machine learning model and the second machine learning model is superior based on the recognition content in the first machine learning model and the recognition content in the second machine learning model. When the first machine learning model is evaluated to be superior to the second machine learning model, the specific information output from the first machine learning model is used for controlling the vehicle control device. When the first machine learning model is evaluated to be superior to the second machine learning model, the second machine learning model is re-learned. configured to transmit update information of the re-learned second machine learning model to the vehicle, a processor mounted on the vehicle, configured to update a second machine learning model stored in a memory mounted on the vehicle according to the received update information, the processor of the data server, when the second machine learning model is evaluated to be superior to the first machine learning model, using a control signal output from the second machine learning model for the control of the vehicle control device, when the second machine learning model is evaluated to be superior to the first machine learning model, re-learning the first machine learning model, configured to transmit update information of the re-learned first machine learning model to the vehicle, a processor mounted on the vehicle, a model update system configured to update a first machine learning model stored in a memory mounted on the vehicle according to the received update information.

Images