Vehicle control device and vehicle-mounted program

Abstract

In a vehicle control device (10) mounted on a vehicle (101), there are provided a data acquisition unit (14) that acquires data related to sensor information acquired by various sensors (20) mounted on the vehicle, a control unit (12) that performs vehicle control based on the sensor information, a preprocessing unit (16) that performs preprocessing on the data acquired by the data acquisition unit in a predetermined period before transmission, and a transmission unit (17) that transmits the data preprocessed by the preprocessing unit to a server (102) via a communication network (103), the predetermined period being set when the control unit does not perform drive control related to travel of the vehicle.

Description

[0001] Cross-references to related applications

[0002] This application is based on Japanese Patent Application No. 2024-069045, filed on April 22, 2024, the contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to vehicle control devices and onboard programs. Background Technology

[0004] In recent years, a method has emerged that involves collecting various data actually obtained from the driving process of vehicles after they are sold via communication networks to a server, analyzing the data collected from the server, and using it for future vehicle development or to update the vehicle's control program.

[0005] As one example, a verification method known as shadow mode is known. To verify the performance and safety of a vehicle control program, the vehicle control program, which is the object of verification, is run during the driving of a vehicle after it has been sold, and its output values ​​are verified. In shadow mode-based verification, for example, the vehicle control program, which is the object of verification, is run in the background, in a manner independent of vehicle control, and data related to its output values ​​is stored. This data is then collected to a server via an external network, etc. The operation of the vehicle control program is then verified by analyzing the collected data. Such techniques related to shadow mode are described, for example, in Patent Document 1, etc.

[0006] Patent Document 1: Japanese Patent Application Publication No. 2023-38697

[0007] However, the process of running multiple vehicle control programs and collecting and sending data has the problem of high processing load. Summary of the Invention

[0008] This disclosure was made in view of the above circumstances, and its main purpose is to provide a vehicle control device and on-board program capable of distributing the load.

[0009] The vehicle control device that solves the above-mentioned problem is a vehicle control device mounted on a vehicle, including: a data acquisition unit that acquires data related to sensor information acquired by various sensors mounted on the vehicle; a control unit that performs vehicle control based on the sensor information; a preprocessing unit that preprocesses the data acquired by the data acquisition unit during a predetermined period; and a transmission unit that transmits the preprocessed data to a server via a communication network, wherein the predetermined period is set when the control unit does not perform drive control related to the driving of the vehicle.

[0010] This allows for the separation of heavy preprocessing and other heavy processing tasks, such as driving-related drive control, from the heavy preprocessing.

[0011] The procedure for solving the above-mentioned problem is executed by a vehicle control device mounted on a vehicle. The procedure causes the vehicle control device to perform: a data acquisition step, acquiring data related to sensor information acquired by various sensors mounted on the vehicle; a control step, implementing vehicle control based on the sensor information; a preprocessing step, preprocessing the data acquired by the data acquisition step for a predetermined period; and a transmission step, transmitting the preprocessed data to a server via a communication network, wherein the predetermined period is set to when drive control related to the driving of the vehicle is not implemented in the control step.

[0012] This allows for the separation of heavy preprocessing and other heavy processing tasks, such as driving-related drive control, from the heavy preprocessing. Attached Figure Description

[0013] The foregoing and other objects, features, and advantages of this disclosure will become more apparent from the following detailed description with reference to the accompanying drawings, which are shown below.

[0014] Figure 1 This is a schematic diagram of the data collection system.

[0015] Figure 2 It is a block diagram representing the functionality of the verification processing device.

[0016] Figure 3 This is a flowchart of data acquisition and processing.

[0017] Figure 4 This is a flowchart of the preprocessing.

[0018] Figure 5 This is a diagram used to illustrate the preprocessing process.

[0019] Figure 6 This is a flowchart of the preprocessing for the variant example. Detailed Implementation

[0020] Hereinafter, embodiments of the vehicle control device and program of this disclosure will be described in detail with reference to the accompanying drawings. Furthermore, in principle, the same or equivalent parts in the drawings will be labeled with the same reference numerals between embodiments and variations, and their descriptions will not be repeated.

[0021] Figure 1 A data collection system 100 using the vehicle control device 10 in this embodiment is shown. The vehicle control device 10 is mounted on a vehicle 101 and performs control or driving assistance for the vehicle 101.

[0022] like Figure 1 As shown, the data collection system 100 includes a server 102 and is configured to communicate with one or more vehicles 101 via a communication network 103 such as the Internet. Figure 1 The diagram shows only one vehicle 101. Vehicle 101 includes a vehicle control device 10, a sensor 20, an actuator 30, etc.

[0023] Sensor 20 includes various sensors for determining the driving state of vehicle 101, such as vehicle speed sensor 21, acceleration sensor 22, yaw rate sensor 23, etc. The sensors for determining the driving state of vehicle 101 may also include sensors other than these, or may not include any one of vehicle speed sensor 21, acceleration sensor 22, yaw rate sensor 23.

[0024] Furthermore, sensor 20 includes various sensors for detecting other vehicles or obstacles, such as camera 24, millimeter-wave radar 25, etc. The sensors for detecting other vehicles or obstacles are not limited to millimeter-wave radar 25, etc., but can also be LiDAR, ultrasonic sensors, etc., and multiple of these sensors can be combined. Furthermore, camera 24 can be a monocular camera or a binocular camera. Furthermore, camera 24 can capture either still images or videos. Furthermore, the number, position, and type of camera 24 can be arbitrarily changed. For example, a front-facing camera with a shooting range in front of the vehicle, a right-side camera with a shooting range to the right of the vehicle, a left-side camera with a shooting range to the left of the vehicle, and a rear-facing camera with a shooting range behind the vehicle can be equipped.

[0025] In addition, sensor 20 includes various sensors for detecting various amounts of driver operation, such as acceleration sensor 26 for detecting the amount of driver accelerator operation, brake sensor 27 for detecting the amount of braking operation, and steering angle sensor 28 for detecting the amount of driver operation on the steering wheel (steering wheel) (steering angle).

[0026] These sensors 20 are connected to the vehicle control unit 10 wirelessly or via wired means, and the measurement results (or detection results) of these sensors 20 are input into the vehicle control unit 10, etc., as sensor information.

[0027] The actuator 30 includes, for example, an actuator for driving the vehicle 101, such as an electric motor that serves as the power source for the vehicle 101. It may also include actuators for controlling the movement of the vehicle 101, such as actuators for operating the steering wheel, the brake pedal, and the accelerator pedal. Furthermore, the actuator 30 may include devices for activating equipment on the vehicle 101 such as displays, speakers, indicator lights, and headlights. Through these actuators 30, the driving and movement of the vehicle 101 are controlled.

[0028] The vehicle control unit 10 is primarily composed of a microcomputer equipped with a processing unit 10a such as a CPU and storage units 10b such as various memories. The functions provided by the microcomputer can be provided through software recorded in a tangible storage device and a computer executing that software, software only, hardware only, or a combination thereof. For example, when the microcomputer is provided by electronic circuitry as hardware, it can be provided by digital circuitry or analog circuitry containing multiple logic circuits. For example, the processing unit 10a of the microcomputer executes a program stored in a non-transitory tangible storage medium, which is itself a storage unit 10b. The program includes, for example, implementations of... Figure 2 The program performs the functions shown. By executing the program, the method corresponding to the program is executed. The storage unit 10b is, for example, a non-volatile memory. Furthermore, the program stored in the storage unit 10b can be downloaded and updated, for example, via OTA (Over The Air) or a communication network 103 such as the Internet.

[0029] The vehicle control unit 10 has various functions (applications) for assisting in driving the vehicle 101, through which the actuator 30 is controlled based on sensor information input from the sensor 20.

[0030] Representative functions used to assist the driving of Vehicle 101 include: Adaptive Cruise Control (ACC), Forward Collision Warning (FCW), Advanced Emergency Braking (AEBS), Night Vision / Pedestrian Detection (NV / PD), Traffic Sign Recognition (TSR), Lane Departure Warning (LDW), Lane Keeping Assist System (LKAS), Rear Cross Traffic Alert (RCTA), Adaptive Front Lighting System (AFS), and Advanced Parking Assist (APA). Vehicle 101 may possess all of these functions, or only some of them. Furthermore, it may also possess other driving assistance functions. Additionally, it may include functions for achieving autonomous driving of Vehicle 101. Furthermore, these functions are implemented by the arithmetic processing unit 10a executing the driving assistance control program stored in the storage unit 10b.

[0031] Furthermore, to verify the performance and safety of the driver assistance control program, the vehicle control device 10 has a function called shadow mode. This means that during the operation of the vehicle 101 after it has been sold—that is, in actual use scenarios—the driver assistance control program (hereinafter referred to as the verification object program) is run and its output values ​​are verified. In shadow mode-based verification, for example, the verification object program is run in the background in a manner independent of vehicle control, and data related to its output values ​​is stored. This data is then collected from the output values ​​to the server 102 via the communication network 103. Furthermore, shadow mode is a data collection mode for collecting data.

[0032] The following describes the configuration and functions related to the shadow mode according to this embodiment. The timing of switching to shadow mode can be arbitrary. For example, it can be switched by the driver's operation or by an instruction from the server 102. Alternatively, it can be switched to shadow mode simultaneously with the ignition switch being turned on.

[0033] In addition, for comparison with the verification object program, the driving assistance control program that actually makes vehicle 101 run will be referred to as the installation program below. Furthermore, in Figure 1 The program is also referred to as the installer. The verification object program is stored in storage unit 10b. In addition, a storage device dedicated to shadow mode can also be set.

[0034] By executing the verification object program, various functions for assisting the driving of vehicle 101 are executed in the background. The functions implemented by executing the verification object program may be the same as the functions actually implemented by the arithmetic processing unit 10a, or some may be omitted. In addition, functions other than those actually implemented by the arithmetic processing unit 10a may also be implemented.

[0035] When the processing unit 10a runs the verification object program in shadow mode, it inputs sensor information from various sensors 20. Then, based on the input sensor information, the processing unit 10a implements various functions of the verification object program and outputs various control signals to activate each actuator 30. These control signals (output results) based on the verification object program are not actually input to the actuators 30, but are stored in the storage unit 10b as data related to the output results. At this time, the processing unit 10a also associates and stores the data related to the sensor information as input values.

[0036] Furthermore, the processing unit 10a stores the control signals output after processing based on the installation program in the storage unit 10b as data related to the output results. In other words, in order to compare the output results in the verification installation program with the output results in the verification target program, the control signals output after processing based on the installation program are also stored.

[0037] Furthermore, the processing unit 10a can also input control signals, processed based on the installation procedure, into the actuator 30, thereby causing sensor information (vehicle speed, yaw rate, acceleration, accelerator pedal operation amount, brake pedal operation amount, steering angle, etc.) detected when the vehicle 101 moves, and store it in the storage unit 10b. In other words, it can also store sensor information related to the actual movement of the vehicle 101 for each scenario, i.e., sensor information required to verify correct movement.

[0038] The vehicle control unit 10 uploads the data stored in the storage unit 10b to the server 102 via the communication network 103 at a predetermined transmission time. The predetermined transmission time is arbitrary, such as when the server 102 issues an upload instruction. Alternatively, it could be when the vehicle 101 is charging, when it is parked or stopped, or when the ignition switch is off, etc.

[0039] However, if all vehicles 101 upload all of their stored data, it may place an excessive burden on the communication network 103. Therefore, it is desirable to compress the data before transmission. However, performing compression in parallel on top of simultaneously executing the installation program and the verification program may result in an excessive processing load on the computing processing unit 10a. Therefore, the vehicle control device 10 of this embodiment performs processing in a distributed manner to avoid placing an excessive load on the computing processing unit 10a. A detailed description will follow.

[0040] like Figure 2 As shown, the arithmetic processing unit 10a of the vehicle control device 10 has the functions of an input unit 11, a control unit 12, a verification unit 13, a data acquisition unit 14, a classification unit 15, a preprocessing unit 16, and a transmission unit 17. These functions are implemented by the arithmetic processing unit 10a by executing the vehicle program stored in the storage unit 10b.

[0041] Input unit 11 inputs sensor information from sensor 20. Then, input unit 11 inputs part or all of the input sensor information as input values ​​for the installation program to control unit 12. In addition, input unit 11 inputs part or all of the input sensor information as input values ​​for the verification target program to verification unit 13. Furthermore, input unit 11 inputs part or all of the input sensor information to data acquisition unit 14.

[0042] The control unit 12 runs the installation program and performs processing based on the input values ​​(sensor information) input from the input unit 11, implementing various functions (application programs) implemented by the installation program. Then, the control unit 12 inputs control signals (control signals of the actuator 30) as the result of its processing to the actuator 30 and the data acquisition unit 14.

[0043] The verification unit 13 runs the verification target program and performs processing based on the input values ​​(sensor information) input from the input unit 11, implementing various functions (application programs) implemented by the verification target program. Then, the verification unit 13 inputs control signals (control signals for the actuator 30) as the result of its processing to the data acquisition unit 14. Furthermore, as described above, these control signals are not input to the actuator 30.

[0044] The data acquisition unit 14 determines whether the acquisition condition is met based on at least one of the input value (sensor information) input from the input unit 11 and the output result (control signal) input from the verification unit 13.

[0045] Acquisition conditions may include thresholds of one or more parameters as constituent elements. Parameters for acquisition conditions may be any of the parameters contained in sensor information such as vehicle speed, yaw rate, accelerator operation amount, braking operation amount, and steering amount, or parameters contained in output results such as requested torque and output indication signals to the display. Furthermore, parameters calculated or estimated from sensor information and output results may also be included. For example, the relative distance to an obstacle (such as a vehicle or pedestrian ahead), the relative speed to the obstacle, and the time-to-collision (TTC) calculated from the recognition results of the camera image or the detection results of the millimeter-wave radar 25 can be set as parameters for acquisition conditions. Additionally, the difference between the output results of the installation program and the output results of the verification object program can be set as a parameter for acquisition conditions. For example, the difference between the braking operation amount output by the processing based on the installation program and the braking operation amount output by the processing based on the verification object program can also be used as a parameter. The threshold may be either an upper or lower limit value of any of these parameters, or both.

[0046] In addition, whether the scenario in which vehicle 101 is driving (hereinafter referred to as the scenario) is a predetermined acquisition scenario can also be used as a constituent element of the acquisition conditions. A scenario refers to various scenarios that are imagined during the driving of vehicle 101, such as the scenario of overtaking a vehicle in front, the scenario of weaving through gaps in a vehicle, the scenario of following a vehicle in front, the scenario of a pedestrian crossing in front of vehicle 101 at night, the scenario of vehicle 101 merging from the acceleration lane into the main lane on a dedicated motor vehicle road, the scenario of parking or stopping, the scenario of waiting for a traffic light, etc.

[0047] These scenarios are estimated based on sensor information. For example, the vehicle control unit 10 can recognize camera images to estimate the scenario. More specifically, sensor information such as camera images can be input into a machine learning inference model such as a deep neural network to estimate the scenario. Furthermore, image recognition does not necessarily have to be performed by the vehicle control unit 10; it can also be performed by an external device of the vehicle control unit 10 (such as an image recognition device), and its results can be used as sensor information input.

[0048] Furthermore, the functions (execution functions) that have been executed in various functions (application programs) based on the verification object program can be used as components of the acquisition conditions. For example, the acquisition conditions can be met if the collision damage mitigation braking control function has been executed. In addition, whether a certain function has been executed can be determined based on the output result (control signal) input from the verification unit 13.

[0049] In this embodiment, the acquisition conditions are set by combining the acquisition scenario, the executed function, and the threshold of the parameter (i.e., AND conditions). For example, it can be set as follows: the acquisition condition is met when the vehicle speed (the parameter of the acquisition condition) is above the threshold (50km / h) in the scenario of following the vehicle in front.

[0050] Furthermore, the combination of components of an acquisition condition can be an acquisition scenario and an execution function, an acquisition scenario and a parameter threshold, or an execution function and a parameter threshold. Additionally, a component of an acquisition condition can include two or more execution functions, in which case the execution functions can be ANDed or ORed. For example, both the forward vehicle approach warning function and the collision damage mitigation braking control function can be executed as a condition, or only one of them can be executed. Similarly, the parameter threshold can also be the threshold of two or more parameters.

[0051] Furthermore, the condition for acquisition being met refers to the situation where all the constituent elements constituting the condition for acquisition are satisfied. For example, if the acquisition scenario of the condition for acquisition is "following the vehicle in front", the execution function of the condition for acquisition is "follow driving function", and the threshold of the parameter of the condition for acquisition is "above 50km / h", then in the scenario of following the vehicle in front, if the follow driving function is executed and the vehicle speed (the parameter of the condition for acquisition) is above the threshold (50km / h), then the condition for acquisition is met.

[0052] Furthermore, the acquisition condition is not limited to one; multiple acquisition conditions can be set. For example, the first acquisition condition can be set as: following a vehicle in front, and the vehicle speed is above a first threshold; the second acquisition condition can be set as: a pedestrian crossing in front of vehicle 101, and the distance between the vehicle and the pedestrian is below a second threshold, and the braking operation amount is above a third threshold.

[0053] If any of the acquisition conditions are met, the data acquisition unit 14 stores the input values ​​and related output results in the storage unit 10b. As described above, the input values ​​include, for example, sensor information input to the processing unit 10a. Furthermore, the output results include, for example, control signals output after processing based on the verification object program and control signals output after processing based on the installation program. Additionally, the output results may also include sensor information related to the actual movement of the vehicle 101 for each scenario. Furthermore, the output results may also include the difference between the output results based on the installation program and the output results based on the verification object program. Moreover, the data to be stored (acquired) can be changed depending on the acquisition conditions that are met. For example, if the acquisition condition is met in a scenario where the vehicle is following another vehicle and the vehicle speed is at or above a first threshold, data related to the vehicle speed is acquired; on the other hand, if the acquisition condition is met in a scenario where a pedestrian is crossing in front of the vehicle 101, the distance between the vehicle and the pedestrian is at or below a second threshold, and the braking operation amount is at or above a third threshold, data related to the braking operation amount can be acquired.

[0054] Furthermore, the data is stored with attached tagging information so that it is aggregated into a single record for each valid acquisition condition. For example, sensor information triggered by the acquisition condition is aggregated into a single data record corresponding to the valid acquisition condition and stored as such.

[0055] Next, the classification unit 15 will be described. Based on the sensor information input from the input unit 11, the classification unit 15 classifies the data acquired (stored) by the data acquisition unit 14 according to the driving scenario of the vehicle 101. The driving scenario of the vehicle 101 is the same as described above. That is, the scenario refers to various scenarios imagined during the driving of the vehicle 101, such as overtaking a vehicle in front, weaving through gaps in a vehicle, following a vehicle in front, a pedestrian crossing in front of the vehicle 101 at night, the vehicle 101 merging from the acceleration lane into the main lane on a dedicated motor vehicle road, parking or stopping, waiting at a traffic light, etc.

[0056] These scenarios are estimated based on sensor information. For example, the vehicle control unit 10 can identify camera images to estimate the scenario. In this case, several benchmark scenarios can be pre-defined, and sensor information representing the characteristics of each scenario can be pre-stored. Then, by comparing the sensor information with each other, the benchmark scenario with the highest similarity among the multiple benchmark scenarios is estimated as the current scenario. More specifically, sensor information such as camera images can be input into a machine learning inference model such as a deep neural network to estimate the scenario.

[0057] Furthermore, image recognition need not be performed by the vehicle control unit 10; it can be performed by an external device of the vehicle control unit 10, such as an image recognition device, and the result can be used as sensor information input. Moreover, scene estimation is performed when the data acquisition unit 14 determines whether the acquisition conditions are met, or when the verification unit 13 performs various functions of the verification object program; therefore, the estimation results can be reused. This reduces the control burden.

[0058] Then, when the data acquisition unit 14 stores the data in the storage unit 10b, it associates the scenes classified (estimated) by the classification unit 15 with the data and stores them together. Thus, the data is stored according to scene classification when it is stored in the storage unit 10b.

[0059] Next, the preprocessing unit 16 will be described. During a predetermined period, the preprocessing unit 16 performs preprocessing on the data acquired by the data acquisition unit 14 and stored in the storage unit 10b before transmission. The preprocessing in this embodiment includes compression processing to reduce the data volume. When performing compression processing, the preprocessing unit 16 compresses the data volume by deleting any data from a plurality of data categorized into predetermined scenes, or by periodically removing a portion of the constituent elements of the data. In the case of deletion, all data as a group is deleted. Furthermore, periodically removing a portion of the constituent elements of the data refers to, for example, reducing the frame rate of video data when the data includes video data, or reducing the sampling rate of audio data when the data includes audio data. That is, when the data is detected in a time sequence at a predetermined period, it means extending the detection period, resulting in digital data with a reduced number of samples. Furthermore, periodically removing a portion of the constituent elements of the data can also be done by reducing color information, etc. Hereinafter, the deletion of any data from a plurality of data, or the periodic removal of a portion of the constituent elements of the data, will be simply referred to as "compression."

[0060] The predetermined period and method for performing preprocessing are described in detail. In this embodiment, the predetermined period for performing preprocessing is during a period when the control unit 12 is not implementing drive control related to the driving of the vehicle 101. The predetermined period for performing preprocessing can be set at any time, as long as drive control related to driving is not implemented; for example, it can be set during a period when the vehicle 101 is parked or stopped, during a period when the vehicle 101 is charging, or during a monitoring mode during which the area around the parked or stopped vehicle 101 is monitored. Furthermore, in this embodiment, the compression processing is preferably performed before the remaining storage capacity of the data in the storage unit 10b falls below a predetermined value.

[0061] During compression, the preprocessing unit 16 compresses any one of the data items categorized into various scenarios. For example, if a priority or urgency level is pre-set for each scenario, the preprocessing unit 16 deletes any one of the data items categorized as low-priority or low-urgency scenarios from the storage unit 10b. Furthermore, for example, the preprocessing unit 16 compresses any one of the data items categorized into scenarios specified by the server 102 (e.g., designated as unnecessary). Additionally, for example, the preprocessing unit 16 compresses the data from the scenario with the largest data volume among the various scenarios categorized in the storage unit 10b.

[0062] The preprocessing unit 16 can arbitrarily change which data among the multiple data contained in the compression target scene is compressed. For example, the preprocessing unit 16 can compress (delete or periodically remove data) the first or last stored data among the multiple data contained in the compression target scene. In addition, it can also compress random data among the multiple data contained in the compression target scene.

[0063] In addition, the preprocessing unit 16 can also set priorities for multiple data classified as compression target scenes according to the order of specific parameters contained in the data, and compress (or delete) the multiple data contained in the compression target scene in order of priority from low to high.

[0064] For example, in the multiple data sets classified as a scene of shuttling through gaps in the workshop (a predetermined scene), the preprocessing unit 16 sets the priority of data with lower vehicle speeds (a specific parameter) to be lower. Then, when the preprocessing unit 16 needs to compress any data set in the multiple data sets classified as a scene of shuttling through gaps in the workshop, it prioritizes deleting data with lower vehicle speeds.

[0065] Furthermore, the amount of data to be compressed can be arbitrarily set. For example, a predetermined amount of data can be compressed. Alternatively, the amount of data to be compressed can be adjusted based on the remaining data storage capacity in the storage unit 10b. That is, the less remaining capacity, the more data can be compressed.

[0066] At a predetermined transmission time, the sending unit 17 transmits data acquired by the data acquisition unit 14 and preprocessed by the preprocessing unit 16 and stored in the storage unit 10b to the server 102 via the communication network 103. The predetermined transmission time is as described above.

[0067] The following is for reference Figure 3 The data acquisition process in this embodiment will be described. The data acquisition process shown below is the data acquisition processing process executed by the computing processing unit 10a. These processes are performed after the shadow mode is set. The timing of setting the shadow mode is arbitrary, such as when the ignition switch is turned on.

[0068] First, the input unit 11 of the processing unit 10a inputs sensor information from the sensor 20 (step S101). The control unit 12 runs the installation program and performs processing based on the input value (sensor information) input from the input unit 11, performing various functions implemented by the installation program (step S102). In step S102, the control unit 12 inputs a control signal as the result of its processing to the actuator 30. At this time, a control signal is also input to the data acquisition unit 14 as needed.

[0069] Furthermore, the verification unit 13 runs the verification target program and performs processing based on various functions implemented by the verification target program, based on the input values ​​(sensor information) input from the input unit 11 (step S103). In step S103, the verification unit 13 inputs a control signal as its processing result to the data acquisition unit 14. It is desirable that the processing of the verification unit 13 and the processing of the control unit 12 be performed in parallel, but if this is difficult to achieve due to processing load, the processing of the control unit 12 can be performed first, and the processing of the verification unit 13 can be performed during periods when the control load is reduced.

[0070] The data acquisition unit 14 determines whether the acquisition condition is met based on at least one of the input value (sensor information) and the output result (control signal, etc.) (step S104).

[0071] If the determination result is positive (if the acquisition condition is met), the classification unit 15 of the processing device 10a classifies the data related to the input value and the output result (step S105). Specifically, the classification unit 15 estimates the scene in which the vehicle 101 is driving based on the sensor information input in step S101. Then, the classification unit 15 classifies the data related to the input value and the output result into the estimated scene.

[0072] Then, the data acquisition unit 14 summarizes and stores (acquires) the data related to the input values ​​and output results in the storage unit 10b (step S105). When storing data, the data acquisition unit 14 stores it in a manner that the data related to the input values ​​and output results is classified into the scenarios classified (estimated) by the classification unit 15 in step S104. In this embodiment, the data to be stored is stored in association with label information about the classification scenario. In addition, the data acquisition unit 14 also stores the acquisition conditions that are met in association with the data. Then, the process ends. On the other hand, if the determination result in step S103 is negative, the arithmetic processing device 10a directly ends the data acquisition process. The data acquisition process is executed at a certain period after the shadow mode is set.

[0073] Furthermore, in this embodiment, step S105 is equivalent to a data acquisition step, and step S102 is equivalent to a control step.

[0074] Next, refer to Figure 4 Preprocessing will be described. Preprocessing is performed at a predetermined cycle. The preprocessing unit 16 determines whether the current period is a predetermined period during which preprocessing can be performed (hereinafter referred to as the preprocessing period) (step S201). The predetermined period is as described above (the predetermined period for performing preprocessing). If the determination result is positive, the preprocessing unit 16 determines the scenario to be compressed and compresses the data stored in the storage unit 10b that is the scenario to be compressed (step S202), and then ends the processing. At this time, the data compression method is as described above. On the other hand, if the determination result in step S201 is negative, the preprocessing ends. In this embodiment, step S202 is equivalent to the preprocessing step.

[0075] At a predetermined transmission time, the sending unit 17 transmits data acquired by the data acquisition unit 14 and stored in the storage unit 10b to the server 102 via the communication network 103. This data also includes data compressed by the preprocessing unit 16. During transmission, the sending unit 17 also transmits information related to the acquisition conditions associated with the data. Furthermore, tag information related to the scene classified by the classification unit 15 may also be transmitted along with the data. In this embodiment, the processing of data transmission by the sending unit 17 is equivalent to the transmission step.

[0076] When server 102 receives data sent from the transmitting unit 17 of each vehicle 101, it stores it in the storage device of server 102. When storing data, it can classify and store the data according to the scene based on scene-related tag information received along with the data. Furthermore, it can also store information related to the acquisition conditions that were met when the data was acquired. That is, based on scene-based classification, the data can be further classified and stored according to acquisition conditions. Thus, the data is grouped by scene, making data analysis easier.

[0077] The above implementation method achieves the following effects.

[0078] During a predetermined period when the control unit 12 is not performing drive control related to the driving of the vehicle 101, the preprocessing unit 16 compresses the data. This allows the heavy workload of compression processing to be distributed among other heavy workloads such as drive control and communication processing related to driving.

[0079] Data is categorized by scenario, determining which scenario will be compressed, and then compressed into any one of the multiple data sets within that scenario. This prevents pressure on storage capacity and reduces communication, processing, and analytical loads. Furthermore, since compression can be adjusted based on scenario, data can be collected without compression for scenarios requiring large amounts of data. Conversely, for scenarios with less data, compression can be used to collect data and perform efficient analysis.

[0080] When implementing various functions or determining whether acquisition conditions are met, the estimated scenario can be used to classify the data according to the scenario. Therefore, classification can be performed in parallel while acquiring data. Consequently, when performing data compression or other tasks later, there is no need to reclassify the data, which reduces the processing burden.

[0081] When the preprocessing unit 16 selects data to be compressed from multiple data sets categorized as compression targets, it compresses data sequentially, starting with the lowest priority data, based on a priority set according to the order of specific parameters contained in the data. Therefore, it can retain as much of the high-priority data that should be collected as possible, enabling efficient analysis.

[0082] The classification unit 15 uses a deep neural network to estimate (identify) similar scenes from multiple predetermined benchmark scenes based on sensor information input to the input unit 11, and classifies the data into the estimated scenes. In this way, since benchmark scenes are predetermined, developers can easily identify the envisioned scenes during analysis, facilitating the analysis process.

[0083] (Modifications of the implementation method)

[0084] In the above embodiments, the preprocessing unit 16 may perform anonymization processing instead of compression processing during preprocessing, or simultaneously with compression processing. Anonymization processing involves processing data such as image data in sensor information acquired as data to make it impossible to identify a specific individual. Specifically, personal information such as an individual's face or license plates of other vehicles in the image data may be pixelated or blurred with black or white. This personal information can be extracted using deep neural networks or the like. Furthermore, this extraction process typically has a high processing load, so anonymization processing is also one of the processes with a high processing load. On the other hand, anonymization processing is a process that does not need to be performed immediately. This anonymization processing can, for example, be performed during preprocessing after the processing (data compression) in step S202.

[0085] In the above embodiment, the timing of data transmission by the transmitting unit 17 can be set during preprocessing. For example, during preprocessing, the data can be transmitted by the transmitting unit 17 after the processing (data compression) in step S202. This is because although the transmission processing of verification data usually has a large processing load, it does not need to be executed immediately.

[0086] ·Reference Figure 5 The periods that can be set as the preprocessing period in the above embodiments will be explained. For example... Figure 5 As shown, the preprocessing period can be set during a period when fewer objects are detected by sensor 20. Figure 5 (The first column in the text). Periods with fewer objects can include, for example, when the ignition switch is off, when driving at high speed on a dedicated motorway, when the automatic brake is engaged, or when the gear lever is shifted to "P" (Park) or "N" (Neutral). Additionally, pre-processing periods can be set during sensor 20 malfunction, or during severe weather such as heavy rain, dense fog, or blizzards. Furthermore, these periods can also include when the vehicle battery is charging, during monitoring mode, or when vehicle 101 is parked or stopped. This is because objects cannot be detected during these periods. Objects include, for example, other vehicles, pedestrians, and structures (signs or traffic lights). Fewer objects refer to situations where the number of objects is below a threshold. The threshold for objects depends on the processing capability of the vehicle control unit 10, which identifies and processes objects. Severe weather conditions can be estimated using image recognition.

[0087] In addition, such as Figure 5 As shown in the second column, the preprocessing period can be set when the processing volume for the target is low. For example, the processing volume is low when driving at low speed, so processing is easier to implement. Low processing volume for the target means, for example, when the utilization rate (CPU utilization rate) of the computing processing unit 10a is below a specified value.

[0088] In addition, such as Figure 5 As shown in the third column, the preprocessing period can be configured when there are fewer applications running in the verification object program or installation program. For example, it can be configured for driving modes where the driver turns off driver assistance-related functions and controls the vehicle 101 independently, or when autonomous driving is stopped. Additionally, it can be configured for when the vehicle battery is charging, the ignition switch is off, monitoring mode is enabled, or the vehicle is parked. Furthermore, by obtaining location information through a navigation system or GPS, when driving on a dedicated road, applications applicable only to general roads can be paused, and preprocessing can be performed instead. Similarly, if the location information indicates driving on a general road, applications applicable only to highways can be paused, and preprocessing can be performed instead.

[0089] In addition, such as Figure 5As shown in the fourth column, the vehicle control device 10 can determine from the camera image whether there is a low processing load and set a preprocessing period. Low processing loads include, for example, when there are few objects or when the vehicle 101 is parked. Furthermore, it can be configured to determine from the camera image whether the vehicle 101 is in a predetermined location (parking location), such as the location of one's own parking lot, and set a preprocessing period if it is in that predetermined location.

[0090] In addition, such as Figure 5 As shown in the fourth column, the processing load of the arithmetic processing unit 10a can be calculated, and preprocessing can be performed when the processing load is low. Low processing load means, for example, when the utilization rate (CPU utilization) of the arithmetic processing unit 10a is below a specified value.

[0091] In the above embodiment, a storage device for the charger, such as a charger, can be provided for temporarily storing the transmitted data. Then, during charging, the data can be saved to the charger's storage device. The saved data is then sent to server 102 via the Internet or the like.

[0092] • In the above embodiment, scenes where the object is below a threshold can be identified from the image data, and when a scene where the object is below the threshold is identified, the preprocessing unit 16 performs preprocessing.

[0093] In the above embodiment, the data acquisition unit 14 may not acquire the control signal (data related to the output result) output after processing based on the installation program. On the other hand, it may acquire sensor information (vehicle speed, yaw rate, acceleration, accelerator pedal operation, brake pedal operation, steering angle, etc.) detected when the vehicle 101 moves, after the acquisition condition is met and until a predetermined time has elapsed. That is, the data related to this sensor information can be stored in the storage unit 10b.

[0094] • In the above embodiment, the verification unit 13 may not perform functions related to driver assistance. That is, the verification unit 13 may not be present. In this case, sensor information (vehicle speed, yaw rate, acceleration, accelerator pedal operation, brake pedal operation, steering angle, etc.) detected when the vehicle 101 moves due to the driver's operation can be acquired after the acquisition conditions are met and until a predetermined time has elapsed. That is, in order to collect this sensor information-related data as correct data, it may also be stored in the storage unit 10b.

[0095] In the above embodiment, the preprocessing unit 16 prioritizes multiple data points classified into predetermined scenarios according to the order of specific parameters (e.g., vehicle speed) contained in the data. These specific parameters are not limited to vehicle speed and can be arbitrarily changed. For example, the specific parameters may include any one of the following: vehicle speed, travel time, weather, road, type of detected object, shape of detected object, position of detected object, distance to detected object, speed of detected object, and collision margin time to detected object. Furthermore, since the specific parameters that should be prioritized differ in different scenarios, the specific parameters can be changed according to the scenario.

[0096] In the above embodiment, the classification unit 15 classifies the data according to the scenario, but it can also classify the data according to the established acquisition conditions. Then, the preprocessing unit 16 can compress the data classified into predetermined acquisition conditions.

[0097] Similarly, the classification unit 15 classifies the data according to scenarios, but it can also classify the data according to the functions (applications) that have been executed among the various functions implemented by the verification object program. Then, the preprocessing unit 16 can compress the data classified as the predetermined execution functions among the implemented functions.

[0098] In the above embodiment, the classification unit 15 estimates which reference scene is similar to the input sensor information. As another example, the classification unit 15 can also classify the data stored in the storage unit 10b according to similar scenes. For example, the classification unit 15 extracts features based on the sensor information (e.g., image data) contained in the data using deep learning or similar methods, and classifies the data according to similar scenes based on the similarity of these features. Then, the classification unit 15 groups the similar scenes together to categorize the data. That is, the classification unit 15 clusters the data based on the sensor information contained in the data and classifies the data according to each cluster consisting of similar scenes.

[0099] In the above embodiment, when the data acquisition unit 14 stores data in the storage unit 10b, it can specify the address of the storage unit 10b according to the priority of the data. Then, when the remaining storage capacity of the storage unit 10b decreases, the preprocessing unit 16 can automatically delete data stored at low-priority addresses, freeing up data areas to increase storage capacity. Accordingly, since the preprocessing unit 16 deletes data based on specified addresses when the remaining storage capacity decreases, compression instructions are not required, simplifying the process. Furthermore, storage fragmentation can be prevented, effectively increasing storage capacity.

[0100] In the above embodiments, responsiveness or time resolution can be preset according to the scenario. A high-responsiveness scenario indicates that data should be sent to server 102 as quickly as possible after acquisition; a low-responsiveness scenario indicates that even if data is sent to server 102 slowly after acquisition, there is no problem. For example, a scenario with a high probability of an accident or a scenario where automatic braking is activated can be considered a high-responsiveness scenario. A low-time-resolution scenario indicates a scenario that changes little over time; a high-time-resolution scenario indicates a scenario that changes significantly over time. For example, a traffic jam on a dedicated road can be considered a low-time-resolution scenario.

[0101] Furthermore, when the responsiveness or time resolution is preset according to the scenario, the transmitting unit 17 can periodically discard (compress) data when transmitting data from scenarios with low responsiveness or time resolution. In addition, the preprocessing unit 16 can compress data sequentially, starting from data classified as low responsiveness or time resolution, based on the remaining storage capacity in the storage unit 10b used for storing data.

[0102] In the above embodiment, the vehicle control device 10 can take into account the processing burden when executing the installation procedure and includes a verification processing unit for executing the verification target program. In this case, some or all of the functions other than the control unit 12, namely the input unit 11, verification unit 13, data acquisition unit 14, classification unit 15, preprocessing unit 16, and transmission unit 17, can be implemented by the verification processing unit. The verification processing unit can have the same performance as the processing unit 10a or be different, as long as it has the performance for executing the verification target program.

[0103] • In the above embodiment, a classification unit 15 is provided to classify data according to scenarios, but data can also be saved directly without classification. In this case, the classification unit 15 is not necessary.

[0104] ·Reference Figure 6 A variation of the preprocessing described in the above embodiments will be explained. Figure 6 The preprocessing shown is performed at a predetermined cycle. Similar to step S201, preprocessing unit 16 determines whether it is within a predetermined period during which preprocessing can be performed (step S301). If the determination result is positive, preprocessing unit 16 performs preprocessing (step S302) and then ends the process. In step S302, data compression processing can be performed in the same way as in step S202, or preprocessing other than compression processing can be performed, such as anonymization processing, data transmission processing, etc.

[0105] On the other hand, if the determination result in step S301 is negative, it is determined whether the time since the last preprocessing has reached or exceeded a time threshold (step S303). That is, it is determined whether the elapsed time since the end of the last preprocessing period has reached or exceeded a time threshold. Furthermore, the time threshold can be set arbitrarily. In addition, for example, the time threshold can be changed according to the capacity of the data that can be stored in the storage unit 10b (the remaining amount of storage capacity) or the amount of data that has been stored. Specifically, when the amount of data that has been stored is large, or when the storage capacity that can be stored is small, the time threshold can be reduced.

[0106] The preprocessing unit 16 determines the process with the lowest processing load among the various processes performed in preprocessing (step S304). For example, when multiple processes such as compression, anonymization, and data transmission are performed in preprocessing, the process with the lowest processing load among these processes is determined. Furthermore, since the processing load varies depending on the amount of data being processed, it is preferable to consider the amount of data being processed when determining the processing load for these processes. Additionally, when the data of the processing object can be divided for processing, the processing load can be determined based on the smallest unit of data.

[0107] Next, the preprocessing unit 16 assesses the remaining capacity of the computing processing unit 10a (step S305). That is, it assesses how much processing capacity remains while prioritizing the processing currently being performed (processing related to driving assistance functions, etc.).

[0108] Then, the preprocessing unit 16 determines whether the capacity of the computing processing device 10a, as determined in step S305, exceeds the processing load of the lowest processing load determined in step S304 (step S306). If the determination result is affirmative, the preprocessing unit 16 performs the lowest processing load determined in step S303 (step S307). For example, if the processing load of the data transmission process is the lowest and the processing capacity capacity exceeds the processing load of the data transmission process, the preprocessing unit 16 transmits data. Furthermore, at this time, data that has already undergone compression processing can be transmitted, or data before compression processing can be transmitted. Furthermore, if the processing load of the compression processing is the lowest and the processing capacity capacity capacity exceeds the processing load of the compression processing, the preprocessing unit 16 performs data compression processing. Then, the processing ends.

[0109] On the other hand, if the determination result in step S306 is negative, the processing device 10a temporarily restricts data collection (step S308). That is, if any processing load is too high to spare and the storage capacity is almost depleted, the acquisition condition is not met. Furthermore, when compression processing or data transmission processing is performed and the storage capacity becomes available, the restriction is lifted.

[0110] The control unit and methods described in this disclosure can be implemented by a special-purpose computer consisting of a processor and memory programmed to perform one or more functions embodied in one or more computer programs. Alternatively, the control unit and methods described in this disclosure can be implemented by a special-purpose computer whose processor consists of one or more special-purpose hardware logic circuits. Alternatively, the control unit and methods described in this disclosure can be implemented by one or more special-purpose computers consisting of a processor and memory programmed to perform one or more functions, and a processor consisting of one or more hardware logic circuits. Furthermore, the computer program can be stored as instructions executable by the computer in a computer-readable, non-transitory tangible recording medium.

[0111] The following describes the technical ideas that can be derived from the above embodiments and variations.

[0112] [Component 1]

[0113] A vehicle control device (10) is mounted on a vehicle (101), the vehicle control device comprising:

[0114] The data acquisition unit (14) acquires data related to sensor information acquired by various sensors (20) mounted on the vehicle.

[0115] The control unit (12) implements vehicle control based on the aforementioned sensor information;

[0116] The preprocessing unit (16) preprocesses the data acquired by the data acquisition unit during a predetermined period; and

[0117] The sending unit (17) sends the data preprocessed by the aforementioned preprocessing unit to the server (102) via the communication network (103).

[0118] The aforementioned predetermined period is set when the aforementioned control unit does not implement drive control related to the driving of the aforementioned vehicle.

[0119] [Component 2]

[0120] According to the vehicle control device of configuration 1, the preprocessing includes compression processing, which is a process of compressing the data by deleting any one of the multiple data acquired by the data acquisition unit or by periodically removing a portion of the constituent elements of the data.

[0121] [Component 3]

[0122] According to the vehicle control device described in configuration 1 or 2, the preprocessing includes anonymization processing, which involves processing image data in the data acquired by the data acquisition unit to make it impossible to identify a specific individual.

[0123] [Composition 4]

[0124] According to any one of the vehicle control devices comprising 1 to 3, the aforementioned transmitting unit transmits the aforementioned data during the aforementioned predetermined period.

[0125] [Component 5]

[0126] According to any one of the vehicle control devices comprising 1 to 4, the predetermined period is set to the period during which the vehicle is parked or stopped, during the period during which the vehicle is charging, or during a monitoring mode during which the vehicle is parked or stopped is monitored.

[0127] [Composition 6]

[0128] According to any one of the vehicle control devices comprising 1 to 5, wherein...

[0129] The control unit described above is configured to identify objects based on the sensor information and perform predetermined processing on the objects.

[0130] The aforementioned predetermined period is set when the number of identified objects is below a threshold or when the processing volume for the aforementioned objects is below a specified value.

[0131] [Component 7]

[0132] According to any one of the vehicle control devices comprising 1 to 6, the predetermined period is set when the position of the vehicle is determined to be a predetermined position based on the sensor information.

[0133] [Composition 8]

[0134] As described in any one of 1 to 7, the predetermined period is set when the weather is determined to be severe based on the sensor information.

[0135] [Composition 9]

[0136] According to the vehicle control device comprising any one of 1 to 8, wherein...

[0137] The aforementioned vehicle control device is configured to implement a shadow mode, which causes the verification object program to run in a manner independent of the control of the aforementioned vehicle.

[0138] When the data acquisition unit performs the verification object procedure using sensor information acquired by various sensors mounted on the vehicle as input values, it acquires data related to the output results and data related to the sensor information, or acquires data related to the output results to replace the data related to the sensor information.

[0139] [Composition 10]

[0140] A program is executed by a vehicle control device (10) mounted on a vehicle (101), wherein,

[0141] The above procedure enables the above vehicle control device to perform:

[0142] The data acquisition step involves acquiring data related to sensor information acquired by various sensors (20) mounted on the aforementioned vehicle;

[0143] The control steps involve implementing vehicle control based on the aforementioned sensor information.

[0144] The preprocessing step involves preprocessing the data acquired through the above data acquisition steps during a predetermined period; and

[0145] In the sending step, the data preprocessed by the above preprocessing steps is sent to the server (102) via the communication network (103).

[0146] The aforementioned predetermined period is set when no drive control related to the driving of the vehicle is implemented during the aforementioned control steps.

[0147] While this disclosure is based on embodiments, it should be understood that this disclosure is not limited to those embodiments or structures. This disclosure also includes various modifications and variations within the same range. Furthermore, various combinations and forms, as well as other combinations and forms that include only one element, more elements, or fewer elements based on these, also fall within the scope and spirit of this disclosure.

Claims

1. A vehicle control device (10) mounted on a vehicle (101), comprising: The data acquisition unit (14) acquires data related to sensor information acquired by various sensors (20) mounted on the vehicle. The control unit (12) implements vehicle control based on the aforementioned sensor information; The preprocessing unit (16) preprocesses the data acquired by the data acquisition unit during a predetermined period; and The sending unit (17) sends the data preprocessed by the aforementioned preprocessing unit to the server (102) via the communication network (103). The aforementioned predetermined period is set when the aforementioned control unit does not implement drive control related to the driving of the aforementioned vehicle.

2. The vehicle control device according to claim 1, wherein, The preprocessing described above includes compression processing, which is a process of compressing the data by deleting any one of the multiple data acquired by the data acquisition unit or by periodically removing a portion of the constituent elements of the data.

3. The vehicle control device according to claim 1, wherein, The aforementioned preprocessing includes anonymization processing, which involves processing image data acquired by the aforementioned data acquisition unit to make it impossible to identify a specific individual.

4. The vehicle control device according to any one of claims 1 to 3, wherein, The aforementioned sending unit will transmit the aforementioned data during the aforementioned scheduled period.

5. The vehicle control device according to any one of claims 1 to 3, wherein, The aforementioned predetermined period is set during the period when the vehicle is parked or stopped, during the period when the vehicle is charging, or during the monitoring mode period when the area around the parked or stopped vehicle is monitored.

6. The vehicle control device according to any one of claims 1 to 3, wherein, The control unit described above is configured to identify objects based on the sensor information and perform predetermined processing on the objects. The aforementioned predetermined period is set when the number of identified objects is below a threshold or when the processing volume for the aforementioned objects is below a specified value.

7. The vehicle control device according to any one of claims 1 to 3, wherein, The aforementioned predetermined period is set when the location of the vehicle is determined to be a predetermined location based on the aforementioned sensor information.

8. The vehicle control device according to any one of claims 1 to 3, wherein, The aforementioned scheduled period is set when the weather is determined to be severe based on the aforementioned sensor information.

9. The vehicle control device according to any one of claims 1 to 3, wherein, The aforementioned vehicle control device is configured to implement a shadow mode, which causes the verification object program to run in a manner independent of the control of the aforementioned vehicle. When the data acquisition unit performs the verification object procedure using sensor information acquired by various sensors mounted on the vehicle as input values, it acquires data related to its output results and data related to the sensor information, or acquires data related to its output results to replace the data related to the sensor information.

10. A program executed by a vehicle control device (10) mounted on a vehicle (101), wherein, The above procedure enables the above vehicle control device to perform: The data acquisition step involves acquiring data related to sensor information acquired by various sensors (20) mounted on the aforementioned vehicle; The control steps involve implementing vehicle control based on the aforementioned sensor information. The preprocessing step involves preprocessing the data acquired through the above data acquisition steps during a predetermined period. as well as In the sending step, the data preprocessed by the above preprocessing steps is sent to the server (102) via the communication network (103). The aforementioned predetermined period is set when no drive control related to the driving of the vehicle is implemented during the aforementioned control steps.

Images