Vehicle control apparatus and on-vehicle program

Abstract

A vehicle control apparatus (10) mounted on a vehicle (101) comprises: a data acquisition unit (14) that acquires data pertaining to sensor information acquired by various sensors (20) mounted on the vehicle; a control unit (12) that performs vehicle control on the basis of the sensor information; a pre-processing unit (16) that performs pre-processing on the data acquired by the data acquisition unit during a prescribed period before transmission; and a transmission unit (17) that transmits the data pre-processed by the pre-processing unit to a server (102) via a communication network (103). The prescribed period is set for times when the control unit does not perform drive control related to traveling of the vehicle.

Description

Vehicle control device and in-vehicle program CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Application No. 2024-069045, filed on April 22, 2024, the contents of which are incorporated herein by reference.

[0002] The present disclosure relates to a vehicle control device and an in-vehicle program.

[0003] In recent years, a method has been adopted in which various data actually acquired from vehicles after they have been sold while they are being driven is collected on a server via a communication network, and the data collected on the server is analyzed and used in future vehicle development or to update the vehicle's control program.

[0004] As one example, a verification method called shadow mode is known, in which a vehicle control program to be verified is run while the vehicle is running after sales to verify the performance and safety of the vehicle control program. In verification using shadow mode, for example, the vehicle control program to be verified is run in the background, i.e., without being involved in vehicle control, and data related to the output values ​​is stored. The data related to the output values ​​is then collected by a server via an external network or the like. The collected data is then analyzed to verify the operation of the vehicle control program. Technology related to such shadow mode is described, for example, in Patent Document 1.

[0005] JP 2023-38697 A

[0006] However, the process of running multiple vehicle control programs to collect and transmit data imposes a heavy processing load.

[0007] The present disclosure has been made in consideration of the above circumstances, and has as its main object to provide a vehicle control device and an in-vehicle program that distribute the processing load.

[0008] A vehicle control device that solves the above problem is a vehicle control device mounted on a vehicle, and includes a data acquisition unit that acquires data related to sensor information acquired by various sensors mounted on the vehicle, a control unit that performs vehicle control based on the sensor information, a pre-processing unit that pre-processes the data acquired by the data acquisition unit during a predetermined period, and a transmission unit that transmits the data pre-processed by the pre-processing unit to a server via a communication network, wherein the predetermined period is set when the control unit is not performing drive control related to the driving of the vehicle.

[0009] This allows the pre-processing, which has a heavy processing load, to be distributed from other processes, such as drive control related to traveling, which have a heavy processing load.

[0010] The program that solves the above problem is a program executed by a vehicle control device mounted on a vehicle, and causes the vehicle control device to perform a data acquisition step of acquiring data related to sensor information acquired by various sensors mounted on the vehicle, a control step of performing vehicle control based on the sensor information, a preprocessing step of preprocessing the data acquired by the data acquisition step during a predetermined period, and a transmission step of transmitting the data preprocessed by the preprocessing step to a server via a communication network, wherein the predetermined period is set when drive control related to the driving of the vehicle is not being performed in the control step.

[0011] This allows the pre-processing, which has a heavy processing load, to be distributed from other processes, such as drive control related to traveling, which have a heavy processing load.

[0012] The above and other objects, features, and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which Fig. 1 is a schematic diagram of a data collection system, Fig. 2 is a block diagram showing functions performed by a verification processing unit, Fig. 3 is a flowchart of data acquisition processing, Fig. 4 is a flowchart of preprocessing, Fig. 5 is a diagram for explaining a preprocessing period, and Fig. 6 is a flowchart of a modified example of preprocessing.

[0013] Hereinafter, embodiments of a vehicle control device and a program according to the present disclosure will be described in detail with reference to the drawings. Note that, between the embodiments and modifications, the same or corresponding parts in the drawings are designated by the same reference numerals, and their description will not be repeated in principle.

[0014] 1 shows a data collection system 100 to which a vehicle control device 10 according to this embodiment is applied. The vehicle control device 10 is mounted on a vehicle 101, and controls the vehicle 101 and performs driving assistance.

[0015] As shown in Fig. 1, a data collection system 100 includes a server 102 and is capable of communicating with one or more vehicles 101 via a communication network 103 such as the Internet. Fig. 1 shows only one vehicle 101. The vehicle 101 includes a vehicle control device 10, a sensor 20, an actuator 30, and the like.

[0016] The sensor 20 includes various sensors for measuring the traveling conditions of the vehicle 101, such as a vehicle speed sensor 21, an acceleration sensor 22, and a yaw rate sensor 23. The sensors for measuring the traveling conditions of the vehicle 101 may include other sensors, or any of the vehicle speed sensor 21, the acceleration sensor 22, and the yaw rate sensor 23 may not be included.

[0017] The sensor 20 also includes various sensors for detecting other vehicles and obstacles, such as a camera 24 and a millimeter-wave radar 25. The sensor for detecting other vehicles and obstacles is not limited to the millimeter-wave radar 25, but may also be a laser radar (LiDAR), an ultrasonic sensor, or a combination of these. The camera 24 may also be a monocular camera or a compound camera. The camera 24 may also capture either still images or videos. The number, position, and type of the cameras 24 may be changed as desired. For example, the vehicle may include a front camera that captures the area in front of the vehicle, a right-side camera that captures the area on the right side of the vehicle, a left-side camera that captures the area on the left side of the vehicle, and a rear camera that captures the area behind the vehicle.

[0018] The sensor 20 also includes various sensors for detecting various amounts of operation by the driver, such as an accelerator sensor 26 that detects the amount of accelerator operation by the driver, a brake sensor 27 that detects the amount of brake operation, and a steering angle sensor 28 that detects the amount of steering (steering angle) of the steering wheel by the driver.

[0019] These sensors 20 are connected to the vehicle control device 10 wirelessly or by wire, and the measurement results (or detection results) of these sensors 20 are input to the vehicle control device 10 or the like as sensor information.

[0020] The actuators 30 include, for example, actuators for driving the vehicle 101, such as a motor that serves as the main engine of the vehicle 101. The actuators 30 may also include actuators for controlling the behavior of the vehicle 101, such as an actuator for operating a steering wheel, an actuator for operating a brake pedal, or an actuator for operating an accelerator pedal. The actuators 30 may also include devices for operating accessories of the vehicle 101, such as a display, a speaker, an indicator, and a headlight. The driving and operation of the vehicle 101 are controlled by the actuators 30.

[0021] The vehicle control device 10 is primarily composed of a microcomputer including a processing unit 10a such as a CPU and a storage unit 10b such as various types of memory. The functions provided by the microcomputer can be provided by software recorded in a physical memory device and a computer executing the software, software alone, hardware alone, or a combination thereof. For example, when the microcomputer is provided by electronic circuits, which are hardware, the functions can be provided by digital circuits including numerous logic circuits or analog circuits. For example, the processing unit 10a of the microcomputer executes programs stored in a non-transitory tangible storage medium (non-transitory tangible storage medium) that serves as the storage unit 10b. The programs include, for example, programs that realize functions shown in FIG. 2 . Execution of the programs results in the execution of methods corresponding to the programs. The storage unit 10b is, for example, a non-volatile memory. The programs stored in the storage unit 10b can be downloaded and updated via a communication network 103, such as the Internet, via over-the-air (OTA) or other means.

[0022] The vehicle control device 10 has various functions (application programs) for assisting the driving of the vehicle 101, and these functions control the actuator 30 based on sensor information input from the sensor 20.

[0023] Typical functions for assisting the driving of the vehicle 101 include, for example, an adaptive cruise control system (ACC), a forward collision warning (FCW), an advanced emergency braking system (AEBS), a night vision / pedestrian detection (NV / PD), a traffic sign recognition (TSR), a lane departure warning (LDW), a lane keeping assist system (LKAS), a rear cross traffic alert (RCTA), an adaptive front lighting system (AFS), and an advanced parking assist (APA). The vehicle 101 may be provided with all or some of these functions. Furthermore, the vehicle 101 may also be provided with other driving assistance functions. Furthermore, the vehicle 101 may also be provided with a function for realizing autonomous driving of the vehicle 101. These functions are realized by the arithmetic processing unit 10a executing a driving assistance control program stored in the storage unit 10b.

[0024] Furthermore, in order to verify the performance and safety of the driving assistance control program, the vehicle control device 10 has a function called a shadow mode in which a driving assistance control program to be verified (hereinafter referred to as a verification target program) is run while the vehicle 101 is running after being sold, that is, in an actual use case, and output values ​​are checked. In verification using the shadow mode, for example, the verification target program is run in the background, that is, in a manner not related to vehicle control, and data related to the output values ​​is stored, and the server 102 collects the data related to the output values ​​via the communication network 103. Note that the shadow mode is a type of data collection mode for collecting data.

[0025] The configuration and functions of the shadow mode according to this embodiment will be described below. The timing of switching to the shadow mode may be any timing. For example, the switching may be performed by an operation by the driver or by an instruction from the server 102. Alternatively, the switching to the shadow mode may be performed when the ignition switch is turned on.

[0026] In the following, the driving assistance control program that actually operates the vehicle 101 will be referred to as an "implemented program" to contrast it with the program to be verified. It is also referred to as an "implemented program" in FIG. 1 . The program to be verified is stored in the storage unit 10b. A storage device dedicated to the shadow mode may be provided.

[0027] Execution of the program to be verified executes various functions in the background to assist the driving of the vehicle 101. The functions implemented by execution of the program to be verified may be the same as the functions actually realized by the arithmetic processing device 10a, or some of the functions may be omitted. Furthermore, functions other than the functions actually realized by the arithmetic processing device 10a may also be implemented.

[0028] When the arithmetic processing device 10a is running the program to be verified in shadow mode, it inputs sensor information from various sensors 20. Then, based on the input sensor information, the arithmetic processing device 10a executes various functions of the program to be verified and outputs various control signals for operating each actuator 30. These control signals (output results) based on the program to be verified are not actually input to the actuators 30, but are stored in the storage unit 10b as data related to the output results. At that time, the arithmetic processing device 10a also associates and stores data related to the sensor information, which is an input value.

[0029] The processing unit 10a also stores the control signals processed and output based on the implementation program in the storage unit 10b as data related to the output results. That is, the processing unit 10a also stores the control signals processed and output based on the implementation program in order to compare and verify the output results of the implementation program and the output results of the program to be verified.

[0030] Furthermore, the arithmetic processing device 10a may input sensor information (vehicle speed, yaw rate, acceleration amount, accelerator pedal operation amount, brake pedal operation amount, steering angle, etc.) detected when a control signal processed and output based on the implementation program is input to the actuator 30 and the vehicle 101 moves as a result, and store the information in the storage unit 10b. In other words, the sensor information related to the actual operation of the vehicle 101 for each scene, that is, the sensor information necessary to verify the correct operation, may also be stored.

[0031] The vehicle control device 10 uploads the data thus stored in the storage unit 10b to the server 102 via the communication network 103 at a predetermined transmission timing. The predetermined transmission timing may be any timing, for example, a timing when an upload instruction is issued from the server 102. Alternatively, the predetermined transmission timing may be, for example, a timing when the vehicle 101 is charging, when the vehicle is parked or stopped, or when the ignition switch is turned off.

[0032] However, if all vehicles 101 upload all of their stored data, there is a risk of placing an excessive burden on the communication network 103. For this reason, it is desirable to compress the data before transmission. However, executing both the implementation program and the verification target program while also performing compression processing in parallel may place an excessive processing burden on the arithmetic processing device 10a. Therefore, the vehicle control device 10 of this embodiment distributes processing so as not to place an excessive burden on the arithmetic processing device 10a. This will be explained in detail below.

[0033] 2, the arithmetic processing device 10a of the vehicle control device 10 has a function as an input unit 11, a function as a control unit 12, a function as a verification unit 13, a function as a data acquisition unit 14, a function as a classification unit 15, a function as a preprocessing unit 16, and a function as a transmission unit 17. These functions are realized by the arithmetic processing device 10a by executing an in-vehicle program stored in the storage unit 10b.

[0034] The input unit 11 inputs sensor information from the sensor 20. Then, the input unit 11 inputs some or all of the input sensor information to the control unit 12 as input values ​​for the implementation program. The input unit 11 also inputs some or all of the input sensor information to the verification unit 13 as input values ​​for the program to be verified. The input unit 11 also inputs some or all of the input sensor information to the data acquisition unit 14.

[0035] The control unit 12 runs the implementation program and performs processing based on various functions (application programs) realized by the implementation program, based on input values ​​(sensor information) input from the input unit 11. The control unit 12 then inputs a control signal (a control signal for the actuator 30) as a result of the processing to the actuator 30 and the data acquisition unit 14.

[0036] The verification unit 13 runs the program to be verified and performs processing based on various functions (application programs) realized by the program to be verified, based on input values ​​(sensor information) input from the input unit 11. The verification unit 13 then inputs control signals (control signals for the actuator 30) as the results of this processing to the data acquisition unit 14. Note that, as described above, these control signals are not input to the actuator 30.

[0037] The data acquisition unit 14 determines whether the acquisition condition is met based on at least one of the input value (sensor information) input from the input unit 11 and the output result (control signal) input from the verification unit 13.

[0038] The acquisition conditions may include, for example, thresholds for one or more parameters as components. The parameters of the acquisition conditions may be, for example, parameters included in sensor information such as vehicle speed, yaw rate, accelerator operation amount, brake operation amount, and steering amount, or parameters included in output results such as required torque and an output instruction signal to a display. The acquisition conditions may also include parameters calculated or estimated from sensor information or output results. For example, the acquisition condition parameters may include the relative distance to an obstacle (such as a preceding vehicle or pedestrian), the relative speed to the obstacle, and the time-to-collision (TTC) calculated from the recognition results of a camera image or the detection results of the millimeter-wave radar 25. The acquisition condition parameters may also include the difference between the output results based on the implemented program and the output results of the program to be verified. For example, the parameter may be the difference between the brake operation amount output by processing based on the implemented program and the brake operation amount output by processing based on the program to be verified. The threshold may be either the upper limit or lower limit of any of these parameters, or both.

[0039] Furthermore, a component of the acquisition condition may be whether or not the scene in which the vehicle 101 is traveling (hereinafter simply referred to as the scene) is a predetermined acquisition scene. The scene refers to various scenes that are expected when the vehicle 101 is traveling, such as a scene in which the vehicle overtakes a preceding vehicle, a scene in which the vehicle passes between vehicles, a scene in which the vehicle follows a preceding vehicle, a scene in which a pedestrian crosses in front of the vehicle 101 at night, a scene in which the vehicle 101 merges from an acceleration lane onto a main lane on a motorway, a scene in which the vehicle is parking or stopping, a scene in which the vehicle is waiting at a traffic light, and the like.

[0040] These scenes are estimated based on sensor information. For example, the vehicle control device 10 may recognize camera images or the like and estimate the scenes. More specifically, the sensor information such as camera images may be input to a machine-learned inference model such as a deep neural network to estimate the scenes. Note that image recognition does not have to be performed by the vehicle control device 10, and may be performed by an external device of the vehicle control device 10, such as an image recognition device, and the results may be input as sensor information.

[0041] Furthermore, among various functions (application programs) based on the program to be verified, an executed function (executed function) may be a component of the acquisition condition. For example, the acquisition condition may be satisfied when a collision damage mitigation braking control function is executed. Note that which function has been executed can be determined based on the output result (control signal) input from the verification unit 13.

[0042] In this embodiment, the acquisition condition is set by combining the acquisition scene, the execution function, and the parameter threshold (i.e., an AND condition). For example, the acquisition condition may be satisfied when the scene is following a preceding vehicle and the vehicle speed (a parameter of the acquisition condition) is equal to or greater than a threshold (50 km / h).

[0043] The combination of components of the acquisition condition may be an acquisition scene and an implementation function, an acquisition scene and a parameter threshold, or an implementation function and a parameter threshold. Furthermore, the components included in one acquisition condition may include two or more implementation functions, and in this case, the implementation functions may be an AND condition or an OR condition. For example, the condition may be that both the forward vehicle approach warning function and the collision damage mitigation braking control function are implemented, or that either one of them is implemented. Similarly, the parameter threshold may be a threshold for two or more types of parameters.

[0044] Furthermore, the acquisition condition is established when all of the conditions of the components that make up the acquisition condition are satisfied. For example, if the acquisition scene of the acquisition condition is "a scene of following a preceding vehicle," the implementation function of the acquisition condition is "a following driving function," and the threshold value of the parameter of the acquisition condition is "50 km / h or more," the acquisition condition is established when the following driving function is implemented in a scene of following a preceding vehicle and the vehicle speed (a parameter of the acquisition condition) is equal to or greater than the threshold value (50 km / h).

[0045] The number of acquisition conditions is not limited to one, and multiple acquisition conditions may be set. For example, a first acquisition condition may be a scene in which the vehicle is following a preceding vehicle and the vehicle speed is equal to or greater than a first threshold, and a second acquisition condition may be a scene in which a pedestrian is crossing in front of the vehicle 101, the distance to the pedestrian is equal to or less than a second threshold, and the amount of brake operation is equal to or greater than a third threshold.

[0046] When any of the acquisition conditions is satisfied, the data acquisition unit 14 stores data related to input values ​​and output results in the storage unit 10b. As described above, the input values ​​include, for example, sensor information input to the arithmetic processing device 10a. Furthermore, the output results include, for example, a control signal processed and output based on the program to be verified or a control signal processed and output based on the implemented program. Furthermore, the output results may include sensor information related to the actual operation of the vehicle 101 for each scene. Furthermore, the output results may include a difference between the output result based on the implemented program and the output result of the program to be verified. The stored (acquired) data may be changed depending on the acquisition condition that is satisfied. For example, when an acquisition condition is satisfied in a scene in which the vehicle is following a preceding vehicle and the vehicle speed is equal to or greater than a first threshold, data related to the vehicle speed may be acquired. On the other hand, when an acquisition condition is satisfied in a scene in which a pedestrian is crossing in front of the vehicle 101 and the distance to the pedestrian is equal to or less than a second threshold and the brake operation amount is equal to or greater than a third threshold, data related to the brake operation amount may be acquired.

[0047] The data is stored with tag information attached so that it is collected and stored for each acquisition condition that has been met. The sensor information and other data acquired in response to an acquisition condition is stored as a single piece of data in association with the acquisition condition that has been met.

[0048] Next, the classification unit 15 will be described. The classification unit 15 classifies the data acquired (stored) by the data acquisition unit 14 for each scene in which the vehicle 101 is traveling, based on the sensor information input by the input unit 11. The scenes in which the vehicle 101 is traveling are the same as those described above. That is, the scenes refer to various scenes that are expected when the vehicle 101 is traveling, such as a scene in which the vehicle overtakes a preceding vehicle, a scene in which the vehicle passes between vehicles, a scene in which the vehicle follows a preceding vehicle, a scene in which a pedestrian crosses in front of the vehicle 101 at night, a scene in which the vehicle 101 merges from an acceleration lane onto a main lane on a motorway, a scene in which the vehicle is parking or stopping, and a scene in which the vehicle is waiting at a traffic light.

[0049] These scenes are estimated based on sensor information. For example, the vehicle control device 10 may recognize camera images or the like and estimate the scene. In this case, several reference scenes are assumed in advance, and sensor information indicating the characteristics of each scene is stored in advance. Then, the sensor information is compared, and the reference scene with the closest similarity between the sensor information among the multiple reference scenes is estimated as the current scene. More specifically, the sensor information such as the camera images may be input into a machine-learned inference model such as a deep neural network, and the scene may be estimated.

[0050] Image recognition does not have to be performed by the vehicle control device 10, but may be performed by an external device to the vehicle control device 10, such as an image recognition device, and the results may be input as sensor information. Furthermore, since scene estimation is performed when the data acquisition unit 14 determines whether acquisition conditions are met or when the verification unit 13 executes various functions of the program to be verified, the results of such estimation may be used. This reduces the control burden.

[0051] When storing the data in the storage unit 10b, the data acquisition unit 14 stores the data in association with the scenes classified (estimated) by the classification unit 15. As a result, when the data is stored in the storage unit 10b, the data is classified by scene and stored.

[0052] Next, the preprocessing unit 16 will be described. The preprocessing unit 16 performs preprocessing before transmission on data acquired by the data acquisition unit 14 during a predetermined period and stored in the storage unit 10b. In this embodiment, preprocessing includes a compression process that compresses the data volume. When performing compression, the preprocessing unit 16 compresses the data volume by deleting any of the multiple data items classified into a predetermined scene or by thinning out some of the data components. When deleting, all of the data that constitutes a group is deleted. Furthermore, thinning out some of the data components refers to, for example, lowering the frame rate of the video data if the data includes video data, or lowering the sampling rate of the audio data if the data includes audio data. In other words, if the data is detected in a time series at a predetermined interval, it refers to lengthening the detection interval to reduce the number of samples and converting the data into digital data. Furthermore, some of the data components may be thinned out by reducing color information, etc. Hereinafter, deleting any of the multiple data items or thinning out some of the data components will simply be referred to as "compression."

[0053] The predetermined period and method for performing preprocessing will be specifically described. In the present embodiment, the predetermined period for performing preprocessing is performed during a period in which the control unit 12 is not performing drive control related to the traveling of the vehicle 101. The predetermined period for performing preprocessing may be set at any timing as long as it is a period in which drive control related to traveling is not being performed. For example, it may be set to a period in which the vehicle 101 is parked or stopped, a period in which the vehicle 101 is charging, or a period in a monitoring mode in which the surroundings of the parked vehicle 101 are monitored. Note that the compression process in this embodiment is preferably performed before the remaining storage capacity of the data in the storage unit 10b falls below a specified value.

[0054] When compressing, the pre-processing unit 16 compresses any of the multiple pieces of data classified into each scene. For example, a priority and a level of urgency are set in advance for each scene, and the pre-processing unit 16 deletes any of the multiple pieces of data classified into scenes with low priority and low urgency from the storage unit 10b. Also, for example, the pre-processing unit 16 compresses any of the multiple pieces of data classified into scenes designated by the server 102 (e.g., designated as unnecessary). Also, for example, the pre-processing unit 16 compresses data for the scene with the largest amount of data among the multiple scenes classified in the storage unit 10b.

[0055] The pre-processing unit 16 may arbitrarily change which data to compress from among the multiple data included in the scene to be compressed. For example, the pre-processing unit 16 may compress (delete or thin out) the first or last stored data from among the multiple data included in the scene to be compressed. Alternatively, the pre-processing unit 16 may compress random data from among the multiple data included in the scene to be compressed.

[0056] In addition, the pre-processing unit 16 may set priorities for multiple data classified into the scene to be compressed in the order of specific parameters contained in the data, and compress (in this embodiment, delete) the multiple data included in the scene to be compressed in order of lowest priority.

[0057] For example, the pre-processing unit 16 sets a lower priority for data that is classified as a scene in which vehicles pass through each other (predetermined scene) and includes a lower vehicle speed (specific parameter).When deleting and compressing any of the data that is classified as a scene in which vehicles pass through each other, the pre-processing unit 16 preferentially deletes data with a lower vehicle speed.

[0058] When compressing data, the amount of data to be compressed may be set arbitrarily. For example, only a predetermined amount of data may be compressed. Also, for example, the amount of data to be compressed may be changed depending on the remaining amount of data storage capacity in the storage unit 10b. In other words, the smaller the remaining amount, the greater the amount of data to be compressed.

[0059] The transmitter 17 transmits, at a predetermined transmission timing, the data acquired by the data acquisition unit 14, preprocessed by the preprocessor 16, and stored in the memory unit 10b via the communication network 103 to the server 102. The predetermined transmission timing is as described above.

[0060] The flow of data acquisition in this embodiment will be described below with reference to FIG. 3. The flow of data acquisition shown below is the flow of data acquisition processing performed by the arithmetic processing device 10a. These processes are performed after the shadow mode is set. The timing at which the shadow mode is set is arbitrary, but may be, for example, the timing at which the ignition switch is turned on.

[0061] First, the input unit 11 of the arithmetic processing device 10a inputs sensor information from the sensor 20 (step S101). The control unit 12 runs an implementation program and performs processing based on various functions realized by the implementation program, based on the input values ​​(sensor information) input from the input unit 11 (step S102). In step S102, the control unit 12 inputs a control signal as a result of the processing to the actuator 30. At this time, the control signal is also input to the data acquisition unit 14 as necessary.

[0062] The verification unit 13 also runs the program to be verified and performs processing based on various functions realized by the program to be verified, based on input values ​​(sensor information) input from the input unit 11 (step S103). In step S103, the verification unit 13 inputs a control signal as a result of the processing to the data acquisition unit 14. It is desirable that the processing by the verification unit 13 and the processing by the control unit 12 be performed in parallel, but if this is difficult due to the processing load, the processing by the control unit 12 may be prioritized, and the processing by the verification unit 13 may be performed during a period when the control load is lighter.

[0063] The data acquiring unit 14 determines whether or not the acquisition condition is met based on at least one of the input value (sensor information) and the output result (control signal, etc.) (step S104).

[0064] If the determination result is positive (if the acquisition condition is met), the classification unit 15 of the arithmetic processing device 10a classifies the data related to the input values ​​and output results (step S105). Specifically, the classification unit 15 estimates the scene in which the vehicle 101 is traveling based on the sensor information input in step S101. Then, the classification unit 15 classifies the data related to the input values ​​and output results into the estimated scene.

[0065] Then, the data acquisition unit 14 stores (acquires) the data regarding the input values ​​and output results in a lump in the storage unit 10b (step S105). When storing the data, the data acquisition unit 14 stores the data such that the data regarding the input values ​​and output results are classified into the scenes classified (estimated) by the classification unit 15 in step S104. In this embodiment, the data to be stored is stored in association with tag information regarding the classified scenes. The data acquisition unit 14 also stores the acquisition conditions that have been met in association with the data. Then, the processing ends. On the other hand, if the determination result in step S103 is negative, the calculation processing device 10a ends the data acquisition process as it is. The data acquisition process is executed at regular intervals after the shadow mode is set.

[0066] In this embodiment, step S105 corresponds to a data acquisition step, and step S102 corresponds to a control step.

[0067] Next, the preprocessing will be described with reference to FIG. 4. The preprocessing is performed at predetermined intervals. The preprocessing unit 16 determines whether or not the present time is a predetermined period during which the preprocessing can be performed (hereinafter referred to as the preprocessing period) (step S201). The predetermined period is as described above (the predetermined period during which the preprocessing is performed). If the determination result is positive, the preprocessing unit 16 determines the scenes to be compressed, compresses the data of the scenes to be compressed among the data stored in the storage unit 10b (step S202), and ends the process. At this time, the data compression method is as described above. On the other hand, if the determination result of step S201 is negative, the preprocessing ends. In this embodiment, step S202 corresponds to the preprocessing step.

[0068] The transmitting unit 17 transmits the data acquired by the data acquiring unit 14 and stored in the storage unit 10b to the server 102 via the communication network 103 at a predetermined transmission timing. This data includes data compressed by the preprocessing unit 16. When transmitting, the transmitting unit 17 also transmits information related to the established acquisition conditions associated with the data. Tag information related to the scenes classified by the classification unit 15 may also be transmitted together with the data. In this embodiment, the process of transmitting data by the transmitting unit 17 corresponds to the transmitting step.

[0069] When the server 102 receives the data transmitted from the transmitter 17 of each vehicle 101, it stores the data in the storage device of the server 102. When storing the data, the data may be classified by scene based on tag information related to the scene received together with the data. The data may also be stored in association with information related to the acquisition conditions that were met when the data was acquired. In other words, after classifying the data by scene, the data may be further classified by the acquisition conditions and stored. This makes it easier to analyze the data because the data is grouped by scene.

[0070] According to the above embodiment, the following effects are achieved.

[0071] The preprocessing unit 16 compresses data during a predetermined period when the control unit 12 is not performing drive control related to the driving of the vehicle 101. This allows a process that imposes a heavy load, such as compression processing, to be distributed from other processes that impose a heavy load, such as drive control related to driving and communication processing.

[0072] The data is classified by scene, and the scene to be compressed is determined. One of the multiple data classified for that scene is compressed for each scene. This prevents storage capacity from being overwhelmed and reduces communication load, processing load, analysis load, and other loads. In addition, because it is possible to change whether or not to compress for each scene, data can be collected without compression for scenes that require a lot of data. In other words, for scenes that do not require much data, compressed data can be collected and analyzed efficiently.

[0073] When performing various functions or determining whether acquisition conditions are met, the estimated scene can be reused to classify data by scene. This allows classification to be performed in parallel with data acquisition. Therefore, there is no need to reclassify the data later, for example, when compressing the data, reducing the processing load.

[0074] When selecting data to compress from among a plurality of data classified into scenes to be compressed, the preprocessing unit 16 compresses data in order of decreasing priority based on the priority set in the order of specific parameters included in the data. This allows data with a high priority to be collected as much as possible to be retained, allowing for efficient analysis.

[0075] The classification unit 15 uses a deep neural network to estimate (recognize) similar scenes from among a plurality of predetermined reference scenes based on the sensor information input to the input unit 11, and classifies the data into the estimated scenes. Since the reference scenes are set in advance in this way, it is easy for the developer to recognize and analyze the expected scenes during analysis.

[0076] (Variations of the Embodiment) In the above embodiment, the preprocessing unit 16 may perform an anonymization process in place of or in addition to the compression process in the preprocessing. The anonymization process processes data, such as image data, from the sensor information acquired as data so that a specific individual cannot be identified. Specifically, the image data is processed by blurring or painting out personal information, such as an individual's face or the license plate of another vehicle, with black or white. This personal information can be extracted using a deep neural network, for example. Note that this extraction process generally requires a large processing load, and therefore the anonymization process is also one of the processes that requires a large processing load. On the other hand, the anonymization process does not need to be performed immediately. For example, the anonymization process can be performed in the preprocessing after the processing of step S202 (after the data is compressed).

[0077] In the above embodiment, the timing for transmitting data by the transmitting unit 17 may be set during the pre-processing period. For example, in the pre-processing, the transmitting unit 17 may transmit the data after the processing of step S202 (after the data is compressed). This is because the transmission process for transmitting verification data generally involves a large processing load, but does not need to be executed immediately.

[0078] A period that can be set as the preprocessing period in the above embodiment will be described with reference to FIG. 5 . As shown in FIG. 5 , the preprocessing period may be set to a period in which few targets are detected by the sensor 20 (the first column in FIG. 5 ). Examples of periods in which few targets are detected include when the ignition switch is turned off, when the vehicle is traveling at high speed on a highway, when the autobrake is activated, and when the shift lever is switched to "P (parking)" or "N (neutral)." The preprocessing period may also be set when the sensor 20 is malfunctioning or during bad weather such as heavy rain, dense fog, or heavy snow. The preprocessing period may also be set when the vehicle battery is charging, when the vehicle is in monitoring mode, or when the vehicle 101 is parked or stopped. This is because targets cannot be found during these periods. Examples of targets include other vehicles, pedestrians, and structures (signs and traffic lights). The term "few targets" refers to a situation in which the number of targets is equal to or less than a threshold. The target threshold depends on the processing capacity of the vehicle control device 10, which recognizes targets and performs processing for the targets. Bad weather can be estimated using image recognition.

[0079] 5, the pre-processing period may be set when the amount of processing for the target is small. For example, when traveling at low speed, the amount of processing is small, making it easier to perform the processing. The amount of processing for the target is small when, for example, the utilization rate (CPU utilization rate) of the arithmetic processing device 10a is equal to or lower than a specified value.

[0080] Furthermore, as shown in the third column in FIG. 5 , the preprocessing period may be set when the verification target program or the implemented program has few application programs running. For example, the preprocessing period may be set during a driving mode in which the driver controls the vehicle 101 by himself with the driving assistance functions turned off, or when autonomous driving is stopped. The preprocessing period may also be set while the vehicle battery is charging, when the ignition switch is turned off, when monitoring mode is set, or when the vehicle is parked or stopped. Furthermore, when location information is acquired using a navigation system or GPS and the vehicle is traveling on a motorway, application programs dedicated to general roads may be paused and preprocessing may be performed instead. Similarly, when it is determined from the location information that the vehicle is traveling on a general road, application programs dedicated to expressways may be paused and preprocessing may be performed instead.

[0081] 5, the vehicle control device 10 may determine a time when the processing load is low from the camera image and set the pre-processing period. A time when the processing load is low is, for example, when there are few targets or when the vehicle 101 is parked. Alternatively, the vehicle control device 10 may be configured to determine from the camera image whether the vehicle 101 is in a predetermined position (parking position), such as the location of a home parking lot, and the pre-processing period may be set when the vehicle 101 is in the predetermined position.

[0082] 5, the processing load of the arithmetic processing device 10a may be calculated, and preprocessing may be performed if the processing load is low, for example, if the utilization rate (CPU utilization rate) of the arithmetic processing device 10a is equal to or lower than a specified value.

[0083] In the above embodiment, a storage device may be provided in the charger to temporarily store data to be transmitted. The data may then be stored in the storage device during charging. The stored data is then transmitted to the server 102 via the Internet or the like.

[0084] In the above embodiment, a scene in which the target is below the threshold may be recognized from the image data, and when it is recognized that the scene is one in which the target is below the threshold, the preprocessing unit 16 may perform preprocessing.

[0085] In the above embodiment, the data acquisition unit 14 does not need to acquire the control signal (data related to the output result) processed and output based on the installed program. However, after the acquisition condition is met, the data acquisition unit 14 may acquire sensor information (vehicle speed, yaw rate, acceleration amount, accelerator pedal operation amount, brake pedal operation amount, steering angle, etc.) detected when the vehicle 101 is operating until a predetermined time has elapsed. In other words, data related to this sensor information may be stored in the memory unit 10b.

[0086] In the above embodiment, the verification unit 13 does not need to perform a function related to driving assistance. In other words, the verification unit 13 may not be necessary. In this case, after the acquisition condition is met, until a predetermined time has elapsed, sensor information (vehicle speed, yaw rate, acceleration amount, accelerator pedal operation amount, brake pedal operation amount, steering angle, etc.) detected when the vehicle 101 is operating due to an operation by the driver may be acquired. In other words, data related to this sensor information may be stored in the memory unit 10b to be collected as correct answer data.

[0087] In the above embodiment, the preprocessing unit 16 prioritizes a plurality of data items classified into a predetermined scene in the order of specific parameters (e.g., vehicle speed) included in the data items. The specific parameters are not limited to vehicle speed and may be changed arbitrarily. For example, the specific parameters may include any of the following: vehicle speed, time of day, weather, road, type of detected target, shape of detected target, position of detected target, distance to detected target, speed of detected target, and time to collision until detected target. Since the specific parameters to be prioritized are likely to differ for each scene, the specific parameters may be changed depending on the scene.

[0088] In the above embodiment, the classification unit 15 classified the data by scene, but the data may be classified by the acquisition condition that is met. Then, the preprocessing unit 16 may compress the data classified into a predetermined acquisition condition among the acquisition conditions.

[0089] Similarly, the classification unit 15 classified the data by scene, but the data may be classified by executed function (application program) among the various functions realized by the program to be verified. Then, the preprocessing unit 16 may compress the data classified into a predetermined executed function among the realized functions.

[0090] In the above embodiment, the classification unit 15 estimates which of the reference scenes the data is similar to based on the input sensor information. As another example, the classification unit 15 may classify the data stored in the storage unit 10b into similar scenes. For example, the classification unit 15 extracts features using deep learning or the like based on the sensor information (e.g., image data) included in the data, and classifies the data into similar scenes based on the similarity of the features. The classification unit 15 then sorts the data into groups of similar scenes. That is, the classification unit 15 clusters the data based on the sensor information included in the data, and classifies the data into clusters of similar scenes.

[0091] In the above embodiment, when storing data in the storage unit 10b, the data acquisition unit 14 may store the data by specifying an address in the storage unit 10b for each priority level of the data. Then, when the remaining storage capacity of the storage unit 10b becomes low, the preprocessing unit 16 may automatically delete data stored in addresses with low priority, freeing up data space and increasing storage capacity. In this way, the preprocessing unit 16 deletes data based on the specified address when the remaining storage capacity becomes low, eliminating the need for a compression command and simplifying processing. Furthermore, fragmentation of the storage area can be prevented, effectively increasing storage capacity.

[0092] In the above embodiment, responsiveness or temporal resolution may be set in advance for each scene. A scene with high responsiveness indicates a scene in which data should be transmitted to the server 102 as soon as possible after acquisition, while a scene with low responsiveness indicates a scene in which data can be transmitted to the server 102 slowly after acquisition. For example, a scene in which there is a high possibility of an accident or a scene in which the automatic brakes are activated can be said to have high responsiveness. A scene with low temporal resolution indicates a scene in which there is little change over time, while a scene with high temporal resolution indicates a scene in which there is a large change over time. For example, a scene in which there is traffic congestion on a highway can be said to have low temporal resolution.

[0093] If the responsiveness or temporal resolution is set in advance for each scene, the transmitter 17 may thin out (compress) the data when transmitting data of a scene with low responsiveness or temporal resolution. Furthermore, the preprocessor 16 may compress data classified as scenes with low responsiveness or temporal resolution in order, depending on the remaining storage capacity of the storage unit 10b that stores the data.

[0094] In the above embodiment, the vehicle control device 10 may be provided with a verification processing device for executing the program to be verified, taking into consideration the processing load when executing the installed program. In this case, the verification processing device may implement some or all of the functions other than the control unit 12, i.e., the input unit 11, the verification unit 13, the data acquisition unit 14, the classification unit 15, the preprocessing unit 16, and the transmission unit 17. The verification processing device may have performance equivalent to or different from that of the processing device 10a, as long as it has the performance required to execute the program to be verified.

[0095] In the above embodiment, the classification unit 15 is provided to classify the data by scene, but the data may be stored without classification. In this case, the classification unit 15 is not necessary. A modified example of the preprocessing in the above embodiment will be described with reference to FIG. 6. The preprocessing shown in FIG. 6 is performed at predetermined intervals. The preprocessing unit 16 determines whether or not a predetermined period in which preprocessing can be performed is in progress (step S301), similar to step S201. If the determination result is positive, the preprocessing unit 16 executes the preprocessing (step S302) and terminates the process. In step S302, data compression processing may be performed, similar to step S202, or preprocessing other than compression processing, such as anonymization processing or data transmission processing, may be performed.

[0096] On the other hand, if the determination result of step S301 is negative, it is determined whether the time since the previous preprocessing was performed is equal to or greater than the time threshold (step S303). In other words, it is determined whether the time elapsed since the end of the previous preprocessing period is equal to or greater than the time threshold. The time threshold may be set arbitrarily. Furthermore, the time threshold may be changed, for example, depending on the capacity (remaining storage capacity) of the storage unit 10b that can store data or the amount of data stored. Specifically, when the amount of stored data is large or the storage capacity is small, the time threshold may be reduced.

[0097] The preprocessing unit 16 identifies the process with the lowest processing load among the various processes executed in the preprocessing (step S304). For example, if multiple processes such as compression, anonymization, and data transmission are performed in the preprocessing, the process with the lowest processing load is identified. Note that the processing load varies depending on the amount of data to be processed, so it is desirable to identify the processing load for these processes taking into account the amount of data to be processed. Furthermore, if the data to be processed can be divided and processed, the processing load may be identified using the smallest unit of data volume.

[0098] Next, the preprocessing unit 16 determines the remaining capacity of the arithmetic processing unit 10a (step S305). That is, the preprocessing unit 16 determines how much remaining processing capacity is available when prioritizing the currently executing process (such as the process related to the driving assistance function).

[0099] The preprocessing unit 16 then determines whether the available capacity of the arithmetic processing unit 10a determined in step S305 exceeds the processing load of the process with the lowest processing load identified in step S304 (step S306). If the determination result is positive, the preprocessing unit 16 performs the process with the lowest processing load identified in step S303 (step S307). For example, if the processing load of the data transmission process is the lowest and the available processing capacity exceeds the processing load of the data transmission process, the preprocessing unit 16 transmits the data. Note that at this time, compressed data may be transmitted, or data before compression may be transmitted. Furthermore, if the processing load of the compression process is the lowest and the available processing capacity exceeds the processing load of the compression process, the preprocessing unit 16 performs data compression processing. Then, the processing ends.

[0100] On the other hand, if the determination result in step S306 is negative, the arithmetic processing device 10a temporarily restricts data collection (step S308). In other words, if there is no spare capacity due to high processing loads and the remaining storage capacity is low, the acquisition condition is not met. Note that if compression processing or data transmission processing is performed and there is spare storage capacity, this restriction is lifted.

[0101] The controller and methods described herein may be implemented by a special-purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the controller and methods described herein may be implemented by a special-purpose computer configured with a processor configured with one or more dedicated hardware logic circuits. Alternatively, the controller and methods described herein may be implemented by one or more special-purpose computers configured with a processor and memory programmed to perform one or more functions in combination with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0102] The following describes technical ideas that can be derived from the above embodiments and modified examples. [Configuration 1] A vehicle control device (10) mounted on a vehicle (101) includes: a data acquisition unit (14) that acquires data related to sensor information acquired by various sensors (20) mounted on the vehicle; a control unit (12) that performs vehicle control based on the sensor information; a pre-processing unit (16) that performs pre-processing on the data acquired by the data acquisition unit during a predetermined period; and a transmission unit (17) that transmits the data pre-processed by the pre-processing unit to a server (102) via a communication network (103), wherein the predetermined period is set when the control unit is not performing drive control related to the traveling of the vehicle. [Configuration 2] The vehicle control device according to Configuration 1, wherein the pre-processing includes a compression process that compresses the data by deleting any of the multiple data acquired by the data acquisition unit or thinning out some of the components of the data. [Configuration 3] The vehicle control device according to configuration 1 or 2, wherein the preprocessing includes anonymization processing of image data among the data acquired by the data acquisition unit so that a specific individual cannot be identified. [Configuration 4] The vehicle control device according to any one of configurations 1 to 3, wherein the transmission unit transmits the data during the predetermined period. [Configuration 5] The vehicle control device according to any one of configurations 1 to 4, wherein the predetermined period is set while the vehicle is parked or stopped, while the vehicle is charging, or during a monitoring mode in which the surroundings of a parked or stopped vehicle are monitored. [Configuration 6] The vehicle control device according to any one of configurations 1 to 5, wherein the control unit is configured to recognize targets based on the sensor information and perform predetermined processing on the targets, and wherein the predetermined period is set when the number of recognized targets is equal to or less than a threshold or when the amount of processing for the targets is equal to or less than a specified value. [Configuration 7] The vehicle control device according to any one of configurations 1 to 6, wherein the predetermined period is set when the position of the vehicle is determined to be a predetermined position based on the sensor information. [Configuration 8] The vehicle control device according to any one of configurations 1 to 7, wherein the predetermined period is set when it is determined that the weather is bad based on the sensor information.[Configuration 9] The vehicle control device according to any one of Configurations 1 to 8, wherein the vehicle control device is configured to be able to execute a shadow mode in which the program to be verified runs in a manner not related to control of the vehicle, and the data acquisition unit acquires data related to an output result of the program to be verified together with or instead of the data related to the sensor information when the program to be verified is executed using sensor information acquired by various sensors mounted on the vehicle as input values. [Configuration 10] A program executed by a vehicle control device (10) mounted on a vehicle (101), the program causing the vehicle control device to execute: a data acquisition step of acquiring data related to sensor information acquired by various sensors (20) mounted on the vehicle; a control step of performing vehicle control based on the sensor information; a preprocessing step of preprocessing the data acquired by the data acquisition step during a predetermined period; and a transmission step of transmitting the data preprocessed by the preprocessing step to a server (102) via a communication network (103), the predetermined period being set when drive control related to driving of the vehicle is not being performed in the control step.

[0103] Although the present disclosure has been described with reference to the embodiments, it is understood that the present disclosure is not limited to the embodiments or structures. The present disclosure also encompasses various modifications and equivalent modifications. In addition, various combinations and forms, including only one element, more than one element, or less than one element, are also within the scope and spirit of the present disclosure.

Claims

1. A vehicle control device (10) mounted on a vehicle (101), comprising: a data acquisition unit (14) that acquires data relating to sensor information acquired by various sensors (20) mounted on the vehicle; a control unit (12) that performs vehicle control based on the sensor information; a pre-processing unit (16) that performs pre-processing on the data acquired by the data acquisition unit during a predetermined period; and a transmission unit (17) that transmits the data pre-processed by the pre-processing unit to a server (102) via a communication network (103), wherein the predetermined period is set when the control unit is not performing drive control related to the running of the vehicle.

2. The vehicle control device according to claim 1, wherein the preprocessing includes a compression process for compressing the data by deleting any of the plurality of data acquired by the data acquisition unit or by thinning out some of the components of the data.

3. The vehicle control device according to claim 1, wherein the preprocessing includes anonymization processing of image data among the data acquired by the data acquisition unit so that a specific individual cannot be identified.

4. A vehicle control device according to any one of claims 1 to 3, wherein the transmission unit transmits the data during the predetermined period.

5. A vehicle control device according to any one of claims 1 to 3, wherein the predetermined period is set while the vehicle is parked or stopped, while the vehicle is charging, or during a monitoring mode in which the surroundings of a parked or stopped vehicle are monitored.

6. A vehicle control device according to any one of claims 1 to 3, wherein the control unit is configured to recognize targets based on the sensor information and perform predetermined processing on the targets, and the predetermined period is set when the number of recognized targets is equal to or less than a threshold value, or when the amount of processing for the targets is equal to or less than a specified value.

7. A vehicle control device according to any one of claims 1 to 3, wherein the predetermined period is set when it is determined that the vehicle's position is a predetermined position based on the sensor information.

8. A vehicle control device according to any one of claims 1 to 3, wherein the predetermined period is set when bad weather is determined based on the sensor information.

9. The vehicle control device is configured to be able to implement a shadow mode in which the program to be verified operates in a manner not related to the control of the vehicle, and the data acquisition unit acquires data on the output results of the program to be verified together with data on the sensor information when the program to be verified is executed using sensor information acquired by various sensors mounted on the vehicle as input values, or acquires data on the output results instead of data on the sensor information. A vehicle control device as described in any one of claims 1 to 3.

10. A program executed by a vehicle control device (10) mounted on a vehicle (101), causing the vehicle control device to perform the following steps: a data acquisition step of acquiring data relating to sensor information acquired by various sensors (20) mounted on the vehicle; a control step of performing vehicle control based on the sensor information; a preprocessing step of performing preprocessing on the data acquired by the data acquisition step during a predetermined period; and a transmission step of transmitting the data preprocessed by the preprocessing step to a server (102) via a communication network (103), wherein the predetermined period is set when drive control related to the driving of the vehicle is not being performed in the control step.

Images