Vehicle control device and program

Abstract

A vehicle control device (10) is mounted in a vehicle (101) and is provided with: a driving assistance implementation unit (13, 113, 10a) that can implement a plurality of functions related to vehicle driving-assistance on the basis of sensor information acquired by a sensor (20, 24, 24a–24d); a compression unit (16, 116) that compresses the data volume of the sensor information by executing at least reduction of some or all image data from one or a plurality of sets of image data included in the sensor information, or reduction of constituent elements of the image data; and a transmission unit (17, 117) that transmits the sensor information compressed by the compression unit to a server (102) via a communication network (103). The image data or the constituent elements thereof to be reduced are determined in accordance with the implementation functions implemented by the driving assistance implementation unit.

Description

Vehicle control device and program CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Application No. 2024-069046 filed on April 22, 2024, the contents of which are incorporated herein by reference.

[0002] The present disclosure relates to a vehicle control device and a program.

[0003] In recent years, a method has been adopted in which various data actually acquired from a sold vehicle while it is in operation is collected in a server via a communication network, and the collected data is analyzed to be used in future vehicle development or to update the vehicle's control program. Such a technique is described, for example, in Patent Document 1.

[0004] Japanese Patent Application Laid-Open No. 2023-53031

[0005] However, when collecting data, if data on all driving conditions in all scenes for all vehicles is collected, the amount of data becomes enormous, which causes problems such as increased communication load, processing load, and analysis load.

[0006] The present disclosure has been made in consideration of the above circumstances, and has as its main object to provide a vehicle control device and a program that suitably compress data.

[0007] A vehicle control device for solving the above problem is a vehicle control device mounted on a vehicle, and includes: a driving assistance implementation unit capable of implementing multiple functions related to driving assistance for the vehicle based on sensor information acquired by sensors mounted on the vehicle; a compression unit that compresses the data volume of the sensor information by reducing a portion of the sensor information; and a transmission unit that transmits the sensor information compressed by the compression unit to a server via a communication network, wherein the portion of the sensor information that is reduced is determined according to the implementation function performed by the driving assistance implementation unit.

[0008] According to the above configuration, the amount of data is compressed by reducing part of the sensor information depending on the implementation function. This reduces the amount of data stored, prevents pressure on storage capacity, and reduces communication load and analysis burden. Furthermore, depending on the implementation function, the amount of data can be compressed while retaining necessary data.

[0009] The program for solving the above problem is a program executed by a vehicle control device mounted on a vehicle, which causes the vehicle control device to perform the following steps: a driving assistance implementation step that can implement multiple functions related to driving assistance for the vehicle based on sensor information acquired by sensors mounted on the vehicle; a compression step that compresses the data volume of the sensor information by reducing a portion of the sensor information; and a transmission step that transmits the sensor information compressed by the compression step to a server via a communication network, wherein the portion of the sensor information that is reduced is determined according to the implementation function performed in the driving assistance implementation step.

[0010] According to the above configuration, the amount of data is compressed by reducing part of the sensor information depending on the implementation function. This reduces the amount of data stored, prevents pressure on storage capacity, and reduces communication load and analysis burden. Furthermore, depending on the implementation function, the amount of data can be compressed while retaining necessary data.

[0011] The above and other objects, features, and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which Fig. 1 is a schematic configuration diagram of a data collection system, Fig. 2 is a block diagram showing functions performed by a verification processing device, Fig. 3 is a diagram showing image data required for each performed function, Fig. 4 is a flowchart of a data acquisition process, Fig. 5 is a block diagram showing functions performed by a processing device of a second embodiment, Fig. 6 is a flowchart of a data acquisition process of the second embodiment, Fig. 7 is a schematic configuration diagram of a data collection system of a third embodiment, and Fig. 8 is a diagram showing sensor information required for each performed function of the third embodiment.

[0012] Hereinafter, embodiments of a vehicle control device and a program according to the present disclosure will be described in detail with reference to the drawings. Note that, between the embodiments and modifications, the same or corresponding parts in the drawings are designated by the same reference numerals, and their description will not be repeated in principle.

[0013] 1 shows a data collection system 100 to which a vehicle control device 10 according to this embodiment is applied. The vehicle control device 10 is mounted on a vehicle 101, and controls the vehicle 101 and performs driving assistance.

[0014] As shown in Fig. 1, a data collection system 100 includes a server 102 and is capable of communicating with one or more vehicles 101 via a communication network 103 such as the Internet. Fig. 1 shows only one vehicle 101. The vehicle 101 includes a vehicle control device 10, a sensor 20, an actuator 30, and the like.

[0015] The sensor 20 includes various sensors for measuring the traveling conditions of the vehicle 101, such as a vehicle speed sensor 21, an acceleration sensor 22, and a yaw rate sensor 23. The sensors for measuring the traveling conditions of the vehicle 101 may include other sensors, or any of the vehicle speed sensor 21, the acceleration sensor 22, and the yaw rate sensor 23 may not be included.

[0016] The sensor 20 also includes various sensors for detecting other vehicles and obstacles, such as a camera 24 and a millimeter-wave radar 25. The sensor for detecting other vehicles and obstacles is not limited to the millimeter-wave radar 25, but may be a laser radar (LiDAR), an ultrasonic sensor, or a combination of these. The camera 24 may be a monocular camera or a compound camera. The camera 24 may capture either still images or videos. The number, position, and type of the cameras 24 may be changed as desired. In this embodiment, the vehicle is equipped with a front camera 24a whose imaging range is in front of the vehicle, a right side camera 24b whose imaging range is on the right side of the vehicle, a left side camera 24c whose imaging range is on the left side of the vehicle, and a rear camera 24d whose imaging range is on the rear of the vehicle.

[0017] The sensor 20 also includes various sensors for detecting various amounts of operation by the driver, such as an accelerator sensor 26 that detects the amount of accelerator operation by the driver, a brake sensor 27 that detects the amount of brake operation, and a steering angle sensor 28 that detects the amount of steering (steering angle) of the steering wheel by the driver.

[0018] These sensors 20 are connected to the vehicle control device 10 wirelessly or by wire, and the measurement results (or detection results) of these sensors 20 are input to the vehicle control device 10 or the like as sensor information.

[0019] The actuators 30 include, for example, actuators for driving the vehicle 101, such as a motor that serves as the main engine of the vehicle 101. The actuators 30 may also include actuators for controlling the behavior of the vehicle 101, such as an actuator for operating a steering wheel, an actuator for operating a brake pedal, or an actuator for operating an accelerator pedal. The actuators 30 may also include devices for operating accessories of the vehicle 101, such as a display, a speaker, an indicator, and a headlight. The driving and operation of the vehicle 101 are controlled by the actuators 30.

[0020] The vehicle control device 10 is primarily composed of a microcomputer including a processing unit 10a such as a CPU and a storage unit 10b such as various types of memory. The functions provided by the microcomputer can be provided by software recorded in a physical memory device and a computer executing the software, software alone, hardware alone, or a combination thereof. For example, when the microcomputer is provided by electronic circuits, which are hardware, the functions can be provided by digital circuits including numerous logic circuits or analog circuits. For example, the processing unit 10a of the microcomputer executes programs stored in a non-transitory tangible storage medium (non-transitory tangible storage medium) that serves as the storage unit 10b. The programs include, for example, programs that realize functions shown in FIG. 2 . Execution of the programs results in the execution of methods corresponding to the programs. The storage unit 10b is, for example, a non-volatile memory. The programs stored in the storage unit 10b can be downloaded and updated via a communication network 103, such as the Internet, via over-the-air (OTA) or other means.

[0021] The vehicle control device 10 has various functions (application programs) for assisting the driving of the vehicle 101, and these functions control the actuator 30 based on sensor information input from the sensor 20.

[0022] Typical functions for assisting the driving of the vehicle 101 include, for example, an adaptive cruise control system (ACC), a forward collision warning (FCW), an advanced emergency braking system (AEBS), a night vision / pedestrian detection (NV / PD), a traffic sign recognition (TSR), a lane departure warning (LDW), a lane keeping assist system (LKAS), a rear cross traffic alert (RCTA), an adaptive front lighting system (AFS), and an advanced parking assist (APA). The vehicle 101 may be provided with all or some of these functions. Furthermore, the vehicle 101 may also be provided with other driving assistance functions. Furthermore, the vehicle 101 may also be provided with a function for realizing autonomous driving of the vehicle 101. These functions are realized by the arithmetic processing unit 10a executing a driving assistance control program stored in the storage unit 10b.

[0023] Furthermore, in order to verify the performance and safety of the driving assistance control program, the vehicle control device 10 has a function called a shadow mode in which a driving assistance control program to be verified (hereinafter referred to as a verification target program) is run while the vehicle 101 is running after being sold, that is, in an actual use case, and output values ​​are checked. In verification using the shadow mode, for example, the verification target program is run in the background, that is, in a manner not related to vehicle control, and data related to the output values ​​is stored, and the server 102 collects the data related to the output values ​​via the communication network 103. Note that the shadow mode is a type of data collection mode for collecting data.

[0024] The configuration and functions of the shadow mode according to this embodiment will be described below. The timing of switching to the shadow mode may be any timing. For example, the switching may be performed by an operation by the driver or by an instruction from the server 102. Alternatively, the switching to the shadow mode may be performed when the ignition switch is turned on.

[0025] In the following, the driving assistance control program that actually operates the vehicle 101 will be referred to as an "implemented program" to contrast it with the program to be verified. It will also be referred to as an "implemented program" in FIG. 1 . The vehicle control device 10 is equipped with a verification processing device 11 for executing the program to be verified, taking into account the processing load when executing the implemented program. The verification processing device 11 may have performance equivalent to or different from that of the processing device 10a, as long as it has the performance to execute the program to be verified. The program to be verified is stored in the storage unit 10b. A storage device dedicated to the shadow mode may also be provided.

[0026] Execution of the program to be verified executes various functions in the background to assist the driving of the vehicle 101. The functions implemented by execution of the program to be verified may be the same as the functions actually realized by the arithmetic processing device 10a, or some of the functions may be omitted. Furthermore, functions other than the functions actually realized by the arithmetic processing device 10a may also be implemented.

[0027] When the verification processing unit 11 is running the program to be verified in shadow mode, it receives sensor information from the various sensors 20, just like the processing unit 10a. Based on the received sensor information, the verification processing unit 11 then performs various functions and outputs various control signals for operating each actuator 30. These control signals (output results) are not actually input to the actuators 30, but are stored in the storage unit 10b as data related to the output results. At that time, the verification processing unit 11 also stores data related to the sensor information, which is an input value, in association with the data.

[0028] The verification processing unit 11 also receives the control signal output from the processing unit 10a after inputting sensor information as an input value and processing it based on the implementation program, and stores the control signal in the storage unit 10b as data related to the output result. That is, the verification processing unit 11 also stores the control signal output from the processing unit 10a after processing it based on the implementation program in order to compare and verify the output result of the implementation program with the output result of the program to be verified.

[0029] Furthermore, the verification processing device 11 may input sensor information (vehicle speed, yaw rate, acceleration amount, accelerator pedal operation amount, brake pedal operation amount, steering angle, etc.) detected when a control signal processed and output based on the implementation program is input to the actuator 30 and the vehicle 101 moves as a result, and store the information in the storage unit 10b. In other words, the sensor information related to the actual operation of the vehicle 101 for each scene, that is, the sensor information necessary to verify the correct operation, may also be stored.

[0030] The vehicle control device 10 uploads the data thus stored in the storage unit 10b to the server 102 via the communication network 103 at a predetermined transmission timing. The predetermined transmission timing may be any timing, for example, a timing when an upload instruction is issued from the server 102. Alternatively, the predetermined transmission timing may be, for example, a timing when the vehicle 101 is charging, when the vehicle is parked or stopped, or when the ignition switch is turned off.

[0031] However, there is a problem in that the storage capacity of the storage unit 10b is likely to be insufficient to store all data while the vehicle 101 is traveling. Also, if all the vehicles 101 upload all of the data they have stored, there is a risk that an excessive load will be placed on the communication network 103. Furthermore, there is also a problem in that the processing load on the server 102 will be too large if all of the uploaded data is analyzed by the server 102. There is also a problem in that there is a lot of waste in acquiring and analyzing all of the data in the first place.

[0032] For this reason, it is desirable to store and transmit only the amount of data necessary for verification and analysis. Therefore, the vehicle control device 10 of this embodiment deletes unnecessary data to reduce the amount of data stored and transmitted. This will be explained in detail below.

[0033] 2, the verification processing device 11 of the vehicle control device 10 has a function as an input unit 12, a function as a processing unit 13, a function as a data acquisition unit 14, a function as an assignment unit 15, a function as a compression unit 16, and a function as a transmission unit 17. These functions are realized by the verification processing device 11 by executing an in-vehicle program stored in the storage unit 10b.

[0034] The input unit 12 inputs sensor information from the sensor 20. Then, the input unit 12 inputs part or all of the input sensor information to the processing unit 13 as input values ​​for the program to be verified. The input unit 12 also inputs part or all of the input sensor information to the data acquisition unit 14.

[0035] The processing unit 13 runs the program to be verified and performs processing based on various functions (application programs) realized by the program to be verified, based on input values ​​(sensor information) input from the input unit 12. Therefore, in this embodiment, the processing unit 13 corresponds to a driving assistance implementation unit. The processing unit 13 then inputs control signals (control signals for the actuator 30) as the processing results to the data acquisition unit 14. As described above, these control signals are not input to the actuator 30.

[0036] The data acquisition unit 14 determines whether the acquisition condition is met based on at least one of the input value (sensor information) input from the input unit 12 and the output result (control signal) input from the processing unit 13.

[0037] The acquisition conditions may include, for example, thresholds for one or more parameters as components. The parameters of the acquisition conditions may be, for example, parameters included in sensor information such as vehicle speed, yaw rate, accelerator operation amount, brake operation amount, and steering amount, or parameters included in output results such as required torque and an output instruction signal to a display. The acquisition conditions may also include parameters calculated or estimated from sensor information or output results. For example, the acquisition condition parameters may include the relative distance to an obstacle (such as a preceding vehicle or pedestrian), the relative speed to the obstacle, and the time-to-collision (TTC) calculated from the recognition results of a camera image or the detection results of the millimeter-wave radar 25. The acquisition condition parameters may also include the difference between the output results based on the implemented program and the output results of the program to be verified. For example, the parameter may be the difference between the brake operation amount output by processing based on the implemented program and the brake operation amount output by processing based on the program to be verified. The threshold may be either the upper limit or lower limit of any of these parameters, or both.

[0038] Furthermore, a component of the acquisition condition may be whether or not the scene in which the vehicle 101 is traveling (hereinafter simply referred to as the scene) is a predetermined acquisition scene. The scene refers to various scenes that are expected when the vehicle 101 is traveling, such as a scene in which the vehicle overtakes a preceding vehicle, a scene in which the vehicle passes between vehicles, a scene in which the vehicle follows a preceding vehicle, a scene in which a pedestrian crosses in front of the vehicle 101 at night, a scene in which the vehicle 101 merges from an acceleration lane onto a main lane on a motorway, a scene in which the vehicle is parking or stopping, a scene in which the vehicle is waiting at a traffic light, and the like.

[0039] These scenes are estimated based on sensor information. For example, the vehicle control device 10 may recognize camera images or the like and estimate the scenes. More specifically, the sensor information such as camera images may be input to a machine-learned inference model such as a deep neural network to estimate the scenes. Note that image recognition does not have to be performed by the vehicle control device 10, and may be performed by an external device of the vehicle control device 10, such as an image recognition device, and the results may be input as sensor information.

[0040] Furthermore, among various functions (application programs) based on the program to be verified, an executed function (executed function) may be a component of the acquisition condition. For example, the acquisition condition may be satisfied when a collision damage mitigation braking control function is executed. Note that which function has been executed can be determined based on the output result (control signal) input from the processing unit 13.

[0041] In this embodiment, the acquisition condition is set by combining the acquisition scene, the execution function, and the parameter threshold (i.e., an AND condition). For example, the acquisition condition may be satisfied when the scene is following a preceding vehicle and the vehicle speed (a parameter of the acquisition condition) is equal to or greater than a threshold (50 km / h).

[0042] The combination of components of the acquisition condition may be an acquisition scene and an implementation function, an acquisition scene and a parameter threshold, or an implementation function and a parameter threshold. Furthermore, the components included in one acquisition condition may include two or more implementation functions, and in this case, the implementation functions may be an AND condition or an OR condition. For example, the condition may be that both the forward vehicle approach warning function and the collision damage mitigation braking control function are implemented, or that either one of them is implemented. Similarly, the parameter threshold may be a threshold for two or more types of parameters.

[0043] Furthermore, the acquisition condition is established when all of the conditions of the components that make up the acquisition condition are satisfied. For example, if the acquisition scene of the acquisition condition is "a scene of following a preceding vehicle," the implementation function of the acquisition condition is "a following driving function," and the threshold value of the parameter of the acquisition condition is "50 km / h or more," the acquisition condition is established when the following driving function is implemented in a scene of following a preceding vehicle and the vehicle speed (a parameter of the acquisition condition) is equal to or greater than the threshold value (50 km / h).

[0044] The number of acquisition conditions is not limited to one, and multiple acquisition conditions may be set. For example, a first acquisition condition may be a scene in which the vehicle is following a preceding vehicle and the vehicle speed is equal to or greater than a first threshold, and a second acquisition condition may be a scene in which a pedestrian is crossing in front of the vehicle 101, the distance to the pedestrian is equal to or less than a second threshold, and the amount of brake operation is equal to or greater than a third threshold.

[0045] When any of the acquisition conditions is met, the data acquisition unit 14 stores data regarding the input values ​​and output results in the storage unit 10b. As described above, the input values ​​include, for example, sensor information input to the verification processing device 11. The output results include, for example, control signals processed and output based on the program to be verified, and control signals processed and output based on the implemented program. The output results may also include sensor information related to the actual operation of the vehicle 101 for each scene. The output results may also include the difference between the output results based on the implemented program and the output results of the program to be verified.

[0046] The data to be stored (acquired) may be changed depending on the acquisition condition that is met. For example, when an acquisition condition is met in a scene where the vehicle is following a preceding vehicle and the vehicle speed is equal to or greater than a first threshold, data related to the vehicle speed may be acquired. On the other hand, when an acquisition condition is met in a scene where a pedestrian is crossing in front of the vehicle 101 and the distance to the pedestrian is equal to or less than a second threshold and the brake operation amount is equal to or greater than a third threshold, data related to the brake operation amount may be acquired. Furthermore, data related to sensor information, etc. acquired upon the satisfaction of an acquisition condition is stored in association with each other so as to be treated as a single set of data for each acquisition condition that is met. Specifically, data related to input values ​​and output results are assigned an ID and stored as a single set of data. Hereinafter, the term "data" simply refers to this data.

[0047] The data acquisition is terminated when a termination condition is met. The termination condition is arbitrary, and may be, for example, terminated when a predetermined time has elapsed since the acquisition condition was met. Alternatively, the data acquisition may be terminated when a started driving assistance function is terminated. Alternatively, the data acquisition may be terminated when the scene at the time of acquisition start is changed to another scene. Alternatively, the data acquisition may be terminated when a parameter included in the sensor information, such as vehicle speed, is compared with a threshold and the threshold is reached. Alternatively, the data acquisition may be terminated when the acquisition condition is no longer met. Note that the termination condition may differ for each type of acquisition condition, for each implemented function, or for each scene.

[0048] Next, the assignment unit 15 will be described. The assignment unit 15 assigns tag information indicating the executed function executed by the processing unit 13 among various functions (application programs) based on the program to be verified to data acquired (stored) by the data acquisition unit 14. This makes it possible to classify the data based on the tag information.

[0049] For example, when the processing unit 13 executes a following driving function, tag information indicating the executed function is assigned. Note that tag information may be assigned when the processing unit 13 executes the following driving function and the forward vehicle approach warning function, respectively. Furthermore, when the occasion (scene) for execution is almost the same, such as when the forward vehicle approach warning function and the collision damage mitigation braking control function are executed, tag information indicating that these functions were executed together may be assigned. Furthermore, the image recognition function is executed before the object recognition function or the traffic light recognition function is executed. When multiple functions are executed in conjunction with each other in this way, tag information indicating that these functions were executed together may be assigned. It is possible to determine which function was executed based on the output result (control signal) input from the processing unit 13.

[0050] When the data acquisition unit 14 stores the data in the storage unit 10b, the data acquisition unit 14 stores the data to which tag information has been added by the adding unit 15. This makes it possible to group the data by the implemented functions using the tag information.

[0051] Next, the compression unit 16 will be described. The sensor information included in each data stored in the storage unit 10b includes image data from cameras 24a to 24d (four image data in this embodiment). The compression unit 16 compresses the amount of data by performing at least one of reducing some or all of the image data and reducing the components of the image data. At this time, the image data or the components of the image data to be reduced are determined according to the function performed by the processing unit 13.

[0052] Specifically, the compression unit 16 identifies the function to be performed from tag information attached to each data, and determines the compression method for the image data or components of the image data to be reduced according to the function to be performed. This will be explained in detail below.

[0053] First, the timing of compression will be described. The compression unit 16 of this embodiment compresses data at a predetermined timing. The predetermined timing may be any timing, for example, the timing when the remaining storage capacity of the data in the storage unit 10b falls below a specified value. The predetermined timing may also be a timing specified by the server 102, or a timing when the processing load on the vehicle control device 10 is reduced, such as when the vehicle 101 is stopped.

[0054] Next, the compression method will be described. As described above, the data includes input values ​​such as sensor information and output results such as control signals. Among these, the data with the largest amount of data is generally image data included in the sensor information. While image data may be still images, it is generally video data, which has a large amount of data (information volume). Furthermore, when multiple cameras 24 (front camera 24a, right side camera 24b, left side camera 24c, rear camera 24d) are provided as in this embodiment, the amount of data related to the image data increases proportionally. Furthermore, the image recognition function may change the exposure when taking a photograph, which also increases the amount of data related to the image data.

[0055] On the other hand, depending on the implemented function, some of the multiple image data are not used as test data, and some image data are hardly used as verification data or have no value when verifying or analyzing. For example, in the case of a forward vehicle approach warning function that outputs an alarm when a forward vehicle approaches, image data other than image data from the front camera 24a (such as image data from the rear camera 24d) is not used. The same is true for a following driving function that causes the vehicle to drive so as to follow a preceding vehicle. On the other hand, in the case of a rear cross traffic alert, image data from the left and right side cameras 24b and 24c and the rear camera 24d are used.

[0056] Furthermore, when the traffic sign recognition function based on image data is implemented, it is sufficient to use the upper image area (e.g., the area above the FOE (Focus of Expansion)) of the image data of the front camera 24a. On the other hand, when the road surface recognition function based on image data is implemented, it is sufficient to use the lower image area (e.g., the area below the FOE) of the image data of the front camera 24a.

[0057] In addition, the image recognition function changes the exposure for each recognition target, so it is sufficient to use only image data with the appropriate exposure for the recognition target. There are also functions that do not use image data at all, such as the function that learns by imitating the driver's driving operations (Machine Learning Planner).

[0058] Therefore, the compression unit 16 of this embodiment compresses the amount of data by retaining image data necessary for the performed function and reducing other image data from the image data included in the data, or by retaining components of image data necessary for the performed function and reducing other components of image data. Note that reducing components of image data means, for example, lowering the sampling rate, deleting color information to create a monochrome image, reducing the size of the image, or deleting and cropping part of the image area.

[0059] With reference to FIG. 3, the image data (necessary image data) to be retained according to the performed function will be described. Here, since the compression unit 16 can identify the performed function from the tag information, it essentially determines the image data to be retained according to the tag information. At this time, if two or more performed functions are associated with the tag information and the image data required for each performed function differs, the image data required for one of the multiple performed functions is retained, and image data or components that are not required for any of the performed functions are deleted. Note that when two or more performed functions are associated with the tag information, the opportunities for performing these performed functions are generally the same, and the image data required (used) is often the same.

[0060] When the function being implemented is a traffic light recognition function using images (function No. 1 in FIG. 3), image data relating to the traffic light to be controlled and the area surrounding the traffic light is retained as necessary image data. Generally, in a traffic light recognition function, in order to recognize a traffic light ahead of the vehicle 101, image data relating to the upper area of ​​the image data from the front camera 24a is retained as necessary image data. Then, image data relating to the lower area of ​​the image data from the front camera 24a, and image data from the left and right side cameras 24b and 24c and the rear camera 24d are deleted.

[0061] Similarly, when the function being executed is an image-based road surface recognition function (function No. 2 in FIG. 3), image data relating to the road surface to be controlled and the surrounding area of ​​the road surface is retained as necessary image data. Generally, in a road surface recognition function, in order to recognize the road surface ahead of the vehicle 101, image data relating to the lower area of ​​the image data from the front camera 24a is retained as necessary image data. Then, image data relating to the upper area of ​​the image data from the front camera 24a, and image data from the left and right side cameras 24b and 24c and the rear camera 24d are deleted.

[0062] Furthermore, when the function being performed is an image-based object recognition function (function No. 3 in FIG. 3), image data relating to the detected object and the area surrounding the object is retained as necessary image data, and the remaining image data is deleted.

[0063] Furthermore, when the function being performed is an image recognition function (function No. 4 in FIG. 3), image data with an appropriate exposure for properly detecting (recognizing) the control target is retained as necessary image data, and image data captured with exposures other than those are deleted.

[0064] In this embodiment, when the traffic light recognition function is implemented, the image recognition function is also implemented as a prerequisite. Therefore, when the traffic light recognition function is implemented, data with appropriate exposure for detecting the traffic light to be controlled is left. Similarly, when the road surface recognition function is implemented, the image recognition function is also implemented as a prerequisite. Therefore, when the road surface recognition function is implemented, data with appropriate exposure for detecting the road surface (road) to be controlled is left. Similarly, when the object recognition function is implemented, the image recognition function is also implemented as a prerequisite. Therefore, when the object recognition function is implemented, data with appropriate exposure for detecting the object to be controlled is left.

[0065] Furthermore, when the implementation function is a function for learning by imitating the driver's driving operations (function No. 5 in FIG. 3), image data is not used much (there is no necessary image data). In other words, when the implementation function is a function for learning by imitating the driver's driving operations, the driver's operation amount (accelerator operation amount, brake operation amount, etc.) is used as test data or verification data. For this reason, when the implementation function is a function for learning by imitating the driver's driving operations, the color information of the image data is deleted, and the image data is reduced and further compressed by lowering the sampling rate. It is also possible to delete all image data.

[0066] Furthermore, when the function being executed is the collision damage mitigation braking control function (function No. 6 in FIG. 3), the image data from the cameras 24a to 24d that captured the approaching obstacle (control target) is retained, and the image data from the other cameras 24a to 24d is reduced. In other words, the necessary image data and the image data to be reduced are differentiated depending on the position of the approaching obstacle (control target).

[0067] For example, when the collision damage mitigation braking control function is implemented and a frontal collision occurs in which an obstacle approaches from the front of the vehicle, the image data from the front camera 24a is retained as necessary image data, and other image data is reduced.Also, when the collision damage mitigation braking control function is implemented and a side collision occurs in which an obstacle approaches from the side of the vehicle, the image data from the side cameras 24b and 24c on the approaching obstacle side and the image data from the front camera 24a are retained as necessary image data.

[0068] Note that an object recognition function that detects and recognizes a control target (approaching obstacle) is implemented as a prerequisite for the collision damage mitigation braking control function. In other words, the implementation of the collision damage mitigation braking control function also includes the implementation of the object recognition function (or the object recognition function is implemented as a prerequisite for the collision damage mitigation braking control function). Furthermore, the object recognition function identifies the position (coordinates) of the control target in the image data, so it is easy to identify which of the cameras 24a to 24d's image data should be retained.

[0069] Furthermore, when the object recognition function is implemented as a prerequisite for the collision damage mitigation braking control function, the image recognition function is also implemented as a prerequisite for that function. Therefore, image data with appropriate exposure is required to properly detect the control target of the collision damage mitigation braking control function.

[0070] Furthermore, when the execution function is the follow-up driving function (function No. 7 in FIG. 3), image data from the camera 24 capturing the image of the controlled object is retained, and the rest is eliminated. In the case of the follow-up driving function, the preceding vehicle that is the controlled object is included in the controlled object. Therefore, when the execution function is the follow-up driving function, the image data from the front camera 24a is retained as necessary image data.

[0071] However, in a scene where another vehicle cuts in front of the vehicle 101 (host vehicle), the number of control targets increases to two. That is, the preceding vehicle and the other vehicle (cutting in vehicle) become control targets. The other vehicle may be detected by the side cameras 24b and 24c. In this case, image data from the side cameras 24b and 24c on the side where the other vehicle is detected is also added and retained. All other image data is deleted. For example, if the other vehicle is present on the right side of the vehicle 101 (host vehicle), image data from the right side camera 24b is retained and image data from the left side camera 24c is deleted.

[0072] Note that an object recognition function for detecting and recognizing obstacles (control targets) is implemented as a prerequisite for the follow-up driving function. In other words, the implementation of the follow-up driving function also includes the implementation of the object recognition function (or the object recognition function is implemented as a prerequisite for the follow-up driving function). The object recognition function identifies the position (coordinates) of the control target in the image data, making it easy to identify which of the cameras 24a to 24d's image data will be retained. As described above, when the object recognition function is implemented as a prerequisite for the follow-up driving function, the image recognition function is also implemented as a prerequisite. Therefore, image data with appropriate exposure is required to properly detect the control target of the follow-up driving function. If there are two or more control targets, image data with appropriate exposure is retained for each of them.

[0073] Furthermore, when the function being implemented is rear cross traffic alert (function No. 8 in FIG. 3 ), image data from the camera 24 capturing an image of the target vehicle is retained, and the rest is deleted. In the case of rear cross traffic alert, image data from the left and right side cameras 24b, 24c and the rear camera 24d are retained as necessary image data. Note that, as with other functions, the position of the other vehicle to be controlled may be detected, and image data from the side camera 24b, 24c on the side where the target vehicle is located may be retained, and image data from the side camera 24b, 24c on the opposite side may be deleted.

[0074] Next, the flow of data acquisition in this embodiment will be described with reference to FIG. 4. The data acquisition flow shown below is the flow of data acquisition processing performed by the verification processing unit 11. This processing is performed after the shadow mode is set. The shadow mode may be set at any timing, such as when the ignition switch is turned on.

[0075] First, the input unit 12 of the verification processing device 11 inputs sensor information from the sensor 20 (step S101). The processing unit 13 runs the program to be verified and performs processing based on various functions realized by the program to be verified, based on the input values ​​(sensor information) input from the input unit 12 (step S102). This step S102 corresponds to the driving assistance implementation step. In step S102, the processing unit 13 inputs a control signal as a result of the processing to the data acquisition unit 14.

[0076] The data acquiring unit 14 determines whether or not an acquisition condition is met based on at least one of the input value (sensor information) and the output result (control signal, etc.) (step S103).

[0077] If the determination result is positive (if the acquisition condition is met), the assigning unit 15 of the verification processing device 11 assigns tag information to the data according to the performed function (step S104). Specifically, the assigning unit 15 identifies the performed function based on the data related to the output result input in step S102. Then, the assigning unit 15 assigns tag information according to the performed function to the data related to the input value and the output result.

[0078] The data acquisition unit 14 then stores (acquires) data relating to the input values ​​and output results in the storage unit 10b (step S105). When storing the data, the data acquisition unit 14 stores the data relating to the input values ​​and output results together. At that time, the tag information assigned by the assignment unit 15 in step S104 is also associated with the data and stored. This makes it possible to classify the data by the performed function. The data acquisition unit 14 also stores the acquisition conditions that were met in association with the data. Then, the processing ends.

[0079] On the other hand, if the determination result in step S103 is negative, the verification processing unit 11 ends the data acquisition process. The data acquisition process is executed at regular intervals after the shadow mode is set.

[0080] After the data acquisition process is completed, the compression unit 16 compresses the data stored in the storage unit 10b at a predetermined timing. As described above, the compression unit 16 compresses the data by leaving the necessary image data and its components according to the implemented function indicated as tag information and deleting unnecessary image data and image data components. This process by the compression unit 16 corresponds to the compression step.

[0081] The transmitting unit 17 transmits the data compressed by the compressing unit 16 and stored in the memory unit 10b to the server 102 via the communication network 103 at a predetermined transmission timing. At this time, the transmitting unit 17 may also transmit information related to the fulfilled acquisition condition associated with the data. The tag information assigned by the assigning unit 15 may also be transmitted together with the data. This processing by the transmitting unit 17 corresponds to the transmitting step.

[0082] When the server 102 receives the data transmitted from the transmitter 17 of each vehicle 101, it stores the data in the storage device of the server 102. When storing the data, the data may be classified by performed function based on the tag information received together with the data and stored. Information regarding the acquisition conditions that were met when the data was acquired may also be associated and stored. In other words, after classifying the data by performed function, the data may be further classified by the acquisition conditions and stored. This makes it easier to analyze the data because the data is grouped by performed function.

[0083] According to the first embodiment, the following effects are achieved.

[0084] The compression unit 16 compresses the amount of data by performing at least one of reducing some or all of the image data included in each data stored in the storage unit 10b and reducing the components of the image data. This prevents storage capacity from being overwhelmed and reduces communication loads, processing loads, analysis loads, and other burdens. The image data or its components to be reduced are determined according to the function being performed. By specifying image data that will not be used in analysis, verification, or the like as the image data or its components to be reduced according to the function being performed, unnecessary image data or its components can be reduced, while necessary image data and its components remain. This allows the data to be appropriately used for verification, etc., even when the amount of data is compressed.

[0085] Furthermore, when the implemented functions include (or are implemented as a prerequisite for) a function for detecting a control target, such as an object recognition function or an image recognition function, the compression unit 16 compresses the amount of data by reducing image data captured by a camera other than the camera that captured the control target. Alternatively, the compression unit 16 compresses the amount of data by reducing components of the image data captured by a camera other than the camera that captured the control target. Image data that does not capture a control target is generally presumed not to be used for analysis, verification, etc., so reducing such image data can suitably compress the amount of data. In other words, even if the amount of data is compressed, the data can still be used appropriately for verification, etc.

[0086] Furthermore, when the implemented functions include (or are implemented as a prerequisite for) a function for detecting a control target, such as an object recognition function or a traffic light recognition function, the compression unit 16 reduces the components of the image data by deleting areas other than the peripheral area of ​​the control target, including the control target. Areas other than the peripheral area of ​​the control target are generally not expected to be used in analysis, verification, or the like, so by deleting such image areas (components), the amount of data can be suitably compressed. In other words, even if the amount of data is compressed, the data can still be appropriately used for verification of the object recognition function, etc.

[0087] In the image recognition function, the camera 24 is configured to capture images of the same object multiple times with different exposures. If the image recognition function is included in the implemented functions (or is implemented as a prerequisite), the compression unit 16 compresses the amount of data by reducing image data captured with exposures other than those predetermined for the object to be controlled by the image recognition function, or by reducing the components of the image data. This makes it possible to compress data while retaining image data used for verification, etc.

[0088] Furthermore, when the execution function is a function that does not use image data, such as a function that learns by imitating the driving operation of a driver, the compression unit 16 compresses the amount of data by deleting all image data or by reducing the components of all image data, thereby enabling efficient data compression.

[0089] The compression unit 16 also reduces the components of the image data by reducing the size of the image data, deleting color information contained in the image data, or, if the image data is video data, lowering the sampling rate, thereby reducing the amount of data while retaining information related to the image data to the extent that the original image data can be at least understood.

[0090] The processing unit 13 is configured to be able to set a shadow mode in which a plurality of functions related to vehicle driving assistance are performed without being involved in the actual control of the vehicle. Specifically, even if the processing unit 13 executes the program to be verified, it does not output the output results to the actuator 30. This makes it possible to execute the program to be verified in an actual use case and collect data related to the output results.

[0091] (Variations of the First Embodiment) In the first embodiment, the control signal (data related to the output result) processed and output based on the implementation program does not need to be acquired (stored). On the other hand, sensor information (vehicle speed, yaw rate, acceleration amount, accelerator pedal operation amount, brake pedal operation amount, steering angle, etc.) detected when the vehicle 101 is operating may be acquired until a predetermined time has elapsed after the acquisition condition is met. In other words, data related to this sensor information may be stored in the storage unit 10b.

[0092] In the above embodiment, whether or not the acquisition condition (trigger condition) is satisfied is determined after the processing by the processing unit 13. However, whether or not the acquisition condition (trigger condition) is satisfied may be determined before the processing by the processing unit 13. Then, only when the acquisition condition is satisfied, the processing unit 13 may be caused to execute the program to be verified and acquire data related to the results. This prevents unnecessary execution of driving assistance functions, thereby reducing the processing load on the processing unit 13.

[0093] Second Embodiment A second embodiment in which the vehicle control device 10 in the first embodiment is partially modified will be described.

[0094] The vehicle control device 10 in the second embodiment is configured to acquire only data related to sensor information and data related to output results based on the installed program without executing the program to be verified. In other words, in the second embodiment, a data collection mode is set in which the program to be verified is not executed.

[0095] The configuration and functions of the shadow mode according to the second embodiment will be described below. In the second embodiment, the verification processing unit 11 is not provided, and the processing unit 10a performs processing related to the data collection mode. As in the first embodiment, the verification processing unit 11 may be provided and may perform processing in place of the processing unit 10a. Furthermore, the verification target program is not stored.

[0096] 5, in the second embodiment, the arithmetic processing device 10a of the vehicle control device 10 has a function as an input unit 112, a function as a processing unit 113, a function as a data acquisition unit 114, a function as an assignment unit 115, a function as a compression unit 116, and a function as a transmission unit 117. These functions are realized by the arithmetic processing device 10a as the arithmetic processing device 10a executes an in-vehicle program stored in the storage unit 10b.

[0097] The input unit 112 inputs sensor information from the sensor 20. The input unit 112 inputs some or all of the input sensor information to the processing unit 113 as input values ​​for the implementation program. The input unit 112 also inputs some or all of the input sensor information to the data acquisition unit 114.

[0098] The processing unit 113 runs the implementation program and performs processing based on various functions (application programs) realized by the implementation program, based on input values ​​(sensor information) input from the input unit 112. The processing unit 113 then inputs control signals (control signals for the actuator 30) as the processing results to the data acquisition unit 114. These control signals are also input to the actuator 30.

[0099] The data acquisition unit 114 determines whether or not an acquisition condition is met based on at least one of the input value (sensor information) input from the input unit 112 and the output result (control signal) input from the processing unit 13. The acquisition condition is the same as in the first embodiment, and therefore a description thereof will be omitted.

[0100] When the acquisition condition is met, the data acquisition unit 114 stores data related to the input values ​​and output results in the storage unit 10b. As described above, the input values ​​include, for example, sensor information input to the arithmetic processing device 10a. The output results include, for example, control signals processed and output based on the implementation program.

[0101] Furthermore, after the acquisition condition is satisfied, until a predetermined period of time has elapsed, the data acquisition unit 114 of the arithmetic processing device 10a may input sensor information (vehicle speed, yaw rate, acceleration amount, accelerator pedal operation amount, brake pedal operation amount, steering angle, etc.) detected when a control signal processed and output based on the implementation program is input to the actuator 30 and the vehicle 101 operates, and store the information in the storage unit 10b. In other words, the sensor information related to the actual operation of the vehicle 101 for each scene, that is, the sensor information necessary to verify the correct operation, may also be stored.

[0102] The assigning unit 115 assigns tag information indicating the executed function executed by the processing unit 113 among various functions (application programs) based on the implementation program to the data acquired (stored) by the data acquiring unit 114. The compression method and the like are the same as those in the first embodiment.

[0103] The compression unit 116 and the transmission unit 117 are similar to the compression unit 16 and the transmission unit 17 in the first embodiment, and therefore a description thereof will be omitted.

[0104] The flow of data acquisition in this embodiment will be described below with reference to Fig. 6. The flow of data acquisition shown below is the flow of data acquisition processing performed by the arithmetic processing device 10a. This processing is performed after the data collection mode is set.

[0105] First, the input unit 112 of the arithmetic processing device 10a inputs sensor information from the sensor 20 (step S201). The processing unit 113 of the arithmetic processing device 10a runs an implementation program and performs processing based on various functions realized by the implementation program, based on the input values ​​(sensor information) input from the input unit 112 (step S202). In step S202, the processing unit 113 inputs control signals as the processing results to the actuator 30 and the data acquisition unit 114.

[0106] The data acquisition unit 114 determines whether or not an acquisition condition is met based on at least one of the input value (sensor information) and the output result (control signal, etc.) (step S203).

[0107] If the determination result is positive (if the acquisition condition is met), the tagging unit 115 of the arithmetic processing device 10a assigns tag information to the data according to the performed function (step S204). Specifically, the tagging unit 115 identifies the performed function based on the data related to the output result input in step S202. Then, the tagging unit 115 assigns tag information according to the performed function to the data related to the input value and the output result.

[0108] The data acquisition unit 114 then stores (acquires) data relating to the input values ​​and output results in the storage unit 10b (step S205). When storing the data, the data acquisition unit 114 stores the data relating to the input values ​​and output results together. At that time, the tag information assigned by the assignment unit 115 in step S204 is also associated with the data and stored. This makes it possible to classify the data by performed function. The data acquisition unit 114 also stores the acquisition conditions that were met in association with the data. Then, the processing ends. On the other hand, if the determination result in step S203 is negative, the arithmetic processing device 10a ends the processing as is.

[0109] The processes of the compression unit 116, the transmission unit 117, and the server 102 are the same as those in the first embodiment, and therefore will not be described here.

[0110] According to the second embodiment, the following effects are achieved.

[0111] Since the program to be verified is not executed, the processing load can be reduced, and therefore the verification processing unit 11 does not need to be provided.

[0112] (Variation of the Second Embodiment) In the second embodiment, the data acquisition unit 114 acquires data related to the output results (control signals) based on the operation of the implementation program. However, it is not necessary to acquire data related to the output results based on the operation of the implementation program. In this case, it is also possible to acquire only sensor information that serves as an input value. Furthermore, it is also possible to acquire sensor information (vehicle speed, yaw rate, acceleration amount, accelerator pedal operation amount, brake pedal operation amount, steering angle, etc.) detected when the control signal processed and output based on the implementation program is input to the actuator 30 and the vehicle 101 operates.

[0113] Third Embodiment A third embodiment will be described, in which the vehicle control device 10 in the first embodiment is partially modified. Note that the same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof will be omitted.

[0114] 7 , the sensor 20 of the data collection system 100 of the third embodiment includes a laser radar 41 (LiDAR), an acoustic sensor 42, and an in-vehicle camera 43. The laser radar 41 is composed of multiple laser radars 41a to 41d with different detection areas. In this embodiment, the system includes a first laser radar 41a whose detection area is in front of the vehicle, a second laser radar 41b whose detection area is on the right side of the vehicle, a third laser radar 41c whose detection area is on the left side of the vehicle, and a fourth laser radar 41d whose detection area is on the rear of the vehicle. Note that the number, positions, types, and detection areas of the laser radars 41 may be changed as desired.

[0115] The acoustic sensor 42 is composed of multiple acoustic sensors 42a to 42c with different detection areas. In this embodiment, the acoustic sensor 42 includes a first acoustic sensor 42a whose detection area is sound from the windshield (front glass, etc.), a second acoustic sensor 42b whose detection area is sound from within the wheel well, and a third acoustic sensor 42c whose detection area is sound from within the vehicle cabin. The number, positions, types, and detection areas of the acoustic sensors 42 may be changed as desired.

[0116] The in-vehicle camera 43 is composed of multiple in-vehicle cameras 43a to 43c with different detection areas. In this embodiment, the in-vehicle camera 43a has the driver's seat as its detection area (image capture range), the second in-vehicle camera 43b has the passenger seat as its detection area (image capture range), and the third in-vehicle camera 43c has the rear seat as its detection area (image capture range). The number, position, type, and detection area of ​​the in-vehicle cameras 43 may be changed as desired.

[0117] The millimeter-wave radar 25 of the third embodiment is composed of multiple millimeter-wave radars 25a to d with different detection areas. In this embodiment, the first millimeter-wave radar 25a has a detection area in front of the vehicle, the second millimeter-wave radar 25b has a detection area on the right side of the vehicle, the third millimeter-wave radar 25c has a detection area on the left side of the vehicle, and the fourth millimeter-wave radar 25d has a detection area on the rear of the vehicle. The number, positions, types, and detection areas of the millimeter-wave radars 25 may be changed as desired.

[0118] Next, the compression unit 16 in the third embodiment will be described. As in the first embodiment, the compression unit 16 in the third embodiment also compresses the amount of data by deleting a portion of the sensor information acquired from the sensor 20. At this time, the portion of the sensor information to be deleted is determined according to the implemented function implemented by the processing unit 13. That is, the compression unit 16 identifies the implemented function from the tag information assigned to each data, and determines the portion of the sensor information to be deleted according to the implemented function. The timing of compression is the same as in the first embodiment. Below, the compression method in the third embodiment will be described in detail with reference to FIG. 8.

[0119] When the executed function is a function that detects a moving object (such as a vehicle or a pedestrian) using the millimeter-wave radar 25 and controls that object (function No. 11 in FIG. 8 ), the sensor information (detection information by the millimeter-wave radar 25) related to the moving object to be controlled is retained as necessary sensor information. On the other hand, the compression unit 16 deletes some or all of the sensor information (detection information by the millimeter-wave radar 25) related to stationary objects. Generally, when the millimeter-wave radar 25 is used, the signal from the millimeter-wave radar 25 is analyzed to detect the speed of the detected object, so it is easy to determine whether the detected object is a moving object or a stationary object based on the speed.

[0120] On the other hand, if the executed function is an advanced parking assist function using the millimeter-wave radar 25, which detects a stationary object (such as an obstacle) using the millimeter-wave radar 25 and controls that object (function No. 12 in FIG. 8 ), the sensor information (detection information by the millimeter-wave radar 25) related to the stationary object to be controlled is left as necessary sensor information. On the other hand, the compression unit 16 deletes some or all of the sensor information (detection information by the millimeter-wave radar 25) related to the moving object.

[0121] When the executed function is a function that detects a moving object (such as a vehicle) ahead using the millimeter-wave radar 25 and controls that object, such as a follow-up driving function using the millimeter-wave radar 25 (function No. 13 in FIG. 8 ), the sensor information relating to the moving object ahead that is the control object, detected by the first millimeter-wave radar 25a, is retained as necessary sensor information. Meanwhile, the compression unit 16 deletes some or all of the sensor information relating to objects to the side or rear that are detected by the second to fourth millimeter-wave radars 25b to 25d. Note that in this case, the sensor information relating to stationary objects ahead that are detected by the first millimeter-wave radar 25a may also be deleted.

[0122] Furthermore, although not shown, when the implemented function detects a rear object using the millimeter-wave radar 25, such as a rear cross traffic alert using the millimeter-wave radar 25, the sensor information related to the rear object detected by the fourth millimeter-wave radar 25d may be retained, while some or all of the sensor information related to the forward or lateral objects detected by the first to third millimeter-wave radars 25a to 25c may be deleted. In this case, the sensor information related to the rear moving object detected by the fourth millimeter-wave radar 25d may be deleted. In other words, the sensor information related to the detection target according to the implemented function may be retained, and the other sensor information may be deleted.

[0123] When the implemented function is a function that detects a moving object (such as a vehicle or a pedestrian) using the laser radar 41 and controls that object (function No. 14 in FIG. 8 ), the sensor information (detection information by the laser radar 41) related to the moving object to be controlled is left as necessary sensor information, while some or all of the sensor information (detection information by the laser radar 41) related to stationary objects is deleted. Generally, when the laser radar 41 is used, the signal from the laser radar 41 is analyzed to detect the speed of the object to be detected, and therefore it is easy to determine whether the object to be detected is a moving object or a stationary object based on the speed.

[0124] On the other hand, if the function being implemented is an advanced parking assist function using the laser radar 41, such as a function that detects stationary objects (such as obstacles) using the laser radar 41 and controls them (function No. 15 in Figure 8), the sensor information (detection information by the laser radar 41) related to the stationary object to be controlled is retained as necessary sensor information, while some or all of the sensor information (detection information by the laser radar 41) related to the moving object is deleted.

[0125] When the function being implemented is a function such as a follow-up travel function using the laser radar 41, which detects a moving object (such as a vehicle) ahead using the laser radar 41 and controls that object (function No. 16 in FIG. 8), the sensor information relating to the moving object ahead that is the control object, detected by the first laser radar 41a, is left as necessary sensor information, while some or all of the sensor information relating to objects to the side or rear that are detected by the second to fourth laser radars 41b to 41d is deleted. In this case, the sensor information relating to stationary objects ahead that are detected by the first laser radar 41a may also be deleted.

[0126] Furthermore, although not shown, when the implemented function detects a rear object using the laser radar 41, such as a rear cross traffic alert using the laser radar 41, the sensor information related to the object detected by the fourth laser radar 41 d may be retained, while some or all of the sensor information related to the forward or lateral objects detected by the first to third laser radars 41 a to 41 c may be deleted. In this case, the sensor information related to the rear moving object detected by the fourth laser radar 41 d may be deleted. In other words, the sensor information related to the detection target corresponding to the implemented function may be retained, and the other sensor information may be deleted.

[0127] When the function being executed is a wind noise detection function that controls wind noise at the windshield (function No. 17 in FIG. 8 ), the sensor information from the first acoustic sensor 42a is retained as necessary sensor information, while some or all of the sensor information from the second and third acoustic sensors 42b and 42c is deleted. The wind noise detection function is executed, for example, when traveling at a speed equal to or greater than a predetermined speed. Similarly, although not shown, when an abnormal noise detection function that controls abnormal noise inside a wheel well is executed, the sensor information from the second acoustic sensor 42b may be retained, while some or all of the sensor information from the first and third acoustic sensors 42a and 42c may be deleted. Similarly, when an interior sound detection function that controls sound inside the vehicle is executed, the sensor information from the third acoustic sensor 42c may be retained, while some or all of the sensor information from the first and second acoustic sensors 42a and 42b may be deleted.

[0128] When the executed function is a driver monitoring function that controls the driver (function No. 18 in FIG. 8), the camera image of the driver's seat taken by the first in-vehicle camera 43a is retained as necessary sensor information, while some or all of the sensor information from the second to third in-vehicle cameras 42b to 42c is deleted. The driver monitoring function is executed, for example, while the vehicle is traveling, to monitor the driver's fatigue, drowsiness, etc.

[0129] The effects of the third embodiment will be described.

[0130] The compression unit 16 compresses the data volume of the sensor information by reducing a portion of the sensor information, and the portion of the sensor information to be reduced is determined according to the implemented function. For example, as described in the third embodiment, if the implemented function includes a function for detecting a predetermined control target from the sensor information, the compression unit 16 reduces the data volume of the sensor information by reducing some or all of the sensor information from sensors 20 other than the sensor 20 that detects the control target. This prevents storage capacity from being overwhelmed and reduces communication loads, processing loads, analysis loads, and other burdens. Furthermore, the compression unit 16 retains sensor information from sensors 20 that detect the control target of the implemented function and deletes sensor information from sensors 20 that do not detect the control target. This allows the data to be appropriately used for verification, etc., even when the data volume is compressed.

[0131] The millimeter-wave radar 25 and the laser radar 41 are configured to detect the speed of an object, and the compressor 16 compresses the data volume of the sensor information by reducing some or all of the sensor information related to a moving object or a stationary object. In the third embodiment, if the implementation function includes a function for detecting a stationary object as a control target from the sensor information, the compressor 16 reduces the data volume of the sensor information by reducing some or all of the sensor information related to the moving object. Furthermore, if the implementation function includes a function for detecting a moving object as a control target from the sensor information, the compressor 16 reduces the data volume of the sensor information by reducing some or all of the sensor information related to the stationary object. This prevents storage capacity from being overwhelmed and reduces communication loads, processing loads, analysis loads, and other burdens. Furthermore, when a moving object is a detection target, the sensor information related to the moving object is retained while the sensor information related to the stationary object is deleted. On the other hand, when a stationary object is a detection target, the sensor information related to the stationary object is retained while the sensor information related to the moving object is deleted. This allows the data to be appropriately used for verification, etc., even when the data volume is compressed.

[0132] (Variation of the Third Embodiment) In the third embodiment, the detection ranges of the millimeter-wave radar 25 and the laser radar 41 have a certain degree of width. For example, the first millimeter-wave radar 25a installed in the front of the vehicle can detect objects within a 120-degree angular range centered on the front. Therefore, the compressor 16 may compress the amount of data by retaining sensor information within a 30-degree angular range centered on the control object and deleting sensor information within other angular ranges. In other words, the compressor 16 may reduce part of the sensor information from the millimeter-wave radar 25 and the laser radar 41 by deleting sensor information relating to areas other than the area surrounding the control object, including the control object. This prevents storage capacity from being overwhelmed and reduces communication loads, processing loads, analysis loads, and other burdens. Furthermore, even if the amount of data is compressed, the data can still be appropriately used for verification, etc.

[0133] The third embodiment and the second embodiment may be combined.

[0134] (Other Modifications) In the above embodiment, the required image data and the components of the required image data may be determined according to the performed function. In other words, the required image data and the components of the required image data may be determined according to the performed function, regardless of the position of the control target. This simplifies the processing.

[0135] In the above embodiment, a portion of the sensor information that is not necessary for data analysis may be specified from the server 102 or the like via the communication network 103. Then, the compression unit 16 may compress the data volume of the sensor information by deleting the portion of the sensor information that is not necessary for data analysis based on the specification.

[0136] For example, if it is desired to improve the recognition process for moving objects, the server 102 may specify that sensor information for stationary objects is not necessary for data analysis. In this case, the compression unit 16 may compress the amount of data by leaving the sensor information for moving objects and deleting the sensor information for stationary objects. If it is desired to improve the recognition process for stationary objects, the opposite can be done.

[0137] In the above embodiment, the tagging unit 15, 115 adds tag information indicating the performed function, but tag information does not have to be added. In this case, the compression unit 16, 116 can identify the performed function from the data and identify the image data and its components to be compressed.

[0138] The processing units 13 and 113 of the arithmetic processing device 10a in the above embodiment function as signal processing units that perform one or more signal processes on signal information acquired from the millimeter-wave radar 25. For example, the processing units 13 and 113 perform signal processing such as converting analog signals detected by the millimeter-wave radar 25 into digital signals, analyzing the digital signals to extract peak signals, detecting point cloud signals from the peak signals, and identifying target signals to be controlled from these signals. Furthermore, multiple signal processes reduce the amount of data. Therefore, the amount of data for the sensor information may be compressed by specifying a signal processing level to be analyzed and using the processing results instead of the sensor information. The signal processing level may be predetermined or may be specified externally (e.g., from the server 102).

[0139] The controller and methods described herein may be implemented by a special-purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the controller and methods described herein may be implemented by a special-purpose computer configured with a processor configured with one or more dedicated hardware logic circuits. Alternatively, the controller and methods described herein may be implemented by one or more special-purpose computers configured with a processor and memory programmed to perform one or more functions in combination with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0140] The following describes technical ideas that can be derived from the above-described embodiment and modifications.

[0141] [Configuration 1] A vehicle control device (10) mounted on a vehicle (101), comprising: a driving assistance implementation unit (13, 113, 10a) capable of implementing a plurality of functions related to driving assistance for the vehicle based on sensor information acquired by sensors (20, 24, 24a to 24d) mounted on the vehicle; a compression unit (16, 116) configured to compress the data volume of the sensor information, the sensor information including one or more pieces of image data, by performing at least one of reducing part or all of the image data and reducing components of the image data, from the one or more pieces of image data included in the sensor information; and a transmission unit (17, 117) configured to transmit the sensor information compressed by the compression unit to a server (102) via a communication network (103), wherein the image data or the components thereof to be reduced are determined according to the implementation function implemented by the driving assistance implementation unit.

[0142] [Configuration 2] The vehicle control device according to Configuration 1, wherein the sensor includes a plurality of cameras (24a, 24b, 24c, 24d) with different imaging areas, and when the implementation function includes a function of detecting a predetermined control object in the image data, the compression unit compresses the data amount of the sensor information by reducing image data captured by cameras other than the camera that captured the control object or by reducing components of the image data captured by cameras other than the camera that captured the control object.

[0143] [Configuration 3] The vehicle control device according to Configuration 1 or 2, wherein, when the implementation function includes a function of detecting a predetermined control object in the image data, the compression unit reduces components of the image data by deleting an area other than a peripheral area of ​​the control object that includes the control object.

[0144] [Configuration 4] The vehicle control device according to any one of configurations 1 to 3, wherein the sensor includes a camera (24, 24a to 24d) that captures the image data, the camera is configured to capture an image of the same object multiple times with different exposures, and when the implementation function includes a function of detecting a predetermined control object in the image data, the compression unit compresses the data amount of the sensor information by reducing image data captured with an exposure other than that predetermined for the control object, or by reducing components of image data captured with an exposure other than that predetermined for the control object.

[0145] [Configuration 5] A vehicle control device according to any one of configurations 2 to 4, wherein a warning function that outputs a warning as a controlled object approaches, a following function that causes the vehicle to follow the controlled object, or a braking avoidance function that causes the vehicle to brake or avoid the controlled object as a controlled object approaches, includes a function that detects the controlled object in the image data, or a function that detects the controlled object in the image data is executed as a prerequisite for these functions.

[0146] [Configuration 6] The vehicle control device according to any one of configurations 1 to 5, wherein when the performed function is a function that does not use the image data, the compression unit compresses the amount of data of the sensor information by deleting all image data or reducing components of all image data.

[0147] [Configuration 7] The vehicle control device according to any one of configurations 1 to 6, wherein the compression unit reduces components of the image data by reducing the size of the image data, by deleting color information included in the image data, or by lowering the sampling rate if the image data is video data.

[0148] [Configuration 8] The vehicle control device according to any one of Configurations 1 to 7, wherein the vehicle control device is configured to be able to set a shadow mode in which a plurality of functions related to driving assistance for the vehicle are performed without being involved in actual control of the vehicle.

[0149] [Configuration 9] An in-vehicle program executed by a vehicle control device (10) mounted on a vehicle (101), causing the vehicle control device to execute the following steps: a driving assistance implementation step capable of implementing a plurality of functions related to driving assistance for the vehicle based on sensor information acquired by sensors (20, 24, 24a to 24d) mounted on the vehicle; a compression step in which, when the sensor information includes one or more pieces of image data, the sensor information is compressed by performing at least one of reducing some or all of the image data and reducing components of the image data, thereby compressing the data volume of the sensor information; and a transmission step in which the sensor information compressed by the compression step is transmitted to a server (102) via a communication network (103), wherein the image data or the components thereof to be reduced are determined according to the implementation function executed in the driving assistance implementation step. [Configuration 11] A vehicle control device (10) mounted on a vehicle (101), comprising: a driving assistance implementation unit (13, 113, 10a) capable of implementing a plurality of functions related to driving assistance of the vehicle based on sensor information acquired by sensors (20, 24, 24a to 24d, 25a to 25d, 41a to 41d, 42a to 42c, 43a to 43c) mounted on the vehicle; a compression unit (16, 116) that compresses the amount of data of the sensor information by reducing a portion of the sensor information; and a transmission unit (17, 117) that transmits the sensor information compressed by the compression unit to a server (102) via a communication network (103), wherein the portion of the sensor information to be reduced is determined according to the implementation function performed by the driving assistance implementation unit. [Configuration 12] The vehicle control device according to Configuration 11, wherein the sensor is composed of a plurality of sensors (24a, 24b, 24c, 24d, 25a to 25d, 41a to 41d, 42a to 42c, 43a to 43c) having different detection areas, and when the implemented function includes a function of detecting a predetermined control object from the sensor information, the compression unit compresses the data amount of the sensor information by reducing some or all of the sensor information from sensors other than the sensor that detected the control object.[Configuration 13] The vehicle control device according to configuration 11 or 12, wherein, when the implementation function includes a function of detecting a predetermined control object from the sensor information, the compression unit reduces a part of the sensor information by deleting sensor information relating to an area other than a peripheral area of ​​the control object that includes the control object. [Configuration 14] The vehicle control device according to any of configurations 11 to 13, wherein the sensor includes a camera (24, 24a to 24d) that captures the image data, and the camera is configured to capture an image of the same object multiple times with different exposures, when the implementation function includes a function of detecting a predetermined control object in the image data, the compression unit compresses the data amount of the sensor information by reducing image data captured with an exposure other than a predetermined exposure corresponding to the control object, or by reducing components of image data captured with an exposure other than a predetermined exposure corresponding to the control object. [Configuration 15] The vehicle control device according to any one of configurations 11 to 14, wherein the sensors include sensors (25a to 25d, 41a to 41d) capable of measuring the speed of a detection target, and the compression unit compresses the data volume of the sensor information by reducing the sensor information related to either a moving object or a stationary object. [Configuration 16] The vehicle control device according to any one of configurations 11 to 14, wherein the sensors include sensors (25a to 25d, 41a to 41d) capable of measuring the speed of a detection target, and when the implementation function includes a function of detecting a stationary object as a control target from the sensor information, the compression unit compresses the data volume of the sensor information by reducing some or all of the sensor information related to a moving object, or when the implementation function includes a function of detecting a moving object as a control target from the sensor information, the compression unit compresses the data volume of the sensor information by reducing some or all of the sensor information related to a stationary object.[Configuration 17] The vehicle control device according to any one of configurations 11 to 16, further comprising: a signal processing unit that performs signal processing one or more times on signal information included in the sensor information, and the compression unit compresses the data amount of the sensor information by replacing the signal information included in the sensor information with a processing result of one or more times of signal processing performed by the signal processing unit. [Configuration 18] The vehicle control device according to any one of configurations 11 to 17, wherein the compression unit compresses the data amount of the sensor information by deleting a part of the sensor information that is not necessary for data analysis. [Configuration 19] The vehicle control device according to any one of configurations 11 to 18, wherein a warning function that outputs a warning as a controlled object approaches, a following function that causes the vehicle to follow the controlled object, or a braking avoidance function that causes the vehicle to brake or avoid the control object as a controlled object approaches includes a function of detecting the controlled object from the sensor information, or a function of detecting the controlled object from the sensor information is executed as a prerequisite for these functions. [Configuration 20] The vehicle control device according to any one of configurations 11 to 19, wherein the sensor information includes image data, and when the performed function is a function that does not use the image data, the compression unit compresses the data amount of the sensor information by deleting all image data or reducing all components of the image data. [Configuration 21] The vehicle control device according to any one of configurations 11 to 20, wherein the sensor information includes image data, and the compression unit reduces the components of the image data by reducing the size of the image data, or by deleting color information included in the image data, or, if the image data is video data, by lowering the sampling rate of the image data. [Configuration 22] The vehicle control device according to any one of configurations 11 to 21, wherein the vehicle control device is configured to be able to set a shadow mode that performs a plurality of functions related to driving assistance for the vehicle without being involved in actual control of the vehicle.[Configuration 23] A program executed by a vehicle control device (10) mounted on a vehicle (101), causing the vehicle control device to perform a driving assistance implementation step capable of implementing a plurality of functions related to driving assistance of the vehicle based on sensor information acquired by sensors (20, 24, 24a to 24d, 25a to 25d, 41a to 41d, 42a to 42c, 43a to 43c) mounted on the vehicle, a compression step of compressing the data volume of the sensor information by reducing a portion of the sensor information, and a transmission step of transmitting the sensor information compressed by the compression step to a server (102) via a communication network (103), wherein the portion of the sensor information to be reduced is determined according to the implemented function performed in the driving assistance implementation step.

[0150] Although the present disclosure has been described with reference to the embodiments, it is understood that the present disclosure is not limited to the embodiments or structures. The present disclosure also encompasses various modifications and equivalent modifications. In addition, various combinations and forms, including only one element, more than one element, or less than one element, are also within the scope and spirit of the present disclosure.

Claims

1. A vehicle control device (10) mounted on a vehicle (101), comprising: a driving assistance implementation unit (13, 113, 10a) capable of implementing a plurality of functions related to driving assistance for the vehicle based on sensor information acquired by sensors (20, 24, 24a to 24d, 25a to 25d, 41a to 41d, 42a to 42c, 43a to 43c) mounted on the vehicle; a compression unit (16, 116) that compresses the data volume of the sensor information by reducing a portion of the sensor information; and a transmission unit (17, 117) that transmits the sensor information compressed by the compression unit to a server (102) via a communication network (103), wherein the reduced portion of the sensor information is determined according to the implemented function performed by the driving assistance implementation unit.

2. The vehicle control device according to claim 1, wherein the sensor is composed of a plurality of sensors (24a, 24b, 24c, 24d, 25a-25d, 41a-41d, 42a-42c, 43a-43c) having different detection areas, and when the implementation function includes a function of detecting a predetermined control object from the sensor information, the compression unit compresses the data volume of the sensor information by reducing some or all of the sensor information from sensors other than the sensor that detected the control object.

3. A vehicle control device as described in claim 1, wherein, when the implementation function includes a function of detecting a predetermined control object from the sensor information, the compression unit reduces a portion of the sensor information by deleting sensor information relating to areas other than the surrounding area of ​​the control object, including the control object.

4. A vehicle control device as described in claim 1, wherein the sensor includes a camera (24, 24a to 24d) that captures the image data, the camera is configured to capture images of the same object multiple times with different exposures, and when the implementation function includes a function to detect a predetermined control object in the image data, the compression unit compresses the data volume of the sensor information by reducing image data captured with an exposure other than that predetermined for the control object, or by reducing components of image data captured with an exposure other than that predetermined for the control object.

5. The vehicle control device according to claim 1, wherein the sensors include sensors (25a to 25d, 41a to 41d) capable of measuring the speed of the object to be detected, and the compression unit compresses the data volume of the sensor information by reducing either the sensor information related to a moving object or the sensor information related to a stationary object.

6. The vehicle control device according to claim 1, wherein the sensors include sensors (25a to 25d, 41a to 41d) capable of measuring the speed of an object to be detected, and when the implementation function includes a function of detecting a stationary object as an object to be controlled from the sensor information, the compression unit compresses the amount of data of the sensor information by reducing some or all of the sensor information related to a moving object, or when the implementation function includes a function of detecting a moving object as an object to be controlled from the sensor information, the compression unit compresses the amount of data of the sensor information by reducing some or all of the sensor information related to a stationary object.

7. A vehicle control device as described in claim 1, further comprising a signal processing unit that performs signal processing one or more times on signal information included in the sensor information, and wherein the compression unit compresses the data volume of the sensor information by replacing the signal information included in the sensor information with the processing result of one or more times of signal processing performed by the signal processing unit.

8. The vehicle control device according to claim 1, wherein the compression unit compresses the amount of data of the sensor information by deleting a portion of the sensor information that is not necessary for data analysis.

9. A vehicle control device according to any one of claims 1 to 8, wherein a warning function that outputs a warning as a controlled object approaches, a following function that causes the vehicle to follow the controlled object, or a braking and avoidance function that causes the vehicle to brake or avoid the controlled object as the controlled object approaches, includes a function that detects the controlled object from the sensor information, or a function that detects the controlled object from the sensor information is executed as a prerequisite for these functions.

10. A vehicle control device as claimed in any one of claims 1 to 8, wherein the sensor information includes image data, and when the implemented function is a function that does not use the image data, the compression unit compresses the amount of data in the sensor information by deleting all image data or reducing the components of all image data.

11. A vehicle control device as claimed in any one of claims 1 to 8, wherein the sensor information includes image data, and the compression unit reduces the components of the image data by reducing the size of the image data, or by deleting color information included in the image data, or, if the image data is video data, by lowering its sampling rate.

12. A vehicle control device according to any one of claims 1 to 8, wherein the vehicle control device is configured to be able to set a shadow mode in which a plurality of functions related to driving assistance for the vehicle are performed without being involved in the actual control of the vehicle.

13. A program executed by a vehicle control device (10) mounted on a vehicle (101), causing the vehicle control device to perform the following steps: a driving assistance implementation step that can implement multiple functions related to driving assistance for the vehicle based on sensor information acquired by sensors (20, 24, 24a to 24d, 25a to 25d, 41a to 41d, 42a to 42c, 43a to 43c) mounted on the vehicle; a compression step that compresses the data volume of the sensor information by reducing a portion of the sensor information; and a transmission step that transmits the sensor information compressed by the compression step to a server (102) via a communication network (103), wherein the portion of the sensor information to be reduced is determined according to the implemented function performed in the driving assistance implementation step.

Images