Vehicle control devices and programs

By incorporating a driving assistance implementation unit, a compression unit, and a transmission unit into the vehicle control device, and compressing sensor information based on driving assistance functions, the load problem caused by the large amount of data is solved, achieving effective data management and reducing communication load.

CN123125633APending Publication Date: 2026-11-17DENSO CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202580029173.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-04-22
Filing Date
2025-03-12
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

During the data collection process, the massive amount of data leads to increased communication load, processing load, and analysis load, and existing technologies struggle to effectively compress and manage vehicle data.

Method used

By setting up a driving assistance implementation unit, a compression unit, and a transmission unit in the vehicle control device, a portion of the sensor information is reduced, the data volume is compressed according to the driving assistance function, and then sent to the server via a communication network.

Benefits of technology

Reduce the amount of stored data, lower the communication load and parsing burden, retain necessary data, and achieve effective data compression.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123125633A_ABST
    Figure CN123125633A_ABST
Patent Text Reader

Abstract

The vehicle control device (10) of the present invention, mounted on a vehicle (101), includes: a driving assistance implementation unit (13, 113, 10a) capable of implementing multiple functions related to driving assistance of the vehicle based on sensor information acquired by sensors (20, 24, 24a to 24d); a compression unit (16, 116) compressing the data volume of the sensor information by performing at least one of reducing a portion or all of the image data contained in the sensor information and reducing the constituent elements of the image data; and a transmission unit (17, 117) transmitting the sensor information compressed by the compression unit to a server (102) via a communication network (103), wherein the reduced image data or its constituent elements are determined according to the implementation functions performed by the driving assistance implementation unit.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application incorporates the contents of Japanese Patent Application No. 2024-069046, filed on April 22, 2024. Technical Field

[0002] This disclosure relates to vehicle control devices and procedures. Background Technology

[0003] In recent years, methods have been developed to collect various data actually acquired during driving from vehicles after they have been sold via communication networks and transmit them to a server, and to analyze the data collected on the server for future vehicle development or to update the vehicle's control program. Such technology is described, for example, in Patent Document 1.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2023-53031

[0005] However, when collecting data, if the goal is to collect data related to all vehicles, all scenarios, and all driving conditions, the amount of data becomes enormous. This large data volume leads to increased overhead in communication, processing, and analysis. Summary of the Invention

[0006] This disclosure was made in view of the above circumstances, and its main purpose is to provide a vehicle control device and program for appropriately compressing data.

[0007] The vehicle control device for solving the above-mentioned problems, in a vehicle control device mounted on a vehicle, includes:

[0008] The driver assistance implementation unit is capable of implementing multiple functions related to driver assistance of the vehicle based on sensor information acquired by sensors mounted on the vehicle.

[0009] The compression unit compresses the amount of sensor information by reducing a portion of the sensor information; and

[0010] The transmitting unit transmits the sensor information compressed by the compression unit to the server via a communication network.

[0011] The amount of sensor information that is reduced is determined based on the implementation functions performed by the aforementioned driver assistance implementation unit.

[0012] Based on the above structure, a portion of the sensor information is reduced according to the implemented functions, thereby compressing the data volume. Therefore, it is possible to reduce the amount of stored data, prevent storage capacity constraints, and alleviate communication load and parsing burden. Furthermore, it is possible to retain only the necessary data and compress the data volume according to the implemented functions.

[0013] The program used to solve the above-mentioned problems is implemented by the vehicle control device installed in the vehicle.

[0014] The above procedure enables the vehicle control device to perform the following steps:

[0015] The driving assistance implementation steps are capable of implementing multiple functions related to the driving assistance of the vehicle based on sensor information acquired by sensors mounted on the vehicle.

[0016] The compression step compresses the amount of sensor information by reducing a portion of it; and

[0017] In the transmission step, the sensor information compressed through the above compression step is sent to the server via a communication network.

[0018] The amount of sensor information that is reduced is determined based on the implementation functions performed in the aforementioned driving assistance implementation steps.

[0019] Based on the above structure, a portion of the sensor information is reduced according to the implemented functions, thereby compressing the data volume. Therefore, it is possible to reduce the amount of stored data, prevent storage capacity constraints, and alleviate communication load and parsing burden. Furthermore, it is possible to retain only the necessary data and compress the data volume according to the implemented functions. Attached Figure Description

[0020] The foregoing and other objects, features, and advantages of this disclosure become clearer from the following detailed description with reference to the accompanying drawings.

[0021] Figure 1 This is a schematic diagram of the data collection system.

[0022] Figure 2 It is a block diagram representing the function of the verification processing device.

[0023] Figure 3 It is a graph representing the image data required for each implemented function.

[0024] Figure 4 This is a flowchart of data acquisition and processing.

[0025] Figure 5 This is a block diagram illustrating the functions of the computational processing device in the second embodiment.

[0026] Figure 6 This is a flowchart of the data acquisition and processing in the second embodiment.

[0027] Figure 7 This is a schematic diagram of the data collection system according to the third embodiment.

[0028] Figure 8 This is a diagram showing the sensor information required for each implementation function of the third embodiment. Detailed Implementation

[0029] Hereinafter, embodiments of the vehicle control device and program of this disclosure will be described in detail with reference to the accompanying drawings. Furthermore, in principle, the same or equivalent parts in the drawings will be labeled with the same reference numerals, and their descriptions will not be repeated between embodiments and variations.

[0030] (First Implementation)

[0031] Figure 1 A data collection system 100 using the vehicle control device 10 in this embodiment is shown. The vehicle control device 10 is mounted on a vehicle 101 and performs vehicle 101 control and driving assistance.

[0032] like Figure 1 As shown, the data collection system 100 includes a server 102 and can communicate with one or more vehicles 101 via a communication network 103 such as the Internet. Figure 1 The image shows only one vehicle 101. Vehicle 101 includes a vehicle control device 10, sensors 20, and actuators 30, etc.

[0033] Sensor 20 includes various sensors used to determine the driving status of vehicle 101, such as vehicle speed sensor 21, acceleration sensor 22, and yaw rate sensor 23. The sensors used to determine the driving status of vehicle 101 may include other sensors, or may not include any one of vehicle speed sensor 21, acceleration sensor 22, or yaw rate sensor 23.

[0034] In addition, sensor 20 includes various sensors such as camera 24 and millimeter-wave radar 25 for detecting other vehicles and obstacles. The sensors used for detecting other vehicles and obstacles are not limited to millimeter-wave radar 25; they can also be LiDAR, ultrasonic sensors, etc., and multiple of these sensors can be combined. Furthermore, camera 24 can be either a monocular camera or a compound-eye camera. Furthermore, camera 24 can capture either still images or videos. Furthermore, the number, position, and type of camera 24 can be arbitrarily changed. In this embodiment, it includes a front camera 24a that captures images of the area in front of the vehicle, a right camera 24b that captures images of the right side of the vehicle, a left camera 24c that captures images of the left side of the vehicle, and a rear camera 24d that captures images of the area behind the vehicle.

[0035] In addition, sensor 20 includes various sensors for detecting various amounts of operation performed by the driver, such as acceleration sensor 26 for detecting the amount of acceleration operation performed by the driver, braking sensor 27 for detecting the amount of braking operation performed by the driver, and steering angle sensor 28 for detecting the amount of steering wheel (steering wheel) steering operation (steering angle) performed by the driver.

[0036] These sensors 20 are connected to the vehicle control unit 10 wirelessly or via wire, and the measurement results (or detection results) of these sensors 20 are input to the vehicle control unit 10, etc., as sensor information.

[0037] The actuator 30 may include, for example, an actuator for driving the vehicle 101, such as a motor that serves as the main unit of the vehicle 101. It may also include actuators for controlling the movement of the vehicle 101, such as actuators for operating the steering wheel, the brake pedal, and the accelerator pedal. Furthermore, the actuator 30 may include devices for activating equipment on the vehicle 101 such as displays, speakers, indicators, and headlights. The driving and movement of the vehicle 101 are controlled by this actuator 30.

[0038] The vehicle control unit 10 is primarily composed of a microcomputer equipped with a processing unit 10a such as a CPU and a storage unit 10b such as various memories. The functions provided by the microcomputer can be provided through software recorded in a physical storage device and a computer executing that software, software only, hardware only, or a combination thereof. For example, when the microcomputer is provided by electronic circuitry as hardware, it can be provided by digital circuitry or analog circuitry containing multiple logic circuits. For example, the processing unit 10a of the microcomputer executes a program stored in a non-transitory tangible storage medium, which is its own storage unit 10b. The program, for example, includes implementations... Figure 2 The program, as shown, performs the functions described above. By executing the program, the corresponding methods are executed. The storage unit 10b is, for example, non-volatile memory. Furthermore, the program stored in the storage unit 10b can be downloaded and updated via communication networks 103 such as OTA (Over-The-Air) or the Internet.

[0039] The vehicle control unit 10 has various functions (applications) for assisting in driving the vehicle 101, through which the actuator 30 is controlled based on sensor information input from the sensor 20.

[0040] Representative functions used to assist the driving of Vehicle 101 include, for example, Adaptive Cruise Control (ACC), Forward Collision Warning (FCW), Advanced Emergency Braking System (AEBS), Night Vision / Pedestrian Detection (NV / PD), Traffic Sign Recognition (TSR), Lane Departure Warning (LDW), Lane Keeping Assist System (LKAS), Rear Cross Traffic Alert (RCTA), Adaptive Front Lighting System (AFS), and Advanced Parking Assist (APA). Vehicles may possess all of these functions or only some of them. Additionally, they may include other driver assistance functions beyond these. Furthermore, they may also include functions for achieving autonomous driving of Vehicle 101. Furthermore, these functions are implemented by the arithmetic processing unit 10a executing the driving assistance control program stored in the storage unit 10b.

[0041] Furthermore, the vehicle control device 10, in order to verify the performance and safety of the driver assistance control program, has a function called "shadow mode" that activates the driver assistance control program (hereinafter referred to as the verification object program) and confirms the output value during the operation of the vehicle 101 after it has been sold—that is, in actual use cases. In shadow mode-based verification, for example, the verification object program is activated in the background, that is, in a manner independent of vehicle control, and data related to its output value is stored, and the server 102 collects the data related to the output value via the communication network 103. Moreover, shadow mode is a data collection mode used for collecting data.

[0042] The structure and function of the shadow mode according to this embodiment will be described below. The switching time to shadow mode can be arbitrary. For example, it can be switched by an operation performed by the driver, or it can be switched according to an instruction from the server 102. Alternatively, it can be switched to shadow mode simultaneously with the ignition switch being turned on.

[0043] Furthermore, to compare the actual driver assistance control program that enables vehicle 101 to operate with the verification program, the following is an installation procedure. Additionally, in Figure 1 The term "installation program" is also used. The vehicle control unit 10, considering the processing burden during the installation process, includes a verification processing unit 11 for implementing the verification target program. The verification processing unit 11 can have performance equivalent to or different from the processing unit 10a, provided it has the performance required to implement the verification target program. The verification target program is stored in the storage unit 10b. Alternatively, a dedicated storage unit for shadow mode may be provided.

[0044] By executing the verification object program, various functions for assisting the driving of vehicle 101 are implemented in the background. The functions implemented by executing the verification object program may be the same as the functions actually implemented by the arithmetic processing unit 10a, or some may be omitted. In addition, functions other than those actually implemented by the arithmetic processing unit 10a may also be implemented.

[0045] When the verification processing unit 11 operates the verification target program in shadow mode, it receives sensor information from various sensors 20, similar to the processing unit 10a. Furthermore, based on the input sensor information, the verification processing unit 11 performs various functions and outputs various control signals to activate each actuator 30. These control signals (output results) are not actually input to the actuators 30, but are stored in the storage unit 10b as data related to the output results. At this time, the verification processing unit 11 also establishes a correlation between the data and the sensor information as input values ​​and stores them in a linked manner.

[0046] In addition, the verification processing unit 11 also inputs a control signal that is input to the processing unit 10a as a sensor information input value, processed and output based on the installation program, and stores the control signal as data related to the output result in the storage unit 10b. That is, in order to compare and verify the output result in the installation program and the output result in the verification target program, the control signal processed and output based on the installation program is also stored.

[0047] Additionally, the verification processing unit 11 can also input control signals processed and output based on the installation procedure to the actuator 30. As a result, sensor information detected when the vehicle 101 moves (vehicle speed, yaw rate, acceleration, accelerator pedal operation, brake pedal operation, steering angle, etc.) is stored in the storage unit 10b. In other words, it can also store the sensor information involved in the actual movement of the vehicle 101 for each scenario, that is, the sensor information required to verify the correct movement.

[0048] The vehicle control unit 10 uploads the data stored in the storage unit 10b to the server 102 via the communication network 103 at a predetermined transmission time. The predetermined transmission time is arbitrary, but for example, it is the time when the server 102 issues an instruction to upload. Alternatively, it could be when the vehicle 101 is charging, when it is parked, or when the ignition switch is turned off, etc.

[0049] However, there is a high possibility that the storage capacity of the storage unit 10b may be insufficient to store all the data from the movement of vehicle 101. Furthermore, if all vehicles 101 upload all the stored data, there is a concern that this would place an excessive burden on the communication network 103. Moreover, if the server 102 were to parse all the uploaded data, there is also a problem of excessively increasing the processing load on the server 102. In addition, there are many wasteful aspects to the original method of acquiring and parsing all the data.

[0050] Therefore, it is preferable to configure the device to store only the amount of data required for verification, transmission, and parsing. Thus, the vehicle control device 10 of this embodiment deletes unnecessary data, reducing the amount of data stored and transmitted. This will be explained in detail below.

[0051] like Figure 2 As shown, the verification processing unit 11 of the vehicle control device 10 has the functions of an input unit 12, a processing unit 13, a data acquisition unit 14, an assignment unit 15, a compression unit 16, and a transmission unit 17. These functions are implemented by the verification processing unit 11 through the execution of the vehicle program stored in the storage unit 10b.

[0052] The input unit 12 inputs sensor information from the sensor 20. Furthermore, the input unit 12 inputs part or all of the input sensor information as input values ​​to the processing unit 13 for the verification target program. Additionally, the input unit 12 inputs part or all of the input sensor information to the data acquisition unit 14.

[0053] The processing unit 13 activates the verification target program and performs processing based on the input values ​​(sensor information) input from the input unit 12, performing various functions (application programs) implemented by the verification target program. Therefore, in this embodiment, the processing unit 13 is equivalent to a driver assistance implementation unit. Furthermore, the processing unit 13 inputs control signals (control signals of the actuator 30) as the result of this processing to the data acquisition unit 14. In addition, as described above, these control signals are not input to the actuator 30.

[0054] The data acquisition unit 14 determines whether the acquisition condition is met based on at least one of the input value (sensor information) input from the input unit 12 and the output result (control signal) input from the processing unit 13.

[0055] Acquisition conditions may include thresholds of one or more parameters as constituent elements. Parameters for acquisition conditions may include, for example, any one of the parameters contained in sensor information such as vehicle speed, yaw rate, acceleration operation amount, braking operation amount, and steering operation amount, or any parameter contained in output results such as requested torque and output indication signals sent to the display. Additionally, parameters calculated or inferred based on sensor information and output results may also be included. For example, the relative distance to an obstacle (vehicle ahead, pedestrian, etc.), the relative speed relative to the obstacle, and the remaining time-to-collision (TTC) calculated based on the recognition results of the camera image and the detection results of the millimeter-wave radar 25 may be set as parameters for acquisition conditions. Furthermore, the difference between the output results of the installation program and the output results of the verification object program may be set as parameters for acquisition conditions. For example, the difference between the braking operation amount output by processing based on the installation program and the braking operation amount output by processing based on the verification object program may be used as a parameter. The threshold may be either an upper limit or a lower limit of any of these parameters, or both.

[0056] Alternatively, whether the scenario in which vehicle 101 is driving (hereinafter referred to as "scenario") is a predetermined acquisition scenario can be used as a constituent element of the acquisition conditions. Scenarios include, for example, scenarios such as overtaking a vehicle in front, weaving between vehicles, following a vehicle in front, pedestrians crossing in front of vehicle 101 at night, vehicle 101 merging into its lane from the acceleration lane on a dedicated motor vehicle road, parking, waiting at a traffic light, and various other scenarios envisioned while vehicle 101 is driving.

[0057] These scenarios are inferred based on sensor information. For example, the vehicle control unit 10 can infer the scenario by recognizing camera images. More specifically, sensor information such as camera images can be input into a machine-learned inference model, such as a deep neural network, to infer the scenario. Furthermore, image recognition does not need to be performed by the vehicle control unit 10; it can be performed by an external device of the vehicle control unit 10, such as an image recognition device, and its results can be used as sensor information input.

[0058] Alternatively, the functions already implemented in various functions (application programs) of the verification object program (implemented functions) can also be used as components of the acquisition conditions. For example, the acquisition conditions can be met when the collision damage mitigation braking control function is implemented. Furthermore, which function is implemented can be determined based on the output result (control signal) input from the processing unit 13.

[0059] In this embodiment, the acquisition conditions are set by combining the acquisition scenario, the function to be implemented, and the threshold of the parameter (that is, by AND conditions). For example, in the scenario of following a vehicle in front, the acquisition condition is met when the vehicle speed (the parameter of the acquisition condition) is above the threshold (50km / h).

[0060] Furthermore, the combination of components of an acquisition condition can be an acquisition scenario and an implemented function, an acquisition scenario and a parameter threshold, or an implemented function and a parameter threshold. Additionally, an implementation function can be two or more components within an acquisition condition; in this case, the implementation function can be either an AND condition or an OR condition. For example, both the forward vehicle approach warning function and the collision damage mitigation braking control function can be implemented as conditions, or either one can be implemented as a condition. Similarly, the parameter threshold can be the threshold of two or more parameters.

[0061] Furthermore, the so-called fulfillment of the acquisition condition means that all the conditions constituting the acquisition condition are met. For example, if the acquisition scenario of the acquisition condition is "following the vehicle in front", the implementation function of the above acquisition condition is "following driving function", and the threshold of the parameter of the above acquisition condition is "above 50km / h", then the above acquisition condition is fulfilled when it is a scenario of following the vehicle in front, when the following driving function is implemented, and when the vehicle speed (parameter of the acquisition condition) is above the threshold (50km / h).

[0062] Furthermore, the acquisition condition is not limited to a single condition; multiple acquisition conditions can be set. For example, the first acquisition condition could be a scenario of following a vehicle in front, with the vehicle speed exceeding a first threshold. The second acquisition condition could be a scenario of a pedestrian crossing in front of vehicle 101, with the distance between the vehicle and the pedestrian below a second threshold and the braking operation amount exceeding a third threshold.

[0063] When any one of the acquisition conditions is met, the data acquisition unit 14 stores data related to the input value and the output result in the storage unit 10b. As described above, the input value includes, for example, sensor information input to the verification processing unit 11. Furthermore, the output result includes, for example, control signals processed and output based on the verification target program, and control signals processed and output based on the installation program. Additionally, the output result may also include sensor information related to the actual movements of the vehicle 101 in each scenario. Furthermore, the output result may also include the difference between the output result based on the installation program and the output result based on the verification target program.

[0064] Furthermore, the stored (acquired) data can be modified based on established acquisition conditions. For example, in a scenario where the vehicle is following another vehicle, if the acquisition condition is that the vehicle speed is above a first threshold, data related to the vehicle speed is acquired. On the other hand, in a scenario where a pedestrian is crossing in front of vehicle 101, if the acquisition condition is that the distance to the pedestrian is below a second threshold and the braking operation amount is above a third threshold, data related to the braking operation amount can also be acquired. Additionally, data related to sensor information acquired based on the establishment of acquisition conditions are stored in a way that treats each established acquisition condition as a whole. Specifically, data related to input values ​​and output results is assigned an ID and stored as a whole. Hereinafter, when referred to simply as data, this refers to that data.

[0065] Furthermore, the termination trigger for data acquisition is configured to occur upon the fulfillment of a termination condition. The termination condition is arbitrary, but for example, it may terminate when a predetermined time has elapsed since the acquisition condition was met. Alternatively, it may terminate upon the termination of an initiated driver assistance function. It may also terminate when the scenario at the start of acquisition changes to another scenario. Furthermore, it may terminate when any parameter included in the sensor information, such as vehicle speed, is compared to a threshold and the threshold is reached. It may also terminate when the acquisition condition is not met. Moreover, the termination condition may vary depending on the type of acquisition condition, the specific implementation function, or the specific scenario.

[0066] Next, the assignment unit 15 will be explained. The assignment unit 15 assigns tag information representing the implementation functions implemented by the processing unit 13 in various functions (applications) based on the verification object program to the data acquired (stored) by the data acquisition unit 14. As a result, the data can be classified based on the tag information.

[0067] For example, when the following driving function is implemented by the processing unit 13, label information indicating the implemented function is assigned. Furthermore, when the following driving function and the forward vehicle approach warning function are implemented by the processing unit 13, separate label information can be assigned. Additionally, when the opportunities (scenarios) for implementation are almost identical, such as the forward vehicle approach warning function and the collision damage mitigation braking control function, label information indicating that these functions are implemented collectively can be assigned. Furthermore, the image recognition function is implemented before the object recognition function and the traffic light recognition function. When multiple functions are implemented in a linked manner, label information indicating that these functions are implemented collectively can also be assigned. Which function is implemented can be determined based on the output result (control signal) input from the processing unit 13.

[0068] Furthermore, when the data acquisition unit 14 stores the data in the storage unit 10b, it also stores the data for which tag information has been assigned by the assignment unit 15. This allows for grouping data according to the tag information for each implementation function.

[0069] Next, the compression unit 16 will be described. The sensor information contained in each data stored in the storage unit 10b includes image data from cameras 24a to 24d (four image data in this embodiment). The compression unit 16 compresses the data volume by performing at least one of the following: reduction of some or all of the image data, or reduction of the constituent elements of the image data. At this time, the image data or the constituent elements of the image data that are reduced are determined according to the implementation function performed by the processing unit 13.

[0070] Specifically, the compression unit 16 determines the implementation function based on the tag information assigned to each data, and determines the compression method for the image data or components of the image data that are to be reduced based on the implementation function. This will be explained in detail below.

[0071] First, the timing of compression will be explained. In this embodiment, the compression unit 16 compresses data at a predetermined time. The predetermined time is arbitrary, but for example, it is when the remaining storage capacity of the data in the storage unit 10b is less than or equal to a predetermined value. Alternatively, it may be a time specified by the server 102, or a time when the processing burden in the vehicle control device 10 is reduced, such as when the vehicle 101 stops.

[0072] Next, the compression method will be explained. As mentioned above, the data includes input values ​​such as sensor information and output results such as control signals. Among these, image data contained in sensor information is usually particularly large in volume. Image data can also be still images, but it is usually video data, which has a large amount of data (information). In addition, when multiple cameras 24 (front camera 24a, right camera 24b, left camera 24c, and rear camera 24d) are set as in this embodiment, the amount of data related to the image data also increases proportionally. Furthermore, in the image recognition function, there are cases where exposure is changed during shooting, in which case the amount of data related to the image data also increases.

[0073] On the other hand, depending on the implementation function, among the various image data, there is also image data that is not used for testing, and that is almost useless for verification or has no value during verification and analysis. For example, in the case of a vehicle approach warning function that outputs an alarm as a vehicle approaches, image data other than the image data of the front camera 24a (such as the image data of the rear camera 24d) is not used. The same applies to follow-up functions that cause the vehicle to move by following the vehicle in front. On the other hand, in the case of a rear cross traffic warning, image data from the left and right side cameras 24b and 24c and the rear camera 24d are used.

[0074] Furthermore, when implementing the traffic sign recognition function based on image data for signal light recognition, it is sufficient to use only the upper image area (e.g., the area above the FOE (Focus of Expansion)) of the image data from the front camera 24a. On the other hand, when implementing the road surface recognition function based on image data, it is sufficient to use only the lower image area (e.g., the area below the FOE) of the image data from the front camera 24a.

[0075] Furthermore, in image recognition functions, since the exposure is changed for each object being recognized, only the image data corresponding to the exposure of the object being recognized is used. Additionally, there are functions that, such as machine learning planners, learn by mimicking a driver's actions and do not use image data to begin with.

[0076] Therefore, the compression unit 16 of this embodiment compresses the data volume by retaining the required image data and reducing other image data in the image data included in the data, or by retaining the required image data components and reducing other image data components according to the implementation function. Furthermore, the reduction of image data components refers to, for example, reducing the sampling rate, deleting color information to create a black and white image, reducing the image size, or cropping by deleting a portion of the image area.

[0077] Reference Figure 3 The image data (required image data) retained according to the implementation function will be explained. Here, since the compression unit 16 can determine the implementation function based on the tag information, the image data to be retained is essentially determined based on the tag information. At this time, when two or more implementation functions are associated with the tag information, and the image data required by each implementation function is different, the image data required in any one of the multiple implementation functions is retained, and image data or components that are not needed in any implementation function are cut off. Furthermore, when two or more implementation functions are associated with the tag information, in most cases, these implementation functions have the same implementation opportunities, and the required (utilized) image data is also the same.

[0078] In the case where the function is an image-based traffic light recognition function (in Figure 3 (Function No. 1) Retains image data of the traffic light being controlled and its surrounding area as the required image data. Typically, in the traffic light recognition function, to recognize the traffic light in front of vehicle 101, the image data of the upper area of ​​the front camera 24a is retained as the required image data. Furthermore, the image data of the lower area of ​​the front camera 24a, as well as the image data of the left and right side cameras 24b and 24c, and the rear camera 24d, are deleted.

[0079] Similarly, in the case of implementing a road surface recognition function based on images (in Figure 3 (Function No. 2) Retains image data related to the road surface and its surrounding area that is the object of control as the required image data. Typically, in the road surface recognition function, in order to recognize the road surface in front of vehicle 101, the image data of the lower area of ​​the front camera 24a is retained as the required image data. Moreover, the image data of the upper area of ​​the front camera 24a, the left and right side cameras 24b and 24c, and the rear camera 24d are deleted.

[0080] Additionally, in the case where the function implemented is image-based object recognition (in... Figure 3 (Function No. 3) retains the image data of the detected object and the area surrounding the object as the required image data. Furthermore, it deletes all other image data.

[0081] Additionally, in cases where the function implemented is image recognition (in Figure 3(Function No. 4) Retains image data that has been properly exposed for the purpose of properly detecting (identifying) the controlled object as the required image data. Furthermore, it deletes image data captured at any other exposure.

[0082] Furthermore, in this embodiment, when implementing the traffic light recognition function, an image recognition function is also implemented as a prerequisite. Therefore, when implementing the traffic light recognition function, data on appropriate exposure for detecting traffic lights that are the objects to be controlled is retained. Similarly, when implementing the road surface recognition function, an image recognition function is also implemented as a prerequisite. Therefore, when implementing the road surface recognition function, data on appropriate exposure for detecting the road surface (road) that is the object to be controlled is retained. Similarly, when implementing the object recognition function, an image recognition function is also implemented as a prerequisite. Therefore, when implementing the object recognition function, data on appropriate exposure for detecting objects that are the objects to be controlled is retained.

[0083] Furthermore, in cases where the function is to learn by mimicking the driver's driving operations (in Figure 3 The function described in section No. 5 makes little use of image data (there is no required image data). That is, in the case of a function that learns to mimic a driver's driving actions, the driver's inputs (acceleration inputs, braking inputs, etc.) are used as test or verification data. Therefore, in the case of implementing a function that learns to mimic a driver's driving actions, the image data is compressed by deleting color information, reducing the image size, and further lowering the sampling rate. Alternatively, all image data can be deleted.

[0084] Additionally, in cases where the function is collision damage mitigation braking control (in Figure 3 (Function No. 6) It retains image data from cameras 24a to 24d obtained from capturing images of approaching obstacles (control objects), and reduces image data from other cameras 24a to 24d. In other words, it makes the required image data and the reduced image data different depending on the position of the approaching obstacle (control object).

[0085] For example, when the collision damage mitigation braking control function is implemented, and in the case of a frontal collision where an obstacle approaches from the front of the vehicle, the image data of the front camera 24a is retained as the required image data, and other image data is cut off. Furthermore, when the collision damage mitigation braking control function is implemented, and in the case of a side collision where an obstacle approaches from the side of the vehicle, the image data of the side cameras 24b and 24c on the side of the approaching obstacle and the image data of the front camera 24a are retained as the required image data.

[0086] Furthermore, as a prerequisite for the collision damage mitigation braking control function, an object recognition function is implemented to detect and identify the controlled object (the approaching obstacle). In other words, the implementation of the collision damage mitigation braking control function also includes the implementation of the object recognition function (or the object recognition function is implemented as a prerequisite for the collision damage mitigation braking control function). Moreover, in the object recognition function, since the position (coordinates) of the controlled object in the image data is determined, it is also easy to determine which camera's image data from 24a to 24d should be retained.

[0087] Furthermore, if an object recognition function is implemented as a prerequisite for the collision damage mitigation braking control function, an image recognition function is further implemented as a prerequisite for that function. Therefore, appropriately exposed image data for properly detecting the controlled object of the collision damage mitigation braking control function is used as the required image data.

[0088] Additionally, when the function is implemented as a follow-driving function (in Figure 3 (Function No. 7) retains the image data of the camera 24 that is currently capturing the controlled object, and cuts off other data. In the case of the follow-up function, the vehicle ahead, which is the follower, is included in the controlled object. Therefore, when the function is implemented as the follow-up function, the image data of the front camera 24a is retained as the required image data.

[0089] However, in scenarios where other vehicles cut in front of vehicle 101 (this vehicle), the number of controlled objects increases to two. That is, the preceding vehicle and the other vehicles (cutting-in vehicles) become controlled objects. If the other vehicles are detected by side cameras 24b and 24c, in this case, the image data from the side cameras 24b and 24c that detected the other vehicle are also retained. All other image data is deleted. For example, if the other vehicle is on the right side of vehicle 101 (this vehicle), the image data from the right camera 24b is retained, and the image data from the left camera 24c is deleted.

[0090] Furthermore, as a prerequisite for the following motion function, an object recognition function is implemented to detect and identify obstacles (controlled objects). In other words, the implementation of the following motion function also includes the implementation of the object recognition function (or the object recognition function is implemented as a prerequisite for the following motion function). Moreover, in the object recognition function, since the position (coordinates) of the controlled object in the image data is determined, it is easy to determine which camera's image data 24a to 24d should be retained. Additionally, similarly to the above, when the object recognition function is implemented as a prerequisite for the following motion function, an image recognition function is further implemented as a prerequisite for that function. Therefore, image data with appropriate exposure for properly detecting the controlled object in the following motion function is used as the required image data. Furthermore, when there are two or more controlled objects, image data with appropriate exposure is retained for each.

[0091] Additionally, when the function is set to provide a rear cross traffic warning (in Figure 3 (Function No. 8) It retains the image data of the camera 24 that is currently capturing the controlled object and cuts off other data. In the case of rear cross traffic warning, it retains the image data of the left and right side cameras 24b and 24c and the rear camera 24d as the required image data. In addition, similar to other functions, it can also detect the position of other vehicles that are the controlled object, retain the image data of the side cameras 24b and 24c on the side where the controlled object is located, and delete the image data of the side cameras 24b and 24c on the opposite side.

[0092] Next, refer to Figure 4 The data acquisition process in this embodiment will be described. The data acquisition process shown below is the data acquisition processing process implemented by the verification processing unit 11. These processes are performed after the shadow mode is set. The timing of setting the shadow mode is arbitrary, for example, when the ignition switch is turned on.

[0093] First, the input unit 12 of the verification processing unit 11 receives sensor information from the sensor 20 (step S101). The processing unit 13 activates the verification target program and performs processing based on the input value (sensor information) received from the input unit 12, performing various functions implemented by the verification target program (step S102). This step S102 corresponds to the driving assistance implementation step. In step S102, the processing unit 13 inputs a control signal, which is the processing result, to the data acquisition unit 14.

[0094] The data acquisition unit 14 determines whether the acquisition condition is met based on at least one of the input value (sensor information) and the output result (control signal, etc.) (step S103).

[0095] If the determination result is positive (if the acquisition condition is met), the assignment unit 15 of the verification processing device 11 assigns tag information to the data according to the implementation function (step S104). Specifically, the assignment unit 15 determines the implementation function based on the data related to the output result input in step S102. Then, the assignment unit 15 assigns tag information corresponding to the implementation function to the data related to the input value and the output result.

[0096] Then, the data acquisition unit 14 stores (acquires) the data related to the input value and the output result to the storage unit 10b (step S105). When storing the data, the data acquisition unit 14 summarizes and stores the data related to the input value and the output result. At this time, the tag information assigned by the assignment unit 15 in step S104 is also stored in association with the data. Thus, the data can be categorized according to each implementation function. Furthermore, the data acquisition unit 14 also stores the data in correspondence with the acquisition conditions that are met. Then, the processing ends.

[0097] On the other hand, if the determination result in step S103 is negative, the verification processing unit 11 directly terminates the data acquisition process. The data acquisition process is executed at regular intervals after the shadow mode is set.

[0098] After the data acquisition process ends, the compression unit 16 compresses the data stored in the storage unit 10b at a predetermined time. At this time, as described above, the compression unit 16 performs compression by retaining the image data and its components required for the implemented functions as indicated by the tag information, and deleting unnecessary image data and its components. The processing performed by this compression unit 16 is equivalent to a compression step.

[0099] At a predetermined transmission time, the sending unit 17 transmits the data compressed by the compression unit 16 and stored in the storage unit 10b to the server 102 via the communication network 103. At this time, the sending unit 17 may also transmit information related to the established acquisition conditions that correspond to the data. Additionally, the tag information assigned by the assignment unit 15 may also be transmitted along with the data. The processing performed by the sending unit 17 is equivalent to the transmission step.

[0100] If server 102 receives data sent from the transmitting unit 17 of each vehicle 101, it stores the data in the storage device of server 102. When storing data, it can classify and store the data according to each implementation function based on the tag information received along with the data. Furthermore, information related to the acquisition conditions established when acquiring the data is also stored in a corresponding relationship. That is, after classifying by each implementation function, the data can be further classified and stored according to the acquisition conditions. Therefore, since the data is grouped by each implementation function, it is easy to parse the data.

[0101] According to the first embodiment described above, the following effects are achieved.

[0102] The compression unit 16 compresses the data volume by performing at least one of the following: reducing a portion or all of the image data contained in each data stored in the storage unit 10b, and reducing the components of the aforementioned image data. Therefore, it is possible to prevent storage capacity constraints and reduce the burden on communication load, processing load, analysis load, etc. Furthermore, the image data or its components to be reduced are determined according to the implementation function. In this case, by specifying image data and its components that are not used in parsing, verification, etc., as the image data and its components to be reduced according to the implementation function, it is possible to reduce unnecessary image data and its components while retaining the necessary image data and its components. Thus, even with data volume compression, the data can be appropriately used for verification, etc.

[0103] Furthermore, in cases where the implementation includes a function to detect a controlled object (or is implemented as a prerequisite), such as object recognition or image recognition, the compression unit 16 compresses the data volume by reducing image data captured by a camera other than the camera that captures the controlled object. Alternatively, the compression unit 16 compresses the data volume by reducing the constituent elements of the image data captured by a camera other than the camera that captures the controlled object. Since image data that does not capture the controlled object is generally assumed not to be used in analysis, verification, etc., reducing such image data allows for appropriate data compression. That is, even with data volume compression, the data can still be appropriately used in verification, etc.

[0104] Furthermore, in cases where the function includes detecting a controlled object (or is implemented as a prerequisite), such as object recognition or traffic light recognition, the compression unit 16 reduces the components of the image data by deleting areas outside the controlled object's periphery. Since areas outside the controlled object's periphery are generally assumed not to be used in analysis, verification, etc., reducing such image areas (components) allows for appropriate data compression. That is, even with data compression, the data can still be appropriately used for verification purposes such as object recognition.

[0105] In the image recognition function, camera 24 is configured to take multiple photos of the same object with varying exposures. Furthermore, when the implementation includes this image recognition function (or is implemented as a prerequisite), compression unit 16 compresses the data volume by reducing image data captured at exposures other than those predetermined based on the controlled object in the image recognition function, or by reducing the constituent elements of the image data. This allows for the retention of image data used for verification, etc., while also compressing the data.

[0106] Furthermore, when functions that do not utilize image data, such as those that mimic driver operation learning, are implemented, the compression unit 16 compresses the data volume by deleting all image data or reducing all components of the image data. This allows for efficient data compression.

[0107] Furthermore, the compression unit 16 reduces the constituent elements of the image data by decreasing its size, deleting color information contained in the image data, or, in the case of video data, reducing its sampling rate. This reduces the amount of data and preserves the information contained in the image data to a minimum level sufficient to understand the original image data.

[0108] The processing unit 13 is configured to be able to set a shadow mode that implements multiple driving assistance functions of the vehicle in a form independent of the actual control of the vehicle. Specifically, even if the processing unit 13 executes the verification object program, it does not output its output results to the actuator 30. Thus, in practical use cases, the verification object program can be executed and data related to its output results can be collected.

[0109] (A variation of the first embodiment)

[0110] • In the first embodiment described above, it is also possible not to acquire (store) the control signals (data related to the output results) processed and output by the installation program. On the other hand, it is also possible to acquire sensor information (vehicle speed, yaw rate, acceleration, accelerator pedal operation, brake pedal operation, steering angle, etc.) detected when the vehicle 101 is in motion after the acquisition conditions are met, until a predetermined time has elapsed. That is, these sensor information-related data can also be stored in the storage unit 10b.

[0111] • In the above embodiment, after the processing performed by the processing unit 13, it is determined whether the acquisition condition (trigger condition) is met. However, it is also possible to determine whether the acquisition condition (trigger condition) is met before the processing performed by the processing unit 13. Moreover, the processing unit 13 can only execute the verification object procedure and acquire data related to its result if the acquisition condition is met. As a result, the driving assistance function is not implemented unnecessarily, and the processing burden of the processing unit 13 can be reduced.

[0112] (Second Implementation)

[0113] The second embodiment, which modifies a portion of the vehicle control device 10 in the first embodiment, will be described.

[0114] In the vehicle control device 10 of the second embodiment, it is configured to acquire only data related to sensor information and data related to the output results based on the installation program without executing the verification target program. In other words, in the second embodiment, a data collection mode is set that does not execute the verification target program.

[0115] The structure and functions of the shadow mode in the second embodiment will be described below. In the second embodiment, the verification processing unit 11 is not provided, and the processing related to the data collection mode is performed by the processing unit 10a. Alternatively, similar to the first embodiment, the verification processing unit 11 may be provided and used instead of the processing unit 10a. In addition, the verification target program is not stored.

[0116] To explain in more detail, as follows: Figure 5 As shown, in the second embodiment, the arithmetic processing unit 10a of the vehicle control device 10 has the functions of an input unit 112, a processing unit 113, a data acquisition unit 114, an assignment unit 115, a compression unit 116, and a transmission unit 117. These functions are implemented by the arithmetic processing unit 10a executing the vehicle program stored in the storage unit 10b.

[0117] Input unit 112 inputs sensor information from sensor 20. Input unit 112 inputs part or all of the input sensor information as input values ​​for the installation program to processing unit 113. In addition, input unit 112 inputs part or all of the input sensor information to data acquisition unit 114.

[0118] The processing unit 113 activates the installation program and performs processing based on the input values ​​(sensor information) input from the input unit 112, performing various functions (application programs) implemented by the installation program. Then, the processing unit 113 inputs control signals (control signals for the actuator 30) as a result of this processing to the data acquisition unit 114. Additionally, these control signals are input to the actuator 30.

[0119] The data acquisition unit 114 determines whether the acquisition condition is met based on at least one of the input value (sensor information) input from the input unit 112 and the output result (control signal) input from the processing unit 13. Since the acquisition condition is the same as in the first embodiment, its description is omitted.

[0120] When the acquisition conditions are met, the data acquisition unit 114 stores data related to the input value and the output result in the storage unit 10b. As described above, the input value includes, for example, sensor information input to the processing unit 10a. Furthermore, the output result includes, for example, control signals processed and output by the installation program.

[0121] Furthermore, the data acquisition unit 114 of the processing unit 10a can also input control signals processed and output by the installation procedure to the actuator 30 after the acquisition conditions are met, input sensor information detected when the vehicle 101 moves (vehicle speed, yaw rate, acceleration, accelerator pedal operation, brake pedal operation, steering angle, etc.), and store it in the storage unit 10b until a predetermined period has elapsed. In other words, it can also store the sensor information involved in the actual movement of the vehicle 101 for each scenario, that is, the sensor information required to verify the correct movement.

[0122] The assignment unit 115 assigns tag information representing the implemented functions (applications) of various functions based on the installer to the data acquired (stored) by the data acquisition unit 114. The compression method, etc., is the same as in the first embodiment.

[0123] Since the compression unit 116 and the transmission unit 117 are the same as those in the first embodiment, their descriptions are omitted.

[0124] The following is for reference Figure 6 The data acquisition process in this embodiment will be described. The data acquisition process shown below is a data acquisition process implemented by the computing processing unit 10a. These processes are performed after a data collection mode is set.

[0125] First, the input unit 112 of the processing unit 10a inputs sensor information from the sensor 20 (step S201). The processing unit 113 of the processing unit 10a activates the installation program and performs processing based on the input value (sensor information) input from the input unit 112, performing various functions implemented by the installation program (step S202). In step S202, the processing unit 113 inputs a control signal, which is the result of its processing, to the actuator 30 and the data acquisition unit 114.

[0126] The data acquisition unit 114 determines whether the acquisition condition is met based on at least one of the input value (sensor information) and the output result (control signal, etc.) (step S203).

[0127] If the determination result is positive (if the acquisition condition is met), the assignment unit 115 of the arithmetic processing device 10a assigns tag information to the data according to the implementation function (step S204). Specifically, the assignment unit 115 determines the implementation function based on the data of the output result input in step S202. Then, the assignment unit 115 assigns tag information corresponding to the implementation function to the data related to the input value and the output result.

[0128] Then, the data acquisition unit 114 stores (acquires) the data related to the input value and the output result to the storage unit 10b (step S205). When storing data, the data acquisition unit 114 summarizes and stores the data related to the input value and the output result. At this time, the tag information assigned by the assignment unit 115 in step S204 is also stored in association with the data. Thus, the data can be classified according to each implementation function. In addition, the data acquisition unit 114 also stores the data in correspondence with the acquired conditions that have been met. Then, the processing ends. On the other hand, if the determination result in step S203 is negative, the arithmetic processing device 10a directly ends the processing.

[0129] The processing of the compression unit 116, the transmission unit 117, and the server 102 is the same as in the first embodiment, so it is omitted.

[0130] According to the second embodiment described above, the following effects are achieved.

[0131] Since no verification object procedure is implemented, the processing burden can be reduced. Consequently, the verification processing unit 11 can also be omitted.

[0132] (A variation of the second embodiment)

[0133] • The data acquisition unit 114 of the second embodiment described above acquires data related to the output result (control signal) of the operation based on the installation program, but it may not acquire data related to the output result of the operation based on the installation program. In this case, only sensor information as input values ​​may be acquired. Alternatively, the control signal processed and output by the installation program may be input to the actuator 30 to acquire sensor information (vehicle speed, yaw rate, acceleration, accelerator pedal operation amount, brake pedal operation amount, steering angle, etc.) detected when the vehicle 101 is in motion.

[0134] (Third Implementation)

[0135] A third embodiment, which modifies a portion of the vehicle control device 10 in the first embodiment, will be described. Furthermore, the same reference numerals are used for structures identical to those in the first embodiment, and descriptions are omitted.

[0136] like Figure 7 As shown, the sensor 20 of the data collection system 100 in the third embodiment includes a lidar 41 (LiDAR), an acoustic sensor 42, and an in-vehicle camera 43. The lidar 41 includes multiple lidars 41a to 41d with different detection areas. In this embodiment, it includes a first lidar 41a that detects the front of the vehicle, a second lidar 41b that detects the right side of the vehicle, a third lidar 41c that detects the left side of the vehicle, and a fourth lidar 41d that detects the rear of the vehicle. Furthermore, the number, position, type, and detection area of ​​the lidars 41 can be arbitrarily changed.

[0137] Furthermore, the acoustic sensor 42 includes multiple acoustic sensors 42a to 42c with different detection areas. In this embodiment, it includes a first acoustic sensor 42a that detects sound from the windshield (front glass, etc.), a second acoustic sensor 42b that detects sound from inside the wheel arch, and a third acoustic sensor 42c that detects sound from inside the vehicle interior. Moreover, the number, position, type, and detection area of ​​the acoustic sensors 42 can be arbitrarily changed.

[0138] Furthermore, the in-vehicle camera 43 includes multiple in-vehicle cameras 43a to 43c with different detection areas. In this embodiment, it includes a first in-vehicle camera 43a that uses the driver's seat as the detection area (shooting range), a second in-vehicle camera 43b that uses the passenger seat as the detection area (shooting range), and a third in-vehicle camera 43c that uses the rear seats as the detection area (shooting range). Moreover, the number, position, type, and detection area of ​​the in-vehicle cameras 43 can be arbitrarily changed.

[0139] Furthermore, the millimeter-wave radar 25 in the third embodiment includes multiple millimeter-wave radars 25a to 25d with different detection areas. In this embodiment, it includes a first millimeter-wave radar 25a that detects the front of the vehicle, a second millimeter-wave radar 25b that detects the right side of the vehicle, a third millimeter-wave radar 25c that detects the left side of the vehicle, and a fourth millimeter-wave radar 25d that detects the rear of the vehicle. Moreover, the number, position, type, and detection area of ​​the millimeter-wave radars 25 can be arbitrarily changed.

[0140] Next, the compression unit 16 in the third embodiment will be described. Similar to the first embodiment, the compression unit 16 in the third embodiment compresses the data volume by deleting a portion of the sensor information acquired from the sensor 20. In this case, the portion of sensor information to be cut is determined based on the implementation function performed by the processing unit 13. That is, the compression unit 16 determines the implementation function based on the tag information assigned to each piece of data, and determines the portion of sensor information to be deleted based on the implementation function. Furthermore, the timing of compression is the same as in the first embodiment. Hereinafter, refer to... Figure 8 The compression method in the third embodiment will be described in detail.

[0141] In cases where the function implemented is a follow-up driving function, a pedestrian detection function, or other functions that use millimeter-wave radar 25 to detect moving objects (vehicles, pedestrians, etc.) and treat these moving objects as control objects (in... Figure 8 (Function No. 11) The compression unit retains the sensor information (detection information of the millimeter-wave radar 25) related to the moving object being controlled as the required sensor information. On the other hand, the compression unit 16 deletes part or all of the sensor information (detection information of the millimeter-wave radar 25) related to the stationary object. Generally, when using the millimeter-wave radar 25, since the signal of the millimeter-wave radar 25 is analyzed and the speed of the detected object is detected, it is possible to easily determine whether the detected object is a moving object or a stationary object based on the speed.

[0142] On the other hand, in cases where the function is an advanced parking assistance function using millimeter-wave radar 25, which detects stationary objects (obstacles, etc.) using millimeter-wave radar 25 and uses those stationary objects as the control target (in... Figure 8 (Function No. 12) The sensor information (detection information of millimeter-wave radar 25) related to the stationary object that is the object of control is retained as the required sensor information. On the other hand, the compression unit 16 deletes part or all of the sensor information (detection information of millimeter-wave radar 25) related to the moving body.

[0143] In cases where the function is a follow-up driving function using millimeter-wave radar 25, the millimeter-wave radar 25 detects a moving object (vehicle, etc.) in front and uses that moving object as the control target (in... Figure 8(Function No. 13) The sensor information related to the moving object in front of the first millimeter-wave radar 25a, which is the object to be controlled, is retained as the required sensor information. On the other hand, the compression unit 16 deletes part or all of the sensor information related to objects to the side or rear detected by the second millimeter-wave radar 25b to the fourth millimeter-wave radar 25d. Furthermore, in this case, sensor information related to stationary objects in front detected by the first millimeter-wave radar 25a can also be deleted.

[0144] Furthermore, although not illustrated, in applications such as rear cross-traffic alerts using millimeter-wave radar 25, where objects are detected behind by millimeter-wave radar 25, sensor information of objects detected behind by the fourth millimeter-wave radar 25d can be retained. Conversely, some or all of the sensor information related to objects in front or to the side detected by the first to third millimeter-wave radars 25a to 25c can be deleted. In this case, sensor information related to moving objects behind detected by the fourth millimeter-wave radar 25d can also be deleted. In other words, sensor information related to the detected object corresponding to the implemented function can be retained, while other sensor information can be deleted.

[0145] In the case where the function implemented is a tracking driving function using LiDAR 41, a pedestrian detection function, etc., where LiDAR 41 detects moving objects (vehicles, pedestrians, etc.) and uses these moving objects as the control object (in... Figure 8 (Function No. 14) It retains the sensor information (detection information of LiDAR 41) of the moving object that is the controlled object as the required sensor information, and deletes part or all of the sensor information (detection information of LiDAR 41) related to the stationary object. Generally, when using LiDAR 41, since the signal of LiDAR 41 is analyzed and the speed of the detected object is detected, it is possible to easily determine whether the detected object is a moving object or a stationary object based on the speed.

[0146] On the other hand, in cases where the function implemented is an advanced parking assistance function using LiDAR 41, which detects stationary objects (obstacles, etc.) using LiDAR 41 and uses those stationary objects as the control target (in... Figure 8 (Function No. 15) It retains the sensor information (detection information of lidar 41) related to the stationary object that becomes the controlled object as the required sensor information, and on the other hand, deletes part or all of the sensor information (detection information of lidar 41) related to the moving object.

[0147] In cases where the function implemented is a following driving function using LiDAR 41, where LiDAR 41 detects a moving object (vehicle, etc.) in front and uses that moving object as the control object (in... Figure 8 (Function No. 16) The sensor information related to the moving object in front of the controlled object detected by the first lidar 41a is retained as the required sensor information. On the other hand, some or all of the sensor information related to objects to the side or rear detected by the second lidar 41b to the fourth lidar 41d is deleted. Furthermore, in this case, the sensor information related to stationary objects in front detected by the first lidar 41a can also be deleted.

[0148] Furthermore, although not illustrated, in applications such as rear cross-traffic alerts using lidar 41, where lidar 41 detects objects behind, sensor information related to objects detected by the fourth lidar 41d can be retained, while some or all of the sensor information related to objects detected in front or to the side by the first lidar 41a to the third lidar 41c can be deleted. In this case, sensor information related to moving objects behind detected by the fourth lidar 41d can also be deleted. In other words, sensor information related to the detected object corresponding to the implemented function can be retained, while other sensor information can be deleted.

[0149] In the case where the function implemented is a wind noise detection function that takes wind noise at the windshield as the control object (in...) Figure 8 (Function No. 17) The sensor information of the first acoustic sensor 42a is retained as the required sensor information, while some or all of the sensor information of the second acoustic sensors 42b to the third acoustic sensors 42c is deleted. The wind noise detection function is implemented, for example, when driving at a speed above a specified speed. Similarly, although not shown, when implementing an abnormal sound detection function that controls abnormal sounds within the wheel arches, the sensor information of the second acoustic sensor 42b can be retained, while some or all of the sensor information of the first acoustic sensors 42a and the third acoustic sensors 42c can be deleted. Similarly, when implementing an in-vehicle acoustic detection function that controls in-vehicle sounds, the sensor information of the third acoustic sensor 42c can be retained, while some or all of the sensor information of the first acoustic sensors 42a to the second acoustic sensors 42b can be deleted.

[0150] In the case where the function of driver monitoring is implemented, which controls the driver (in...) Figure 8(Function No. 18) It retains the camera image of the driver's seat taken by the first in-vehicle camera 43a as the required sensor information, while deleting part or all of the sensor information from the second in-vehicle cameras 42b to the third in-vehicle cameras 42c. The driver monitoring function is implemented, for example, while the vehicle is in motion, to monitor the driver's fatigue, drowsiness, etc.

[0151] The effects of the third embodiment will be explained.

[0152] The compression unit 16 compresses the amount of sensor information by reducing a portion of the sensor information, and the portion of sensor information to be reduced depends on the implementation function. For example, as described in the third embodiment, when the implementation function includes a function that detects a predetermined control object based on the sensor information, the compression unit 16 compresses the amount of sensor information by reducing a portion or all of the sensor information from sensors 20 other than the sensor 20 that detects the control object. Therefore, it is possible to prevent storage capacity constraints and reduce the burden on communication load, processing load, analysis load, etc. In addition, sensor information from the sensor 20 that detects the control object of the implementation function is retained, while sensor information from sensors 20 that do not detect the control object is deleted. Thus, even with data volume compression, the data can be appropriately used for verification, etc.

[0153] The millimeter-wave radar 25 and the lidar 41 are configured to detect the speed of an object. The compression unit 16 compresses the amount of sensor information by reducing part or all of the sensor information related to either the moving object or the stationary object. In the third embodiment, when the function includes detecting a stationary object as a control object based on sensor information, the compression unit 16 compresses the amount of sensor information by reducing part or all of the sensor information related to the moving object. Similarly, when the function includes detecting a moving object as a control object based on sensor information, the compression unit 16 compresses the amount of sensor information by reducing part or all of the sensor information related to the stationary object. Therefore, it is possible to prevent storage capacity constraints and reduce the burden on communication, processing, and analysis loads. Furthermore, when a moving object is detected, sensor information related to the moving object is retained, while sensor information related to the stationary object is deleted. Conversely, when a stationary object is detected, sensor information related to the stationary object is retained, while sensor information related to the moving object is deleted. Thus, even with data compression, the data can be appropriately used for verification, etc.

[0154] (A variation of the third embodiment)

[0155] • In the third embodiment described above, the detection range of the millimeter-wave radar 25 and the lidar 41 has a certain width. For example, in the case of the first millimeter-wave radar 25a installed at the front of the vehicle, detection can be performed within a 120-degree angle range centered on the front. Therefore, the compression unit 16 can also retain sensor information within a 30-degree angle range centered on the controlled object and delete sensor information within other angle ranges to compress the data volume. In other words, the compression unit 16 can also reduce the sensor information by deleting sensor information from the millimeter-wave radar 25 and the lidar 41 that pertains to areas outside the controlled object's periphery. This prevents storage capacity constraints and reduces the burden on communication load, processing load, analysis load, etc. Furthermore, even with data volume compression, the data can be appropriately used for verification, etc.

[0156] • It can also be implemented by combining the third embodiment and the second embodiment described above.

[0157] (Other variations)

[0158] • In the above embodiments, the system can also be configured to determine the required image data and its constituent elements based on the implemented function. In other words, it can be configured to determine the required image data and its constituent elements based on the implemented function, regardless of the position of the controlled object. This simplifies the process.

[0159] • In the above embodiment, a portion of the sensor information that is not needed in the data parsing can also be specified from the server 102 or the like via the communication network 103. Furthermore, the compression unit 16 can compress the amount of sensor information data by deleting the portion of sensor information that is not needed in the data parsing based on this specification.

[0160] For example, if you want to improve the recognition processing of moving objects, you can specify from server 102 that sensor information related to stationary objects is not needed in data parsing. In this case, compression unit 16 can also retain the sensor information related to moving objects and delete the sensor information related to stationary objects to compress the data volume. If you want to improve the recognition processing of stationary objects, you can perform the opposite operation.

[0161] • In the above embodiment, the assignment units 15 and 115 are assigned label information indicating the implemented function, but they may not be assigned label information. In this case, the compression units 16 and 116 determine the implemented function based on the data, and determine the image data to be compressed and its constituent elements.

[0162] • The processing units 13 and 113 of the computational processing apparatus 10a described above have the function of performing one or more signal processing operations on the signal information acquired from the millimeter-wave radar 25. For example, they perform signal processing such as converting analog signals detected by the millimeter-wave radar 25 into digital signals, analyzing the digital signals to extract peak signals, detecting point group signals from the peak signals, and determining the target signals that are the objects to be controlled from these signals. Moreover, the amount of data is reduced if signal processing is superimposed. Therefore, the amount of sensor information data can be compressed by specifying the desired level of signal processing and using the processing result as a substitute for sensor information. The signal processing level can be predetermined or specified from an external source (such as server 102).

[0163] The control unit and method described in this disclosure can be implemented by a dedicated computer, which is provided by comprising a processor programmed to perform one or more functions embodied in a computer program and a memory. Alternatively, the control unit and method described in this disclosure can also be implemented by a dedicated computer, which is provided by comprising a processor composed of one or more dedicated hardware logic circuits. Or, the control unit and method described in this disclosure can also be implemented by one or more dedicated computers, which are composed of a combination of a processor programmed to perform one or more functions, a memory, and a processor composed of one or more hardware logic circuits. Furthermore, the computer program can also be stored as instructions executable by a computer on a computer-readable non-transferable tangible recording medium.

[0164] The following describes the technical ideas that can be derived from the above embodiments and variations.

[0165] [Structure 1]

[0166] A vehicle control device, comprising a vehicle control device (10) mounted on a vehicle (101), including:

[0167] The driver assistance implementation unit (13, 113, 10a) is capable of implementing multiple functions related to driver assistance of the vehicle based on sensor information acquired by the sensors (20, 24, 24a to 24d) mounted on the vehicle.

[0168] The aforementioned sensor information includes one or more image data.

[0169] The compression unit (16, 116) compresses the data volume of the sensor information by performing at least one of the following: reducing a portion or all of the image data contained in one or more of the image data, and reducing the constituent elements of the image data; and

[0170] The transmitting unit (17, 117) transmits the sensor information compressed by the compression unit to the server (102) via the communication network (103).

[0171] The amount of the aforementioned image data or its constituent elements that are reduced is determined based on the implementation functions performed by the aforementioned driving assistance implementation unit.

[0172] [Structure 2]

[0173] According to the vehicle control device described in structure 1, wherein...

[0174] The aforementioned sensors include multiple cameras (24a, 24b, 24c, 24d) in different shooting areas.

[0175] In cases where the aforementioned implementation functions include the function of detecting a predetermined control object in the aforementioned image data, the compression unit compresses the amount of sensor information data by reducing the image data captured by a camera other than the camera capturing the control object, or by reducing the constituent elements of the image data captured by a camera other than the camera capturing the control object.

[0176] [Structure 3]

[0177] According to the vehicle control device described in structure 1 or 2, wherein...

[0178] In the case where the above-described implementation functions include the function of detecting a predetermined control object in the above-described image data, the compression unit reduces the constituent elements of the above-described image data by deleting areas other than the area surrounding the control object containing the control object.

[0179] [Structure 4]

[0180] The vehicle control device according to any one of structures 1 to 3, wherein...

[0181] The aforementioned sensors include cameras (24, 24a to 24d) that capture the aforementioned image data.

[0182] The aforementioned photographic equipment was used to take multiple shots of the same subject with varying exposures.

[0183] In cases where the aforementioned implementation includes the function of detecting a predetermined control object in the aforementioned image data, the compression unit compresses the amount of sensor information by reducing image data captured at an exposure other than the exposure predetermined for the control object, or by reducing the constituent elements of image data captured at an exposure other than the exposure predetermined for the control object.

[0184] [Structure 5]

[0185] The vehicle control device according to any one of structures 2 to 4, wherein...

[0186] The functions that include detecting the controlled object in the image data include either a warning function that outputs a warning as the controlled object approaches, a follow function that causes the vehicle to follow the controlled object, or a braking or avoidance function that causes the vehicle to brake or avoid the controlled object as it approaches. Alternatively, the function of detecting the controlled object in the image data may be performed as a prerequisite for these functions.

[0187] [Structure 6]

[0188] The vehicle control device according to any one of structures 1 to 5, wherein...

[0189] When the function described above is to not use the image data, the compression unit compresses the amount of sensor information by deleting all image data or reducing all components of the image data.

[0190] [Structure 7]

[0191] The vehicle control device according to any one of structures 1 to 6, wherein...

[0192] The compression unit reduces the constituent elements of the image data by reducing its size, deleting color information contained in the image data, or, in the case of video data, reducing its sampling rate.

[0193] [Structure 8]

[0194] The vehicle control device according to any one of structures 1 to 7, wherein...

[0195] The aforementioned vehicle control device is configured to be able to set a shadow mode, in which multiple functions related to the driving assistance of the vehicle are implemented in a manner unrelated to the actual control of the vehicle.

[0196] [Structure 9]

[0197] An in-vehicle program implemented by a vehicle control device (10) mounted on the vehicle (101),

[0198] The above procedure enables the vehicle control device to perform the following steps:

[0199] The driving assistance implementation steps are capable of implementing multiple functions related to the driving assistance of the aforementioned vehicle based on sensor information acquired by the sensors (20, 24, 24a to 24d) mounted on the aforementioned vehicle.

[0200] The aforementioned sensor information includes one or more image data.

[0201] The compression step compresses the data volume of the sensor information by performing at least one of the following: reducing a portion or all of the image data contained in one or more of the image data, and reducing the constituent elements of the image data; and

[0202] In the transmission step, the sensor information compressed by the above compression step is sent to the server (102) via the communication network (103).

[0203] The amount of the aforementioned image data or its constituent elements that are reduced is determined based on the implementation function performed in the aforementioned driving assistance implementation steps.

[0204] [Structure 11]

[0205] A vehicle control device, comprising a vehicle control device (10) mounted on a vehicle (101), including:

[0206] The driver assistance implementation unit (13, 113, 10a) is capable of implementing multiple functions related to driver assistance of the vehicle based on sensor information acquired by sensors (20, 24, 24a-24d, 25a-25d, 41a-41d, 42a-42c, 43a-43c) mounted on the vehicle.

[0207] The compression unit (16, 116) compresses the data volume of the sensor information by reducing a portion of the sensor information; and

[0208] The transmitting unit (17, 117) transmits the sensor information compressed by the compression unit to the server (102) via the communication network (103).

[0209] The amount of sensor information that is reduced is determined based on the implementation functions performed by the aforementioned driver assistance implementation unit.

[0210] [Structure 12]

[0211] According to the vehicle control device described in structure 11, wherein...

[0212] The aforementioned sensors include multiple sensors with different detection areas (24a, 24b, 24c, 24d, 25a-25d, 41a-41d, 42a-42c, 43a-43c).

[0213] In cases where the aforementioned implementation includes the function of detecting a predetermined controlled object based on the aforementioned sensor information, the compression unit compresses the amount of sensor information by reducing part or all of the sensor information from sensors other than the sensor that detected the controlled object.

[0214] [Structure 13]

[0215] According to the vehicle control device described in structure 11 or 12, wherein,

[0216] In cases where the aforementioned implementation includes a function to detect a predetermined controlled object based on the aforementioned sensor information, the compression unit reduces a portion of the aforementioned sensor information by deleting sensor information relating to the area outside the area surrounding the controlled object, which contains the aforementioned controlled object.

[0217] [Structure 14]

[0218] The vehicle control device according to any one of structures 11 to 13, wherein...

[0219] The aforementioned sensors include cameras (24, 24a to 24d) that capture the aforementioned image data.

[0220] The aforementioned photographic equipment was used to take multiple shots of the same subject with varying exposures.

[0221] In cases where the aforementioned implementation includes the function of detecting a predetermined control object in the aforementioned image data, the compression unit compresses the amount of sensor information by reducing image data captured at an exposure other than the exposure predetermined for the control object, or by reducing the constituent elements of image data captured at an exposure other than the exposure predetermined for the control object.

[0222] [Structure 15]

[0223] The vehicle control device according to any one of structures 11 to 14, wherein...

[0224] The aforementioned sensors include sensors (25a-25d, 41a-41d) capable of measuring the speed of the object being detected.

[0225] The compression unit compresses the amount of sensor information by reducing the sensor information of either the moving object or the stationary object.

[0226] [Structure 16]

[0227] The vehicle control device according to any one of structures 11 to 14, wherein...

[0228] The aforementioned sensors include sensors (25a-25d, 41a-41d) capable of measuring the speed of the object being detected.

[0229] When the above-described implementation function includes the function of detecting a stationary object as a control object based on the above-described sensor information, the compression unit compresses the data amount of the sensor information by reducing part or all of the sensor information related to the moving object.

[0230] [Structure 17]

[0231] The vehicle control device according to any one of structures 11 to 16, wherein,

[0232] The device includes a signal processing unit that performs one or more signal processing operations on the signal information contained in the sensor information.

[0233] The compression unit compresses the data volume of the sensor information by replacing the signal information contained in the sensor information with the processing result of one or more signal processing operations performed by the signal processing unit.

[0234] [Structure 18]

[0235] The vehicle control device according to any one of structures 11 to 17, wherein...

[0236] The compression unit compresses the amount of sensor information by deleting a portion of the sensor information that is not needed during data parsing.

[0237] [Structure 19]

[0238] The vehicle control device according to any one of structures 11 to 18, wherein...

[0239] The warning function that outputs a warning as the controlled object approaches, the following function that causes the vehicle to follow the controlled object, or the braking or avoidance function that causes the vehicle to brake or avoid the controlled object as it approaches includes the function of detecting the controlled object based on the sensor information, or the function of detecting the controlled object based on the sensor information is executed as a prerequisite for these functions.

[0240] [Structure 20]

[0241] The vehicle control device according to any one of structures 11 to 19, wherein...

[0242] The aforementioned sensor information includes image data.

[0243] When the function described above is to not use the image data, the compression unit compresses the amount of sensor information by deleting all image data or reducing all components of the image data.

[0244] [Structure 21]

[0245] The vehicle control device according to any one of structures 11 to 20, wherein,

[0246] The aforementioned sensor information includes image data.

[0247] The compression unit reduces the constituent elements of the image data by reducing its size, deleting color information contained in the image data, or, in the case of video data, reducing its sampling rate.

[0248] [Structure 22]

[0249] According to any one of structures 11 to 21, the vehicle control device is configured to be able to set a shadow mode, in which multiple functions related to driving assistance of the vehicle are implemented in a form independent of the actual control of the vehicle.

[0250] [Structure 23]

[0251] A procedure implemented by a vehicle control device (10) mounted on a vehicle (101),

[0252] The above procedure enables the vehicle control device to perform the following steps:

[0253] The driving assistance implementation steps are capable of implementing multiple functions related to the driving assistance of the aforementioned vehicle based on sensor information acquired by sensors (20, 24, 24a-24d, 25a-25d, 41a-41d, 42a-42c, 43a-43c) mounted on the aforementioned vehicle.

[0254] The compression step compresses the amount of sensor information by reducing a portion of it; and

[0255] In the transmission step, the sensor information compressed by the above compression step is sent to the server (102) via the communication network (103).

[0256] The amount of sensor information that is reduced is determined based on the implementation functions performed in the aforementioned driving assistance implementation steps.

[0257] This disclosure has been described based on embodiments, but it should be understood that this disclosure is not limited to those embodiments or structures. This disclosure also includes various modifications and equivalent variations. Furthermore, various combinations and forms, and even other combinations and forms that include only one, more than, or less than one of these elements, are also included in the scope and spirit of this disclosure.

Claims

1. A vehicle control device, comprising, in a vehicle control device (10) mounted on a vehicle (101): The driver assistance implementation unit (13, 113, 10a) is capable of implementing multiple functions related to driver assistance of the vehicle based on sensor information acquired by sensors (20, 24, 24a-24d, 25a-25d, 41a-41d, 42a-42c, 43a-43c) mounted on the vehicle. The compression unit (16, 116) compresses the data volume of the sensor information by reducing a portion of the sensor information; and The transmitting unit (17, 117) transmits the sensor information compressed by the compression unit to the server (102) via the communication network (103). The amount of sensor information that is reduced is determined based on the implementation functions performed by the aforementioned driver assistance implementation unit.

2. The vehicle control device according to claim 1, wherein, The aforementioned sensors include multiple sensors with different detection areas (24a, 24b, 24c, 24d, 25a-25d, 41a-41d, 42a-42c, 43a-43c). In cases where the aforementioned implementation includes the function of detecting a predetermined controlled object based on the aforementioned sensor information, the compression unit compresses the amount of sensor information by reducing part or all of the sensor information from sensors other than the sensor that detected the controlled object.

3. The vehicle control device according to claim 1, wherein, In cases where the aforementioned implementation includes a function to detect a predetermined controlled object based on the aforementioned sensor information, the compression unit reduces a portion of the aforementioned sensor information by deleting sensor information relating to the area outside the area surrounding the controlled object, which contains the aforementioned controlled object.

4. The vehicle control device according to claim 1, wherein, The aforementioned sensors include cameras (24, 24a to 24d) that capture the aforementioned image data. The aforementioned photographic equipment was used to take multiple shots of the same subject with varying exposures. In cases where the aforementioned implementation includes the function of detecting a predetermined control object in the aforementioned image data, the compression unit compresses the amount of sensor information by reducing the image data captured at an exposure other than the exposure predetermined for the control object, or by reducing the constituent elements of the image data captured at an exposure other than the exposure predetermined for the control object.

5. The vehicle control device according to claim 1, wherein, The aforementioned sensors include sensors (25a-25d, 41a-41d) capable of measuring the speed of the object being detected. The compression unit compresses the amount of sensor information by reducing the sensor information of either the moving object or the stationary object.

6. The vehicle control device according to claim 1, wherein, The aforementioned sensors include sensors (25a-25d, 41a-41d) capable of measuring the speed of the object being detected. When the above-described implementation function includes the function of detecting a stationary object as a control object based on the above-described sensor information, the compression unit compresses the data amount of the sensor information by reducing part or all of the sensor information related to the moving object.

7. The vehicle control device according to claim 1, wherein, The device includes a signal processing unit that performs one or more signal processing operations on the signal information contained in the sensor information. The compression unit compresses the data volume of the sensor information by replacing the signal information contained in the sensor information with the processing result of one or more signal processing operations performed by the signal processing unit.

8. The vehicle control device according to claim 1, wherein, The compression unit compresses the amount of sensor information by deleting a portion of the sensor information that is not needed during data parsing.

9. The vehicle control device according to any one of claims 1 to 8, wherein, The warning function that outputs a warning as the controlled object approaches, the following function that causes the vehicle to follow the controlled object, or the braking or avoidance function that causes the vehicle to brake or avoid the controlled object as it approaches includes the function of detecting the controlled object based on the sensor information, or the function of detecting the controlled object based on the sensor information is executed as a prerequisite for these functions.

10. The vehicle control device according to any one of claims 1 to 8, wherein, The aforementioned sensor information includes image data. When the function described above is to not use the image data, the compression unit compresses the amount of sensor information by deleting all image data or reducing all components of the image data.

11. The vehicle control device according to any one of claims 1 to 8, wherein, The aforementioned sensor information includes image data. The compression unit reduces the constituent elements of the image data by reducing its size, deleting color information contained in the image data, or, in the case of video data, reducing its sampling rate.

12. The vehicle control device according to any one of claims 1 to 8, wherein, The aforementioned vehicle control device is configured to be able to set a shadow mode, in which multiple functions related to the driving assistance of the vehicle are implemented in a manner unrelated to the actual control of the vehicle.

13. A procedure implemented by a vehicle control device (10) mounted on a vehicle (101), The above procedure enables the vehicle control device to perform the following steps: The driving assistance implementation steps are capable of implementing multiple functions related to the driving assistance of the aforementioned vehicle based on sensor information acquired by sensors (20, 24, 24a-24d, 25a-25d, 41a-41d, 42a-42c, 43a-43c) mounted on the aforementioned vehicle. The compression step compresses the amount of sensor information by reducing a portion of it; and In the transmission step, the sensor information compressed by the above compression step is sent to the server (102) via the communication network (103). The amount of sensor information that is reduced is determined based on the implementation functions performed in the aforementioned driving assistance implementation steps.

Citation Information

Patent Citations

  • Systems and methods for obtaining training data

    JP2023053031A

  • Hot water supply system

    JP2024069046A