DATA PROCESSING APPARATUS, DATA PROCESSING METHOD, AND COMPUTER PROGRAM FOR DATA PROCESSING

Abstract

To provide a data processing device that can obtain least-deteriorated data from a sensor and use the data in processing.SOLUTION: A data processing device repeatedly executes a series of processing including a first processing for obtaining data from a sensor every cycle time in a predetermined order; estimates a deterioration removal time range showing a range of time during which a cause of deterioration of the obtained data in the first processing, by operating an instrument arranged near the senor; and when an execution scheduled time for the first processing included in the series of processing that is to be started at a predetermined time is not included in the deterioration removal time range, sets, before executing the first processing, a delay time during which any processing included in the series of processing is not executed, where a length of a period of time, during which a subsequent series of processing following the series of processing that are to be started at a predetermined time after a finishing scheduled time of the series of processing, set as an upper limit.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to a data processing device, a data processing method, and a computer program for data processing that processes data acquired from a sensor. [Background technology]

[0002] When the driving of a vehicle is controlled based on data acquired by a sensor, in order to appropriately control the driving of the vehicle, it is preferable to perform processing related to the control of the driving of the vehicle on data generated by the sensor that is as free of noise and omissions as possible.

[0003] Patent document 1 describes a wiper control device for a vehicle that uses a camera to capture images of objects outside the vehicle through the windshield, from which raindrops are wiped away by the operation of the wipers, and that changes the wiping operation speed of the wipers when the predicted position of the wipers at the time of capturing the image overlaps with the imaging field of view. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 8-58529 Summary of the Invention [Problem to be solved by the invention]

[0005] When the windshield wipers are operating to wipe raindrops off the windshield, the sensors located on the inside of the windshield may output degraded data about the conditions outside the vehicle, depending on the timing. It is desirable to prevent such degraded data from being used in processing related to the control of vehicle driving.

[0006] An object of the present disclosure is to provide a data processing device that can acquire data from a sensor with as little degradation as possible and use the data for processing. [Means for solving the problem]

[0007] The data processing device according to the present disclosure comprises a cycle execution unit that repeatedly executes a series of processes including a first process for acquiring data from a sensor in a predetermined order at cycle times; an estimation unit that estimates a degradation elimination time range indicating a time range in which the cause of degradation of data acquired in the first process is eliminated by the operation of an equipment arranged near the sensor; and a delay unit that, if the scheduled execution time of a first process included in the series of processes starting at a predetermined time is not included in the degradation elimination time range, sets a delay time before execution of the first process during which none of the processes included in the series of processes are executed, with the delay time being up to the length of the period from the scheduled end time of the series of processes to the start time of the next series of processes in the series of processes starting at the predetermined time.

[0008] It is preferable that the data processing device according to the present disclosure further includes a prohibition unit that prohibits the execution of a process that uses data obtained by the delayed first process when the scheduled execution time of the first process delayed by setting a delay time is before the degradation removal time range.

[0009] In the data processing device according to the present disclosure, it is preferable that the delay section sets a delay time with a lower limit equal to the length of the period from the scheduled execution time of the first process to the start time of the degradation removal time range.

[0010] The data processing method according to the present disclosure includes repeatedly executing a series of processes including a first process for acquiring data from a sensor in a predetermined order at cycle times, estimating a degradation elimination time range indicating the range of time in which the cause of degradation of the data acquired in the first process is eliminated by the operation of an equipment located near the sensor, and if the scheduled execution time of a first process included in the series of processes starting at a predetermined time is not included in the degradation elimination time range, setting a delay time before execution of the first process during which none of the processes included in the series of processes are executed, with the delay time being up to the length of the period from the scheduled end time of the series of processes to the start time of the next series of processes in the series of processes starting at the predetermined time.

[0011] The computer program for data processing according to the present disclosure causes a computer to repeatedly execute a series of processes including a first process for acquiring data from a sensor in a predetermined order at cycle times, estimate a degradation elimination time range indicating the range of time in which the cause of degradation of the data acquired in the first process is eliminated by the operation of an equipment located near the sensor, and, if the scheduled execution time of a first process included in the series of processes starting at a predetermined time is not included in the degradation elimination time range, set a delay time before execution of the first process during which none of the processes included in the series of processes are executed, with the delay time being up to the length of the period from the scheduled end time of the series of processes to the start time of the next series of processes in the series of processes starting at the predetermined time.

[0012] According to the data processing device of the present disclosure, data with as little degradation as possible can be acquired from the sensor and used for processing. [Brief description of the drawings]

[0013] [Figure 1] 1 is a schematic configuration diagram of a vehicle in which a data processing device is implemented; [Diagram 2] FIG. 2 is a hardware configuration diagram of the data processing device. [Diagram 3] FIG. 11 is a diagram showing an example of a schedule for cyclic execution of a plurality of processes. [Figure 4] FIG. 11 is a diagram illustrating an example of a timing chart of a wiper operation. [Diagram 5] FIG. 2 is a functional block diagram of a processor included in the data processing device. [Figure 6] FIG. 11 is a diagram illustrating an example of cycle processing. [Figure 7] 1 is a flowchart of data processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, with reference to the drawings, a data processing device capable of acquiring data from a sensor without being affected by the operation of a device arranged near the sensor will be described in detail. The data processing device repeatedly executes a series of processes at cycle times for executing the series of processes in a predetermined order. The series of processes includes a first process for acquiring data from the sensor. The data processing device estimates a degradation elimination time range indicating a time range in which a cause of degradation of data acquired in the first process is eliminated by the operation of a device arranged near the sensor. If the scheduled execution time of the first process included in the series of processes started at a predetermined time is not included in the degradation elimination time range, the data processing device provides a delay time during which none of the processes included in the series of processes is executed before execution of the first process. The delay time is set with an upper limit of the length of the period from the scheduled end time of the series of processes to the start time of the next series of processes of the series of processes started at the predetermined time.

[0015] FIG. 1 is a schematic diagram of a vehicle 1 in which a data processing device is implemented.

[0016] The vehicle 1 has a surrounding camera 2, a wiper device 3, and a data processing device 4. The surrounding camera 2, the wiper device 3, and the data processing device 4 are communicatively connected via an in-vehicle network that complies with a standard such as a controller area network.

[0017] The surrounding camera 2 is an example of a sensor for generating data representing the surrounding conditions of the vehicle 1. The surrounding camera 2 has a two-dimensional detector configured with an array of photoelectric conversion elements sensitive to visible light, such as a CCD or C-MOS, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. The surrounding camera 2 is disposed, for example, at the front upper part of the vehicle interior facing forward, photographs the surrounding conditions of the vehicle 1 through the windshield at a predetermined photographing period (for example, 1 / 30 second to 1 / 10 second), and outputs data representing the surrounding conditions of the vehicle 1.

[0018] The wiper device 3 is an example of a device arranged near a sensor. The peripheral camera 2 and the wiper device 3 are both arranged near the windshield. That is, the wiper device 3 is arranged near the peripheral camera 2.

[0019] The wiper device 3 includes a wiper 31 and a wiper controller 32. The wiper 31 includes a rod-shaped soft member such as rubber, and wipes away dirt from the outer surface of the windshield by operating the soft member at a predetermined cycle so as to abut against the outer surface of the windshield, using a predetermined position near the periphery of the windshield as a fulcrum. The wiper controller 32 controls the operation of the wiper 31 so that the wiper 31 operates more frequently as the amount of rainfall detected by a raindrop sensor (not shown) that detects the amount of rainfall around the vehicle 1 increases. The wiper controller 32 may control the operation of the wiper 31 in response to a driver's operation received by a wiper switch (not shown).

[0020] The operation of the wiper device 3 wipes away dirt on the outer surface of the windshield through which the light rays pass when the peripheral camera 2 captures the surrounding conditions of the vehicle 1, thereby eliminating the cause of degradation of the data output by the peripheral camera 2.

[0021] The data processing device 4 is an ECU (Electronic Control Unit) equipped with a communication interface, a memory, and a processor. The data processing device 4 repeatedly executes a series of processes in a predetermined order at cycle times, such as processing data acquired from the peripheral camera 2 at the time when dirt on the windshield is wiped off by the wiper device 3.

[0022] 2 is a schematic hardware diagram of the data processing device 4. The data processing device 4 includes a communication interface 41, a memory 42, and a processor 43.

[0023] The communication interface 41 is an example of a communication unit, and includes a communication interface circuit for connecting the data processing device 4 to an in-vehicle network. The communication interface 41 supplies the received data to the processor 43. The communication interface 41 also outputs the data supplied from the processor 43 to the outside.

[0024] The memory 42 is an example of a storage unit, and includes a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 42 stores various data used in processing by the processor 43, such as a schedule that associates the expected required time for each of a series of processes with the order of cycle execution, a cycle time that indicates the interval when a series of processes are repeatedly executed, and a timing chart that indicates the operation timing for each operation mode of the wiper device 3. The memory 42 also stores various application programs, such as a data processing program that executes data processing.

[0025] 3 is a diagram showing an example of a schedule for cyclic execution of a series of processes. The memory 42 stores a schedule 421 that associates, for each of the series of processes, an estimated required time with the order of the cyclic execution.

[0026] The schedule 421 includes, as a series of processes, a process A, a first process, and a process B. For example, the process A is a process for communicating with an external device via a data communication module (not shown), the first process is a process for acquiring data from the peripheral camera 2, and the process B is a process for converting the data acquired in the first process. In this specification, the series of processes that are cyclically executed according to the schedule 421 is also collectively referred to as cyclic processes.

[0027] Process A is associated with a sequence number of "1" and an expected duration of "30 ms." Process A is therefore started at the beginning of a cycle time of a given length (e.g., 150 ms), i.e., 0 ms after the start. Process A is also expected to end 30 ms after the start of process A, which is 30 ms after the start of the cycle time.

[0028] The first process is associated with a sequence number of "2" and an expected time required of "50 ms." Thus, the first process starts in the cycle time at the time when the process with sequence number "1" (process A) ends, i.e., 30 ms after the start of the cycle time. The first process is also expected to end 50 ms after the start of the first process, or 80 ms after the start of the cycle time.

[0029] Process B is associated with a sequence number of "3" and an expected time required of "20 ms." Therefore, process B starts at the time when the process with sequence number "2" (first process) ends in the cycle time, i.e., 80 ms after the start of the cycle time. Process B is also expected to end 20 ms after the start of process B, 100 ms after the start of the cycle time.

[0030] 4 is a diagram showing an example of a timing chart of the wiper operation. The memory 42 stores a timing chart 422 showing the operation timing of the wiper device 3 for each operation mode.

[0031] The timing chart 422 includes, as operation modes of the wiper device 3, a "high" mode with a relatively high wiping frequency and a "low" mode with a relatively low wiping frequency. The "high" mode is associated with "0 ms - 400 ms" as the time range of the forward operation (the time range from the storage position to the end of the movement of the wiper 31), and "400 ms - 800 ms" as the time range of the return operation (the time range from the end of the movement to the return of the wiper 31 to the storage position). In addition, "800 ms - 900 ms" is associated with the time range of the pause operation (the time range from the return of the wiper 31 from the end of the movement to the storage position to the start of the movement to the end of the movement). The time range of each operation is expressed by determining the start time and the end time using the time from the start of the operation in the corresponding mode. After the series of operations from the forward movement to the pause is completed, the next series of operations is performed with the end time of the pause (900 ms after the start of the operation) as the new start time, and the operations are continued in the same manner.

[0032] The "Low" mode is associated with a forward time range of "0ms - 600ms", a return time range of "600ms - 1200ms", and a pause time range of "1200ms - 1400ms".

[0033] In the timing chart 422, the operation timing for each operation mode may be represented using the duration of each operation (forward operation, return operation, and pause operation) of the wiper 31. That is, for example, the operation timing for the "high" mode may be represented as "the length of the forward operation is 400 ms, the length of the return operation is 400 ms, and the length of the pause operation is 100 ms."

[0034] In the timing chart 422, the operation mode is further associated with a degradation removal time range. The degradation removal time range is appropriately set to a time range in which dirt on the outer surface of the windshield is wiped away by the operation of the wiper device 3 and the cause of degradation of the data output by the peripheral camera 2 is removed. In the timing chart 422, the degradation removal time range is set so as to correspond to a time range from when the wiper 31, returning from the moving end to the storage position, passes the position of the peripheral camera 2 until a predetermined time has elapsed. The degradation removal time range is expressed by determining a start time and an end time using the time from the start of operation in the mode. The degradation removal time range may be expressed by determining a start time using a period from the start of operation in the mode and determining a length between the start time and the end time. For example, the degradation removal time range of the "high" mode may be expressed as "the start time is 500 ms after the start of operation and the length is 300 ms."

[0035] The wiper controller 32 starts controlling the operation of the wiper 31 in a predetermined operation mode according to the timing chart 422, with a predetermined time as the operation start time, in response to the amount of rainfall detected by the raindrop sensor or the operation of the wiper switch. The data processing device 4 can obtain the operation mode and operation start time of the wiper 31 from the wiper controller 32, and estimate the operation state of the wiper 31 by referring to the timing chart 422.

[0036] FIG. 5 is a functional block diagram of the processor 43 included in the data processing device 4. As shown in FIG.

[0037] The processor 43 of the data processing device 4 has, as functional blocks, a cycle processing unit 431, an estimation unit 432, a delay unit 433, and an inhibition unit 434. Each of these units in the processor 43 is a functional module implemented by a computer program stored in the memory 42 and executed on the processor 43. A computer program for realizing the functions of each unit in the processor 43 may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. Alternatively, each of these units in the processor 43 may be implemented in the data processing device 4 as an independent integrated circuit, a microprocessor, or firmware.

[0038] The cycle processing unit 431 repeatedly executes a series of processes including a first process of acquiring data from the peripheral camera 2 at cycle times (e.g., 150 ms) in an order defined in a schedule 421 stored in the memory 42. The cycle processing unit 431 may be realized as a function of a real-time OS that operates on the processor 43. In addition to the first process, the series of processes may include sending a control signal to a driving mechanism (not shown) of the vehicle 1, sending display data to a meter display (not shown), communication with external devices via a data communication module (not shown), format conversion of acquired data, and the like.

[0039] FIG. 6 is a diagram showing an example of cycle processing. iIn the cycle process starting from , it is scheduled to execute process A first, and then execute the first process after the end of process A.

[0040] The estimation unit 432 estimates a degradation removal time range indicating a time range in which the cause of degradation of the data acquired from the surrounding camera 2 is removed by the operation of the wiper device 3.

[0041] The estimation unit 432 acquires the operation mode and operation start time of the wiper 31 from the wiper controller 32 via the in-vehicle network. The estimation unit 432 calculates the elapsed time from the operation start time to a predetermined time, and can estimate the operation state of the wiper 31 at that time by referring to the timing chart 422 stored in the memory 42.

[0042] The estimation unit 432 estimates the operation state of the wiper 31 at each time included in the time range shown in FIG. 6 and estimates the operation state of the wiper 31 at each time included in the time range shown in FIG. cs From time t ce (for example, time t i 70 ms after time t i It is estimated that the time is 190 ms after the

[0043] The estimation unit 432 may change the deterioration removal time range from the range stored in the memory 42 according to the amount of rainfall detected by the raindrop sensor. For example, if the amount of rainfall detected is greater than the standard amount of rainfall in the operation mode applied to the operation of the wiper 31, it is considered that the time until raindrops adhere to the windshield again after the dirt in the vicinity of the peripheral camera 2 is wiped away by the return operation of the wiper 31 will be shorter than expected. In this case, the estimation unit 432 may change the end time of the deterioration removal time range to an earlier timing. Conversely, if the amount of rainfall detected is less than the standard amount of rainfall in the operation mode applied to the operation of the wiper 31, the estimation unit 432 may change the end time of the deterioration removal time range to a later timing.

[0044] When the scheduled execution time of a first process included in a cycle process that starts at a specified time is not included in the degradation removal time range, a delay unit 433 provides a delay time before the execution of the first process during which none of the processes included in the cycle process are executed.

[0045] In the example of Figure 6, time t i The first process in the cycle process is scheduled to be executed at time t ip1 (Time t i The delay unit 433 delays the scheduled execution time of the first process until time t ip1 6, the delay unit 433 sets the delayed scheduled execution time of the first process to a time t cs (Time t i The timing is set to 70 ms after the

[0046] The delay unit 433 determines the scheduled end time of the cycle process (time t i 100 ms after t i The time t at which the next cycle process of the cycle process that starts at i+1 (Time t i A delay time is provided with an upper limit of the length of the period (50 ms) from the start of the first process to the start of the next cycle process (150 ms after the start of the first process). By providing a delay time in this manner, the data processing device 4 can start the next cycle process without being affected by the execution of the delayed first process.

[0047] The delay unit 433 delays the first process at a scheduled execution time t ip1 (Time t i 30 ms after t cs (Time t i A delay time may be set with a lower limit of the length of the period from the time when the first process is scheduled to be executed until 70 ms after the time when the first process is scheduled to be executed (40 ms). By setting a delay time in this manner, the data processing device 4 can obtain data with as little degradation as possible and use it for processing, so that the scheduled execution time of the first process is included in the degradation removal time range.

[0048] When the scheduled execution time of the first process delayed by providing the delay time is before the degradation removal time range, the prohibition unit 434 prohibits the execution of a process that uses data acquired by the delayed first process.

[0049] In the example of Fig. 6, process B corresponds to a process that uses data acquired by the first process. In the example of Fig. 6, the scheduled execution time of the delayed first process is included in the degradation elimination time range, so prohibition unit 434 does not prohibit the execution of process B. However, if the scheduled execution time of the delayed first process is not included in the degradation elimination time range, prohibition unit 434 prohibits the execution of process B.

[0050] Fig. 7 is a flow chart of data processing. The data processing device 4 repeatedly executes the process shown in Fig. 7 at cycle time intervals while cycle processing is requested.

[0051] First, the estimation unit 432 of the processor 43 of the data processing device 4 estimates a degradation removal time indicating a time range in which the cause of degradation of data acquired from the peripheral camera 2 is removed by the operation of the wiper device 3 (step S1).

[0052] Next, the delay unit 433 of the processor 43 determines whether or not the start time of a first process included in a series of processes that starts at a predetermined time is included in the degradation removal time range (step S2). The first process is a process of acquiring data from the peripheral camera 2.

[0053] If it is determined that the start time of the first process is not included in the degradation removal time range (step S2: N), the delay unit 433 sets a delay time during which none of the processes included in the series of processes is executed before executing the first process (step S3), and proceeds to step S4 described below.

[0054] If it is determined that the start time of the first process is included in the degradation elimination time range (step S2: Y), the cycle processing unit 431 executes a series of processes in a predetermined order (step S4) and ends the data processing for that cycle time.

[0055] As described above, the data processing device 4 operates as described above, and thereby can obtain data from the sensors with as little degradation as possible and use the data for processing.

[0056] According to a modified example, the vehicle 1 may have a LiDAR (Light Detection and Ranging) sensor as a sensor for generating data representing the situation around the vehicle 1. The LIDAR sensor generates a distance image in which each pixel has a value corresponding to the distance to an object represented by that pixel, based on the situation around the vehicle 1. In addition, the window on which the sensor is attached is not limited to the front window, and the sensor may be attached to a rear window or a side window.

[0057] The vehicle 1 may have, as a device arranged near the sensor, a window heater that removes fogging from the window by heating a heating element attached to the window of the vehicle 1. The estimation unit 432 estimates, as the degradation elimination time range, a time range in which the fogging on the window that causes data degradation is removed by the operation of the window heater. The estimation unit 432 may estimate the degradation elimination time range such that the greater the difference in air temperature between the interior and exterior of the vehicle 1, the longer the time interval from the generation of heat from the heating element to the start time of the degradation elimination time range.

[0058] The vehicle 1 may have, as a device arranged near the sensor, a window washer that removes dirt on the window by spraying cleaning liquid on the window of the vehicle 1. The estimation unit 432 estimates, as the degradation removal time range, a time range in which the dirt on the window that causes data degradation is removed by the operation of the window washer.

[0059] It should be understood that those skilled in the art can make various changes, substitutions, and alterations thereto without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0060] 1 vehicle 2 Peripheral Camera 3. Wiper device 4 Data Processing Device 431 Cycle Processing Unit 432 Estimation Department 433 Delay Division 434 Prohibited part

Claims

1. a cycle execution unit that repeatedly executes a series of processes including a first process of acquiring data from a sensor in a predetermined order at cycle times; an estimation unit that estimates a degradation removal time range indicating a time range in which a cause of degradation of the data acquired in the first process is removed by an operation of a device disposed near the sensor; a delay unit which, when a scheduled execution time of the first process included in the series of processes started at a predetermined time is not included in the degradation removal time range, provides a delay time before the execution of the first process during which none of the processes included in the series of processes is executed, the delay time being up to a period from a scheduled end time of the series of processes to a start time of the series of processes next to the series of processes started at the predetermined time; A data processing device comprising:

2. 2. The data processing device according to claim 1, further comprising a prohibition unit that prohibits execution of a process that uses data acquired by the delayed first process when a scheduled execution time of the first process delayed by setting the delay time is earlier than the degradation removal time range.

3. The data processing device according to claim 1 , wherein the delay section sets the delay time with a lower limit being a length of a period from a scheduled execution time of the first process to a start time of the degradation removal time range.

4. Repeating a series of processes including a first process of acquiring data from a sensor in a predetermined order at cycle times; estimating a degradation removal time range indicating a time range in which a cause of degradation of the data acquired in the first process is removed by an operation of a device disposed near the sensor; when a scheduled execution time of the first process included in the series of processes started at a predetermined time is not included in the degradation removal time range, a delay time during which none of the processes included in the series of processes is executed is provided before the execution of the first process, the delay time being up to a length of a period from a scheduled end time of the series of processes to a start time of the series of processes next to the series of processes started at the predetermined time; (c) Processing the data.

5. Repeating a series of processes including a first process of acquiring data from a sensor in a predetermined order at cycle times; estimating a degradation removal time range indicating a time range in which a cause of degradation of the data acquired in the first process is removed by an operation of a device disposed near the sensor; when a scheduled execution time of the first process included in the series of processes started at a predetermined time is not included in the degradation removal time range, a delay time during which none of the processes included in the series of processes is executed is provided before the execution of the first process, the delay time being up to a length of a period from a scheduled end time of the series of processes to a start time of the series of processes next to the series of processes started at the predetermined time; A computer program for processing data that causes a computer to carry out the following:

Images