Information processing device, vehicle and program

Abstract

To improve the accuracy of an inference operation that is performed periodically.SOLUTION: An information processing device includes a processing unit that executes an inference operation in each execution period. The inference operation is performed by inputting input data including time-series data into a neural network. An interval between acquisition times of constituent data of the time-series data input in one inference operation is longer than an execution period.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing device, a vehicle, and a program. [Background technology]

[0002] In autonomous driving, a technology is known that uses a model generated by machine learning to perform behavior planning. Patent Document 1 describes that in autonomous driving control processing, a curvature is calculated using multiple time series data of estimated coordinate values ​​calculated from the past to the present. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-127098 A Summary of the Invention [Problem to be solved by the invention]

[0004] When an inference operation using time series data as input is executed periodically, if the interval between the acquisition times of the time series data is equal to the execution period of the inference operation, the accuracy of the inference operation may be reduced. One aspect of the present invention aims to provide a technique for improving the accuracy of the inference operation executed periodically. [Means for solving the problem]

[0005] In view of the above problem, there is provided an information processing device comprising a processing means for executing an inference operation for each execution period, the inference operation being executed by inputting input data including time series data into a neural network, and the interval between the acquisition times of the constituent data of the time series data input in one inference operation is longer than the execution period. Effect of the Invention

[0006] By this means, the accuracy of the inference operations performed periodically is improved. [Brief description of the drawings]

[0007] [Figure 1] 1 is a block diagram illustrating an example of the configuration of a vehicle according to a first embodiment. [Diagram 2] FIG. 4 is a flowchart illustrating an example of the operation of the ECU according to the first embodiment. [Diagram 3] FIG. 2 is a schematic diagram illustrating an example of a neural network according to the first embodiment. [Figure 4] FIG. 3 is a schematic diagram illustrating an example of input data according to the first embodiment. [Diagram 5] FIG. 1 is a block diagram illustrating an example of the arrangement of an information processing apparatus according to a first embodiment. [Figure 6] FIG. 4 is a flow diagram illustrating an example of the operation of the information processing device according to the first embodiment. [Figure 7] FIG. 3 is a schematic diagram illustrating an example of learning data according to the first embodiment. [Figure 8] FIG. 11 is a schematic diagram illustrating an example of a neural network according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims, and not all combinations of features described in the embodiments are essential to the invention. Two or more features among the multiple features described in the embodiments may be arbitrarily combined. In addition, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0009] First Embodiment Fig. 1 is a block diagram of a vehicle 1 according to a first embodiment of the present invention. In Fig. 1, the vehicle 1 is shown generally in a plan view and a side view. As an example, the vehicle 1 is a sedan-type four-wheeled passenger car. The vehicle 1 may be such a four-wheeled vehicle, or may be a two-wheeled vehicle or another type of vehicle.

[0010] The vehicle 1 includes a vehicle control device 2 (hereinafter, simply referred to as the control device 2) that controls the vehicle 1. The control device 2 includes a plurality of ECUs 20 to 29 that are communicatively connected by an in-vehicle network. Each ECU includes a processor such as a CPU, a memory such as a semiconductor memory, an interface with an external device, and the like. Therefore, each ECU may be referred to as an information processing device. The memory stores a program executed by the processor and data used by the processor for processing, and the like. Each ECU may include a plurality of processors, memories, interfaces, and the like. For example, the ECU 20 includes a processor 20a and a memory 20b. The processor 20a executes an instruction included in a program stored in the memory 20b, whereby the processing by the ECU 20 is performed. Alternatively, the ECU 20 may include a dedicated integrated circuit such as an ASIC for performing the processing by the ECU 20. The same applies to the other ECUs.

[0011] The following describes the functions and the like handled by each of the ECUs 20 to 29. Note that the number of ECUs and the functions handled by them can be designed appropriately, and they can be divided more finely or integrated than in this embodiment.

[0012] The ECU 20 executes control related to the automatic driving of the vehicle 1. In the automatic driving, at least one of the steering and the acceleration / deceleration of the vehicle 1 is automatically controlled. The automatic driving by the ECU 20 may include automatic driving that does not require driving operation by the driver (which may also be called automatic driving) and automatic driving that assists the driving operation by the driver (which may also be called driving assistance).

[0013] The ECU 21 controls the electric power steering device 3. The electric power steering device 3 includes a mechanism for steering the front wheels in response to a driver's driving operation (steering operation) on a steering wheel 31. The electric power steering device 3 also includes a motor that generates a driving force for assisting the steering operation and automatically steering the front wheels, a sensor that detects the steering angle, and the like. When the driving state of the vehicle 1 is autonomous driving, the ECU 21 automatically controls the electric power steering device 3 in response to an instruction from the ECU 20, and controls the traveling direction of the vehicle 1.

[0014] ECUs 22 and 23 control detection units 41-43 that detect the surrounding conditions of the vehicle and process information on the detection results. Detection unit 41 is a camera that captures images ahead of vehicle 1 (hereinafter, may be referred to as camera 41), and in this embodiment, is attached to the inside of the passenger compartment of the windshield at the front of the roof of vehicle 1. By analyzing images captured by camera 41, it is possible to extract the contours of targets and lane markings (white lines, etc.) on the road.

[0015] The detection unit 42 is a LIDAR (Light Detection and Ranging) (hereinafter, may be referred to as LIDAR 42) and detects targets around the vehicle 1 and measures the distance to the targets. In the present embodiment, five LIDARs 42 are provided, one at each corner of the front of the vehicle 1, one at the rear center, and one on each side of the rear. The detection unit 43 is a millimeter wave radar (hereinafter, may be referred to as RADAR 43) and detects targets around the vehicle 1 and measures the distance to the targets. In the present embodiment, five RADARs 43 are provided, one at the front center of the vehicle 1, one at each front corner, and one at each rear corner.

[0016] The ECU 22 controls one camera 41 and each lidar 42 and processes information on the detection results. The ECU 23 controls the other camera 41 and each radar 43 and processes information on the detection results. By providing two sets of devices that detect the surrounding conditions of the vehicle, the reliability of the detection results can be improved, and by providing different types of detection units such as a camera, lidar, and radar, the surrounding environment of the vehicle can be analyzed from multiple perspectives.

[0017] The ECU 24 controls the gyro sensor 5, the GPS sensor 24b, and the communication device 24c, and processes information on the detection results or communication results. The gyro sensor 5 detects the rotational motion of the vehicle 1. The path of the vehicle 1 can be determined based on the detection results of the gyro sensor 5, the wheel speed, etc. The GPS sensor 24b detects the current position of the vehicle 1. The communication device 24c wirelessly communicates with a server that provides map information and traffic information to obtain this information. The ECU 24 can access a database 24a of map information constructed in a memory, and the ECU 24 performs route searches from the current location to a destination, etc. The ECU 24, the map database 24a, and the GPS sensor 24b constitute a so-called navigation device.

[0018] The ECU 25 includes a communication device 25a for vehicle-to-vehicle communication. The communication device 25a performs wireless communication with other surrounding vehicles and exchanges information between the vehicles.

[0019] The ECU 26 controls the power plant 6. The power plant 6 is a mechanism that outputs a driving force to rotate the drive wheels of the vehicle 1, and includes, for example, an engine and a transmission. The ECU 26 controls the output of the engine in response to a driving operation (accelerator operation or acceleration operation) of the driver detected by an operation detection sensor 7a provided on the accelerator pedal 7A, for example, and switches the gear stage of the transmission based on information such as the vehicle speed detected by a vehicle speed sensor 7c. When the driving state of the vehicle 1 is autonomous, the ECU 26 automatically controls the power plant 6 in response to an instruction from the ECU 20, and controls the acceleration and deceleration of the vehicle 1.

[0020] The ECU 27 controls lighting devices (headlights, taillights, etc.) including turn signals 8 (blinkers). In the example of Fig. 1, the turn signals 8 are provided at the front, door mirrors, and rear of the vehicle 1.

[0021] The ECU 28 controls the input / output device 9. The input / output device 9 outputs information to the driver and accepts information input from the driver. The audio output device 91 notifies the driver of information by audio. The display device 92 notifies the driver of information by displaying an image. The display device 92 is arranged on the surface of the driver's seat, for example, and constitutes an instrument panel or the like. Note that, although audio and display are exemplified here, information may be notified by vibration or light. Information may be notified by a combination of audio, display, vibration, or light. Furthermore, the combination or the notification mode may be changed depending on the level of the information to be notified (for example, urgency). The input device 93 is a group of switches arranged in a position operable by the driver and used to give instructions to the vehicle 1, but may also include an audio input device.

[0022] The ECU 29 controls the brake device 10 and the parking brake (not shown). The brake device 10 is, for example, a disk brake device provided on each wheel of the vehicle 1, and applies resistance to the rotation of the wheel to decelerate or stop the vehicle 1. The ECU 29 controls the operation of the brake device 10 in response to the driver's driving operation (brake operation) detected by, for example, an operation detection sensor 7b provided on the brake pedal 7B. When the driving state of the vehicle 1 is automatic driving, the ECU 29 automatically controls the brake device 10 in response to an instruction from the ECU 20, and controls the deceleration and stop of the vehicle 1. The brake device 10 and the parking brake can also be operated to maintain the stopped state of the vehicle 1. In addition, when the transmission of the power plant 6 is equipped with a parking lock mechanism, this can also be operated to maintain the stopped state of the vehicle 1.

[0023] An example of the operation by the ECU 20 for performing automatic driving control will be described with reference to FIG. 2. Each step of this operation may be executed by the processor 20a executing an instruction included in a program stored in the memory 20b, for example. Alternatively, at least some steps of the operation in FIG. 2 may be executed by a dedicated integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The operation in FIG. 2 is started when the vehicle 1 starts automatic driving control. In FIG. 2, the description of operations that are not used in the operation of this embodiment is omitted. Existing operations may be performed for the operations whose description is omitted.

[0024] In step S201, the ECU 20 acquires external world data related to the outside of the vehicle 1 and stores the data in a storage unit (for example, the memory 20b). The external world data includes data related to traffic participants (vehicles, pedestrians, bicycles, etc.) and static objects (buildings, roads, signs, etc.) around the vehicle 1. The external world data is acquired, for example, using the above-mentioned detection units 41 to 43. For example, the external world data may be an image of the surroundings of the vehicle 1 captured by the camera 41. As will be described later, the external world data is acquired repeatedly. Therefore, the external world data is stored in the memory 20b as time-series data. That is, each piece of external world data becomes constituent data of the time-series data. When storing external world data acquired at a new time, the ECU 20 may delete past external world data that is no longer necessary from the storage unit.

[0025] In step S202, the ECU 20 acquires input data to be used in the inference operation in step 203. The input data may include, for example, vehicle data at the time of execution of step 202 and time-series data of external environment data. The vehicle data may include, for example, the speed and acceleration of the vehicle. The type of external environment data to be acquired will be described later.

[0026] In step S203, the ECU 20 predicts external data at a future time point using the input data acquired in step S202. Specifically, the ECU 20 executes an inference operation by inputting the input data to a trained neural network. The inference result (output result) of this inference operation becomes the prediction result of the external data.

[0027] A specific example of a neural network used in the inference operation of step S203 will be described with reference to FIG. 3. The neural network 300 has an input layer 301, an intermediate layer 302, and an output layer 303. In the example of FIG. 3, an example in which the number of layers of the intermediate layer 302 is three is shown, but the number of layers of the intermediate layer 302 is not limited to this. The neural network 300 may be a so-called deep neural network in which the number of layers of the intermediate layer 302 is large (for example, five or more). The neural network 300 is trained before the start of the operation of FIG. 2, and is stored in a storage unit (for example, memory 20b) of the vehicle 1. The training method will be described later. Vehicle data and time-series external data are input to the neural network 300.

[0028] The input layer 301 may have an individual node for each pixel value of the external world data. Alternatively, the input layer 301 may perform a convolution process on the external world data. That is, the neural network 300 may be a convolutional neural network. Furthermore, the neural network 300 may be a three-dimensional convolutional neural network. In the input layer 301 of the three-dimensional convolutional neural network, multiple pieces of external world data acquired at different times are convoluted.

[0029] In step S204, the ECU 20 controls the traveling of the vehicle 1 based on the inference result of the inference operation (specifically, the predicted external environment data). This step may be performed using existing technology, so a detailed description will be omitted. For example, the ECU 20 generates an action plan for the vehicle 1 based on the predicted external environment data, and controls each actuator of the vehicle 1 according to this action plan.

[0030] In step S205, the ECU 20 determines whether to end the automatic driving control. If the ECU 20 determines that the automatic driving control is to be ended ("YES" in step S205), the ECU 20 ends the process, and otherwise ("NO" in step S205), the ECU 20 returns the process to step S201. For example, the automatic driving control may be determined to be ended according to an instruction from the driver, or may be determined to be ended in response to the driver taking over the driving.

[0031] As described above, ECU 20 repeatedly executes the operations of steps S201 to S204. In this embodiment, ECU 20 executes steps S201 to S204 periodically, that is, so that the execution interval of the operations of the same step is constant. This execution interval is hereinafter referred to as an operation cycle, and its length is represented as P. That is, for each operation cycle P, ECU 20 acquires external data in step S201 and executes an inference operation in step S203.

[0032] The input data used in the inference operation will be specifically described with reference to Fig. 4. In Fig. 4, the circles (white and black circles) shown next to "inference operation" indicate the time when the inference operation was performed, and the black circles indicate the inference operation performed at the time of interest. The circles (white and black circles) shown next to "vehicle data" indicate the time when the vehicle data was acquired, and the black circles indicate the vehicle data used in the inference operation performed at the time of interest. The circles (white and black circles) shown next to "external world data" indicate the time when the external world data was acquired, and the black circles indicate the external world data used in the inference operation performed at the time of interest.

[0033] The upper diagram of FIG. 4 explains the inference operation executed at time t13. The ECU 20 executes the inference operation at each of times t1 to t12 until time t13. Therefore, each time interval (for example, t2-t1) between times t1 to t12 is equal to the above-mentioned operation cycle P. At time t13, the ECU 20 executes an inference operation for predicting the external data at time t14 by inputting the vehicle data acquired at time t13 and the external data acquired at times t1, t5, t9, and t13 to the neural network 300. In the inference operation at time t13, the external data acquired at times other than t1, t5, t9, and t13 among the time series data of the external data is not used. Thus, in this embodiment, the interval between the acquisition times of the external data of the time series data input in one inference operation is longer than the execution cycle P. Hereinafter, the interval between the acquisition times of the external data of the time series data input in one inference operation is represented as an acquisition interval I. In the example of FIG. 4, the acquisition interval I is four times the execution period P.

[0034] The technical effect of making the acquisition interval I longer than the execution cycle P in this way will be described. The execution cycle P of the inference operation is preferably short so as to be able to quickly respond to changes in the outside world. Since the execution cycle P cannot be shorter than the time required for the inference operation, for example, in a certain environment, the execution cycle P is about 50 to 150 ms. In the following description, it is assumed that the execution cycle P is 100 ms. It is assumed that a pedestrian is walking around the vehicle 1 at a speed of 4 km per hour. In this case, the pedestrian moves about 0.1 m in 100 ms. Therefore, if the acquisition interval I is made equal to the execution cycle P, the change between the environmental data at adjacent times will be buried in the observation noise, and the accuracy of the inference operation will decrease. On the other hand, in this embodiment, by making the acquisition interval I longer than the execution cycle P, the change between the environmental data at adjacent times can be made larger, thereby improving the accuracy of the inference operation.

[0035] In the example of FIG. 4, the acquisition interval I is four times the execution period P. Alternatively, the acquisition interval I may be another integer multiple of the execution period P that is two or more. By setting this ratio as an integer multiple, no adjustment of the period is required between the acquisition of external data and the execution of the inference operation, making calculations easier. The multiple is appropriately determined depending on the relationship between the execution period P and the input data. Furthermore, the acquisition interval I does not have to be an integer multiple of the execution period P. For example, when the acquisition of external data and the execution of the inference operation are executed asynchronously, the acquisition interval I may not be an integer multiple of the execution period P.

[0036] The lower diagram of Fig. 4 explains the inference operation executed at time t14. At time t14, the ECU 20 executes an inference operation for predicting external data at time t15 by inputting the vehicle data acquired at time t14 and the external data acquired at times t2, t6, t10, and t14 to the neural network 300. In the inference operation at time t14, the external data acquired at times other than t2, t6, t10, and t14 among the time-series data of the external data is not used. Thus, the acquisition interval I is longer than the execution period P for the inference operation executed at time t14 as well.

[0037] The external world data used in the inference operation at time 14 includes external world data acquired at new time t14, compared with the external world data used in the inference operation at time 13. In this way, the ECU 20 updates the time series data of the external world data included in the input data every time the inference operation is executed so as to include the external world data acquired at the new time. This enables prediction based on the most recent situation of the outside world. Furthermore, in this update, the ECU 20 updates the time series data of the external world data included in the input data so that each external world data is advanced by the execution period P. Specifically, the acquisition times t2, t6, t10, and t14 of the external world data used in the inference operation at time 14 are advanced by the execution period P from the acquisition times t1, t5, t9, and t13 of the external world data used in the inference operation at time 13. This makes it possible to suppress variation in the accuracy of inference for each execution of the inference operation.

[0038] A configuration example of an information processing device 500 for training the neural network 300 will be described with reference to the block diagram of Fig. 5. The information processing device 500 is realized by an information processing device such as a personal computer or a workstation. The information processing device 500 may be realized by a single device, or may be realized by a plurality of devices connected to each other via a network.

[0039] The information processing device 500 has the components shown in Fig. 5. The processor 501 controls the overall operation of the information processing device 500. The processor 501 is realized, for example, by a CPU (Central Processing Unit) or a combination of a CPU and a GPU (Graphics Processing Unit). The memory 502 stores programs and temporary data used in the operation of the information processing device 500. The memory 502 is realized, for example, by a ROM (Read Only Memory) or a RAM (Random Access Memory).

[0040] The input device 503 is used by the user of the information processing device 500 to input to the information processing device 500, and is realized by, for example, a mouse or a keyboard. The output device 504 is used by the user of the information processing device 500 to check the output from the information processing device 500, and is realized by, for example, an output device such as a display or an audio device such as a speaker. The communication device 505 provides a function for the information processing device 500 to communicate with other devices, and is realized by, for example, a network card. The communication with other devices may be wired communication or wireless communication. The storage device 506 is used to store data used in the processing of the information processing device 500, and is realized by, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0041] An example of an operation of the information processing device 500 for performing a learning operation will be described with reference to Fig. 6. Each step of this operation may be performed by, for example, the processor 501 executing an instruction included in a program stored in the memory 502. Alternatively, at least some of the steps of the operation in Fig. 6 may be performed by a dedicated integrated circuit such as an ASIC or an FPGA. The operation in Fig. 6 may be started by a user's instruction to start learning.

[0042] In step S601, the information processing device 500 acquires learning data. Past time series data of external environment data related to the external environment of the vehicle 1 and past time series data of vehicle data of the vehicle 1 are stored in advance in the storage device 506 of the information processing device 500. The information processing device 500 selects and acquires the learning data from the data stored in the storage device 506.

[0043] In step S602, the information processing device 500 uses the training data to train the neural network 300. Since an existing algorithm may be used for the specific algorithm of the training method, a detailed description thereof will be omitted.

[0044] The learning data used in the learning operation will be specifically described with reference to Fig. 7. In Fig. 4, the black circle next to "learning operation" indicates the reference time for executing the learning operation. The circles (white and black circles) next to "vehicle data" indicate the time when the vehicle data was acquired, and the black circle indicates the vehicle data used in the learning operation. The circles (white and black circles) next to "external world data" indicate the time when the external world data was acquired, and the black circle indicates the external world data used in the learning operation. All of times t21 to t35 are times earlier than the time when the learning operation is executed.

[0045] In the example of FIG. 7, time t33 is the reference time for the learning operation. In this learning operation, the ECU 20 executes an inference operation for predicting the external data at time t34 by inputting the vehicle data acquired at time t33 and the external data acquired at times t21, t25, t29, and t33 to the neural network 300, and makes the neural network 300 learn by comparing the inference result with the external data at time t35. The information processing device 500 executes the learning operation for various reference times. The interval between the acquisition times of the external data input to the neural network 300 in the learning operation is longer than the execution period P of the inference operation performed using the neural network 300. This allows the neural network 300 used in the operation of FIG. 2 to learn with high accuracy.

[0046] <Second embodiment> A second embodiment of the present invention will be described with reference to Fig. 8. The second embodiment differs from the first embodiment in that a neural network 800 is used instead of the neural network 300, but may be similar to the first embodiment in other respects.

[0047] The neural network 800 differs from the neural network 300 in that it has a feedback path 801 in addition to the input layer 301, the intermediate layer 302, and the output layer 303. The neural network 800 is a so-called recurrent neural network. In the neural network 800, the output of the intermediate layer 302b is fed back to the intermediate layer 302a. As described in the first embodiment, the ECU 20 periodically performs an inference operation. The ECU 20 feeds back the output of the intermediate layer 302b at a time that is two or more integer multiples of the execution period P before to the intermediate layer 302a. For example, during the execution of the inference operation at time t13, the ECU 20 feeds back the output of the intermediate layer 302b of the inference operation at time t9 (a time that is four times the execution period P before) to the intermediate layer 302a.

[0048] <Modification> In the above-described embodiment, the neural network 300 is trained by supervised learning, and an inference operation is performed using the neural network 300. Alternatively, the neural network 300 may be used for unsupervised learning or reinforcement learning. For example, the ECU 20 of the vehicle 1 may execute an inference operation for performing an action plan for the vehicle 1 using a model obtained by performing reinforcement learning. In this case, the neural network 300 may be used for a DQN (Deep Q-Network). The above-described embodiment has been described in the context of automatic vehicle driving control. However, the above-described embodiment is also applicable to other contexts in which an inference operation is periodically executed.

[0049] <Summary of the embodiment> <Item 1> An information processing device (20), A processing means (20a) for executing an inference operation in each execution cycle, The inference operation is performed by inputting input data including time series data into a neural network (300, 800); An information processing device, wherein an interval (I) between acquisition times of constituent data of the time-series data input in one inference operation is longer than the execution period (P). This item makes it easier to capture changes in time-series data, improving the accuracy of inference operations. <Item 2> 2. The information processing device according to item 1, wherein an interval between acquisition times of constituent data of the time-series data input in one inference operation is an integer multiple of the execution period that is equal to or greater than two. According to this item, the acquisition period of the configuration data and the execution period of the inference operation can be synchronized, which facilitates the calculation. <Item 3> 3. The information processing device according to item 1 or 2, wherein the processing means updates the time series data included in the input data so as to include configuration data acquired at a new time each time the inference operation is executed. This item allows inferences to be made based on the most recent situation. <Item 4> 3. The information processing device according to item 1 or 2, wherein the processing means updates the time series data included in the input data each time the inference operation is executed so that each piece of configuration data has a time advanced by the execution period. This item makes it possible to suppress variation in the accuracy of inference each time an inference operation is executed. <Item 5> the neural network is a recurrent neural network (800); The information processing device according to any one of items 1 to 4, wherein the processing means feeds back to a second hidden layer an output from a first hidden layer of the recurrent neural network at a time that is an integer multiple of two or more of the execution period before the time. According to this item, the accuracy of the inference operation is further improved. <Item 6> 5. The information processing device according to any one of items 1 to 4, wherein the neural network is a three-dimensional convolutional neural network. According to this item, the accuracy of the inference operation is further improved. <Item 7> A vehicle (1) equipped with the information processing device according to any one of items 1 to 6. This item improves the accuracy of inference operations in the vehicle. <Item 8> The vehicle further includes a detection means (41 to 43) for acquiring external environment data relating to an external environment of the vehicle, 8. The vehicle according to item 7, wherein the processing means acquires the external data as constituent data of the time series data. According to this item, the accuracy of inference operations using external data is improved. <Item 9> 9. The vehicle according to item 8, wherein the detection means acquires the external environment data for each execution cycle. According to this item, external data can be acquired in synchronization with the inference operation. <Item 10> The vehicle according to any one of items 7 to 9, further comprising a driving control means (20) for controlling driving of the vehicle based on an inference result of the inference operation. According to this item, the quality of driving control is improved. <Item 11> An information processing device (500), A processing means (501) for training a neural network (300, 800) by inputting input data including time series data to the neural network, An information processing device, wherein an interval between acquisition times of constituent data of the time series data is longer than an execution period of an inference operation performed using the neural network. This item makes it easier to capture changes in time-series data, improving the accuracy of inference operations. <Item 12> 2. The information processing device according to item 1, wherein an interval between acquisition times of the constituent data of the time series data is an integer multiple of two or more of an execution period of an inference operation performed using the neural network. According to this item, the acquisition period of the configuration data and the execution period of the inference operation can be synchronized, which facilitates the calculation. <Item 13> the neural network is a recurrent neural network (800); Item 13. The information processing device according to item 11 or 12, wherein the processing means feeds back to a second hidden layer an output from a first hidden layer of the recurrent neural network at a time that is an integer multiple of two or more of the execution period before the time. According to this item, the accuracy of the inference operation is further improved. <Item 14> Item 13. The information processing device according to item 11 or 12, wherein the neural network is a three-dimensional convolutional neural network. According to this item, the accuracy of the inference operation is further improved. <Item 15> A program for causing a computer to function as each of the means of the information processing device according to any one of items 1 to 6 and 11 to 14. According to this section, the invention is embodied in the form of a program.

[0050] The invention is not limited to the above-described embodiment, and various modifications and variations are possible within the scope of the gist of the invention. [Explanation of symbols]

[0051] 1 vehicle, 2 control device, 300 neural network, 500 information processing device

Claims

1. An information processing device, A processing means for executing an inference operation in each execution cycle is provided, The inference operation is performed by inputting input data, including time series data, into a neural network; An information processing device, wherein an interval between acquisition times of constituent data of the time-series data input in one inference operation is longer than the execution period.

2. The information processing apparatus according to claim 1 , wherein an interval between acquisition times of constituent data of the time-series data input in one inference operation is an integer multiple of the execution period, which is equal to or greater than two.

3. 3 . The information processing apparatus according to claim 1 , wherein the processing means updates the time-series data included in the input data so as to include configuration data acquired at a new time each time the inference operation is executed.

4. 3 . The information processing apparatus according to claim 1 , wherein the processing means updates the time series data included in the input data each time the inference operation is executed so that each piece of configuration data has a time advanced by the execution period.

5. the neural network is a recurrent neural network; 5 . The information processing device according to claim 1 , wherein the processing means feeds back to a second hidden layer an output from a first hidden layer of the recurrent neural network at a time that is an integer multiple of two or more of the execution period before the first hidden layer.

6. The information processing device according to claim 1 , wherein the neural network is a three-dimensional convolutional neural network.

7. A vehicle comprising the information processing device according to any one of claims 1 to 6.

8. The vehicle further includes a detection means for acquiring external environment data relating to an external environment of the vehicle, The vehicle according to claim 7 , wherein the processing means acquires the external world data as constituent data of the time series data.

9. The vehicle according to claim 8 , wherein the detection means acquires the external environment data for each execution period.

10. The vehicle according to claim 7 , further comprising a driving control means for controlling driving of the vehicle based on an inference result of the inference operation.

11. An information processing device, A processing means for training a neural network by inputting input data including time series data to the neural network, An information processing device, wherein an interval between acquisition times of constituent data of the time series data is longer than an execution period of an inference operation performed using the neural network.

12. The information processing apparatus according to claim 11 , wherein an interval between acquisition times of the constituent data of the time-series data is an integer multiple of two or more of an execution period of an inference operation performed using the neural network.

13. the neural network is a recurrent neural network; 13 . The information processing apparatus according to claim 11 , wherein the processing means feeds back to a second hidden layer an output from a first hidden layer of the recurrent neural network at a time that is an integer multiple of two or more of the execution period before the time.

14. The information processing device according to claim 11 , wherein the neural network is a three-dimensional convolutional neural network.

15. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 6 and 11 to 14.

Images