Information processing devices, vehicles, and storage media

Abstract

This invention provides an information processing device, a vehicle, and a storage medium, with the aim of improving the accuracy of periodically executed inference actions. The information processing device includes a processing unit that performs inference actions in each execution cycle. The inference actions are performed by inputting input data containing time-series data into a neural network. In one inference action, the interval between the acquisition times of the constituent data of the time-series data input is longer than the execution cycle.

Description

Technical Field

[0001] This invention relates to information processing devices, vehicles, and storage media. Background Technology

[0002] In autonomous driving, techniques are known to use models generated through machine learning for action planning. Patent document 1 describes a method for calculating curvature in autonomous driving control processing using multiple time-series data of estimated coordinate values ​​calculated over a period from the past to the present.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-127098 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] When performing inference actions that take time-series data as input periodically, if the interval between the acquisition of the time-series data is equal to the execution cycle of the inference action, the accuracy of the inference action may decrease. One aspect of this invention aims to provide a technique for improving the accuracy of periodically executed inference actions.

[0008] Methods for solving problems

[0009] In view of the above-mentioned problems, according to one embodiment, an information processing apparatus is provided, which includes a processing unit that performs an inference action in each execution cycle. The inference action is performed by inputting input data containing time-series data into a neural network, wherein the interval between the acquisition times of the constituent data of the time-series data input in one inference action is longer than the execution cycle. According to another embodiment, an information processing apparatus is provided, which includes a processing unit that enables the neural network to learn by inputting input data containing time-series data into a neural network, wherein the interval between the acquisition times of the constituent data of the time-series data is longer than the execution cycle of the inference action performed using the neural network.

[0010] Invention Effects

[0011] Through the above methods, the accuracy of periodically executed inference actions is improved. Attached Figure Description

[0012] Figure 1 This is a block diagram illustrating a structural example of the vehicle according to the first embodiment.

[0013] Figure 2This is a flowchart illustrating an example of the operation of the ECU involved in the first embodiment.

[0014] Figure 3 This is a schematic diagram illustrating an example of the neural network involved in the first embodiment.

[0015] Figure 4 This is a schematic diagram illustrating an example of input data involved in the first embodiment.

[0016] Figure 5 This is a block diagram illustrating a structural example of the information processing apparatus according to the first embodiment.

[0017] Figure 6 This is a flowchart illustrating an example of the operation of the information processing apparatus according to the first embodiment.

[0018] Figure 7 This is a schematic diagram illustrating an example of the learning data involved in the first embodiment.

[0019] Figure 8 This is a schematic diagram illustrating an example of the neural network involved in the second embodiment. Detailed Implementation

[0020] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Furthermore, the following embodiments do not limit the invention to which the technical solution pertains, and all combinations of features described in the embodiments are not limited to those essential to the invention. Two or more features from the plurality of features described in the embodiments may be arbitrarily combined. Additionally, the same or identical structures are labeled with the same reference numerals, and repeated descriptions are omitted.

[0021] <First Implementation Method>

[0022] Figure 1 This is a block diagram of vehicle 1 according to the first embodiment of the present invention. Figure 1 In this diagram, the outline of vehicle 1 is represented by a top view and a side view. As an example, vehicle 1 is a four-wheeled passenger car of sedan type. Vehicle 1 can be such a four-wheeled vehicle, or it can be a two-wheeled vehicle or other types of vehicles.

[0023] Vehicle 1 includes a vehicle control unit 2 (hereinafter referred to as control unit 2) for controlling vehicle 1. Control unit 2 includes multiple ECUs 20 to 29 that can be connected via an in-vehicle network. Each ECU includes a processor (such as a CPU), a memory such as semiconductor memory, and an interface for external devices. Therefore, each ECU can also be referred to as an information processing unit. The memory stores the program executed by the processor, the data used by the processor in processing, etc. Each ECU may also have multiple processors, memories, and interfaces. For example, ECU 20 has a processor 20a and a memory 20b. The processor 20a executes the commands contained in the program stored in the memory 20b, thereby performing the processing of ECU 20. Alternatively, ECU 20 may also have a dedicated integrated circuit such as an ASIC for performing the processing of ECU 20. The same applies to other ECUs.

[0024] The functions of each ECU 20 to 29 will be explained below. It should be noted that the number of ECUs and their functions can be appropriately designed and can be further subdivided or integrated than in this embodiment.

[0025] ECU 20 performs controls related to the autonomous driving of vehicle 1. In autonomous driving, it automatically controls at least one of the steering and acceleration / deceleration of vehicle 1. The autonomous driving of ECU 20 may also include autonomous driving that does not require driver operation (also known as autonomous driving) and autonomous driving that assists driver operation (also known as driving assistance).

[0026] ECU 21 controls the electric power steering system 3. The electric power steering system 3 includes a mechanism for steering the front wheels according to the driver's driving operation (steering operation) on the steering wheel 31. Additionally, the electric power steering system 3 includes a motor that provides driving force for assisting steering operation or automatically steering the front wheels, a sensor for detecting the steering angle, etc. When the vehicle 1 is in automatic driving mode, ECU 21 automatically controls the electric power steering system 3 in accordance with instructions from ECU 20 to control the direction of travel of the vehicle 1.

[0027] ECU 22 and ECU 23 control the detection units 41-43 that detect the surrounding conditions of the vehicle and process the information of the detection results. Detection unit 41 is a camera (hereinafter, sometimes referred to as camera 41) that takes pictures of the front of the vehicle 1. In this embodiment, it is mounted on the front of the vehicle 1 inside the passenger compartment of the windshield. By analyzing the images captured by camera 41, it is possible to extract the outline of targets and lane markings (white lines, etc.) on the road.

[0028] The detection unit 42 is a light-emitting radar (LED) (hereinafter sometimes referred to as LED 42), which detects objects around the vehicle 1 or measures the distance to objects. In this embodiment, five LEDs 42 are provided: one at each corner of the front of the vehicle 1, one at the center of the rear, and one on each side of the rear. The detection unit 43 is a millimeter-wave radar (hereinafter sometimes referred to as radar 43), which detects targets around the vehicle 1 or measures the distance to targets. In this embodiment, five radars 43 are provided: one at the center of the front of the vehicle 1, one at each corner of the front, and one at each corner of the rear.

[0029] ECU 22 controls one of the cameras 41 and each of the optical radars 42, and processes the information from the detection results. ECU 23 controls the other camera 41 and each of the radars 43, and processes the information from the detection results. By having two sets of devices for detecting the vehicle's surroundings, the reliability of the detection results can be improved. In addition, by having different types of detection units such as cameras, optical radars, and radars, the vehicle's surrounding environment can be analyzed from multiple perspectives.

[0030] ECU 24 controls the gyroscope sensor 5, GPS sensor 24b, and communication device 24c, and processes the detection or communication results. The gyroscope sensor 5 detects the rotational motion of vehicle 1. It can determine the vehicle 1's route based on the detection results of the gyroscope sensor 5, wheel speed, etc. The GPS sensor 24b detects the current position of vehicle 1. The communication device 24c wirelessly communicates with a server providing map and traffic information to acquire this information. ECU 24 can access a map information database 24a built in its memory, and performs route searches from the current location to the destination. ECU 24, the map information database 24a, and the GPS sensor 24b constitute a navigation device.

[0031] The ECU25 is equipped with a communication device 25a for inter-vehicle communication. The communication device 25a communicates wirelessly with other vehicles in the vicinity to exchange information between vehicles.

[0032] ECU 26 controls power unit 6. Power unit 6 is a mechanism that outputs driving force to rotate the drive wheels of vehicle 1, and includes, for example, an engine and a transmission. ECU 26 controls the engine output, for example, in response to driver operations (accelerator or acceleration) detected by the operation detection sensor 7a located on the accelerator pedal 7A, or switches transmission gears based on information such as vehicle speed detected by the vehicle speed sensor 7c. When vehicle 1 is in automatic driving mode, ECU 26 automatically controls power unit 6 in response to instructions from ECU 20, controlling the acceleration and deceleration of vehicle 1.

[0033] ECU27 controls the lighting devices (headlights, taillights, etc.), including the turn indicator 8. Figure 1 In the case of this example, the direction indicator 8 is located at the front of the vehicle 1, the door rearview mirror, and the rear.

[0034] ECU 28 controls the input / output device 9. The input / output device 9 outputs information to the driver and receives information from the driver. The sound output device 91 reports information to the driver via sound. The display device 92 reports information to the driver via image display. The display device 92 may be located in front of the driver's seat, forming part of the instrument panel, etc. It should be noted that sound and display are exemplified here, but information can also be reported via vibration or light. Furthermore, multiple methods of reporting information can be combined, such as sound, display, vibration, or light. The combination or notification method can also differ depending on the level of the information to be notified (e.g., urgency). The input device 93 is located in a position operable by the driver and is a switch group for instructing the vehicle 1, but may also include a sound input device.

[0035] ECU 29 controls the braking device 10 and the parking brake (not shown). The braking device 10 is, for example, a disc brake, installed on each wheel of the vehicle 1, which decelerates or stops the vehicle 1 by applying resistance to the rotation of the wheels. ECU 29 controls the operation of the braking device 10 in accordance with the driver's driving operation (braking operation) detected by the operation detection sensor 7b installed on the brake pedal 7B. When the vehicle 1 is in automatic driving mode, ECU 29 automatically controls the braking device 10 in accordance with the instructions from ECU 20, controlling the deceleration and stopping of the vehicle 1. The braking device 10 and the parking brake can also operate to maintain the vehicle 1 in a stopped state. In addition, if the transmission of the power unit 6 is equipped with a parking lock mechanism, it can also be operated to maintain the vehicle 1 in a stopped state.

[0036] Reference Figure 2An example of the operation of the ECU 20 used for automatic driving control will be described. Each step of this operation can also be executed, for example, by the processor 20a executing commands contained in a program stored in the memory 20b. Instead, Figure 2 At least some of the steps of the operation can also be performed by application-specific integrated circuits such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays). Figure 2 The action begins simultaneously with the initiation of automatic driving control on vehicle 1. Figure 2 In this embodiment, descriptions of actions not used are omitted. Existing actions may also be performed for the actions for which descriptions have been omitted.

[0037] In step S201, ECU 20 acquires external data related to the external environment of vehicle 1 and stores it in a storage unit (e.g., memory 20b). External data includes, for example, data related to traffic participants (vehicles, pedestrians, bicycles, etc.) and static objects (buildings, roads, signs, etc.) located around vehicle 1. External data is acquired, for example, using the detection units 41-43 described above. For example, external data may also be images of the area around vehicle 1 captured by camera 41. As described later, external data is acquired repeatedly. Therefore, external data is stored in memory 20b as time-series data. That is, each piece of external data becomes constituent data of the time-series data. ECU 20 may also delete past external data that is no longer needed from the storage unit when storing external data acquired at a new time.

[0038] In step S202, ECU 20 acquires input data for use in the inference action of step 203. The input data may include, for example, timing data of vehicle data and external data at the execution time of step 202. Vehicle data may include, for example, vehicle speed, acceleration, etc. The specific types of external data acquired will be described later.

[0039] In step S203, ECU20 uses the input data acquired in step S202 to predict external data at future points in time. Specifically, ECU20 performs an inference action by inputting input data into the learned neural network. The inference result (output result) of this inference action becomes the prediction result of the external data.

[0040] Reference Figure 3 A specific example of the neural network used in the inference action of step S203 will be explained. The neural network 300 includes an input layer 301, an intermediate layer 302, and an output layer 303. Figure 3The example shown illustrates a case where the intermediate layer 302 has 3 layers, but the number of intermediate layers 302 is not limited to this. The neural network 300 can be a so-called deep neural network with many intermediate layers 302 (e.g., 5 or more). Figure 2 The actions are learned before they begin and stored in the storage unit of vehicle 1 (e.g., memory 20b). The learning method will be described later. Vehicle data and timing-based external data are input into neural network 300.

[0041] The input layer 301 can also have a separate node for each pixel value of the external data. Alternatively, convolution processing of the external data can be performed in the input layer 301. That is, the neural network 300 can also be a convolutional neural network. Furthermore, the neural network 300 can also be a three-dimensional convolutional neural network. In the input layer 301 of the three-dimensional convolutional neural network, multiple external data acquired at different times are convolved.

[0042] In step S204, ECU 20 controls the driving of vehicle 1 based on the inferred results of the inferred action (specifically, the predicted external data). This step can also be performed using existing technology, so detailed description is omitted. For example, ECU 20 generates a driving plan for vehicle 1 based on the predicted external data and controls the actuators of vehicle 1 according to the driving plan.

[0043] In step S205, ECU 20 determines whether to terminate automatic driving control. If ECU 20 determines that automatic driving control has terminated ("Yes" in step S205), it terminates the process; otherwise ("No" in step S205), it returns the process to step S201. For example, it may determine that automatic driving control terminated according to the driver's instructions, or it may determine that driving has been handed over to the driver.

[0044] As described above, ECU 20 repeatedly executes steps S201 to S204. In this embodiment, ECU 20 executes steps S201 to S204 periodically, that is, with a constant execution interval for the same step. This execution interval is referred to as the execution cycle, and its length is denoted as P. That is, according to each execution cycle P, ECU 20 acquires external data in step S201 and performs a deduction action in step S203.

[0045] Reference Figure 4 This section provides a detailed explanation of the input data used in the inference process. Figure 4In the diagram, the circles (white and black) next to "Inference Action" indicate the time when the inference action was performed; the black circle indicates the inference action performed at the time of focus. The circles (white and black) next to "Vehicle Data" indicate the time when vehicle data was acquired; the black circle indicates the vehicle data used in the inference action performed at the time of focus. The circles (white and black) next to "External Data" indicate the time when external data was acquired; the black circle indicates the external data used in the inference action performed at the time of focus.

[0046] Figure 4 The diagram above illustrates the inference action performed at time t13. Before time t13, ECU 20 performs inference actions at times t1 to t12. Therefore, the time intervals between each of times t1 to t12 (e.g., t2 to t1) are equal to the aforementioned execution period P. At time t13, ECU 20 performs an inference action to predict external data for time t14 by inputting vehicle data acquired at time t13 and external data acquired at times t1, t5, t9, and t13 into neural network 300. In the inference action at time t13, external data acquired outside of times t1, t5, t9, and t13 from the time-series data of external data is not used. Thus, in this embodiment, the interval between the acquisition times of the external data of the time-series data input in one inference action is longer than the execution period P. Hereinafter, the interval between the acquisition times of the external data of the time-series data input in one inference action will be denoted as the acquisition interval I. Figure 4 In the example, the acquisition interval I is 4 times the execution cycle P.

[0047] The effect of making the acquisition interval I longer than the execution period P will be explained. To respond quickly to changes in the external environment, the shorter the execution period P of the inference action, the better. Since the execution period P cannot be shorter than the time required for the inference action, for example, in a certain environment, the execution period P is approximately 50 to 150 ms. In the following explanation, the execution period P is 100 ms. Assume a pedestrian is walking around vehicle 1 at a speed of 4 km / h. In this case, the pedestrian moves approximately 0.1 m in 100 ms. Therefore, if the acquisition interval I is equal to the execution period P, the changes in environmental data between adjacent moments are buried by observation noise, reducing the accuracy of the inference action. On the other hand, in this embodiment, by making the acquisition interval I longer than the execution period P, the changes in environmental data between adjacent moments can be increased, thereby improving the accuracy of the inference action.

[0048] exist Figure 4In the example, the acquisition interval I is 4 times the execution cycle P. Alternatively, the acquisition interval I can also be any other integer multiple of the execution cycle P, greater than 2. By setting this ratio as an integer multiple, there is no need for periodic adjustments between acquiring external data and executing inference actions, thus simplifying the computation. The multiple is appropriately determined based on the relationship between the execution cycle P and the input data. Furthermore, the acquisition interval I may not be an integer multiple of the execution cycle P. For example, if the acquisition of external data and the execution of inference actions are performed asynchronously, the acquisition interval I may not be an integer multiple of the execution cycle P.

[0049] Figure 4 The diagram below illustrates the inference action performed at time t14. At time t14, ECU 20 performs an inference action to predict the external data for time t15 by inputting vehicle data acquired at time t14 and external data acquired at times t2, t6, t10, and t14 into neural network 300. In the inference action at time t14, external data acquired at times t2, t6, t10, and beyond are not used. Therefore, for the inference action performed at time t14, the acquisition interval I is longer than the execution period P.

[0050] The external data used in the inference action at time 14 includes external data acquired at the new time t14, compared to the external data used in the inference action at time 13. Thus, each time an inference action is performed, the ECU 20 updates the timing data of the external data included in the input data to include the external data acquired at the new time. This enables prediction based on the most recent external conditions. Furthermore, in this update, the ECU 20 updates the timing data of the external data included in the input data to reflect the time when each external data point has advanced by execution cycle P. Specifically, the acquisition times t2, t6, t10, and t14 of the external data used in the inference action at time 14 are such that the acquisition times t1, t5, t9, and t13 of the external data used in the inference action at time 13 have each advanced by execution cycle P. This suppresses deviations in the accuracy of the inference each time an inference action is performed.

[0051] Reference Figure 5 The block diagram illustrates an example of the structure of an information processing device 500 used to enable the neural network 300 to learn. The information processing device 500 is implemented, for example, by an information processing device such as a personal computer or a workstation. The information processing device 500 can be implemented as a single device or as multiple devices interconnected via a network.

[0052] Information processing device 500 has Figure 5The components shown are as follows. The processor 501 controls the overall operation of the information processing device 500. The processor 501 is implemented, for example, by a CPU (Central Processing Unit), a combination of a CPU and a GPU (Graphics Processing Unit), etc. The memory 502 stores programs, temporary data, etc., used for the operation of the information processing device 500. The memory 502 is implemented, for example, by ROM (Read Only Memory), RAM (Random Access Memory), etc.

[0053] Input device 503 is used by the user of information processing device 500 to input data into information processing device 500, such as through a mouse or keyboard. Output device 504 is used by the user of information processing device 500 to confirm output from information processing device 500, such as through an output device like a display or an audio device like a speaker. Communication device 505 provides the function for information processing device 500 to communicate with other devices, such as through a network card. Communication with other devices can be wired or wireless. Storage device 506 is used to store data used in the processing of information processing device 500, such as through HDD (Hard Disk Drive) or SSD (Solid State Drive).

[0054] Reference Figure 6 An example of the operation of the information processing device 500 used for performing learning actions will be described. Each step of this operation can also be executed, for example, by the processor 501 executing commands contained in a program stored in the memory 502. Instead, Figure 6 At least some of the steps in the operation can also be performed by application-specific integrated circuits such as ASICs and FPGAs. Figure 6 The action can also be initiated based on the start instructions learned from the user.

[0055] In step S601, the information processing device 500 acquires learning data. The information processing device 500's storage device 506 stores prior-time data of external data related to the vehicle 1's external environment and prior-time data of the vehicle 1's vehicle data. The information processing device 500 selects and acquires learning data from the data stored in the storage device 506.

[0056] In step S602, the information processing device 500 uses the learning data to learn the neural network 300. Regarding the specific algorithm for the learning method, existing algorithms can also be used, therefore detailed descriptions are omitted.

[0057] Reference Figure 7 This provides a detailed explanation of the learning data used in the learning actions. Figure 7 In the diagram, the black dot next to "Learning Action" indicates the reference time for executing the learning action. The circles (white and black) next to "Vehicle Data" indicate the time when vehicle data was acquired; the black circle indicates the vehicle data used in the learning action. The circles (white and black) next to "External Data" indicate the time when external data was acquired; the black circle indicates the external data used in the learning action. Times t21 to t35 are all times preceding the execution time of the learning action.

[0058] exist Figure 7 In the example, time t33 serves as the reference time for the learning action. In this learning action, ECU 20 inputs vehicle data acquired at time t33 and external data acquired at times t21, t25, t29, and t33 into neural network 300 to perform an inference action to predict external data at time t34. The neural network 300 learns by comparing the inference result with external data at time t35. Information processing device 500 performs learning actions for various reference times. The interval between the acquisition times of external data input into neural network 300 during the learning action is longer than the execution cycle P of the inference action performed using neural network 300. Therefore, high-precision learning is possible at various reference times. Figure 2 The neural network 300 used in the action.

[0059] <Second Implementation Method>

[0060] Reference Figure 8 The second embodiment of the present invention will now be described. In the second embodiment, the difference from the first embodiment is that a neural network 800 is used instead of a neural network 300; other aspects are the same as in the first embodiment.

[0061] Neural network 800 differs from neural network 300 in that it has a feedback path 801 in addition to the input layer 301, intermediate layer 302, and output layer 303. Neural network 800 is a so-called recurrent neural network. In neural network 800, the output of intermediate layer 302b is fed back to intermediate layer 302a. As described in the first embodiment, ECU 20 performs inference actions periodically. ECU 20 feeds back the output of intermediate layer 302b at a time before an integer multiple of 2 of the execution period P to intermediate layer 302a. For example, during the inference action at time t13, ECU 20 feeds back the output of intermediate layer 302b of the inference action at time t9 (a time before 4 times the execution period P) to intermediate layer 302a.

[0062] <Variation Example>

[0063] In the above-described embodiment, the neural network 300 learns through supervised learning and performs inference actions using the neural network 300. Alternatively, the neural network 300 can also be used for unsupervised learning or reinforcement learning. For example, the ECU 20 of vehicle 1 can also use a model obtained through reinforcement learning to perform inference actions for planning the actions of vehicle 1. In this case, the neural network 300 can also be used with DQN (Deep Q-Network). Furthermore, the above-described embodiment is described in the context of automatic driving control of a vehicle. However, the above-described embodiment can also be applied to other contexts where inference actions are performed periodically.

[0064] <Summary of Implementation Methods>

[0065] Project 1

[0066] An information processing device, wherein,

[0067] The information processing device (20) includes a processing unit (20a) that performs inference actions in each execution cycle.

[0068] The inference action is performed by feeding input data containing time-series data into a neural network (300, 800).

[0069] The interval (I) between the acquisition times of the time sequence data input in one of the inference actions is longer than the execution cycle (P).

[0070] According to this project, changes in time-series data are easily captured, thus improving the accuracy of action inferences.

[0071] Project 2

[0072] According to the information processing apparatus of Project 1, the interval between the acquisition times of the constituent data of the time sequence data input in one of the inference actions is an integer multiple of 2 or more of the execution cycle.

[0073] According to this project, the data acquisition cycle can be synchronized with the inference action execution cycle, thus making the calculation easier.

[0074] Project 3

[0075] According to the information processing apparatus described in Project 1 or 2, the processing unit updates the time-series data contained in the input data to the constituent data acquired at the new time each time the inference action is performed.

[0076] According to this project, inferences can be made based on the most recent situation.

[0077] Project 4

[0078] According to the information processing apparatus described in Project 1 or 2, the processing unit updates the time-series data contained in the input data to the time after each constituent data has advanced the execution cycle each time the inference action is performed.

[0079] According to this project, it is possible to suppress the deviation in the accuracy of inferences each time an inference action is performed.

[0080] Project 5

[0081] The information processing apparatus according to any one of items 1 to 4, wherein...

[0082] The neural network is a recurrent neural network (800).

[0083] The processing unit feeds back the output of the first intermediate layer of the recursive neural network at a time before an integer multiple of 2 of the execution cycle to the second intermediate layer.

[0084] According to this project, the accuracy of inference actions has been further improved.

[0085] Project 6

[0086] The information processing apparatus according to any one of items 1 to 4, wherein the neural network is a three-dimensional convolutional neural network.

[0087] According to this project, the accuracy of inference actions has been further improved.

[0088] <Project 7>

[0089] A vehicle (1) having an information processing device as described in any one of items 1 to 6.

[0090] According to this project, the accuracy of inference actions in vehicles has been improved.

[0091] <Project 8>

[0092] The vehicle also includes a detection unit (41-43) for acquiring external data related to the vehicle's external environment.

[0093] According to the vehicle described in Project 7, the processing unit acquires the external data as constituent data of the time-series data.

[0094] According to this project, the accuracy of inferences made using external data has been improved.

[0095] Project 9

[0096] According to the vehicle described in Project 8, the detection unit acquires the external data in each execution cycle.

[0097] According to this project, external data can be acquired synchronously with the inference process.

[0098] <Project 10>

[0099] The vehicle according to any one of items 7 to 9, wherein the vehicle further comprises a driving control unit (20) that controls the driving of the vehicle based on the inference result of the inference action.

[0100] The quality of driving control has been improved as a result of this project.

[0101] <Project 11>

[0102] An information processing device, wherein,

[0103] The information processing device (500) includes a processing unit (501) that enables the neural network (300, 800) to learn by inputting input data containing time-series data into the neural network.

[0104] The interval between the acquisition times of the constituent data of the time series data is longer than the execution cycle of the inference action performed using the neural network.

[0105] According to this project, changes in time-series data are easily captured, thus improving the accuracy of action inferences.

[0106] <Project 12>

[0107] According to the information processing apparatus of Project 1, the interval between the acquisition times of the constituent data of the time series data is an integer multiple of 2 or more of the execution cycle of the inference action performed using the neural network.

[0108] According to this project, the data acquisition cycle can be synchronized with the inference action execution cycle, thus making the calculation easier.

[0109] <Project 13>

[0110] According to the information processing apparatus described in item 11 or 12, wherein,

[0111] The neural network is a recurrent neural network (800).

[0112] The processing unit feeds back the output of the first intermediate layer of the recursive neural network at a time before an integer multiple of 2 of the execution cycle to the second intermediate layer.

[0113] According to this project, the accuracy of inference actions has been further improved.

[0114] <Project 14>

[0115] According to the information processing apparatus described in item 11 or 12, the neural network is a three-dimensional convolutional neural network.

[0116] According to this project, the accuracy of inference actions has been further improved.

[0117] <Project 15>

[0118] A program for enabling a computer to function as a unit of an information processing apparatus as described in any of items 1 to 6 and 11 to 14.

[0119] According to this project, the invention is implemented in the form of a procedure.

[0120] This invention is not limited to the above-described embodiments, and various modifications and alterations can be made within the scope of the invention's intent.

Claims

1. An information processing device, wherein, The information processing device includes a processing unit that performs inference actions in each execution cycle. The inference action is performed by inputting time-series data, which includes vehicle data and external data acquired by the vehicle's detection unit, into a neural network. The interval between the acquisition times of the constituent data of the time sequence data input in one of the inference actions is longer than the execution cycle.

2. The information processing apparatus according to claim 1, wherein, The interval between the acquisition times of the time sequence data input in one of the inference actions is an integer multiple of 2 or more of the execution cycle.

3. The information processing apparatus according to claim 1, wherein, Each time the inference action is performed, the processing unit updates the time-series data contained in the input data to the constituent data acquired at the new time.

4. The information processing apparatus according to claim 1, wherein, Each time the inference action is performed, the processing unit updates the timing data contained in the input data to the time after each constituent data has advanced the execution cycle.

5. The information processing apparatus according to claim 1, wherein, The neural network is a recurrent neural network. The processing unit feeds back the output of the first intermediate layer of the recursive neural network at a time before an integer multiple of 2 of the execution cycle to the second intermediate layer.

6. The information processing apparatus according to claim 1, wherein, The neural network is a three-dimensional convolutional neural network.

7. A vehicle, wherein, The vehicle is equipped with an information processing device. The information processing device includes a processing unit that performs inference actions in each execution cycle. The inference action is performed by feeding input data containing time-series data into a neural network. The interval between the acquisition times of the constituent data of the time sequence data input in one of the inference actions is longer than the execution cycle.

8. The vehicle according to claim 7, wherein, The vehicle also has a detection unit that acquires external data related to the vehicle's external environment. The processing unit acquires the external data as constituent data of the time-series data.

9. The vehicle according to claim 8, wherein, The detection unit acquires the external data in each execution cycle.

10. The vehicle according to claim 7, wherein, The vehicle also includes a driving control unit that controls the driving of the vehicle based on the inference result of the inference action.

11. A storage medium, wherein, The storage medium stores programs for enabling the computer to function as the various units of the information processing apparatus of claim 1.

12. An information processing apparatus, wherein, The information processing device includes a processing unit that inputs input data, including vehicle data and time-series data obtained from external data acquired by the vehicle's detection unit, into a neural network to enable the neural network to learn. The interval between the acquisition times of the constituent data of the time series data is longer than the execution cycle of the inference action performed using the neural network.

13. The information processing apparatus according to claim 12, wherein, The interval between the acquisition times of the constituent data of the time series data is an integer multiple of 2 or more of the execution cycle of the inference action performed using the neural network.

14. The information processing apparatus according to claim 12, wherein, The neural network is a recurrent neural network. The processing unit feeds back the output of the first intermediate layer of the recursive neural network at a time before an integer multiple of 2 of the execution cycle to the second intermediate layer.

15. The information processing apparatus according to claim 12, wherein, The neural network is a three-dimensional convolutional neural network.

16. A storage medium, wherein, The storage medium stores programs for enabling the computer to function as the various units of the information processing apparatus of claim 12.

Images