Control device, control method, and control program

Abstract

To achieve high-performance and high-speed autonomous travelling control.SOLUTION: A control device 101 calculates an estimated value of a controlled quantity of a mobile object, from sensor information of a controlled object, using a transformed mobile object prediction model Mc obtained by transformation of a mobile object prediction model M. The mobile object prediction model M is a model which has a multi-layered neuron structure and activating functions of a rectified linear unit (ReLU) structure, and predicts dynamic behavior of the mobile object. The transformed mobile object prediction model Mc is a model for which weight coefficients, biases, upper and lower limit values of input / output variables of neurons are set, on the basis of the mobile object prediction model M and in which the activating functions of the neurons are transformed using a linear inequality function including binary variables. The control device 101 calculates an estimate value of the controlled quantity of an object to be controlled, from a candidate value of an operation quantity for controlling the controlled quantity of the object to be controlled. The control device 101 determines a value of the operation quantity, on the basis of a target value of the controlled quantity of the object to be controlled, the estimated value of the controlled quantity of the object to be controlled, and an estimated value of the controlled quantity of the mobile object.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a control device, a control method, and a control program.

Background Art

[0002] Conventionally, in the autonomous driving control of a vehicle, a moving object prediction model that detects a moving object (obstacle) from information of an in-vehicle camera or radar and predicts future behavior by reproducing the moving object prediction model using deep learning of AI (Artificial Intelligence) has been used to achieve high-performance control. The autonomous driving control of a vehicle is realized, for example, by performing multivariable optimal control using a model of a multi-layer neural network (NN) (a multi-input multi-output model of a moving object).

[0003] As a prior art, for example, for the modeling of an industrial process, there is one that constructs a combined MLD system by merging two interconnected MLD (Mixed Logical Dynamical) subsystems.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the prior art, multivariable optimal control using a model of a multi-layer neural network takes a lot of calculation time, and it is difficult to perform real-time control of autonomous driving of a vehicle or the like.

[0006] In one aspect, an object of the present invention is to achieve high-performance and high-speed autonomous driving control.

Means for Solving the Problems

[0007] In one embodiment, based on a first model that has a multi-layer neuron structure and an activation function of the ReLU structure, and predicts the dynamic behavior of a moving body around the control target using the information of sensors mounted on the control target as input, the weight coefficients, biases, and upper and lower limit values of the input / output variables of each neuron are set, and the activation function of each neuron is converted using a linear inequality function including binary variables. A converted first model, the information of the sensor, and the target value of the controlled quantity of the control target are obtained. Using the converted first model, an estimated value of the controlled quantity of the future moving body is calculated from the information of the sensor, and an estimated value of the controlled quantity of the future control target is calculated from candidate values of the operation quantity for controlling the controlled quantity of the control target. Based on the target value of the controlled quantity of the control target, the estimated value of the controlled quantity of the control target, and the estimated value of the controlled quantity of the moving body, a control unit that determines the value of the operation quantity for controlling the controlled quantity of the control target is provided.

Advantages of the Invention

[0008] According to one aspect of the present invention, there is an effect that high-performance and high-speed autonomous driving control can be realized.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2A

Figure 2B

Figure 3

Figure 4

Figure 5

Figure 6

[0010] Hereinafter, embodiments of a control device, a control method, and a control program according to the present invention will be described in detail with reference to the drawings.

[0011] (Embodiment) FIG. 1 is an explanatory diagram showing an example of an embodiment of a control method according to the embodiment. In FIG. 1, a control device 101 is a computer that controls a control target. The control target is, for example, a moving body such as an automobile or a drone (unmanned aerial vehicle). Controlling the control target means, for example, controlling the braking amount, the accelerator amount, the steering amount, and the like.

[0012] Here, when detecting a moving body (obstacle) around an automobile and predicting its future behavior using a moving body prediction model of a multi-layer neural network (NN), the moving body (obstacle) may be, for example, another automobile, a pedestrian, a bicycle, or the like. On the other hand, in order to accurately reproduce the behavior of the moving body (obstacle), the number of variables to be considered increases, and the scale of the moving body prediction model becomes large. For this reason, in the prior art, when performing multivariable optimal control using a moving body prediction model, a large amount of calculation time is required.

[0013] In addition, an existing moving body prediction model is a dynamic system including characteristics such as non-linearity, first-order delay, second-order delay, and dead time, and it is difficult to derive a mathematical model of an inverse function by mathematical processing. For this reason, there is a problem that the problem cannot be solved analytically and the calculation load becomes high. Therefore, in the prior art, it is difficult to implement a moving body prediction model in, for example, an autonomous driving control unit of an automobile and perform real-time control on board.

[0014] Therefore, in the present embodiment, a moving body prediction model M (first model) having an activation function with a ReLU structure is converted into a converted moving body prediction model Mc (converted first model) using a linear inequality function including binary variables, and the dynamic behavior of the moving body is predicted, and an operation amount for controlling a controlled amount of a control target is searched for, thereby realizing a high-performance and high-speed autonomous driving control method. Hereinafter, a processing example of the control device 101 will be described.

[0015] The control device 101 acquires a converted first model obtained by converting the first model. Here, the first model is a model that predicts the dynamic behavior of a moving body around a control target. The first model has a multi-layer neuron structure and an activation function with a ReLU (Rectified Linear Unit) structure.

[0016] The ReLU structure is a function in which the output value is 0 when the input value to the function is 0 or less, and the output value changes linearly when the input value is greater than 0. The first model takes as input the information of sensors mounted on the control target and outputs the value (estimated value) of the controlled amount of the moving body in the future. The controlled amount of the moving body is, for example, a trajectory, a speed, a fuel consumption, etc. The sensor is a device for detecting obstacles around the control target, and is, for example, a radar, a camera, etc.

[0017] According to the first model, the dynamic behavior of the moving body around the control target can be reproduced with high accuracy. On the other hand, if the first model is used as it is for autonomous driving control, the calculation time for optimal control may increase. Therefore, the control device 101 acquires a converted first model obtained by converting the first model in order to realize high-performance and high-speed autonomous driving control.

[0018] The first model after transformation is a model in which, based on the first model, the weight coefficients, biases, and upper and lower limit values of the input and output variables of each neuron (node) are set, and the activation function of each neuron is transformed using a linear inequality function including binary variables. The weight coefficients and biases represent the relationship between the input and output of each neuron (node). The weight coefficients indicate the weights for the inputs. The bias is used to bias the input values within a certain range.

[0019] The input and output variables are, for example, variables input to the model itself, variables output from the model itself, variables output from each neuron within the model, and the like. The upper and lower limit values of the input and output variables are set based on, for example, the maximum and minimum values of each input and output variable obtained by giving the first model a Chirp signal or an APRBS (Amplitude Pseudo Random Binary Signal) signal, which is an exhaustive test pattern.

[0020] In the following description, the first model may be denoted as "mobile object prediction model M", and the first model after transformation may be denoted as "transformed mobile object prediction model Mc".

[0021] Here, with reference to FIGS. 2A and 2B, an example of the transformation of the mobile object prediction model M will be described. The mobile object prediction model M is, for example, created in advance.

[0022] FIG. 2A is an explanatory diagram showing an example of the mobile object prediction model. In FIG. 2A, the mobile object prediction model 200 is an example of the mobile object prediction model M and has a multi-layer neuron structure and an activation function of the ReLU structure. The mobile object prediction model 200 takes image information (current value, past value) and radar information (current value, past value) as inputs and outputs an estimated value of the controlled quantity of the future mobile object. The past value is, for example, the value one step before.

[0023] Here, as an example, the case of outputting the trajectory of the moving object (predicted value after one step) as an estimated value of the controlled quantity of the moving object will be described. The image information is, for example, the image information of an in-vehicle camera C mounted on a vehicle (automobile) to be controlled. The in-vehicle camera C is an imaging device that captures the surroundings of the vehicle. The radar information is, for example, the information of a radar R mounted on a vehicle (automobile) to be controlled.

[0024] The radar R is a device that emits radio waves, hits an obstacle, and measures the direction and position by receiving the reflected wave. The radar R is, for example, at least any one of LiDAR (Light Detection And Ranging), millimeter-wave radar, and ultrasonic sensor. The trajectory of the moving object is the path along which the moving object moves over a period from the current time to the future.

[0025] In the moving object prediction model 200, for example, the nodes 201, 202 are an example of neurons including the ReLU function. The ReLU function is an activation function of the ReLU structure. Also, ω 11 (1) represents the weight applied to the input from u1 to the node 201. Also, b1 (1) represents the bias of the node 201.

[0026] Figure 2B is an explanatory diagram showing a conversion example of the moving object prediction model. In Figure 2B, the moving object prediction model 210 is a simplified representation of the moving object prediction model 200 shown in Figure 2A. Note that in Figure 2B, a part of the moving object prediction model 210 is extracted and shown.

[0027] In the moving object prediction model 210, for example, the ReLU function 211 represents the activation function of the node 201 (see Figure 2A). w1 represents the weight applied to the input from u to the node 201. b1 represents the bias of the node 201. Also, the ReLU function 212 represents the activation function of the node 202 (see Figure 2A). w2 represents the weight applied to the input from the node 201 to the node 202. b2 represents the bias of the node 202. Note that I represents a linear function.

[0028] The control device 101 transforms each ReLU function, for example, using propositional logic. The ReLU function is represented, for example, by the following formula (1).

[0029]

Equation

[0030] In order to express the conditional branching conditions, the control device 101 introduces a binary variable δ into the ReLU function as in the following formulas (2) and (3). The binary variable δ is a variable that can only take two values, 0 and 1 (a 0-1 variable).

[0031] [f(x) ≤ 0] ⇔ [δ = 1] ···(2) [f(x) > 0] ⇔ [δ = 0] ···(3)

[0032] The above propositional logic is equivalent to the inequalities in the following formulas (4) and (5) holding. However, M is the maximum value of f(x). m is the minimum value of f(x). ε is the computer precision. The control device 101 sets the maximum value of the variable output from each neuron of the pre-conversion mobile prediction model 210 as M (the upper limit value of the variable output from each neuron). Also, the control device 101 sets the minimum value of the variable output from each neuron of the pre-conversion mobile prediction model 210 as m (the lower limit value of the variable output from each neuron).

[0033] f(x) ≤ M(1 - δ) ···(4) f(x) ≥ ε + (m - ε)δ ···(5)

[0034] By using the binary variable δ, y can be expressed as in the following formula (6).

[0035] y = (1 - δ)f(x) ···(6)

[0036] The inequalities equivalent to the above formula (6) are, for example, the following formulas (7) to (10).

[0037] y ≤ M(1 - δ) ···(7) y ≥ m(1 - δ) ···(8) y ≤ f(x) - mδ ···(9) y ≥ f(x) - Mδ ···(10)

[0038] The control device 101 converts the moving body prediction model 210, for example, by applying all the above transformations to all ReLU functions.

[0039] The moving body prediction model 220 is an example of the transformed moving body prediction model Mc, and is the transformed moving body prediction model obtained by transforming the moving body prediction model 210. The moving body prediction model 220 takes image information (current value, past value) and radar information (current value, past value) as inputs and outputs the trajectory (estimated value) of a future moving body. In FIG. 2B, a part of the moving body prediction model 220 is extracted and displayed.

[0040] The control device 101 sets the weight coefficients and biases of each neuron of the transformed moving body prediction model 220 as the weight coefficients and biases (for example, w1, b1) of each neuron (for example, node 201) of the moving body prediction model 210 before transformation. Also, the control device 101 sets the upper and lower limit values of the variables input to the moving body prediction model 210 before transformation as the upper and lower limit values of the variables (for example, u) input to the transformed moving body prediction model 220. Further, the control device 101 sets the upper and lower limit values of the variables output from the moving body prediction model 210 before transformation as the upper and lower limit values of the variables (for example, y) output from the transformed moving body prediction model 220.

[0041] Thereby, the control device 101 can obtain the transformed moving body prediction model 220 obtained by transforming the moving body prediction model 210. According to the moving body prediction model 220, since an algorithm capable of quickly solving the problem (mixed integer programming problem) can be used, the computational load can be suppressed. Also, since the problem space can be limited, the solution space becomes smaller and the time required for search can be shortened.

[0042] However, the conversion of the moving body prediction model 210 may be performed on another computer different from the control device 101. In this case, the control device 101 acquires the converted moving body prediction model 220, for example, by a user's operation input or by receiving it from another computer.

[0043] The control device 101 acquires information of sensors mounted on the controlled object and target values of the controlled amounts of the controlled object. Specifically, for example, the control device 101 acquires image information from an in-vehicle camera C mounted on the controlled object. The image information is information indicating an image captured by the in-vehicle camera C, for example.

[0044] In addition, the control device 101 acquires radar information from a radar R mounted on the controlled object. The radar information is, for example, a measurement result indicating the direction and position of a moving body (obstacle) existing around the controlled object. The controlled amount of the controlled object is, for example, the trajectory, speed, fuel consumption, etc. of the controlled object.

[0045] The trajectory of the controlled object is the path along which the controlled object moves during a period from the current time to the future. The speed of the controlled object is the speed at which the controlled object moves during a period from the current time to the future. The fuel consumption of the controlled object is the fuel consumption (fuel efficiency) consumed by the controlled object during a period from the current time to the future.

[0046] The target value of the controlled amount is, for example, the value of the controlled amount set according to the road conditions. The target value of the controlled amount is, for example, the target trajectory and target speed of a vehicle heading to a destination. Specifically, for example, the control device 101 acquires the target trajectory and target speed (target values of the controlled amounts) of the vehicle to be controlled from the navigation system of the vehicle.

[0047] The control device 101 calculates an estimated value of the controlled quantity of the future mobile body from the acquired sensor information using the converted mobile body prediction model Mc. Specifically, for example, the control device 101 inputs the acquired image information of the in-vehicle camera C and the radar information of the radar R into the converted mobile body prediction model Mc to calculate an estimated value of the controlled quantity of the mobile body. The controlled quantity of the mobile body is, for example, the trajectory, speed, fuel consumption, etc. of the mobile body.

[0048] The control device 101 calculates an estimated value of the controlled quantity of the future controlled object from candidate values of the operation quantity for controlling the controlled quantity of the controlled object. The operation quantity for controlling the controlled quantity of the controlled object is, for example, the steering quantity, braking quantity, accelerator quantity, etc. The candidate values of the operation quantity are, for example, random values.

[0049] Specifically, for example, the control device 101 uses the second model to calculate an estimated value of the controlled quantity of the future controlled object from candidate values of the operation quantity for controlling the controlled quantity of the controlled object. Here, the second model is a model that predicts the dynamic behavior of the controlled object with the operation quantity of the controlled object as the input.

[0050] The second model is a mathematical model constructed based on physical laws such as equations of motion. For example, the second model takes the steering quantity, braking quantity, and accelerator quantity of the vehicle to be controlled as inputs and outputs an estimated value of the controlled quantity of the vehicle. Any existing model may be used as the second model.

[0051] The control device 101 determines the value of the operation quantity for controlling the controlled quantity of the controlled object based on the acquired target value of the controlled quantity of the controlled object, the calculated estimated value of the controlled quantity of the controlled object, and the calculated estimated value of the controlled quantity of the mobile body. Specifically, for example, the control device 101 calculates a control evaluation value for each candidate value of the operation quantity based on the estimated value of the controlled quantity of the controlled object, the target value of the controlled quantity of the controlled object, and the estimated value of the controlled quantity of the mobile body.

[0052] Here, the control evaluation value represents the cost for controlling the controlled quantity of the control target. The control evaluation value indicates, for example, that the lower the value, the higher the evaluation. Then, based on the calculated control evaluation value, the control device 101 determines the value of the operation amount for controlling the controlled quantity of the control target from among the candidate values of the operation amount.

[0053] More specifically, for example, the control device 101 uses the optimization solver sv to calculate the estimated value of the controlled quantity of the control target by using the converted moving body prediction model Mc while changing the candidate values of the operation amount. The optimization solver sv is software that solves problems, for example, using the dual-simplex algorithm. Next, the control device 101 calculates the control evaluation value based on, for example, the estimated value of the controlled quantity of the control target, the target value of the controlled quantity of the control target, and the estimated value of the controlled quantity of the moving body by using the evaluation function ef.

[0054] The evaluation function ef is a cost function that includes, for example, a term that considers the error between the estimated value of the controlled quantity (trajectory, speed) of the control target and the target value of the controlled quantity (trajectory, speed) of the control target, and a term that considers the positional relationship between the control target and the moving body. The positional relationship between the control target and the moving body is specified from the estimated value of the controlled quantity (trajectory, speed) of the control target and the estimated value of the controlled quantity (trajectory, speed) of the moving body. Each term may be normalized to appropriately compare the values of the terms with each other, or may be weighted according to the priority of each term.

[0055] Then, the control device 101 determines the candidate value of the operation amount when the control evaluation value (cost) is minimized as the value of the operation amount for controlling the controlled quantity of the control target in the next step. Thereby, for example, it is possible to determine the value of the operation amount for bringing the controlled quantity of the control target as close as possible to the target value while avoiding a collision with the moving body (obstacle).

[0056] The control device 101 outputs the determined value of the operation amount. Specifically, for example, the control device 101 outputs operation amount information 110 to the vehicle to be controlled as information on the operation amount when controlling the controlled amount in the following steps. Here, the operation amount information 110 is information indicating the determined value of the operation amount, and for example, indicates the values of the steering amount, the brake amount, and the accelerator amount.

[0057] In this way, according to the control device 101, it is possible to realize high-performance and high-speed autonomous driving control by using a model (the converted moving body prediction model Mc) capable of accurately predicting the dynamic behavior of the moving bodies around the control target. For example, according to the control device 101, it is possible to perform autonomous driving control that follows the trajectory and speed of the vehicle to be controlled as closely as possible to the target values while avoiding collisions with moving bodies (obstacles).

[0058] In addition, according to the control device 101, by formulating the moving body prediction model of the multi-layer neural network using a linear inequality function, the optimization problem can be solved analytically. Further, as activation functions that can be transformed into linear inequality functions, for example, there are ReLU, Hadlims, Satlins, tribas, and the like. By using the activation function with the ReLU structure among these activation functions for the moving body prediction model, it is possible to realize highly accurate prediction of the dynamic behavior of the moving body.

[0059] (Hardware configuration example of the control device 101) Next, a hardware configuration example of the control device 101 will be described.

[0060] FIG. 3 is a block diagram showing a hardware configuration example of the control device 101. In FIG. 3, the control device 101 includes a CPU (Central Processing Unit) 301, a memory 302, a disk drive 303, a disk 304, a communication I / F (Interface) 305, a portable recording medium I / F 306, and a portable recording medium 307. Further, each component is connected by a bus 300.

[0061] Here, the CPU 301 is in charge of the overall control of the control device 101. The CPU 301 may have a plurality of cores. The memory 302 has, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), and a flash ROM. Specifically, for example, the flash ROM stores the OS program, the ROM stores the application program, and the RAM is used as the work area of the CPU 301. The program stored in the memory 302 is loaded into the CPU 301 to cause the CPU 301 to execute the coded processing.

[0062] The disk drive 303 controls the read / write of data to / from the disk 304 according to the control of the CPU 301. The disk 304 stores the data written under the control of the disk drive 303. Examples of the disk 304 include a magnetic disk and an optical disk.

[0063] The communication I / F 305 is connected to the network 310 through a communication line and is connected to an external computer via the network 310. Then, the communication I / F 305 serves as the interface between the network 310 and the inside of the device and controls the input / output of data from / to the external computer. For the communication I / F 305, for example, a modem or a LAN adapter can be adopted.

[0064] The portable recording medium I / F 306 controls the read / write of data to / from the portable recording medium 307 according to the control of the CPU 301. The portable recording medium 307 stores the data written under the control of the portable recording medium I / F 306. Examples of the portable recording medium 307 include a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disk), and a USB (Universal Serial Bus) memory.

[0065] Note that, in addition to the above-described components, the control device 101 may have, for example, an input device, a display, etc. Further, the control device 101 may not have, for example, the disk drive 303, the disk 304, the portable recording medium I / F 306, and the portable recording medium 307 among the above-described components.

[0066] (Functional configuration example of control device 101) FIG. 4 is a block diagram showing a functional configuration example of the control device 101. In FIG. 4, the control device 101 includes an acquisition unit 401, a first estimation unit 402, a second estimation unit 403, an optimization unit 404, and an evaluation value calculation unit 405. The acquisition unit 401 to the evaluation value calculation unit 405 are functions that serve as a control unit. Specifically, for example, by causing the CPU 301 to execute a program stored in a storage device such as the memory 302, the disk 304, and the portable recording medium 307 shown in FIG. 3, or by means of the communication I / F 305, their functions are realized. The processing results of each functional unit are stored in a storage device such as the memory 302 and the disk 304, for example.

[0067] The acquisition unit 401 acquires information of sensors mounted on the control target. In the following description, "automobile (vehicle)" will be taken as an example of the control target for explanation. Further, "in-vehicle camera C" and "radar R" will be taken as examples of sensors mounted on the control target for explanation.

[0068] Specifically, for example, the acquisition unit 401 acquires image information from the in-vehicle camera C mounted on the host vehicle. The image information is, for example, information indicating an image captured by the in-vehicle camera C, and is, for example, a moving image (time-series data) for a certain period of time from the past to the current time. Further, the acquisition unit 401 acquires radar information from the radar R mounted on the host vehicle. The radar information is, for example, information indicating the direction and position of a moving object (obstacle) existing around the host vehicle measured by the radar R, and is, for example, the measurement result (time-series data) for a certain period of time.

[0069] In addition, the acquisition unit 401 acquires the target value of the controlled quantity of the host vehicle (the object to be controlled). The target value of the controlled quantity is, for example, the target trajectory, target speed, etc. of the host vehicle. The target trajectory of the host vehicle is the route that the host vehicle moves along during a period from the current time to the future (e.g., 5 seconds). The target speed of the host vehicle is the target speed when the host vehicle moves during a period from the current time to the future. The target speed may be, for example, the average speed, or may be the speed at regular intervals (such as 1 second).

[0070] Specifically, for example, the acquisition unit 401 acquires the target trajectory and target speed of the host vehicle from a navigation system (not shown) of the host vehicle. The navigation system is a system that supports the driver's driving by displaying the current position of the vehicle on a map, displaying the distance and direction to the destination, etc. In the navigation system, for example, the target trajectory and target speed of the host vehicle are calculated according to the road conditions on which the host vehicle travels.

[0071] The first estimation unit 402 uses the transformed moving object prediction model Mc to calculate an estimated value of the controlled quantity of a future moving object from the acquired image information of the in-vehicle camera C and the radar information of the radar R. The moving object is a moving object (obstacle) existing around the host vehicle. The controlled quantity of the moving object is, for example, the trajectory, speed, fuel consumption, etc. of the moving object.

[0072] The trajectory of the moving object is the route that the moving object moves along during a period from the current time to the future (e.g., 5 seconds). The speed of the moving object is the speed when the moving object moves during a period from the current time to the future. The fuel consumption of the moving object is the fuel consumption that the moving object consumes during a period from the current time to the future.

[0073] The transformed moving object prediction model Mc is obtained by transforming the moving object prediction model M (see, for example, FIGS. 2A and 2B). The moving object prediction model M has a multi-layer neuron structure and an activation function of the ReLU structure, and is a model for predicting the dynamic behavior of moving objects around the host vehicle.

[0074] The moving object prediction model M is, for example, a model that takes as input the image information (current value, past values) of the in-vehicle camera C and the radar information (current value, past values) of the radar R, and outputs an estimated value of the controlled quantity of a future moving object. The image information (current value, past values) and the radar information (current value, past values) are, for example, information for a certain period of time immediately before.

[0075] Based on the moving object prediction model M, the converted moving object prediction model Mc is a model in which the weight coefficients, biases of each neuron, and the upper and lower limit values of the input / output variables are set, and the activation function of each neuron is converted using a linear inequality function including binary variables. The moving object prediction model M is, for example, created in advance and stored in a storage device such as the memory 302 and the disk 304.

[0076] Specifically, for example, the first estimation unit 402 sets the weight coefficients and biases of each neuron of the converted moving object prediction model Mc as the weight coefficients and biases of each neuron of the moving object prediction model M before conversion, based on the moving object prediction model M. Also, the first estimation unit 402 sets the upper and lower limit values of the variables input to the converted moving object prediction model Mc as the upper and lower limit values of the variables input to the moving object prediction model M.

[0077] Also, the first estimation unit 402 sets the upper and lower limit values of the variables output from the converted moving object prediction model Mc as the upper and lower limit values of the variables output from the moving object prediction model M. Also, the first estimation unit 402 sets the upper and lower limit values of the variables output from each neuron (activation function) of the converted moving object prediction model Mc as the upper and lower limit values of the variables output from each neuron of the moving object prediction model M.

[0078] The upper and lower limit values of the input / output variables are set based on the maximum and minimum values of each input / output variable obtained, for example, by giving a Chirp signal or an APRBS signal as an input to the moving body prediction model M. A Chirp signal is a sine wave signal whose frequency component changes continuously according to time. An APRBS signal is a signal in which the amplitudes of rectangular waves are randomly combined. According to the Chirp signal and the APRBS signal, an exhaustive test pattern can be realized, and the upper and lower limit values of the input / output variables can be set accurately. Also, by appropriately limiting the range of each input / output variable, a high-speed and high-performance model (the converted moving body prediction model Mc) can be generated.

[0079] However, the conversion of the moving body prediction model M may be performed in the control device 101, or may be performed in another computer different from the control device 101. When it is performed by another computer, the control device 101 acquires the converted moving body prediction model Mc, for example, by a user's operation input or by receiving it from another computer.

[0080] The second estimation unit 403 calculates an estimated value of a controlled quantity of the host vehicle in the future from a search value of an operation amount for controlling the controlled quantity of the host vehicle. The controlled quantity of the host vehicle is, for example, the trajectory, speed, fuel consumption, etc. of the host vehicle. The trajectory of the host vehicle is the route along which the host vehicle moves during a period from the current time to the future. The speed of the host vehicle is the speed at which the host vehicle moves during a period from the current time to the future. The fuel consumption of the host vehicle is the fuel consumption consumed by the host vehicle during a period from the current time to the future.

[0081] The operation amount is, for example, a steering amount, a brake amount, an accelerator amount, etc. The search value of the operation amount corresponds to a candidate value of the operation amount for controlling the controlled quantity. The search value of the operation amount is searched, for example, according to a search rule predetermined by the optimization unit 404 and given from the optimization unit 404 to the second estimation unit 403. The estimated value of the controlled quantity of the host vehicle in the future represents, for example, the controlled quantity of the next step when controlling the host vehicle.

[0082] Specifically, for example, the second estimation unit 403 calculates an estimated value of a controlled quantity of the host vehicle in the future from a search value of an operation amount for controlling the controlled quantity of the host vehicle using a host vehicle model. Here, the host vehicle model is a model that predicts the dynamic behavior of the host vehicle with the operation amount of the host vehicle as an input.

[0083] The host vehicle model is a mathematical model constructed based on physical laws such as equations of motion. The host vehicle model corresponds to the "second model" described in FIG. 1. For example, the host vehicle model takes as inputs the steering amount, braking amount, and accelerator amount of the host vehicle, and outputs estimated values of the trajectory, speed (vehicle speed), and fuel consumption (fuel efficiency) of the host vehicle.

[0084] More specifically, for example, the second estimation unit 403 inputs the operation amount (past value) from k steps before (k is a natural number of 2 or more) to 1 step before and the search value (current value) of the operation amount at the current time into the host vehicle model, thereby calculating an estimated value of the controlled quantity (trajectory, speed, fuel consumption) of the host vehicle over a period from the current time to the future (for example, 5 seconds). The operation amount (past value) at each past step is stored in a storage device such as the memory 302 and the disk 304, for example.

[0085] The optimization unit 404 determines a value of the operation amount for controlling the controlled quantity of the host vehicle based on the target value of the controlled quantity of the host vehicle, the estimated value of the controlled quantity of the host vehicle, and the estimated value of the controlled quantity of the moving body (obstacle). Specifically, for example, the optimization unit 404 may determine the value of the operation amount based on the error between the estimated value of the controlled quantity (trajectory, speed) of the host vehicle and the target value of the controlled quantity (trajectory, speed) of the host vehicle. Thereby, the value of the operation amount can be determined in consideration of the followability to the target values (target trajectory, target vehicle speed) of the controlled quantity (trajectory, speed) of the host vehicle.

[0086] Further, the optimization unit 404 may determine the value of the operation amount based on, for example, the positional relationship between the host vehicle and the moving object, which is specified from the estimated value of the controlled amount (trajectory, speed) of the host vehicle and the estimated value of the controlled amount (trajectory, speed) of the moving object. Thereby, the value of the operation amount can be determined in consideration of the possibility of collision between the host vehicle and the moving object.

[0087] Further, the optimization unit 404 may determine the value of the operation amount based on, for example, the estimated value of the controlled amount (fuel consumption) of the host vehicle and the estimated value of the controlled amount (fuel consumption) of the host vehicle. Thereby, the value of the operation amount can be determined in consideration of the fuel consumption of the host vehicle and the moving object.

[0088] Further, the optimization unit 404 may determine the value of the operation amount based on, for example, the change amount of the operation amount. Here, the change amount of the operation amount is represented by, for example, the difference between the current value of the operation amount and the search value of the operation amount. Thereby, the value of the operation amount can be determined in consideration of the change amounts such as the steering amount, the brake amount, and the accelerator amount.

[0089] More specifically, for example, the optimization unit 404 determines, from among the search values of the operation amount, the value of the operation amount for controlling the controlled amount of the host vehicle based on the control evaluation value calculated by the evaluation value calculation unit 405 for each search value of the operation amount. The control evaluation value represents the cost for controlling the controlled amount of the host vehicle.

[0090] Here, the evaluation value calculation unit 405 calculates the control evaluation value based on the estimated value of the controlled amount of the host vehicle, the target value of the controlled amount of the host vehicle, and the estimated value of the controlled amount of the moving object, which are calculated for each search value of the operation amount. The control evaluation value indicates that, for example, the lower the value, the higher the evaluation.

[0091] Specifically, for example, the evaluation value calculation unit 405 may calculate a control evaluation value based on the error between the estimated value and the target value of the controlled quantity (trajectory, speed) of the host vehicle, and the positional relationship between the host vehicle and the moving object. Here, the error between the estimated value and the target value of the controlled quantity (trajectory, speed) is represented by, for example, the error between the position of the host vehicle based on the target trajectory and the target vehicle speed, and the position of the host vehicle based on the estimated values of the trajectory and speed of the host vehicle.

[0092] For example, each trajectory (target trajectory, estimated value of the trajectory) is set as the trajectory for 5 seconds from the current time. In this case, the evaluation value calculation unit 405 calculates the position of the host vehicle every 1 second based on the target trajectory and the target vehicle speed. Further, the evaluation value calculation unit 405 calculates the position of the host vehicle every 1 second based on the estimated values of the trajectory and speed of the host vehicle. Then, the evaluation value calculation unit 405 may calculate the error between the estimated value and the target value of the controlled quantity (trajectory, speed) by summing the errors between the positions every 1 second.

[0093] Further, the error between the estimated value and the target value of the controlled quantity (trajectory, speed) may be the sum of the difference between the estimated value of the trajectory and the target trajectory and the difference between the estimated value of the speed and the target vehicle speed. The difference between the estimated value of the trajectory and the target trajectory may be represented by, for example, the similarity of the shapes of the trajectories.

[0094] The positional relationship between the host vehicle and the moving object is represented by, for example, the presence or absence of an intersection between the estimated value of the trajectory of the host vehicle and the estimated value of the trajectory of the moving object. Thereby, the control evaluation value can be calculated in consideration of the possibility of collision between the host vehicle and the moving object due to the intersection of the trajectories.

[0095] Further, the positional relationship between the host vehicle and the moving object may be represented by the error between the position of the host vehicle based on the estimated values of the trajectory and speed of the host vehicle and the position of the moving object based on the estimated values of the trajectory and speed of the moving object. Thereby, the control evaluation value can be calculated in consideration of the possibility of collision due to the distance between the host vehicle and the moving object.

[0096] Further, the evaluation value calculation unit 405 may further calculate a control evaluation value based on an estimated value of the controlled quantity (fuel consumption) of the host vehicle and an estimated value of the controlled quantity (fuel consumption) of the moving object. Further, the evaluation value calculation unit 405 may further calculate a control evaluation value based on the change amount of the operation amount (the difference between the current value and the search value of the operation amount).

[0097] More specifically, for example, the evaluation value calculation unit 405 calculates a control evaluation value for each search value of the operation amount using the evaluation function ef. The evaluation function ef includes, for example, a first term C1, a second term C2, a third term C3, a fourth term C4, and a fifth term C5 as shown in the following formula (11), and is a cost function in which weights are assigned to each of the terms C1 to C5. Here, E is the control evaluation value. a, b, c, d, and e are weight coefficients.

[0098] E = a * C1 + b * C2 + c * C3 + d * C4 + e * C5 ··· (11)

[0099] Here, the first term C1 is a term that considers the error between the estimated value of the controlled quantity (trajectory, speed) of the host vehicle and the target value of the controlled quantity (trajectory, speed) of the host vehicle. The value of the first term C1 increases, for example, as the error between the estimated value and the target value of the controlled quantity (trajectory, speed) increases.

[0100] The second term C2 is a term that considers the presence or absence of an intersection between the estimated trajectory of the host vehicle and the estimated trajectory of the moving object. The value of the second term C2 is, for example, "1" when the trajectories of the host vehicle and the moving object intersect, and "0" when they do not intersect. The third term C3 is a term that considers the error between the position of the host vehicle based on the estimated trajectory and speed of the host vehicle and the position of the moving object based on the estimated trajectory and speed of the moving object. The value of the third term C3 increases, for example, as the error between the positions of the host vehicle and the moving object increases.

[0101] The fourth term C4 is a term that takes into account the estimated fuel consumption of the host vehicle and the estimated fuel consumption of the moving body. The value of the fourth term C4 increases, for example, as the fuel consumption of the host vehicle and the moving body increases. The fifth term C5 is a term that takes into account the difference between the current value of the operation amount and the search value of the operation amount. The value of the fifth term C5 increases, for example, as the change amount of the operation amount increases.

[0102] Each of the weight coefficients a to e can be arbitrarily set. For example, each of the weight coefficients a to e is set (normalized) to appropriately compare the values between the terms. Also, each of the weight coefficients a to e is set according to the priority of each term. For example, the larger the value of the weight coefficient b, the larger the control evaluation value when the trajectories of the host vehicle and the moving body intersect.

[0103] Then, the optimization unit 404 determines, for example, based on the calculated control evaluation value, the value of the operation amount for controlling the controlled amount from among the search values of the operation amount. More specifically, for example, the optimization unit 404 determines the search value with the minimum control evaluation value among the search values of the operation amount as the value of the operation amount for controlling the controlled amount of the host vehicle.

[0104] Note that the search for the operation amount by the optimization unit 404 may be performed, for example, until the change amount of the minimum control evaluation value becomes almost zero, or may be performed for a specified number of iterations.

[0105] Also, the optimization unit 404 outputs the determined value of the operation amount. Specifically, for example, the optimization unit 404 outputs the determined value of the operation amount as the operation amount for controlling the controlled amount in the next step to the autonomous driving control unit. The autonomous driving control unit is a control unit for controlling autonomous driving.

[0106] The autonomous driving control unit controls the host vehicle, for example, by adjusting the steering amount, brake amount, and accelerator amount of the host vehicle based on the value of the operation amount output from the control device 101 (optimization unit 404). The autonomous driving control unit is, for example, an ECU (Engine Control Unit).

[0107] Specifically, for example, in the autonomous driving control unit, according to the control of the controller, the operation amounts (steering amount, braking amount, accelerator amount) of the next step are adjusted to the values of the output operation amounts by various actuators and the like. As a result, the controlled amounts of the automobile (for example, trajectory, speed, fuel consumption) are controlled.

[0108] Also, the activation function of the ReLU structure included in the moving body prediction model M may be, for example, a Leaky ReLU structure. The Leaky ReLU structure is a function in which the output value changes linearly both when the input value to the function is 0 or less and when it is greater than 0, and the slope is different with 0 as the boundary. By using the activation function of the Leaky ReLU structure for the activation function included in the moving body prediction model M, the accuracy of predicting the dynamic behavior of the moving body can be improved.

[0109] Note that the control device 101 may be realized, for example, by the autonomous driving control unit of an automobile (control target). In this case, the control device 101 controls the host vehicle by adjusting the steering amount, braking amount, and accelerator amount of the host vehicle based on the determined operation amount value, for example. Also, the control device 101 may be realized by another computer that can communicate with the autonomous driving control unit of the automobile (control target). The other computer may be, for example, a server, or a smartphone, a PC (Personal Computer), or the like.

[0110] Also, the control device 101 may be realized, for example, by an application-specific IC such as a standard cell or a structured ASIC (Application Specific Integrated Circuit), or a PLD (Programmable Logic Device) such as an FPGA.

[0111] (Control processing procedure of control device 101) Next, the control processing procedure of the control device 101 will be described.

[0112] Figure 5 is a flowchart showing an example of the control processing procedure of the control device 101. In the flowchart of Figure 5, first, the control device 101 acquires image information from the in-vehicle camera C mounted on the host vehicle (step S501). Next, the control device 101 acquires radar information from the radar R mounted on the host vehicle (step S502).

[0113] Next, the control device 101 acquires the target value of the controlled quantity from the navigation system of the host vehicle (step S503). Then, the control device 101 calculates the estimated value of the controlled quantity of the future moving object from the acquired image information and radar information using the converted moving object prediction model Mc (step S504).

[0114] Next, the control device 101 executes an optimization process (step S505). The optimization process is a process of optimizing the operation amount for controlling the controlled quantity of the host vehicle. The specific processing procedure of the optimization process will be described later with reference to Figure 6.

[0115] Then, the control device 101 outputs the value of the operation amount, which is the temporary optimal value stored in the optimization process (step S506), and ends the series of processes according to this flowchart.

[0116] Next, with reference to Figure 6, the specific processing procedure of the optimization process in step S505 shown in Figure 5 will be described.

[0117] Figure 6 is a flowchart showing an example of the specific processing procedure of the optimization process. In the flowchart of Figure 6, first, the control device 101 acquires a search value of the operation amount for controlling the controlled quantity of the host vehicle (step S601). Then, the control device 101 calculates the estimated value of the controlled quantity of the future host vehicle from the acquired search value of the operation amount using the host vehicle model (step S602).

[0118] Next, the control device 101 calculates a control evaluation value using the evaluation function ef based on the estimated values of the controlled quantities (trajectory, speed, fuel consumption) of the host vehicle, the target values of the controlled quantities (trajectory, speed) of the host vehicle, the estimated values of the controlled quantities (trajectory, speed, fuel consumption) of the moving body, and the change amount of the operation amount (step S603). The change amount of the operation amount is represented by, for example, the difference between the current value of the operation amount and the search value of the operation amount.

[0119] Then, the control device 101 determines whether the calculated control evaluation value is smaller than the control evaluation value min (step S604). The control evaluation value min is "null" in the initial state. In step S604, when the control evaluation value min is in the initial state, the control device 101 proceeds to step S605.

[0120] Here, when the control evaluation value is greater than or equal to the control evaluation value min (step S604: No), the control device 101 proceeds to step S607. On the other hand, when the control evaluation value is smaller than the control evaluation value min (step S604: Yes), the control device 101 saves the obtained search value of the operation amount as a temporary optimal value (step S605).

[0121] Then, the control device 101 sets the calculated control evaluation value as the control evaluation value min (step S606). Next, the control device 101 determines whether to end the search for the operation amount (step S607). The search for the operation amount is performed, for example, until the change amount of the control evaluation value min becomes almost zero or for a specified number of iterations.

[0122] Here, when the search for the operation amount is not ended (step S607: No), the control device 101 returns to step S601 to obtain a new search value of the operation amount. On the other hand, when the search for the operation amount is ended (step S607: Yes), the control device 101 returns to the step where the optimization process was called.

[0123] As a result, the control device 101 can highly accurately and rapidly predict the dynamic behavior of a moving body by using the transformed moving body prediction model Mc obtained by transforming a moving body prediction model M having a ReLU-structured activation function by using a linear inequality function including binary variables. Further, the control device 101 can search for an operation amount for causing a controlled amount of the host vehicle to follow a target value in consideration of the dynamic behavior of the moving body.

[0124] As described above, according to the control device 101 according to the embodiment, image information of the in-vehicle camera C mounted on the host vehicle and radar information of the radar R are acquired, and an estimated value of a controlled amount of a future moving body can be calculated from the acquired image information and radar information by using the transformed moving body prediction model Mc. Further, according to the control device 101, an estimated value of a controlled amount of a future host vehicle can be calculated from a search value of an operation amount for controlling the controlled amount of the host vehicle. The estimated value of the controlled amount of the host vehicle is calculated, for example, from the search value of the operation amount by using a host vehicle model that predicts the dynamic behavior of the host vehicle with the operation amount of the host vehicle as an input. Further, according to the control device 101, a target value of the controlled amount of the host vehicle can be acquired. Then, according to the control device 101, a value of an operation amount for controlling the controlled amount of the host vehicle is determined based on the target value of the controlled amount of the host vehicle, the estimated value of the controlled amount of the host vehicle, and the estimated value of the controlled amount of the moving body, and the determined value of the operation amount can be output. The operation amount is, for example, a steering amount, a brake amount, and an accelerator amount.

[0125] As a result, the control device 101 can realize high-performance and high-speed autonomous driving control by using a model (transformed moving body prediction model Mc) capable of highly accurately predicting the dynamic behavior of a moving body (obstacle) around a control target. For example, the control device 101 can perform autonomous driving control for avoiding a collision with the moving body and causing the controlled amount of the host vehicle to follow the target value as much as possible.

[0126] Further, according to the control device 101, for each search value of the operation amount, a control evaluation value representing the cost for controlling the controlled amount of the host vehicle is calculated based on the target value of the controlled amount of the host vehicle, the estimated value of the controlled amount of the host vehicle, and the estimated value of the controlled amount of the moving object, and based on the calculated control evaluation value, an operation amount value for controlling the controlled amount of the host vehicle can be determined from among the search values of the operation amount.

[0127] Thereby, the control device 101 can determine, as the operation amount value for controlling the controlled amount of the host vehicle, a search value with a low cost for controlling the controlled amount of the host vehicle from among the search values of the operation amount.

[0128] Further, according to the control device 101, the control evaluation value can be calculated based on the error between the estimated value of the controlled amount (trajectory, speed) of the host vehicle and the target value of the controlled amount (trajectory, speed) of the host vehicle, and the positional relationship between the host vehicle and the moving object.

[0129] Thereby, the control device 101 can determine the operation amount value for controlling the controlled amount of the host vehicle in consideration of the followability of the controlled amount (trajectory, speed) of the host vehicle to the target value (target trajectory, target vehicle speed) and the possibility of collision between the host vehicle and the moving object.

[0130] Further, according to the control device 101, the control evaluation value can be calculated based on the presence or absence of an intersection between the estimated trajectory of the host vehicle and the estimated trajectory of the moving object.

[0131] Thereby, the control device 101 can determine the operation amount value for controlling the controlled amount of the host vehicle in consideration of the possibility of collision between the host vehicle and the moving object due to the intersection of the trajectories.

[0132] Further, according to the control device 101, the control evaluation value can be calculated based on the error between the position of the host vehicle based on the estimated trajectory and speed of the host vehicle and the position of the moving object based on the estimated trajectory and speed of the moving object.

[0133] As a result, the control device 101 can determine the value of the operation amount for controlling the controlled amount of the host vehicle in consideration of the possibility of collision depending on the distance between the host vehicle and the moving object.

[0134] Further, according to the control device 101, a control evaluation value can be calculated based on the estimated value of the controlled amount (fuel consumption) of the host vehicle and the estimated value of the controlled amount (fuel consumption) of the moving object.

[0135] As a result, the control device 101 can determine the value of the operation amount for controlling the controlled amount of the host vehicle so as to suppress the fuel consumption of the host vehicle and the moving object.

[0136] Further, according to the control device 101, a control evaluation value can be calculated based on the difference between the current value and the search value of the operation amount.

[0137] As a result, considering that there is a possibility that the actuator or the like may malfunction if the operation amount changes abruptly, the control device 101 can determine the value of the operation amount for controlling the controlled amount of the host vehicle so as to suppress a sharp change in the operation amount.

[0138] Further, according to the control device 101, the activation function included in the moving object prediction model M can be an activation function having a Leaky ReLU structure.

[0139] As a result, the control device 101 can improve the accuracy of predicting the dynamic behavior of the moving objects around the host vehicle.

[0140] From these facts, according to the control device 101 according to the embodiment, the trajectory during autonomous driving can be optimized to improve the control performance of the vehicle. Further, for example, by implementing the control device 101 in the autonomous driving control unit, it becomes possible to perform real-time control of the vehicle on board.

[0141] Note that the control method described in this embodiment can be realized by executing a pre-prepared program on a computer such as a personal computer or a workstation. This control program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, a DVD, or a USB memory, and is executed by being read from the recording medium by the computer. Further, this control program may be distributed via a network such as the Internet.

[0142] Regarding the above-described embodiment, the following additional remarks are disclosed.

[0143] (Supplementary Note 1) Based on a first model that has a multi-layer neuron structure and an activation function of the ReLU structure, and predicts the dynamic behavior of a moving body around the control target using the information of a sensor mounted on the control target, the weight coefficients, biases, and upper and lower limit values of the input / output variables of each neuron are set, and the activation function of each neuron is converted using a linear inequality function including binary variables. The converted first model, a control unit that acquires the information of the sensor and the target value of the controlled quantity of the control target, calculates an estimated value of the controlled quantity of the future moving body from the information of the sensor using the converted first model, calculates an estimated value of the controlled quantity of the future control target from candidate values of the operation amount for controlling the controlled quantity of the control target, and determines a value of the operation amount for controlling the controlled quantity of the control target based on the target value of the controlled quantity of the control target, the estimated value of the controlled quantity of the control target, and the estimated value of the controlled quantity of the moving body; A control device characterized by comprising the above.

[0144] (Supplementary Note 2) The estimated value of the controlled quantity of the control target is calculated from candidate values of the operation amount using a second model that predicts the dynamic behavior of the control target with the operation amount of the control target as an input. The control device according to Supplementary Note 1, characterized by this.

[0145] (Supplementary Note 3) The control unit For each candidate value of the operation amount, a control evaluation value representing a cost for controlling the controlled amount of the control target is calculated based on the estimated value of the controlled amount of the control target that has been calculated, the target value of the controlled amount of the control target, and the estimated value of the controlled amount of the moving body. Based on the calculated control evaluation value, a value of the operation amount for controlling the controlled amount of the control target is determined from among the candidate values of the operation amount. The control device according to appended note 1, characterized in that.

[0146] (Appended note 4) The controlled amount of the control target includes the trajectory and speed of the control target. The controlled amount of the moving body includes the trajectory and speed of the moving body. The control unit The control evaluation value is calculated based on the error between the estimated value of the controlled amount of the control target and the target value of the controlled amount of the control target, and the positional relationship between the control target and the moving body specified from the estimated value of the controlled amount of the control target and the estimated value of the controlled amount of the moving body. The control device according to appended note 3, characterized in that.

[0147] (Appended note 5) The positional relationship is represented by the presence or absence of an intersection between the estimated trajectory of the control target and the estimated trajectory of the moving body. The control device according to appended note 4, characterized in that.

[0148] (Appended note 6) The positional relationship is represented by the error between the position of the control target based on the estimated trajectory and speed of the control target and the position of the moving body based on the estimated trajectory and speed of the moving body. The control device according to appended note 4, characterized in that.

[0149] (Appended note 7) The controlled amount of the control target includes the fuel consumption of the control target. The controlled amount of the moving body includes the fuel consumption of the moving body. The control unit Furthermore, the control evaluation value is calculated based on the estimated value of the fuel consumption of the control target and the estimated value of the fuel consumption of the moving body. The control device according to appended note 4, characterized in that.

[0150] (Appendix 8) The control unit further calculates the control evaluation value based on the difference between the current value of the operation amount and the candidate value of the operation amount. The control device according to Appendix 4 is characterized by this.

[0151] (Appendix 9) The operation amount is a steering amount, a brake amount, and an accelerator amount. The control device according to Appendix 1 is characterized by this.

[0152] (Appendix 10) The control unit outputs the determined value of the operation amount. The control device according to Appendix 1 is characterized by this.

[0153] (Appendix 11) The information of the sensor includes the image information of the camera mounted on the control target. The control device according to Appendix 1 is characterized by this.

[0154] (Appendix 12) The information of the sensor includes at least any one of the information of the LiDAR, millimeter wave radar, and ultrasonic sensor mounted on the control target. The control device according to Appendix 1 is characterized by this.

[0155] (Appendix 13) The upper and lower limit values of the input / output variables are set based on the maximum value and the minimum value of the input / output variables obtained by inputting a Chirp signal or an APRBS signal to the first model. The control device according to Appendix 1 is characterized by this.

[0156] (Appendix 14) The activation function has a Leaky ReLU structure. The control device according to Appendix 1 is characterized by this.

[0157] (Appendix 15) The target value of the controlled quantity of the control target is obtained from the navigation system mounted on the control target. The control device according to Appendix 1 is characterized by this.

[0158] (Appendix 16) Obtain the information of the sensor mounted on the control target and the target value of the controlled quantity of the control target, Based on a first model that has a multi-layer neuron structure and an activation function with a ReLU structure, and predicts the dynamic behavior of a moving object around the control target using the information of the sensor as input, the weight coefficients, biases, and upper and lower limit values of the input and output variables of each neuron are set, and the activation function of each neuron is converted using a linear inequality function including binary variables. Using the converted first model, an estimated value of the controlled quantity of the moving object in the future is calculated from the acquired sensor information. An estimated value of the controlled quantity of the control target in the future is calculated from candidate values of the operation quantity for controlling the controlled quantity of the control target. Based on the acquired target value of the controlled quantity of the control target, the calculated estimated value of the controlled quantity of the control target, and the calculated estimated value of the controlled quantity of the moving object, a value of the operation quantity for controlling the controlled quantity of the control target is determined. A control method characterized in that a computer executes the processing.

[0159] (Appendix 17) Information of a sensor mounted on a control target and a target value of the controlled quantity of the control target are acquired. Based on a first model that has a multi-layer neuron structure and an activation function with a ReLU structure, and predicts the dynamic behavior of a moving object around the control target using the information of the sensor as input, the weight coefficients, biases, and upper and lower limit values of the input and output variables of each neuron are set, and the activation function of each neuron is converted using a linear inequality function including binary variables. Using the converted first model, an estimated value of the controlled quantity of the moving object in the future is calculated from the acquired sensor information. An estimated value of the controlled quantity of the control target in the future is calculated from candidate values of the operation quantity for controlling the controlled quantity of the control target. Based on the acquired target value of the controlled quantity of the control target, the calculated estimated value of the controlled quantity of the control target, and the calculated estimated value of the controlled quantity of the moving object, a value of the operation quantity for controlling the controlled quantity of the control target is determined. A control program characterized in that the processing is executed by a computer.

Explanation of Signs

[0160] 101 Control device 110 Operation amount information 200, 210, M Moving body prediction model 220, Mc Transformed moving body prediction model 300 Bus 301 CPU 302 Memory 303 Disk drive 304 Disk 305 Communication I / F 306 Portable recording medium I / F 307 Portable recording medium 310 Network 401 Acquisition unit 402 First estimation unit 403 Second estimation unit 404 Optimization unit 405 Evaluation value calculation unit C In-vehicle camera R Radar

Claims

1. Based on a first model that has a multi-layer neuron structure and an activation function of the ReLU structure, and predicts the dynamic behavior of a moving object around the control target using the information of sensors mounted on the control target as input, the weight coefficients, biases, and upper and lower limit values of input and output variables of each neuron are set, and the activation function of each neuron is converted using a linear inequality function including binary variables. The converted first model, obtains the information of the sensor and the target value of the controlled quantity of the control target, uses the converted first model to calculate an estimated value of the controlled quantity of the future moving object from the information of the sensor, and calculates an estimated value of the controlled quantity of the future control target from candidate values of the operation quantity for controlling the controlled quantity of the control target. A control unit that determines a value of the operation quantity for controlling the controlled quantity of the control target based on the target value of the controlled quantity of the control target, the estimated value of the controlled quantity of the control target, and the estimated value of the controlled quantity of the moving object, A control device characterized by comprising:

2. The estimated value of the controlled quantity of the control target is calculated from candidate values of the operation quantity using a second model that predicts the dynamic behavior of the control target with the operation quantity of the control target as input. The control device according to claim 1, characterized in that

3. The control unit For each candidate value of the operation quantity, calculates a control evaluation value representing the cost for controlling the controlled quantity of the control target based on the calculated estimated value of the controlled quantity of the control target, the target value of the controlled quantity of the control target, and the estimated value of the controlled quantity of the moving object, Based on the calculated control evaluation value, determines a value of the operation quantity for controlling the controlled quantity of the control target from among the candidate values of the operation quantity. The control device according to claim 1, characterized in that

4. The controlled quantity of the control target includes the trajectory and speed of the control target, The controlled quantity of the moving object includes the trajectory and speed of the moving object, The control unit Calculates the control evaluation value based on the error between the estimated value of the controlled quantity of the control target and the target value of the controlled quantity of the control target, and the positional relationship between the control target and the moving object specified from the estimated value of the controlled quantity of the control target and the estimated value of the controlled quantity of the moving object. The control device according to claim 3, characterized in that

5. The controlled quantity of the control target includes the fuel consumption of the control target, The controlled quantity of the moving object includes the fuel consumption of the moving object, The control unit further calculates the control evaluation value based on an estimated value of the fuel consumption of the control target and an estimated value of the fuel consumption of the moving body, The control device according to claim 4, characterized in that. **Claim 6** The control unit further calculates the control evaluation value based on a difference between the current value of the operation amount and a candidate value of the operation amount, The control device according to claim 4, characterized in that. **Claim 7** The operation amount is a steering amount, a brake amount, and an accelerator amount, The control device according to claim 1, characterized in that. **Claim 8** The control unit outputs the determined value of the operation amount, The control device according to claim 1, characterized in that. **Claim 9** acquires information of a sensor mounted on a control target and a target value of a controlled amount of the control target, Based on a first model having a multi-layer neuron structure and an activation function of ReLU structure, predicting the dynamic behavior of a moving body around the control target with the information of the sensor as an input, the weight coefficient, bias and upper and lower limit values of the input / output variables of each neuron are set, using a linear inequality function including binary variables to calculate an estimated value of a future controlled amount of the moving body from the acquired sensor information using the transformed first model in which the activation function of each neuron is transformed, calculates an estimated value of a future controlled amount of the control target from candidate values of an operation amount for controlling the controlled amount of the control target, determines a value of an operation amount for controlling the controlled amount of the control target based on the acquired target value of the controlled amount of the control target, the calculated estimated value of the controlled amount of the control target, and the calculated estimated value of the controlled amount of the moving body, A control method characterized in that a computer executes the process. **Claim 10** acquires information of a sensor mounted on a control target and a target value of a controlled amount of the control target, Based on a first model having a multi-layer neuron structure and an activation function of ReLU structure, predicting the dynamic behavior of a moving body around the control target with the information of the sensor as an input, the weight coefficient, bias and upper and lower limit values of the input / output variables of each neuron are set, using a linear inequality function including binary variables to calculate an estimated value of a future controlled amount of the moving body from the acquired sensor information using the transformed first model in which the activation function of each neuron is transformed, calculates an estimated value of a future controlled amount of the control target from candidate values of an operation amount for controlling the controlled amount of the control target, Based on the target value of the controlled quantity of the controlled object obtained, the estimated value of the controlled quantity of the controlled object calculated, and the estimated value of the controlled quantity of the moving body calculated, determine the value of the operation amount for controlling the controlled quantity of the controlled object. A control program characterized by causing a computer to execute the process.

Images