Remote control device, remote control method, remote control system, and moving object

Abstract

The present disclosure relates to a remote control device for controlling at least one moving body via a transmission path including at least a network, the remote control device comprising: a transmission delay distribution estimation unit for estimating a transmission delay distribution; the transmission delay distribution estimation unit estimates transmission delay distribution information including a time-varying probability distribution of the estimated transmission delay on the basis of transmission delay information in a transmission path acquired beforehand and transmission delay information acquired online. A trajectory generation unit that generates a target trajectory for the at least one moving body on the basis of the surroundings information about the surroundings of the at least one moving body; and a moving body control unit that generates a control amount for at least one moving body on the basis of the transfer delay distribution information, the target trajectory, and the moving body information, the moving body control unit comprising: a gain setting unit that sets a gain on the basis of the transfer delay distribution information; and a control amount calculation unit that generates a control amount on the basis of the target trajectory, the control gain, and the moving object information.

Description

Technical Field

[0001] The present disclosure relates to a remote control device that controls one or more moving objects via a network, and to a remote control device that takes transmission delay into consideration. Background Art

[0002] In recent years, the development of remote control devices that realize automatic driving such as valet parking and automatic transportation by sending and receiving data with a mobile body at a remote end has been developed. Wireless communication and a network composed of the Internet are used for sending and receiving data, but in this case, transmission delay occurs when sending and receiving data due to the distance between the remote control device and the mobile body and obstacles. When trying to control the mobile body in this environment, there is a possibility that the mobile body will fall into an unstable state.

[0003] Patent Document 1 discloses a method of stably controlling a moving object by regarding transmission delay as a probability variable and designing a control gain based on the probability distribution of the transmission delay.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent No. 6940036

[0007] Non-patent literature

[0008] Non-patent document 1: Yohei Hosoe et al. "On second moment stability of discrete-time linear systems with general stochastic dynamics", IEEE Trans. Automatic Control, 2021

[0009] Non-Patent Literature 2: Daiki Takita, Yohei Hosoe, Tomomichi Hagiwara, "Stabilization of Discrete-Time Linear Systems with Determined Dynamic Characteristics by Hidden Markov Models", 9th Multi-Specialty Symposium of the Control Division of the Japan Society for Measurement and Automation Control, 2022 (Daiki Takita, Yohei Hosoe, Tomomichi Hagiwara, "Stabilization of Discrete-Time Linear Systems with Determined Dynamic Characteristics by Hidden Markov Models", 9th Multi-Specialty Symposium of the Control Division of the Japan Society for Measurement and Automation Control, 2022) Summary of the invention

[0010] However, in Patent Document 1, the control gain is designed under the assumption that the probability distribution of the transmission delay does not change with respect to time, that is, it is invariant with respect to time (time invariant), and the mode of the value has no time dependence. This is the assumption that the transmission delay follows an independent and identical distribution ("Independent and identical distribution", hereinafter abbreviated as "iid") with respect to the time in mathematics, and the mathematical treatment is relatively simple, so there is an advantage that the design of the control gain becomes simple. In Patent Document 1, when the transmission delay meets this condition, stability can be guaranteed.

[0011] When the probability distribution followed by each probability variable, that is, the surrounding probabilities are the same and they are independent, the probability variables can be said to follow independent and identical distributions, but there is no such distribution as independent and identical distributions.

[0012] Such an assumption is valid in a small-scale network, that is, when the frequency of change of packet path control is small and when the network users are small, but when using a large-scale network such as the Internet, the packet path is frequently changed. In this case, the assumption that the probability distribution changes with time (hereinafter sometimes described as "time-varying"), that is, becomes iid with respect to the time, sometimes does not hold.

[0013] In addition, Patent Document 1 states that “the distribution of transmission delays can be estimated and can be estimated online while being remotely controlled”, but a specific method is not shown.

[0014] An object of the present disclosure is to provide a remote control device that suppresses unstable behavior of a mobile object even in a large-scale network environment.

[0015] The present disclosure provides a remote control device that controls at least one moving body via a transmission path that includes at least a network, and the remote control device includes: a transmission delay distribution estimation unit that estimates transmission delay distribution information including a time-varying probability distribution of the estimated transmission delay based on transmission delay information in the transmission path obtained in advance and the transmission delay information obtained online; a trajectory generation unit that generates a target trajectory of the at least one moving body based on surrounding information around the at least one moving body; and a moving body control unit that generates a control amount of the at least one moving body based on the transmission delay distribution information obtained from the transmission delay distribution estimation unit, the target trajectory obtained from the trajectory generation unit, and the moving body information obtained from the at least one moving body, the moving body control unit including: a gain setting unit that sets a control gain based on the transmission delay distribution information; and a control amount calculation unit that generates the control amount based on the target trajectory, the control gain, and the moving body information.

[0016] According to the remote control device according to the present disclosure, it is possible to realize remote control of a moving object while suppressing unstable behavior even in an environment with transmission delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a block diagram showing the configuration of a remote control device and a remote control system according to Embodiment 1 of the present disclosure.

[0018] Figure 2 This is a block diagram showing the configuration of a remote control device and a remote control system according to Embodiment 1 of the present disclosure.

[0019] Figure 3 It is a block diagram showing the structure of the first moving body control unit.

[0020] Figure 4 It is a block diagram showing the structure of the first moving body control unit.

[0021] Figure 5 It is a block diagram showing the structure of the first moving object.

[0022] Figure 6 This is a diagram showing an example of a configuration in which the first moving object is a vehicle.

[0023] Figure 7 It is a diagram showing an example of the configuration of the object information acquisition unit.

[0024] Figure 8 It is a diagram showing a target path for the first moving body to avoid a stationary object.

[0025] Fig. 9It is a diagram showing an example of the arrangement of the object information acquisition unit and the environment information acquisition unit.

[0026] Fig.10 This is a diagram showing an example of a target speed generated in the remote control device according to Embodiment 1 of the present disclosure.

[0027] Fig.11 This is a diagram showing an example of a method of generating target trajectories for two or more moving objects in a trajectory generating unit.

[0028] Fig.12 This is a diagram showing an example of a method of generating target trajectories for two or more moving objects in a trajectory generating unit.

[0029] Fig.13 This is a block diagram showing an example of the configuration of a transmission delay distribution estimation unit.

[0030] Fig.14 It is a diagram showing an example of a time series of transmission delay.

[0031] Fig.15 This is a diagram showing an example of modeling using HMM.

[0032] Fig.16 This is a block diagram showing an example of a control system in which the remote control device according to the first embodiment of the present disclosure controls a first moving object in a transmission delay environment.

[0033] Fig.17 This is a diagram showing an example of a target path when the first moving object is a vehicle.

[0034] Fig.18 This is a block diagram showing the configuration of a remote control device and a remote control system according to Embodiment 2 of the present disclosure.

[0035] Fig.19 This is a block diagram showing an example of the configuration of a transmission delay distribution estimation unit.

[0036] Fig. 20 This is a block diagram showing an example of the configuration of a transmission delay distribution estimation unit of a remote control device according to Embodiment 3 of the present disclosure.

[0037] Fig.21 : is a diagram showing a hardware configuration for realizing a remote control device.

[0038] Fig. 22 : is a diagram showing a hardware configuration for realizing a remote control device. DETAILED DESCRIPTION

[0039] <Implementation Method 1>

[0040] <Overall Structure>

[0041] Figure 1 This is a block diagram showing an example of the configuration of the remote control device 1000 according to the first embodiment of the present disclosure and the configuration of a remote control system RCS1 of a mobile object MV that is remotely controlled via a network NW.

[0042] like Figure 1 As shown, the remote control system RCS1 is configured such that a mobile object MV, a remote control device 1000 , an object information acquisition unit 200 , and an environment information acquisition unit 300 are connected to a network NW. In addition, a map database is connected to the remote control device 1000 .

[0043] The network NW can connect a plurality of components to each other by cables and radio waves, etc., and transmit and receive data. The network NW is composed of LAN (Local Area Network), WAN (Wide Area Network), Internet, telephone lines, wireless communications, etc., but the network NW is not limited to these, and any example can be used as long as it is a medium that can transmit and receive data between a remote control device and a mobile object at a remote end.

[0044] Regarding the moving body MV, in the present first embodiment, one moving body is used as the control object, but in order to distinguish it from the case where multiple moving bodies are controlled, it is called the first moving body 100. The first moving body 100 moves according to the control amount sent from the sending unit 1004 of the remote control device 1000, and outputs the state amount of the moving body detected by the built-in sensor (described later) composed of the speed sensor etc. as the state information of the first moving body 100, that is, the moving body 1 information. The structure of the first moving body 100 will be described later. Figure 4 Detailed description.

[0045] The object information acquisition unit 200 is composed of one or more sensors mounted around the first moving body 100 or on the first moving body 100. When the moving body is a car and is traveling on a road, the object information acquisition unit 200 is installed at, for example, a traffic light, a utility pole, or a light at an intersection. In addition, there are also cases where it is installed on the side of the road. In the case of other moving bodies, such as a moving body moving indoors, the object information acquisition unit is sometimes installed on the ceiling and the wall. The object information acquisition unit 200 acquires the position and speed of obstacles such as other vehicles, bicycles, and pedestrians around the first moving body 100 as object information. In addition, the object information acquisition unit 200 can acquire the position and speed of the first moving body 100 itself as moving body information. In this case, the moving body information is a part of the object information. The object information acquisition unit 200 sends the moving body information to the receiving unit 1012 in the remote control device 1000 via the network NW. In addition, when a built-in sensor is installed on the first moving body 100, the moving body information can also be acquired from the built-in sensor. In this case, the moving body information corresponds to the moving body 1 information. Therefore, the moving body information can be acquired from the object information acquisition unit 200 or the first moving body 100.

[0046] The object information acquisition unit 200 includes a time synchronization unit 201. The time synchronization unit 201 has a function of synchronizing the timing of data transmission and reception in cooperation with a time synchronization unit (not shown) in the first moving body 100, a time synchronization unit 310 in the environment information acquisition unit 300, and a time synchronization unit 1011 in the remote control device 1000.

[0047] Each of the above-mentioned time synchronization units can be synchronized by using a GNSS (Global Navigation Satellite System) sensor when outdoors. GNSS is a global time synchronization system and is a well-known technology, so it is easy to synchronize time using GNSS. On the other hand, when indoors, time synchronization can be performed by accessing an NTP (Network Time Protocol) server set on the network NW.

[0048] The environmental information acquisition unit 300 is composed of one or more sensors arranged around the first moving body 100, similarly to the object information acquisition unit 200. The environmental information acquisition unit 300 is also arranged inside or outside the house. The environmental information acquisition unit 300 acquires environmental information such as traffic lights and stop lines. The environmental information acquisition unit 300 sends the environmental information to the receiving unit 1012 in the remote control device 1000 via the network NW. In addition, the environmental information can also be acquired by the object information acquisition unit 200. In all embodiments, the object information and the environmental information are combined as the surrounding information. However, for example, in the case where the moving body is a robot, the surrounding information can also include only the object information without including the environmental information. In addition, the sensor used in the environmental information acquisition unit 300 can also be mounted on the first moving body 100.

[0049] The environment information acquisition unit 300 includes a time synchronization unit 301. The time synchronization unit 301 has a function of synchronizing the timing of data transmission and reception in cooperation with a time synchronization unit (not shown) in the first moving body 100, a time synchronization unit 201 in the object information acquisition unit 200, and a time synchronization unit 1011 in the remote control device 1000.

[0050] Examples of sensors used in the object information acquisition unit 200 and the environment information acquisition unit 300 include cameras, LiDAR (Light Detection And Ranging), radars, and the like.

[0051] The camera is installed at a position capable of capturing images of the front, sides, and rear, and acquires, from the captured images, for example, the demarcation lines around the first moving body 100 and the position and speed of obstacles.

[0052] LiDAR detects the position of an object by detecting the time difference between irradiating laser light to the surrounding area and reflecting it from the surrounding objects and returning.

[0053] The radar irradiates the surroundings with radar light and detects the reflected waves, thereby measuring the relative distance and relative speed of obstacles in the surroundings to the radar and outputting the measurement results.

[0054] In addition, in the case where each obstacle is equipped with a GNSS sensor capable of detecting the absolute position of obstacles around the first mobile body 100, and in the case where the first mobile body 100 is equipped with a GNSS sensor capable of detecting the absolute position of obstacles around the first mobile body 100, and in the case where the GNSS sensor can send the absolute position information to the remote control device 1000 via the network NW, the object information can be detected by GNSS, so the object information acquisition unit 200 can be omitted.

[0055] The map database 500 stores map data of the surroundings of the first moving object 100. Figure 1 In the embodiment, the track generation unit 1002 is connected to the map database 500, but the present invention is not limited thereto, and each component in the remote control device 1000 can access the map database 500. When the first moving body 100 is a vehicle, the map database 500 often includes data related to driving, such as the center coordinate information of the road, the information of the stop line, the information of the white line, and the drivable area.

[0056] <Remote Control Device>

[0057] Next, the components of the remote control device 1000 are described. Figure 1 As shown, the remote control device 1000 includes a transmission delay distribution estimation unit 1001 , a trajectory generation unit 1002 , a moving body control unit 1003 , a transmission unit 1004 , a time synchronization unit 1011 , a reception unit 1012 , and a transmission delay measurement unit 1013 .

[0058] The time synchronization unit 1011 has a function of synchronizing the timing of data transmission and reception in cooperation with a time synchronization unit (not shown) in the first moving body 100 , the time synchronization unit 201 in the object information acquisition unit 200 , and the time synchronization unit 301 in the environment information acquisition unit 300 .

[0059] The receiving unit 1012 receives object information from the object information acquisition unit 200, environmental information from the environmental information acquisition unit 300, and mobile body 1 information from the first mobile body 100. In addition, the surrounding information is information that combines the object information and the environmental information as described above, and is recorded as surrounding information in the figure. The mobile body information includes a first state quantity, a second state quantity, and time information. The first state quantity is a state quantity such as the position and speed, acceleration, angular velocity, etc. of the first mobile body 100 obtained by using a sensor. The second state quantity is a state quantity that cannot be obtained by a sensor and can be estimated by a state estimation unit described later. The time information includes, for example, the time synchronized by the time synchronization unit 1011 and information used for time synchronization processing.

[0060] The trajectory generation unit 1002 generates a target trajectory of the first moving body 100, that is, a moving body 1 target trajectory, based on the map data of the surroundings of the first moving body 100 obtained from the map database 500 and the surrounding information obtained via the network NW. Here, the target trajectory can be set as information that combines the target path and the target speed. Alternatively, the target trajectory can be set as information that combines the target path and the target position. In addition, it is not limited to the target speed or the target position, and anything that is a state quantity of the first moving body 100 can be combined with the target path. In addition, the trajectory generation unit 1002 can also generate the target trajectory based only on the surrounding information. The method for the trajectory generation unit 1002 to generate the target trajectory will be described later. Figure 13 to Figure 15 Detailed description.

[0061] The mobile body control unit 1003 includes a first mobile body control unit 1031. The first mobile body control unit 1031 calculates a control amount for making the first mobile body 100 follow the target track based on the mobile body information obtained from the network NW via the receiving unit 1012 and the mobile body 1 target track obtained from the track generating unit 1002. When the first mobile body 100 is a vehicle, the control amount is, for example, a target steering amount and a target acceleration / deceleration amount, and is output to the network NW via the sending unit 1004 as the mobile body 1 control amount. In addition, regarding the first mobile body control unit 1031, the control amount is used later. Figure 3 , Figure 4 Detailed description.

[0062] The transmission delay measuring unit 1013 measures the transmission delay generated between the first mobile body 100 and the remote control device 1000, i.e., the transmission delay time, using the time synchronized by the time synchronization unit 1011, and outputs it to the transmission delay distribution estimating unit 1001 as the transmission delay information of the first mobile body 100, i.e., the mobile body 1 transmission delay information. The transmission delay time can be obtained based on the difference between the transmission time included in the mobile body 1 information output from the first mobile body 100 and the reception time when the mobile body 1 information is received by the remote control device 1000.

[0063] In addition, when the time synchronization unit is not provided on the remote control device 1000 and the first mobile body 100, the transmission delay can be measured as described below. That is, first, a packet is sent from the remote control device 1000 to the first mobile body 100, and the time is recorded at the same time. When the first mobile body 100 receives the packet, it sends it to the remote control device 1000 at the same time, and the transmission delay can be calculated based on the difference between the time received by the remote control device 1000 and the time sent. The transmission delay calculated in this way can be called RTT (Round Trip Time). Similarly, if the time is recorded on the first mobile body 100 side, the RTT observed from the first mobile body 100 can also be calculated.

[0064] The transmission delay distribution estimation unit 1001 uses the transmission delay information from the transmission delay measurement unit 1013 to output the transmission delay distribution information related to the first moving body 100, that is, the transmission delay distribution information of the moving body 1. The transmission delay distribution information refers to information estimated from a transmission delay model such as a transmission delay pattern in addition to the probability distribution of the transmission delay. The structure and operation of the transmission delay distribution estimation unit 1001 will be described later. Figure 6 To explain.

[0065] The transmission unit 1004 transmits the moving object 1 control amount from the first moving object control unit 1031 to the first moving object 100 via the network NW.

[0066] <Control of Multiple Moving Objects>

[0067] Figure 2 This is a block diagram showing an example of the configuration of the remote control device 1000 in the case of controlling two or more moving objects, and the configuration of a remote control system RCS1A for the moving object MV which is remotely controlled via the network NW.

[0068] like Figure 2 As shown, the remote control system RCS1A has a configuration in which a mobile object MV, a remote control device 1000 , an object information acquisition unit 200 , and an environment information acquisition unit 300 are connected to a network NW.

[0069] In the remote control device 1000, in order to control a plurality of moving bodies, the moving body control unit 1003 includes a first moving body control unit 1031 and a second moving body control unit 1032. The moving bodies MV to be controlled are the first moving body 100 and the second moving body 101. Figure 2 About and use Figure 1 The same components as those of the remote control device 1000 described above are denoted by the same reference numerals, and repeated descriptions are omitted.

[0070] The first moving body 100 and the second moving body 101 move respectively according to the moving body 1 control amount and the moving body 2 control amount sent from the sending unit 1004 of the remote control device 1000, and the state quantity of the moving body detected by the built-in sensors composed of speed sensors and the like respectively carried by the first moving body 100 and the second moving body 101 is output as moving body 1 information and moving body 2 information.

[0071] The object information acquisition unit 200 is composed of one or more sensors mounted around the first moving body 100 and the second moving body 101 or on the first moving body 100 and the second moving body 101. When the moving body is a car and is traveling on a road, the object information acquisition unit 200 is installed at, for example, a traffic light, a telephone pole, a lamp, etc. at an intersection. In addition, there are also cases where it is installed on the side of the road. In the case of other moving bodies, such as a moving body moving indoors, the object information acquisition unit is sometimes installed on the ceiling and the wall. The object information acquisition unit 200 acquires the position and speed of obstacles such as other vehicles, bicycles, and pedestrians around the first moving body 100 and the second moving body 101 as object information. In addition, the object information acquisition unit 200 can acquire the position and speed of the first moving body 100 itself as moving body information, and can acquire the position and speed of the second moving body 101 itself as moving body information. In this case, the moving body information is a part of the object information. The object information acquisition unit 200 transmits the moving body information to the receiving unit 1012 in the remote control device 2000 via the network NW. In addition, when the first moving body 100 is provided with a built-in sensor, the moving body information can also be obtained from the built-in sensor, and when the second moving body 101 is provided with a built-in sensor, the moving body information can also be obtained from the built-in sensor. In this case, the moving body information is equivalent to the moving body 1 information and the moving body 2 information. Therefore, the moving body information can be obtained from the object information acquisition unit 200, and can also be obtained from the first moving body 100 and the second moving body 101.

[0072] The environment information acquisition unit 300 is composed of one or more sensors installed around the first moving body 100 and one or more sensors installed around the second moving body 101, similarly to the object information acquisition unit 200. The environment information acquisition unit 300 acquires environment information such as traffic lights and stop lines. The environment information acquisition unit 300 transmits the environment information to the receiving unit 1012 in the remote control device 2000 via the network NW.

[0073] <Remote Control Device>

[0074] Next, the components of the remote control device 1000 are described. Figure 2 As shown, the remote control device 1000 includes a transmission delay distribution estimation unit 1001 , a trajectory generation unit 1002 , a moving body control unit 1003 , a transmission unit 1004 , a time synchronization unit 1011 , a reception unit 1012 , and a transmission delay measurement unit 1013 .

[0075] They have Figure 1The remote control device 1000 shown in the figure has the same functions, but the transmission delay measuring unit 1013 uses the time synchronized by the time synchronization unit 1011 to measure the various transmission delays generated between the first mobile body 100 and the second mobile body 101 and the remote control device 1000, and outputs them to the transmission delay distribution estimation unit 1001 as the mobile body 1 transmission delay information of the first mobile body 100 and the mobile body 2 transmission delay information of the second mobile body 101.

[0076] In addition, the moving body control unit 1003 includes a first moving body control unit 1031 and a second moving body control unit 1032. The first moving body control unit 1031 calculates a control amount for making the first moving body 100 follow the target orbit, that is, a moving body 1 control amount, based on the moving body 1 information obtained from the network NW via the receiving unit 1012 and the moving body 1 target orbit obtained from the orbit generating unit 1002. The second moving body control unit 1032 calculates a control amount for making the second moving body 101 follow the target orbit, that is, a moving body 2 control amount, based on the moving body 2 information obtained from the network NW via the receiving unit 1012 and the moving body 2 target orbit obtained from the orbit generating unit 1002. In addition, when there are three or more moving bodies, the moving body control unit 1003 is configured to be additionally provided with a third moving body control unit and the like corresponding to the number of moving bodies.

[0077] The transmission delay profile estimation unit 1001 uses the transmission delay information of the mobile body 1 and the transmission delay information of the mobile body 2 from the transmission delay measurement unit 1013 to output the transmission delay profile information of the mobile body 1 related to the first mobile body 100 and the transmission delay profile information of the mobile body 2 related to the second mobile body 101 .

[0078] Furthermore, when the network environment and surrounding conditions of the plurality of moving bodies are substantially the same and the tendency of the transmission delay is considered to be the same, the transmission delay distribution estimation unit 1001 considers the transmission delay information of the first moving body 100 and the transmission delay information of the second moving body 101 to be the same, groups the first moving body 100 and the second moving body 101, and estimates a common transmission delay distribution using the transmission delay information of either one of them and outputs the estimated distribution to the trajectory generation unit 1002. The same is true when there are three or more moving bodies.

[0079] This makes it sufficient to calculate the estimated transmission delay distribution only once, thereby reducing the calculation load.

[0080] The receiving unit 1012 receives the moving body information from the object information acquisition unit 200, the environmental information from the environmental information acquisition unit 300, the moving body 1 information from the first moving body 100, and the moving body 2 information from the second moving body 101. The surrounding information is information obtained by combining the object information and the environmental information as described above, and is described as surrounding information in the figure.

[0081] The trajectory generation unit 1002 generates a target trajectory of the first moving body 100, i.e., a moving body 1 target trajectory, and a target trajectory of the second moving body 101, i.e., a moving body 2 target trajectory, based on the map data from the map database 500 and the surrounding information from the receiving unit 1012. The method by which the trajectory generation unit 1002 generates the target trajectory of each of two or more moving bodies will be described later. Fig.11 as well as Fig.12 Detailed description.

[0082] The transmission unit 1004 transmits the control amount of the moving body 1 from the first moving body control unit 1031 and the control amount of the moving body 2 from the second moving body control unit 1032 to the first moving body 100 and the second moving body 101 , respectively, via the network NW.

[0083] <Mobile body control unit>

[0084] Below, use Figure 3 The first moving body control unit 1031 of the moving body control unit 1003 will be described. Figure 3 This is a block diagram showing an example of the configuration of the first moving object control unit 1031 .

[0085] like Figure 3 As shown, the first mobile body control unit 1031 includes a mobile body estimation unit 311, a control amount calculation unit 312, a gain setting unit 313, and a control possibility determination unit 314. When the remote control device 1000 remotely controls two or more mobile bodies, Figure 2 The second moving body control unit 1032 shown also has the same structure as described above.

[0086] The moving body estimation unit 311 can estimate the probability distribution of the coefficients of the state quantity of the first moving body 100, that is, the coefficient distribution, based on the moving body 1 information from the receiving unit 1012. Here, the coefficients refer to the mass and inertia moment of the first moving body 100, and the cornering stiffness, etc., when the first moving body 100 is a vehicle. These coefficients also affect the control stability in the same way as the transmission delay of the network NW and are variable. The coefficients are estimated based on the state equation and the state quantity related to the first moving body 100.

[0087] In addition, when the value and probability distribution of the coefficient of the state quantity for the first moving body 100 are known, the gain can be set using this information, and therefore the moving body estimation unit 311 can be omitted.

[0088] The probability distribution of known coefficients can be obtained based on data obtained in advance and design values. As a case where it is obtained based on data obtained in advance, for example, the cornering stiffness represents the relationship between the road surface and the tire, and this probability distribution can be obtained by driving on the road only once and obtaining data. In addition, as a case where it is obtained based on design values, for example, in a case where the load of goods and personnel is determined to be 100kg to 200kg as the specification of the first mobile body 100, the distribution of mass is made into a probability distribution of 100kg to 200kg, and this can also be set as a uniform distribution. In addition, for example, in a case where the frequency of carrying 150kg of goods is high, it can also be modeled as a normal distribution with 150kg as the peak. In the case where the probability distribution is obtained in advance, the control gain can be designed based on the probability distribution obtained in this way.

[0089] The gain setting unit 313 sets the control gain based on the transmission delay profile information of the moving object 1 from the transmission delay profile estimation unit 1001 .

[0090] Furthermore, the gain setting unit 313 sets the control gain based on the transmission delay distribution information of the moving object 1 and the coefficient distribution of the state quantity for the first moving object 100. In this case, it is possible to cope with the case where the state equation related to the first moving object 100 has a probabilistic deviation other than the transmission delay.

[0091] The control amount calculation unit 312 calculates the control amount for making the first moving body 100 follow the target track based on the moving body 1 information from the receiving unit 1012 and the control gain from the gain setting unit 313. The method of setting the control gain by the gain setting unit 313 and the method of calculating the control amount by the control amount calculation unit 312 will be described later. Fig.17 And the disclosures of non-patent documents 1 and 2 are explained in detail.

[0092] The control possibility determination unit 314 determines whether to continue or stop the control on the first moving body 100 based on the transmission delay profile information of the moving body 1 from the transmission delay profile estimation unit 1001. Alternatively, the control possibility determination unit 314 determines whether to continue or stop the control on the first moving body 100 based on the transmission delay profile information of the moving body 1 and the distribution of the coefficient of the state quantity for the first moving body 100.

[0093] When the result of the determination is "continue control", the control possibility determination unit 314 outputs the control amount for controlling the first moving body 100, that is, the control amount from the control amount calculation unit 312, to the transmission unit 1004. When the result of the determination is "stop control", the control possibility determination unit 314 sets a value such that the first moving body 100 is stopped as the control amount, and outputs it to the transmission unit 1004. The method of determining whether to continue control or stop control will be described in detail later.

[0094] Figure 4 1031 is a block diagram showing another example of the structure of the first moving body control unit 1031. Figure 3 As shown, the first mobile body control unit 1031 includes a mobile body estimation unit 311, a control amount calculation unit 312, a gain setting unit 313, a control availability determination unit 314, and a state amount estimation unit 315. When the remote control device 1000 remotely controls two or more mobile bodies, Figure 2 The second moving body control unit 1032 shown also has the same structure as described above.

[0095] In the point that the first moving body control unit 1031 has the state quantity estimation unit 315, Figure 3 The state quantity estimation unit 315 estimates a second state quantity different from the first state quantity obtained by the sensor, such as position, speed, acceleration and angular velocity, among the state quantities of the first moving body 100, based on the moving body 1 information from the receiving unit 1012. The second state quantity is a state quantity that cannot be obtained by the sensor.

[0096] The state quantity estimation unit 315 estimates the second state quantity using an observer, a Kalman filter, a particle filter, etc., based on the state equation related to the first moving body 100 and the moving body 1 information. The remote control device 1000 also controls the first moving body 100 using the second state quantity that cannot be obtained by the sensor, so the first moving body 100 can be remotely controlled with higher accuracy.

[0097] In addition, although Figure 4 Although not shown in the figure, the second state quantity can also be estimated probabilistically using transmission delay distribution information.

[0098] When estimating the coefficient distribution of the state quantity for the first moving object 100 , the moving object estimation unit 311 can use not only the moving object 1 information from the receiving unit 1012 but also the second state quantity from the state quantity estimation unit 315 .

[0099] The control amount calculation unit 312 calculates the control amount based on the moving body 1 information from the receiving unit 1012 , the second state amount from the state amount estimation unit 315 , and the control gain from the gain setting unit 313 .

[0100] <Mobile body>

[0101] Next, use Figure 5 The structure of the first moving body 100 will be described. Figure 5 1 is a block diagram showing the structure of the first moving body 100. Figure 5 As shown, the first moving body 100 includes a built-in sensor 401, a command value calculation unit 402, an actuator 403, a receiving unit 404, a transmitting unit 405, and a time synchronization unit 406. When the remote control device 1000 remotely controls two or more moving bodies, Figure 2 The second moving body 101 shown has the same structure as described above.

[0102] The built-in sensor 401 is an IMU (Inertial Measurement Unit) sensor, a speed sensor, an acceleration sensor, a steering angle sensor, and a steering torque sensor, etc., which detects internal information of the first moving body 100 and outputs it as moving body 1 information and inputs it into the network NW via the sending unit 405.

[0103] The command value calculation unit 402 obtains the control amount of the mobile body 1 calculated by the mobile body control unit 1003 of the remote control device 1000 via the receiving unit 404 and converts it into an actuator command value that can be input to the actuator 403. For example, if it is a target steering angle, it is converted into a control current value of the electric power steering (EPS) or the like. The actuator 403 is composed of a motor that actually moves the first mobile body 100. In addition, the command value calculation unit 402 calculates the driving force and braking force of the vehicle required to make the acceleration of the vehicle follow the target acceleration and deceleration, and outputs the calculation results to the vehicle drive device and the brake control device. Regarding the electric motor, the vehicle drive device and the brake control device, Figure 6 Detailed description.

[0104] The time synchronization unit 406 has a function of synchronizing the timing of data transmission and reception in cooperation with the time synchronization unit 201 in the object information acquisition unit 200 , the time synchronization unit 310 in the environment information acquisition unit 300 , and the time synchronization unit 1011 in the remote control device 1000 .

[0105] The transmission unit 405 transmits the information of the moving object 1 from the built-in sensor 401 to the reception unit 1012 of the remote control device 1000 via the network NW. The reception unit 404 receives the control amount from the transmission unit 1004 of the remote control device 1000.

[0106] The first moving body 100 includes, for example, a vehicle, an aircraft, an unmanned aerial vehicle, a detection aircraft, and agricultural machinery. When there are multiple moving bodies, they can also be combined. When the first moving body 100 is a moving body other than a vehicle, the object information acquisition unit 200 obtains the positions and speeds of other moving bodies or pedestrians as object information when there are other moving bodies or pedestrians around the first moving body 100. In addition, when the first moving body 100 is a moving body other than a vehicle, the environment information acquisition unit 300 accesses the map database 500 to obtain the movable area of ​​the first moving body 100 as map data.

[0107] Figure 6 1 is a diagram showing an example of a structure in which the first mobile body 100 is a vehicle. A steering wheel 1 provided for a driver, i.e., a driver, to operate the vehicle is engaged with a steering shaft 2. The steering shaft 2 is engaged with a pinion shaft 13 of a rack and pinion mechanism 4. A rack shaft 14 of the rack and pinion mechanism 4 freely moves back and forth according to the rotation of the pinion shaft 13, and front knuckle arms 6 are connected to the left and right ends thereof via tie rods 5. The front knuckle arms 6 rotatably support a front wheel 15 as a steering wheel, and are steerably supported to a vehicle body frame.

[0108] The torque generated by the driver's operation of the steering wheel 1 rotates the steering shaft 2, and the rack and pinion mechanism 4 moves the rack shaft 14 to the left and right according to the rotation of the steering shaft 2. The front knuckle arm 6 rotates around a steering kingpin axis (not shown) by the movement of the rack shaft 14, thereby steering the front wheel 15 to the left and right. Therefore, the driver can change the lateral momentum of the vehicle by operating the steering wheel 1 when the vehicle moves forward or backward.

[0109] Furthermore, in the case of fully automated vehicles and non-human-carrying mobile objects such as drones, components for driver operation such as a steering wheel are not required.

[0110] The first moving body 100 is provided with a vehicle speed sensor 20 , an IMU sensor 21 , a steering angle sensor 22 , a steering torque sensor 23 , and the like as built-in sensors 401 for recognizing the running state of the first moving body 100 .

[0111] The command value calculation unit 402 uses Figure 5As described above, the control amount of the mobile body 1 is converted into an actuator command value that can be input to the actuator 403, and the actuator command value is input to the acceleration / deceleration control device 9 and the steering control device 12. The command value calculation unit sometimes constitutes a local feedback for controlling the actuator with high accuracy, but in this case, the command value calculation unit uses a sensor value obtained by a built-in sensor. For example, when the actuator is an electric motor described later, a steering angle sensor and a steering torque sensor are used to calculate an actuator command value with good accuracy.

[0112] The first moving body 100 is provided with actuators such as an electric motor 3 for realizing lateral movement of the first moving body 100 , a vehicle driving device 7 for controlling movement of the first moving body 100 in the front-rear direction, and a brake control device 10 .

[0113] The acceleration / deceleration control device 9 controls the vehicle drive device 7 and the brake control device 10 , and the steering control device 12 controls the electric motor 3 .

[0114] The electric motor 3 is generally composed of a motor and a gear, and can freely rotate the steering shaft 2 by providing torque to the steering shaft 2. That is, the electric motor 3 can freely steer the front wheels 15 independently of the driver's steering wheel operation.

[0115] The vehicle drive device 7 is an actuator for driving the first mobile body 100 in the front-rear direction. The vehicle drive device 7 provides a driving force obtained by a driving source such as an engine or a motor via a gearbox and an axle (not shown), so as to rotate the front wheels 15 and the rear wheels 16. Thus, the vehicle drive device 7 can freely control the driving force of the first mobile body 100.

[0116] On the other hand, the brake control device 10 is an actuator for braking the first mobile body 100, and controls the braking amount of the brakes 11 provided on each of the front wheels 15 and the rear wheels 16 of the first mobile body 100. A general brake uses hydraulic pressure to press a brake disc against a disc rotor that rotates together with the front wheels 15 and the rear wheels 16, thereby generating a braking force.

[0117] The built-in sensors and other devices form a network using a CAN (Controller Area Network) or a LAN (Local Area Network) in the first moving body 100 . Figure 5 Each device in the first mobile body 100 shown can obtain each information via the network. In addition, the built-in sensors can send and receive data to each other via the network. In addition, when the first mobile body is a mobile body other than a vehicle, it also has the same structure as the actuator, built-in sensor, command value calculation unit, etc.

[0118] <Object Information Acquisition Unit and Trajectory Generation>

[0119] Next, use Figure 7 as well as Figure 8 An example of the configuration of the object information acquisition unit 200 and an example of the target trajectory generated in the remote control device 1000 are described below. Figure 7 , as an example of the arrangement of the object information acquisition unit 200, the case where the external sensors 42 and 43 are arranged on the road side on which the first moving body 100 travels is shown, and there is a stationary object OB in front of the first moving body 100. The detection ranges of the external sensors 42 and 43 are ranges R42 and R43, respectively.

[0120] Figure 8 1 is a diagram showing a target path TR for generating a target trajectory for the first moving body 100 to avoid the stationary object OB when there is a stationary object OB in front of the first moving body 100 .

[0121] The external sensors 42 and 43 are composed of cameras, LiDAR, radar, sonar, infrared cameras, etc., and detect the position and speed of the first moving body 100 and other objects. Figure 7 Although it is arranged on the road side, it can also be mounted on the first moving body 100.

[0122] use Figure 7 The external sensor 42 in the apparatus detects the relative position and relative speed of the first moving body 100 with respect to the external sensor 42. In addition, the relative position and relative speed of the stationary object OB with respect to the external sensors 42 and 43 are detected by the external sensors 42 and 43. The object information acquisition unit 200 transforms the information of the relative position and speed of the moving body and the stationary object OB and the external sensors 42 and 43 into the information of the relative position and speed of the stationary object OB observed from the first moving body 100. Alternatively, the information of the relative position and speed of the first moving body 100 and the stationary object OB and the external sensors 42 and 43 is transformed into a coordinate system unified in the first moving body 100 and the stationary object OB, such as a geographic coordinate system used in GNSS, etc., thereby calculating the relative position and speed of the first moving body 100 and the stationary object OB.

[0123] The trajectory generation unit 1002 of the remote control device 1000 generates the following trajectory based on the information: Figure 8 The target path TR is a path such that the first moving body 100 avoids the stationary object OB and travels within the drivable area RR. Although not shown here, the trajectory generation unit 1002 also generates a target speed for the first moving body 100 and combines it with the target path TR as a target trajectory.

[0124] As an example, the trajectory generation unit 1002 generates the target speed so that the first moving body 100 decreases its speed when avoiding the stationary object OB. The trajectory generation unit 1002 generates a target trajectory (avoidance trajectory) that combines the target path and the target speed.

[0125] Fig. 9 is a diagram showing an example of the configuration of the object information acquisition unit 200 and the environment information acquisition unit 300. Fig. 9 , the following case is shown: as an example of the arrangement of the object information acquisition unit 200 when there is a stop line STL and a traffic light TL in front of the first moving body 100, the external sensor 42 is arranged on the road side where the first moving body 100 travels, and as an example of the arrangement of the environment information acquisition unit 300, the external sensor 52 is arranged at a position where the light emission colors of the stop line STL and the traffic light TL can be detected. The detection ranges of the external sensors 42 and 52 are ranges R42 and R52, respectively.

[0126] use Fig. 9 The external sensor 42 in the apparatus detects the relative position and relative speed of the first moving body 100 with respect to the external sensor 42 , and the external sensor 52 detects the relative positions of the stop line STL and the traffic light TL with respect to the external sensor 52 .

[0127] use Fig. 9 The external sensor 42 in the apparatus detects the relative position and relative speed of the first moving body 100 with respect to the external sensor 42 , and the external sensor 52 detects the light emission colors of the stop line STL and the signal light TL.

[0128] The trajectory generation unit 1002 of the remote control device 1000 generates a target path TR as shown by a dashed line based on these information. The target path TR is a path in which the first moving body 100 moves straight toward the stop line STL.

[0129] Fig.10 It is shown as Fig. 9 FIG. 1 is a diagram showing an example of a target speed generated in the remote control device 1000 when there is a stop line STL and a traffic light TL ahead. Fig.10 In FIG. 1 , the horizontal axis represents the moving distance when the first movable body 100 moves toward the stop line STL, and the vertical axis represents the speed of the first movable body 100 .

[0130] The track generation unit 1002 is set at Fig. 9 The target speed TV is shown as a single-dot chain line in FIG. The target speed TV is such that the speed of the first moving body 100 is gradually reduced and Figure 8The trajectory generation unit 1002 generates a target trajectory (stop trajectory) that combines the target path and the target speed.

[0131] like Figure 7 to Figure 9 As shown in FIG. 1 , the target trajectory is an avoidance trajectory for the stopped object OB and a stop trajectory until the first moving body 100 stops. The target trajectory is not limited to these two trajectories, and there are various target trajectories depending on the road on which the first moving body 100 travels. It is also conceivable that the first moving body 100 itself generates the target trajectory, but in terms of the versatility of the first moving body 100, it is preferable that the trajectory generating unit 1002 generates the target trajectory. As a result, the structure of the first moving body 100 becomes simple.

[0132] In addition, Figure 7 to Figure 9 In FIG. 1 , a case where there is one remotely controlled moving object is shown. However, even if there are two or more remotely controlled moving objects, respective target trajectories can be generated by the same method. Fig.11 as well as Fig.12 An example is given below.

[0133] Fig.11 1 is a diagram showing an example of a method for generating target trajectories of two or more moving objects in the trajectory generating unit 1002 . Fig.11 1 is a diagram for explaining a method for generating a target trajectory when the first moving body 100 and the second moving body 101 travel at an intersection. Fig.11 , the external sensor 42 of the object information acquisition unit 200 and the external sensor 52 of the environment information acquisition unit 300 are arranged on the road side where the first moving body 100 is traveling, around the first moving body 100 and the second moving body 101. In addition, there is a stop line STL in front of the road where the second moving body 101 is traveling.

[0134] The detection ranges of the external sensors 42 and 52 are ranges R42 and R52, respectively. The external sensors 42 and 52 are arranged at intervals such that the detection ranges R42 and R52 indicated by dotted lines partially overlap, and the external sensor 42 covers the first moving body 100 and the second moving body 101 approaching the intersection.

[0135] The relative positions and relative speeds of the first moving body 100 and the second moving body 101 with respect to the external sensor 42 are detected by the external sensor 42, and the relative position of the stop line STL with respect to the external sensor 52 is detected by the sensor 52. The track generation unit 1002 generates a target path TR1 of the first moving body 100 based on this information. Although not shown in the figure, the track generation unit 1002 also generates a target speed of the first moving body 100. The track generation unit 1002 generates the target speed in such a way that the first moving body 100 has a constant speed along the target path TR1.

[0136] The track generation unit 1002 generates a target path TR2 for the second moving body 101. Although not shown, the track generation unit 1002 also generates a target speed for the second moving body 101. The target speed for the second moving body 101 is a speed that gradually decreases as it approaches the stop line STL and becomes zero at the stop line STL.

[0137] The trajectory generation unit 1002 generates a target trajectory that combines the target path TR1 and the target speed for the first moving body 100. Similarly, the trajectory generation unit 1002 generates a target trajectory that combines the target path TR2 and the target speed for the second moving body 101.

[0138] In addition, Fig.11 , the track generation unit 1002 generates the target track in a manner taking into account the priority of the travel of the first moving body 100. That is, the target tracks for the first moving body 100 and the second moving body 101 are generated in a manner that increases the priority of the travel of the first moving body 100 using the stop line STL detected by the sensor 52.

[0139] Fig.12 1 is a diagram showing an example of a method for generating target trajectories of two or more moving objects in the trajectory generating unit 1002 . Fig.12 1 is a diagram for explaining a method of generating a target trajectory when the first moving body 100 and the second moving body 101 are traveling in a line. Fig.12 In the embodiment, the external sensor 42 of the object information acquisition unit 200 is installed on the road side where the first moving body 100 and the second moving body 101 travel. The detection range of the external sensor 42 is a range R42, which covers the first moving body 100 and the second moving body 101.

[0140] The relative position and relative speed of the first moving body 100 and the second moving body 101 with respect to the external sensor 42 are detected by the external sensor 42. The track generation unit 1002 generates a target track of the first moving body 100 based on this information. That is, the track generation unit 1002 generates a target path TR1 and a target speed (not shown) of the first moving body 100. As an example, the track generation unit 1002 generates the target speed in such a way that the first moving body 100 has a constant speed along the target path TR1.

[0141] In addition, the track generation unit 1002 generates a target track for the second moving body 101. That is, the track generation unit 1002 generates a target path TR2 and a target speed (not shown) for the second moving body 101. The track generation unit 1002 generates the target tracks for the first moving body 100 and the second moving body 101 in such a manner that the position of the second moving body 101 is a predetermined distance away from the first moving body 100 in the rear. That is, the track generation unit 1002 generates the target track such that the second moving body 101 forms a queue with respect to the first moving body 100 which is the leading one among the moving bodies.

[0142] In this case, the target speed for the second moving body 101 is the same as the target speed for the first moving body 100 , and the target path TR1 of the first moving body 100 and the target path TR2 of the second moving body 101 are also the same.

[0143] Furthermore, the trajectory generation unit 1002 can generate the target trajectory so that the target path TR1 and the target path TR2 are different from each other according to the situation of obstacles around the first moving body 100 and the second moving body 101 .

[0144] If used Fig.11 as well as Fig.12 As described above, the trajectory generation unit 1002 generates target trajectories for a plurality of moving objects. It is also possible to generate a target trajectory for each moving object, but the trajectory generation unit 1002 generates the target trajectory all at once to achieve efficiency improvement and reduction in calculation load.

[0145] <Transmission Delay Distribution Estimation Unit>

[0146] Fig.13 2 is a block diagram showing an example of the configuration of the transmission delay distribution estimation unit 1001. In this example, the transmission delay distribution estimation unit 1001 is composed of a transmission delay preprocessing unit and a transmission delay model unit.

[0147] The transmission delay preprocessing unit 111 has a function of converting the transmission delay information from the transmission delay measuring unit 1013 into a transmission delay feature quantity referred to by the transmission delay model unit 112. The transmission delay feature quantity may be, for example, the mean, variance, or higher-order moment of the transmission delay distribution within a predetermined time interval. Alternatively, the maximum value or minimum value of the transmission delay within the time interval may also be used as the transmission delay feature quantity.

[0148] The transmission delay model unit 112 has the following function: using the transmission delay feature calculated by the transmission delay preprocessing unit 111, referencing the pre-constructed transmission delay model, thereby estimating at least the probability distribution of the current transmission delay, and outputting it as transmission delay distribution information. In addition, the transmission delay distribution information can also include information other than the above-mentioned probability distribution, such as the current or past mode in the hidden Markov model described later. Various models can be used in the transmission delay model unit 112, but in this disclosure, a hidden Markov model ("Hidden Markov Model", hereinafter abbreviated as "HMM") is described as an example of a transmission delay model.

[0149] HMM is a probability model in which the output follows a sequence of discrete or continuous probability distributions, and the modes (states) are constructed so that transitions occur between the modes according to a determined transition probability. Hereinafter, the probability distribution corresponding to each mode in the HMM is referred to as an output distribution.

[0150] Explain the output of HMM and the migration of modes. For example, when the HMM is mode A at a certain moment, the output follows the probability distribution of mode A. On the other hand, there are also cases where the mode migrates to other modes following a certain migration probability and the probability distribution of the output changes. For example, when migrating from mode A to mode B, in the time interval of mode B, the output follows the probability distribution of mode B. In HMM, it is impossible to directly observe which mode is currently in, but only its output sequence can be observed, so it is "hidden".

[0151] Next, use Fig.14 The reason why transmission delay can be modeled using HMM is explained. Fig.14 is a diagram showing an example of a time series of transmission delays. Fig.14 , which is a sequence in which the horizontal axis represents time and the vertical axis represents the transmission delay amount, can be easily obtained using the output of the transmission delay measurement unit 1013.

[0152] Regarding transmission delay, when a dedicated line is not used, it is generally not a constant value, but always has a varying value. Fig.14 As shown, it is found that the deviation pattern changes for each time interval. Fig.14The time interval 1, the time interval 3, or the time interval 4, and the time interval 6 show the same degree of deviation pattern, and the time interval 2 and the time interval 5 are intervals where a significantly large transmission delay is likely to occur.

[0153] Here, regarding the transmission delay, when considering the HMM output sequence, the time intervals showing the same tendency of deviation patterns can be regarded as the same mode in the HMM. That is, it is considered that time intervals 1 and time intervals 3, time intervals 2 and time intervals 4 are the same mode and are set as mode 1 and mode 2 respectively. Similarly, it is considered that time intervals 2 and time intervals 5 are set as mode 3 and mode 4 respectively.

[0154] When these are expressed using HMM, Fig.15 Modeled as shown. Fig.15 , there are a total of 4 modes in HMM, and each delay mode can be expressed as follows.

[0155] Mode 1: Normal distribution with small mean and variance

[0156] Mode 2: Normal distribution with large mean and variance

[0157] Mode 3: Exponential distribution with small mean

[0158] Mode 4: Exponential distribution with large mean

[0159] However, it should be noted that these distributions are only representational and actual communication delays will not take negative values ​​like a normal distribution.

[0160] exist Fig.15 In the ij (i=1, 2, 3, 4, j=1, 2, 3, 4) is set as the probability of transition from mode i of the transition source to mode j of the transition destination. For example, when the mode of the transition source is set to 1, p11, p12, and p13 respectively represent the probability of transition from mode 1 to mode 1, the probability of transition from mode 1 to mode 2, and the probability of transition from mode 1 to mode 3. The same is true for mode 2, mode 3, and mode 4.

[0161] In this way, as an example of a transmission delay model, the transmission delay can be modeled using HMM.

[0162] In addition, regarding the reason why the transmission delay becomes such a model, if the inventor's opinion is described, in the case of a general network, path control is performed in order to efficiently and with high reliability deliver packets, which are units of data transmission and reception, to the correct destination. Due to this path control, the transmission path of the packet may sometimes change, and it can be interpreted that the migration of the mode represents the situation of the switching of the transmission path. Regarding such path control, in the case of a small-scale network, its frequency is low, but in the case of a large-scale network, the switching frequency of the transmission path and the change in the deviation pattern of the transmission delay become large. Such a switching of the transmission path can be interpreted as the switching of the mode in the HMM.

[0163] However, in reality, other factors, such as the surrounding conditions of the moving body, etc. may also be the cause. The countermeasures are described in Embodiment 2.

[0164] In addition, in the present disclosure, in the description of the HMM, Fig.15 is used as the mode, but the increase or decrease of the mode and the probability distribution of each mode can be arbitrarily set.

[0165] <Difference from i.i.d.>

[0166] In Patent Document 1, it is assumed that the transmission delay is i.i.d. with respect to time. In the case of the transmission delay as shown in Fig.14 where the deviation pattern changes, the probability distribution of the transmission delay changes significantly. That is, the probability distribution of the transmission delay is time-varying and is a probability distribution with time-dependence in the way of values, and it can be said that the assumption of i.i.d. does not hold.

[0167] Thus, in Patent Document 1, the control gain is set under the assumption that the transmission delay is i.i.d. with respect to time, so there is room for improvement in terms of improving the performance of remote control.

[0168] In addition, as already described, in the case where the network scale is small and there are few obstacles around the moving body and it is considered that the transmission delay follows i.i.d. with respect to time, the method of Patent Document 1 is effective.

[0169] <Method for creating and referring to HMM>

[0170] Hereinafter, the method for creating the HMM will be described. First, in the transmission delay measurement unit 1013, sequence data of the transmission delay is periodically obtained at intervals of, for example, 0.01 seconds or non-periodically with a certain time width such as 1 hour in advance, and one set of the obtained sequence data is defined as one group, and multiple such groups are obtained. The multiple obtained data is defined as "prior information".

[0171] For the obtained data set, a certain time interval, such as about 1 second, is determined, and the average, variance, maximum value, minimum value, etc., which can be used to estimate the mode of transmission delay, are used as transmission delay feature quantities for each time interval. In addition, the transmission delay feature quantity can also be defined as "prior information". For each transmission delay feature quantity, the pattern is classified by clustering or other techniques. Multiple data sets are classified, and the transition probability between patterns and the output distribution of each pattern are calculated, so that the final HMM can be obtained.

[0172] In addition, the transmission delay information obtained online from the transmission delay measurement unit 1013 when actually controlling the first moving body 100 is distinguished from the advance information and defined as "post-event information". Post-event information refers to the transmission delay information obtained when the remote control device 1000 performs remote control of the first moving body 100. By creating an HMM based on the advance information, the time required for HMM creation can be shortened, and by creating an HMM based on the post-event information, an HMM that is more in line with reality can be created. In Embodiment 3, a method for creating an HMM based on post-event information is described.

[0173] If such an HMM is prepared in advance, in the case of online, the transmission delay feature amount from the transmission delay measurement unit 1013 is used to obtain transmission delay distribution information such as which mode the transmission delay is currently in and the output distribution in that mode.

[0174] Furthermore, HMM is widely used in the field of speech recognition, and methods for creating HMM such as the Baum-Welch algorithm have been widely developed, so HMM can be created using these techniques.

[0175] In addition, when the transmission delay feature amount is not required in the transmission delay model unit 112, that is, when the data of the transmission delay is used as a reference value of the HMM as it is, the transmission delay preprocessing unit 111 can be omitted.

[0176] <Stability of Probability System>

[0177] Here, in order to facilitate understanding of the HMM creation method in the present disclosure, with reference to Non-Patent Document 1, the definition of stability of a control object (hereinafter referred to as a "probabilistic system") having deviations such as transmission delay and the like and an evaluation formula for evaluating its stability are explained.

[0178] First, let k be an integer representing a time. k is a real vector of dimension Z, which represents the probability variable following a certain probability distribution at time k. k The probability process (ξ k ) is recorded as ξ. In addition, the probability process ξ before time k0 is recorded as ξ k0―, the probability process ξ after time k0 is recorded as ξ k0+ After considering the time k0, we get ξ k0― So, we can get the value of ξ k0― Recorded as ξ h k0― When, after time k0, the initial condition of the probability process is expressed by the following formula (1).

[0179] [Formula 1]

[0180]

[0181] The conditional expected value under the condition that the event Ap occurs is expressed as E[(·)|Ap]. In order to define the stability of the probability system, the conditional expected value E expressed by the following formula (2) is introduced: k0 [·].

[0182] [Formula 2]

[0183]

[0184] That is, formula (2) means that the value of the probability process ξ up to time k0 is ξ h (k0-1)― Expected value under such conditions.

[0185] Next, a discrete-time state equation of a probabilistic system is expressed as the following formula (3).

[0186] [Formula 3]

[0187] x k+1 =A(ξ k )x k …(3)

[0188] In formula (3), x k is an n-dimensional vector representing the state of the moving object at time k, A k (ξ k ) is composed of k Determine the n×n random matrix. Regarding this random matrix, in order to define stability, it is assumed that for any time k, there exists a positive real number M that satisfies the following formula (4): A (k).

[0189] [Formula 4]

[0190] E k [A ij (ξ k ) 2 ]≤M A (k)…(4)

[0191] Here, i, j = 1...n, A ij (ξ k ) is A(ξ k ) of the (i, j) elements. Formula (4) means that there exists A(ξ k ) of the various elements that produce ξ h k0 --conditional expected value under the condition of.

[0192] According to non-patent document 1, regarding the probability system of formula (3), when a is set to a real number taking a positive value and λ is set to a real number becoming 0<λ<1, under the assumption of formula (4), when there exist a and λ such as those satisfying the following formula (5), the probability system becomes quadratic moment exponentially stable, that is, stable.

[0193] [Formula 5]

[0194]

[0195] Here, ||x k || is x k The Euclidean norm of , k0 is the moment that satisfies k>k0.

[0196] In non-patent document 1, under the assumption that the probability of formula (3) is formula (4), it is proved that the exponential stability of the second-order moment is equivalent to satisfying the following conditions. That is, when ε d and ε u When ξ is a real number taking a positive value, λ is a real number satisfying 0<λ<1, and P(ξ) is a mapping that makes the probability process ξ correspond to an n×n-dimensional symmetric matrix, there exists ε that satisfies the following formulas (6) and (7): d and ε u , λ, and P, the probability system is quadratic moment exponentially stable.

[0197] [Formula 6]

[0198] ∈ d I n×n ≤E k0 [P(S k0 ξ k0+ )]≤∈ u I n×n …(6)

[0199] [Formula 7]

[0200] E k0 [λ 2 P(s k0 ξ k0+ )-A(ξ k0 ) T E k0 [P(Sk0+1 ξ (k0+1)+ )|f k0 A(ξ k0 )]≥0

[0201] …(7)

[0202] where S k0 is a time shift operator, defined such that applying it to ξ k0+ results in ζ = S k0 ξ k0 yielding ζ0 = ξ k0+ , ζ1 = ξ k0 ,... Additionally, I k0+1 is an n×n identity matrix, and F n×n is the σ-additive family (also called the complete additive family, etc.) generated by ξ k ,..., ξ k0 . Regarding formulas (6) and (7), since P contains S k ξ k0 , etc., it is found that they become conditional equations联立 for an infinite time. k0+ Formulas (6) and (7) hold generally for probability systems that satisfy the assumptions of formula (4). Therefore, if for various levels of probability processes such as probability distributions without upper bounds and time-varying probability processes that are not i.i.d., formula (4) is satisfied and the conditions of formulas (6) and (7) hold, then the probability system can be said to be second-moment exponentially stable. In cases where there are the above-mentioned HMMs and martingales, etc. in the level of time-varying probability processes, and the probability processes whose transmission delays are considered to follow these levels, the control gain can be designed based on the above stability conditions. Below, referring to Non-Patent Document 2, the stability conditions of HMMs and the design method of control gain will be described.

[0203] <Stability Conditions of HMMs and Design of Control Gain>

[0204] <Let the HMM be composed of N models, and each model is called Model 1, Model 2,..., Model N. The output distributions of each model are D1, D2,..., D

[0205] Let the model at each time k be σ N (i.e., σ k takes values 1, 2,..., N). Additionally, let η k be the probability distribution of the HMM output at time k, and η k follows any of the probability distributions in D1, D2,..., D k at each time. Let the time-invariant transition probability from model i to model j be p N ij ​Regarding the transition between the modes of HMM, when following a regular and non-periodic Markov chain, it is expressed by the following formula (8).

[0206] [Formula 8]

[0207]

[0208] In addition, η at the same time as the mode k At this time, regarding the probability process ξ, the real number vector ξ related to the time k is expressed by the following formula (9): k The sequence is provided.

[0209] [Formula 9]

[0210]

[0211] According to non-patent document 2, when the probability process ξ is taken as formula (9), and the probability system that satisfies the assumption of formula (4) becomes quadratic moment exponentially stable, the following conditions are satisfied. That is, when λ is set to a real number that satisfies 0<λ<1, it becomes a positive definite matrix (equivalent to real numbers whose eigenvalues ​​are all positive) P i (i=1, ..., N), when there are λ, P that satisfy the following formula (10): i When , the probability system is quadratic moment exponentially stable.

[0212] [Formula 10]

[0213]

[0214] Here, the superscript T represents the transpose of the matrix, and the symbol combining 0 and × represents the Kronecker product.

[0215] G j ' is a matrix obtained as follows. First, let row(A) be a row vector that arranges the elements of matrix A in order from the first row, (j) By following the probability variable η of the distribution Dj (j) , becomes a probability variable expressed by the following formula (11).

[0216] [Formula 11]

[0217]

[0218] About G j ', first, n 2 ×n 2 The matrix E[row(A(ξ (j) ))Trow(A(ξ (j)))] is decomposed as follows: n j The matrix G j .

[0219] [Formula 12]

[0220]

[0221] G j It is expressed by the following formula (13).

[0222] [Formula 13]

[0223] G j :=[G 1j , ..., G nj ]…(13)

[0224] Therefore, regarding G j ', use n j ×n 2 The matrix G 1j , ..., G nj , defined by the following formula (14) as n·n j A matrix of size n×n.

[0225] [Formula 14]

[0226]

[0227] Since the transmission delay can be expressed by HMM as described above, the stability of the controlled object can be evaluated using the evaluation formula of formula (10).

[0228] Next, a method for designing control gain using equation (10) is described. First, consider the discrete-time state equation represented by the following equation (15) obtained by adding a control input term to the probability system of equation (3).

[0229] [Formula 15]

[0230] x k+1 =A(ξ k )x k +B(ξ k ) k …(15)

[0231] Here, uk is an m-dimensional vector representing the control input. Regarding B(ξk), similarly to the assumption in formula (4), it is assumed that for any time k, there exists a positive real number M satisfying the following formula (16): B (k).

[0232] [Formula 16]

[0233] Ek [B ij (ξ k ) 2 ]≤M B (k)…(16)

[0234] Here, i=1…n, j=1…m, B ij (ξ k ) is B(ξ k )’s (i, j) elements.

[0235] In the following, it is assumed that the probability of formula (15) satisfies the assumptions of formula (4) and formula (16), and it is called the control object.

[0236] Regarding the design guidelines of the control gain, the following three methods are described as examples in this disclosure.

[0237] Method 1: Using the current model

[0238] Method 2: Method without using patterns

[0239] Method 3: Using past patterns

[0240] <Method 1: Using the current mode>

[0241] At the same time as the mode, the output distribution is considered to be iid relative to the time k. Therefore, it is possible to use the transmission delay distribution estimation unit 1001 ( Figure 1 ) obtains the mode information of the transmission delay and controls the gain according to the mode switching method at each moment.

[0242] Let the probability variable representing the transmission delay at the time of mode i be η k (i) , set the control gain to F (i) When the control input u is in mode i, the discrete-time state equation of the control object and the control input u are expressed by the following formula (17) and formula (18): k .

[0243] [Formula 17]

[0244]

[0245] [Formula 18]

[0246] u k =F (i) x k…(18)

[0247] Here, F (i)It is an m×n matrix. At the time of mode i, it is considered iid, so the control gain F of each mode can be obtained by the method of Patent Document 1. (i) In actual control, the control gain is switched for each mode that changes every moment.

[0248] <Method 2: Method without using patterns>

[0249] According to Non-Patent Document 2, it is possible to design a control gain F that does not depend on a pattern expressed by the following formula (19).

[0250] [Formula 19]

[0251] u k =Fx k …(19)

[0252] Here, F is an m×n matrix. Under this control gain, formula (15) is expressed by the following formula (20).

[0253] [Formula 20]

[0254] x k+1 =(A(ξ k )+B(ξ k )F)x k …(20)

[0255] Hereinafter, the coefficient matrix of the closed-loop system is expressed as the following formula (21).

[0256] [Formula 21]

[0257] A cl (ξ k )=A(ξ k )+B(ξ k )F…(21)

[0258] In formula (20), if the design variable is set to F and the control gain F can be obtained so that the quadratic moment exponential stability of formula (10) is obtained, the controlled object can be stabilized. The method for obtaining F is derived from non-patent document 2. That is, when λ is set to a real number that satisfies 0<λ<1, X is set to an n×n positive definite matrix, and Y is set to an m×n matrix, when there are λ, X, and Y that satisfy the conditions expressed by the following formula (22), there is a control gain F that stabilizes the controlled object.

[0259] [Formula 22]

[0260]

[0261] In particular, F = YX -1Here, in the case where ξ(j) is written as formula (11), in order to define the matrix H' Aj , matrix H' Bj First, define the matrix H as follows: j .

[0262] [Formula 23]

[0263]

[0264] Moreover, the matrix H that satisfies formula (23) j It becomes an nj×n(n+m) matrix expressed by the following formula (24).

[0265] [Equation 24]

[0266] H j =[H Aj1 , ..., H Ajn , H Bjn , ..., H Bjn ]…(twenty four)

[0267] Matrix H' Aj And the matrix H' Bj These are nj·n×n matrix and nj·n×m matrix defined by the following formula (25) and formula (26) respectively for formula (24). Hereinafter, j is assumed to be j=1, ..., N.

[0268] [Formula 25]

[0269]

[0270] [Equation 26]

[0271]

[0272] Formula (22) is a combination of linear matrix inequalities (Linear Matrix, Inequality, hereinafter referred to as "LMI") for modes 1 to N. It is possible to use a tool for solving linear matrix inequalities such as MATLAB (registered trademark) to fix λ and find the values ​​of X and Y, so the control gain F can be calculated using the found X and Y. In addition, by minimizing λ using a bisection method or the like, it is also possible to design a control gain F that accelerates the convergence speed.

[0273] By controlling the controlled object using the control gain F obtained in this way, the control system can be stabilized under the HMM.

[0274] <Method 3: Using past patterns>

[0275] Non-patent document 2 describes a method that also utilizes past patterns. That is, a control gain F that depends on past patterns is used. σk-1 , as shown in the following formula (27) k .

[0276] [Formula 27]

[0277] u k =F σk-1 x k …(27)

[0278] Here, F σk-1 is an m×n matrix. The meaning of formula (27) is: i , for example, in the past the mode was j (i.e. σ k-1 =j), the current mode is i (i.e. σ k = i), the control gain F designed in mode j is used at the current moment. j Such meaning.

[0279] If the design variable is set to F i And it is possible to find the evaluation formula F that satisfies formula (10) i , then the controlled object can be stabilized. i That is, when λ is set to a real number satisfying 0<λ<1 and X i Assume that it is an n×n positive definite matrix, and replace Y i When it is an m×n matrix, there are λ and X that satisfy the conditions expressed by the following formula (28): i , Y i When there is a design variable F that stabilizes the control object i .

[0280] [Equation 28]

[0281]

[0282] In particular, F i =Y i X i -1 Become one of them. Aj , H' Bj This is a matrix obtained in the same manner as in method 2.

[0283] Formula (28) is the joint LMI of patterns 1 to N, and can be solved in the same way as in the case of method 2 which does not use a pattern.

[0284] This disclosure describes a method of designing a control gain using three methods. In addition, a method using both a current pattern and a past pattern is also conceivable, and these methods can be used.

[0285] In addition, in the previous method of determining the system, various LMIs such as H2 performance and H∞ performance are derived. By solving LMI in combination with formula (22) and formula (28) in accordance with each purpose, it is possible to design control gains that satisfy H2 performance and H∞ performance while achieving quadratic moment exponential stability, and to design control gains for multiple purposes.

[0286] In addition, in non-patent documents 1 and 2, in addition to the assumption of formula (4), in A(ξ k ), B(ξ k ) is below a certain value. Under the assumption that the output distribution of each mode in the HMM has upper and lower limits, the control gain can be designed under such an assumption. In this case, the condition does not include the calculation of the expected value, and the effect of being able to simply design the control gain is obtained.

[0287] <Remote Control of Mobile Objects>

[0288] Next, a method of remotely controlling a moving object will be described based on the above-mentioned method of designing a control gain. Fig.16 1 is a block diagram showing an example of a control system in which the remote control device 1000 according to the first embodiment controls the first moving body 100 which is a control target under a transmission delay environment. Fig.16 In the figure, the solid line represents the input and output of the signal represented by continuous values, and the dotted line represents the input and output of the signal represented by discrete values. c and u c The moving body information of the first moving body 100 obtained by various sensors is a discrete value, so the moving body information is equivalent to the output value of the sampler S. The moving body information is sent to the remote control device 1000 via the network NW, so a transmission delay occurs at this time, and here, an upload transmission delay D occurs. UP The mobile information is delayed by the upload transmission delay D up The controller ψ outputs a control amount calculated using a control gain based on the moving body information. This control amount is equivalent to the control amount output by the control amount calculation unit 312 of the first moving body control unit 1031. The control amount is sent to the first moving body 100 via the network NW, so a transmission delay occurs at this time, and in this case, a download transmission delay D occurs. dwThe control amount input to the first moving body 100 at a certain time becomes a constant value by the holder H until it is input next. That is, the holder H has a function of 0th order holding. The 0th order held control amount is input to the first moving body 100 as the control object Pc.

[0289] Fig.16 The control system is a closed-loop system, so in order to ensure control stability, it is necessary to consider the transmission delay, that is, the upload transmission delay and the download transmission delay to set the control gain and calculate the control amount. The following describes the control object using the probability distribution to represent the transmission delay. First, the continuous-time state equation of the control object Pc is expressed by the following formula (29).

[0290] [Equation 29]

[0291] x c · =A c x c (t)+B c u c (t)…(29)

[0292] Here, x c 、u c is the state and input in continuous time. x c · Represents x c Hereinafter, · represents the time derivative. Fig.16 The sampler S and the holder H shown are at the sampling time t that satisfies the following formula (30): k The sampler and 0th order hold of the next action.

[0293] [Formula 30]

[0294]

[0295] When the sampling interval is set to h k =t k+1 -t k When Fig.16 In the network control system k It is not constant and becomes non-periodic sampling. Fig.16 Upload transmission delay D up and download transmission delay D dw is the arrival delay time τ from the transmission source to the transmission destination at each time k uk and τ dk delay factor.

[0296] Consider a probabilistic process ξ following HMM etc. Let ξ=[ξ uk , dk], using the constant ε u >0 and the constant ε d >0, the transmission delay is expressed by the following formula (31).

[0297] [Equation 31]

[0298] τ uk =ξ uk +∈ u , τ dk =ξ dk +∈ d …(31)

[0299] At this time, the sampling interval h k It is expressed by the following formula (32).

[0300] [Formula 32]

[0301] h k =τ uk +τ dk =ξ uk +ξ dk +∈ u +∈ d …(32)

[0302] Here, ε u , ε d This is a physically determined transmission delay in addition to the transmission delay that varies probabilistically.

[0303] pass Fig.16 With the sampler S and the holder H, formula (29) is transformed into a discrete-time state equation as shown in the following formula (33).

[0304] [Formula 33]

[0305] x k+1 =A k x k +B k u k-1 …(33)

[0306] Here, the relationship between the continuous signal and the discrete signal is expressed by the following formula (34).

[0307] [Equation 34]

[0308] x k =x c (t k ),u c (t) = u k-1 (t∈[t k , t k+1 ))…(34)

[0309] At this time, Ak and B k Provided by the following formula (35).

[0310] [Formula 35]

[0311]

[0312] Therefore, A k , B k Become dependent on k In formula (33), the control input does not become u k But become u k-1 , it is impossible to convert the k Decision k As the input at time k, we can find it. Therefore, we add a new state x e，k The expansion system is expressed by the following formula (36).

[0313] [Equation 36]

[0314]

[0315] Here, replace the obtained formula (36) with formula (15), and use the time-varying probability distribution such as HMM to calculate h k Modeling is performed so that the control gain design method described above can be applied.

[0316] <When the moving object is a vehicle>

[0317] When the moving body is a vehicle, a continuous-time state equation such as formula (29) is obtained for each dynamic of the moving body. In the present disclosure, a remote control device applicable to various moving bodies is provided, but a vehicle is used as an example to explain it in detail. In addition, although many methods for controlling a moving body are proposed, in the present disclosure, a method for decomposing the movement into lateral movement and front-rear movement and controlling each direction separately is explained.

[0318] First, use Fig.17 The state equation representing the lateral dynamics of the moving body will be described. Fig.17 is a diagram showing an example of a sequence of positions in a target path TR, that is, a target track, when the first moving body 100 is a vehicle. The target path TR is represented by an absolute coordinate system having an X-axis and a Y-axis. The lateral offset and the deflection angle of the first moving body 100 relative to the target path TR are represented by e y and e θ express.

[0319] In this case, the state equation of the first moving body 100 in the lateral direction is expressed by the following formula (37).

[0320] [Formula 37]

[0321]

[0322] v x :Vehicle speed [m / s]

[0323] δ: Rudder angle [rad]

[0324] m: mass [kg]

[0325] L f : Distance between center of gravity and front wheel axle [m]

[0326] L r : Distance between center of gravity and rear wheel axle [m]

[0327] I z : Moment of inertia about the yaw axis [kg·m 2 ]

[0328] C f : Front wheel cornering stiffness [N / rad]

[0329] C r : Rear wheel cornering stiffness [N / rad]

[0330] e y : Lateral deviation of the vehicle's center of gravity to the target path [m]

[0331] e θ : The deviation angle from the center of gravity of the vehicle to the target path [rad]

[0332] e y · : The time differential associated with the lateral displacement of the moving body

[0333] e θ · : The time differential related to the deflection angle of the moving object

[0334] Cornering stiffness is a proportionality coefficient indicating the relationship between the lateral force and the lateral slip angle generated in the moving body, and is a value that changes depending on the state of the contact surface between the moving body and the road surface, such as a dry surface, a wet surface, or an icy surface.

[0335] When formula (37) is written in the same manner as formula (29), the continuous-time state equation can be expressed by the following formula (38).

[0336] [Equation 38]

[0337] x c =[e y e y · eθ e θ · ] T

[0338]

[0339] If this continuous-time state equation is used so that e y 、e θ 、e y · 、e θ · If the control is set to 0, the moving object can follow the target path.

[0340] The state equation for the front-rear direction of the first moving body 100 is: α To vehicle speed v x The state equation is taken as the time constant T a When modeling a first-order delay system, the forward and backward acceleration α can be used. x , as modeled by the following formula (39).

[0341] [Equation 39]

[0342]

[0343] Formula (39) is introduced as a reference model for setting the desired response when the target acceleration is provided, and the state equation is constructed with the deviation from the reference model as the state, so that Formula (39) can be used as a regulator problem. Thus, the control gain F can be designed such that the state converges to 0 using the method disclosed in the present invention.

[0344] <Controllability Judgment Section>

[0345] In addition, there may be a case where there is no control gain F that satisfies the stability of the quadratic moment index. In this case, there is a possibility that unexpected transmission delay will occur, and the stability of the control of the moving body cannot be guaranteed. Therefore, the control determination unit 314 ( Figure 3 ) When the stability of the control of the moving body cannot be guaranteed and the transmission delay exceeds a predetermined value, a decision is made to stop the control of the moving body. Thus, when an unexpected transmission delay occurs, the control of the moving body can be stopped to ensure the safety of the control of the moving body.

[0346] In order to determine whether the closed-loop system is quadratic moment exponentially stable, it is determined whether there is a solution in each LMI. Alternatively, if a transmission delay that cannot be modeled by the HMM occurs, it is determined to stop the control.

[0347] <Implementation Method 2>

[0348] <Overall Structure>

[0349] Fig.18 This is a block diagram showing an example of the configuration of the remote control device 2000 according to the second embodiment of the present disclosure and the configuration of a remote control system RCS2 of a mobile object MV which is remotely controlled via a network NW.

[0350] like Fig.18 As shown, the remote control device 2000 of the remote control system RCS2 becomes Figure 1 The remote control device 1000 shown in the figure is configured to also use surrounding information, mobile body 1 information, and map data of the map database 500 from the receiving unit 1012 as inputs, compared to the transmission delay distribution estimation unit 1001. Other than these, the remote control device 1000 is the same as the remote control device 1000, and therefore repeated descriptions are omitted.

[0351] In addition, Fig.18 In order to facilitate the description, only the first moving body 100 is controlled. However, for two or more moving bodies, it is also possible to control the first moving body 100 by setting the first moving body 100 to the first moving body 100. Figure 2 In this case, the transmission delay profile estimation unit 1001 receives the moving object information of two or more moving objects from the receiving unit 1012 , and outputs the transmission delay profile information of each of the two or more moving objects to the moving object control unit 1003 .

[0352] Furthermore, when the network environments and surrounding conditions of the plurality of mobile bodies are substantially the same and the tendencies of the transmission delays are considered to be of the same degree, the transmission delay distribution estimation unit 1001 considers the transmission delays of the plurality of mobile bodies to be the same and simplifies the calculation.

[0353] <Transmission Delay Distribution Estimation Unit>

[0354] Fig.19 1 is a block diagram showing an example of the configuration of the transmission delay distribution estimation unit 1001. In this example, the transmission delay distribution estimation unit 1001 is composed of a transmission delay preprocessing unit 111, a transmission delay model unit 112, and an environment preprocessing unit 113.

[0355] The transmission delay preprocessing unit 111 has a function of converting the transmission delay information from the transmission delay measuring unit 1013 into a transmission delay feature amount referenced by the transmission delay model unit 112 .

[0356] The transmission delay model unit 112 is modeled in advance using the transmission delay feature amount calculated by the transmission delay preprocessing unit 111, and calculates transmission delay distribution information by referring to the transmission delay feature amount and the environment feature amount.

[0357] The environment preprocessing unit 113 calculates the characteristic amount of the environment information other than the transmission delay information. That is, it has a function of calculating the environment characteristic amount that characterizes the environment based on the map data from the map database 500, the mobile body 1 information from the receiving unit 1012, and the surrounding information. In addition, since the transmission delay preprocessing unit 111 and the transmission delay characteristic amount are the same as those in the first embodiment, the description thereof is omitted.

[0358] The transmission delay feature is a feature directly obtained from the transmission delay sequence, while the environmental feature is a feature representing the surrounding conditions of the moving body, and is calculated based on physically measurable values ​​such as the current time, the condition of the radio waves around the moving body, the presence or absence of structures around the moving body, the distance from the moving body, the conductors around the moving body, obstacles, radio wave strength, and raffic. In the environmental preprocessing unit 113, the environmental feature is calculated using the map data, the moving body information, and the surrounding information.

[0359] For example, if there is a building near a moving object, the location of the building can be detected from map data, and the location of the moving object can be detected from GNSS, so the relative distance can be digitized as an environmental feature. In addition, radio wave strength can also be detected by antennas and receivers, so it can be used as an environmental feature.

[0360] The advantages of such a structure are described. As described above, the transmission delay varies due to the deviation of the switching of the path, but in addition, it is caused by the load of the network NW such as the status of the line users, the amount of communication, and the characteristics of the router. In addition, since the moving body is moving, the transmission delay may also be caused by the presence or absence of obstacles and interference on the radio wave transmission path, the presence or absence of conductors around the moving body, etc. It is considered that these factors are overlapped to become the final transmission delay.

[0361] By inputting environmental feature quantities expressing such situations into a transmission delay model, it is possible to create a transmission delay model and estimate transmission delay distribution with higher accuracy.

[0362] Specifically, the transmission delay model in the second embodiment is constructed as follows: Fig.15 Use p in ij The transition probability between the modes represented by (i=1, 2, 3, 4, j=1, 2, 3, 4) varies according to the environmental feature quantity. For example, if the user moves to a place where there are many structures such as buildings that refract or shield radio waves, the transition probability is increased so that it is easy to transition to a mode with a large transmission delay. Alternatively, the user may consider modeling such as reducing the transition probability so that it is difficult to transition to a mode with a large transmission delay because the communication volume decreases at night. In such a structure, it is possible to design a control gain with the transition probability as a parameter.

[0363] <Implementation Method 3>

[0364] <Transmission Delay Distribution Estimation Unit>

[0365] Fig. 20 1 is a block diagram showing an example of the structure of the transmission delay distribution estimation unit 1001 of the remote control device 3000 according to the third embodiment of the present disclosure. In this example, the transmission delay distribution estimation unit 1001 is composed of the model unit 115. In addition, except for the transmission delay distribution estimation unit 1001, the other structures are the same as those of FIG. Fig.18 The remote control device 2000 of the second embodiment shown in FIG. 1 is the same as that of the remote control device 2000 of the second embodiment shown in FIG. Fig.18 The same, repeated descriptions are omitted.

[0366] The model unit 115 receives input of the transmission delay information from the transmission delay measurement unit 2013 , the map data from the map database 500 , the surrounding information acquired via the network NW, and the information of the moving body 1 , and calculates the transmission delay distribution information through machine learning.

[0367] In recent years, machine learning technologies using AI (Artificial Intelligence) have developed significantly, led by deep learning technology.

[0368] In this embodiment, a method for learning a transmission delay model using a machine learning technique and designing a control gain using the learned model and a method for estimating transmission delay distribution information online are provided, thereby obtaining a highly accurate transmission delay model and online transmission delay distribution information.

[0369] Fig. 20 The transmission delay distribution estimation unit 1001 is configured to output transmission delay distribution information online using a learned model learned by machine learning. When learning the transmission delay model, it is first necessary to obtain learning data. In order to obtain learning data, the first moving body 100 is moved, and the transmission delay information of the moving body 1 from the transmission delay measurement unit 1013, the surrounding information obtained via the network NW, and the moving body information are obtained and stored as a data group. The model unit 115 can perform learning using the stored data group and the map database 500.

[0370] The learning method of setting the transmission delay model as the HMM model has been widely studied in the field of speech recognition. By using this method, the HMM model can be learned. In the case of wanting to learn a more general transmission delay model, the machine learning method of LSTM (Long Short Time Memory) using the learning time series can be used for learning.

[0371] The transmission delay may include at least any one of the transmission delay amount, the average value of the transmission delay in a predetermined time interval, the variance of the transmission delay, the maximum value, and the minimum value of the transmission delay.

[0372] The surrounding information may include at least the time, the state of radio waves around the moving body, the presence or absence of structures around the moving body, the distance between the structures and the moving body, conductors and obstacles around the moving body, radio wave strength, weather, and communication volume.

[0373] The map data may include at least the shape of roads around the moving object, and the positions and shapes of surrounding structures.

[0374] In machine learning, if there is a correlation between input and output, learning is possible, and the transmission delay feature quantity, environmental feature quantity, and map data are input to the model unit 115, thereby outputting transmission delay distribution information. For example, in "Freesound Sound Recognition System (2nd Edition)" by Masahiro Araki and Morihoku Publishing Co., Ltd., a method for learning HMM using deep learning is disclosed.

[0375] <Modification>

[0376] The HMM described in Embodiments 1 to 3 is a non-hierarchical hidden Markov model, but a hierarchical hidden Markov model obtained by hierarchizing the HMM can be used as a more precise transmission delay model. Both the non-hierarchical and hierarchical hidden Markov models can accurately predict transmission delays.

[0377] In addition, as the probability distribution that the transmission delay follows, the level of the martingale is also considered. The martingale is a level where the expected value at the current time is consistent with the value at the previous time. Non-patent document 1 also shows the stability condition for the level of the martingale, and by using this, the control gain can be designed.

[0378] In addition, in this disclosure, a method of evaluating stability using the quadratic moment exponential stability is described. Generally speaking, the quadratic moment exponential stability is the strongest indicator of stability, but it can be used similarly for other stabilities.

[0379] In the present disclosure, a remote control device and a mobile body are used as Fig.16As described in the operation of the sampler S and the holder H of FIG. 1 , the structure in which the next signal is immediately sent to the other side upon receiving the signal is basically described. However, in this structure, when the transmission delay is very small, there is a possibility that the sampling interval is too short compared to the responsiveness of the mobile body as the control object, and the communication volume of the network is unnecessarily increased. This problem can be dealt with by manually setting the minimum delay time in advance according to the responsiveness of the mobile body, etc., and when the actual communication delay is less than the minimum delay time, the mobile body or the remote control device side waits until the set minimum delay time has passed before sending the signal.

[0380] For example, when the minimum delay is determined to be 50 msec, when a transmission delay of less than 50 msec occurs, by waiting until 50 msec including the transmission delay has passed before sending a signal, a situation where the sampling interval is always greater than 50 msec can be formed. This waiting can be performed on either the mobile body side or the remote control device side.

[0381] By designing the control gain according to the transmission delay under the premise of such a response, control that responds to the above-mentioned problem can be realized. In addition, the minimum delay can be set to a fixed value, but can also be changed according to information on the actual transmission delay.

[0382] For example, the minimum delay may be switched for each transmission delay pattern, or may be changed in accordance with the time and surrounding information.

[0383] In addition, in the present disclosure, it is simply explained that the operation time in the remote control device can be ignored, but if it cannot be ignored, the time can also be included in the transmission delay. When the minimum delay is set as described above, the waiting time can also be used for some operations.

[0384] <Hardware Structure>

[0385] In addition, each component of the remote control device 1000 to 3000 of the above-described embodiments 1 to 3 can be configured using a computer and implemented by the computer executing a program. Fig.21 The processing circuit 60 shown in the figure is implemented. As the processing circuit 60, a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor) is applied, and the functions of each part are realized by executing a program stored in a storage device.

[0386] In addition, dedicated hardware may also be applied as the processing circuit 6. When the processing circuit 60 is dedicated hardware, the processing circuit 60 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0387] The remote control devices 1000 to 3000 may realize the functions of the respective components by independent processing circuits, or may integrate these functions and realize them by one processing circuit.

[0388] In addition, Fig. 22 , a hardware structure is shown in the case where a processor is used to constitute the processing circuit 60. In this case, the functions of each part of the remote control device 1000 to 3000 are realized by a combination of software, etc. (software, firmware, or software and firmware). The software, etc. are described as programs and stored in the memory 62. The processor 61 that functions as the processing circuit 60 realizes the functions of each part by reading and executing the program stored in the memory 62 (storage device). That is, the program can be called a program of the process and method of causing the computer to execute the actions of the constituent elements of the remote control devices 1000 to 3000.

[0389] Here, the memory 62 can be set to, for example, RAM, ROM, flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory) and other non-volatile or volatile semiconductor memories, HDD (Hard Disk Drive), magnetic disk, floppy disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc) and its drive device, etc., or any storage medium that will be used in the future.

[0390] The above describes a structure in which the functions of the components of the remote control devices 1000 to 3000 are implemented by either hardware or software. However, the present invention is not limited thereto, and some of the components of the remote control devices 1000 to 3000 can be implemented by dedicated hardware, and other components can be implemented by software. For example, the functions of some components can be implemented by the processing circuit 60 as dedicated hardware, and the functions of other components can be implemented by the processing circuit 60 as the processor 61 reading and executing the program stored in the memory 62.

[0391] As described above, the remote control devices 1000 to 3000 can realize the above-mentioned functions through hardware, software, etc., or a combination thereof.

[0392] Although the present disclosure has been described in detail, the above description is by way of example only in all aspects and the present disclosure is not limited thereto. It should be understood that numerous modifications not shown in the examples may be conceived without departing from the scope of the present disclosure.

[0393] In addition, the present disclosure can freely combine the various embodiments or appropriately modify or omit the various embodiments within the scope of the disclosure.

Claims

1. A remote control device for controlling at least one moving object via a transmission path including at least a network, the remote control device comprising: a transmission delay distribution estimation unit that estimates transmission delay distribution information including a time-varying probability distribution of the estimated transmission delay based on the transmission delay information in the transmission path acquired in advance and the transmission delay information acquired online; a trajectory generating unit that generates a target trajectory of the at least one moving object based on surrounding information of the at least one moving object; and a moving body control unit that generates a control amount of the at least one moving body based on the transmission delay distribution information obtained from the transmission delay distribution estimation unit, the target trajectory obtained from the trajectory generation unit, and the moving body information obtained from the at least one moving body, The mobile body control unit includes: a gain setting unit that sets a control gain based on the transmission delay distribution information; and A control amount calculation unit generates the control amount based on the target trajectory, the control gain, and the moving object information.

2. A remote control device for controlling at least one moving object via a transmission path including at least a network, the remote control device comprising: a transmission delay distribution estimation unit that estimates transmission delay distribution information including a probability distribution of estimated transmission delays and a current pattern or a past pattern of the transmission delays based on transmission delay information in the transmission path acquired in advance and the transmission delay information acquired online; a trajectory generating unit, which generates a target trajectory of the at least one moving object based on surrounding information of the at least one moving object; as well as a moving body control unit that generates a control amount of the at least one moving body based on the transmission delay distribution information obtained from the transmission delay distribution estimation unit, the target trajectory obtained from the trajectory generation unit, and the moving body information obtained from the at least one moving body, The mobile body control unit includes: a gain setting unit that sets a control gain based on the transmission delay distribution information; and A control amount calculation unit generates the control amount based on the target trajectory, the control gain, and the moving object information.

3. The remote control device according to claim 1 or 2, wherein: The transmission delay distribution estimation unit estimates the transmission delay distribution information based on the transmission delay information and an environmental feature value around the at least one moving object.

4. The remote control device according to claim 1 or 2, wherein: The transmission delay distribution estimation unit estimates the transmission delay distribution information based on the transmission delay model learned using machine learning.

5. The remote control device according to claim 1 or 2, wherein: The moving object information includes a state quantity of the at least one moving object obtained by a sensor, The moving body control unit includes a moving body estimation unit that estimates a coefficient distribution as a probability distribution of the coefficients for the state quantity. The gain setting unit sets the control gain based on the transmission delay distribution information and the coefficient distribution.

6. The remote control device according to claim 1, wherein: The transmission delay distribution estimation unit models the time-varying probability distribution using a hierarchical or non-hierarchical hidden Markov model.

7. The remote control device according to claim 2, wherein: The transmission delay distribution estimation unit models the probability distribution using a hierarchical or non-hierarchical hidden Markov model.

8. The remote control device according to any one of claims 1 to 7, wherein: The moving body control unit includes a control possibility determination unit that determines whether to continue to control or stop the at least one moving body based on the transmission delay distribution information. The control possibility determination unit stops the control of the at least one moving object when the transmission delay exceeds a predetermined value and when the stability of the control of the at least one moving object cannot be ensured.

9. The remote control device according to any one of claims 1 to 8, wherein: The at least one moving object is a plurality of moving objects, The trajectory generating unit generates the target trajectory for each of the plurality of moving objects.

10. The remote control device according to claim 9, wherein: When the transmission delay information of each of the plurality of moving objects is considered to be the same, the transmission delay distribution estimation unit groups the plurality of moving objects and estimates the probability distribution of the common transmission delay using any of the transmission delay information.

11. The remote control device according to any one of claims 1 to 10, wherein: When a signal is received with a delay time shorter than a preset minimum delay time, the gain setting unit sets the control gain based on the transmission delay distribution information taking the minimum delay time into consideration, The control amount is sent after the minimum delay time has passed.

12. A mobile body communicating with the remote control device according to claim 11, wherein: When receiving a signal with a delay time shorter than the minimum delay time, the moving object waits until the minimum delay time elapses.

13. A remote control system comprising: The remote control device according to claim 1 or 2; the network; and The at least one moving object, The remote control device controls the at least one moving object based on the control amount.

14. A remote control method for controlling at least one moving object via a transmission path including at least a network, wherein: Estimate transmission delay distribution information including a time-varying probability distribution of the estimated transmission delay based on the transmission delay information in the transmission path acquired in advance and the transmission delay information acquired online, generating a target trajectory of the at least one moving object based on surrounding information of the at least one moving object, generating a control amount of the at least one moving object based on a control gain set based on the transmission delay distribution information, the target trajectory, and moving object information of the at least one moving object, The at least one moving object is controlled according to the control amount.

15. A remote control method for controlling at least one moving object via a transmission path including at least a network, wherein: Estimate the transmission delay distribution information including the estimated probability distribution of the transmission delay and the current pattern or past pattern of the transmission delay based on the transmission delay information in the transmission path acquired in advance and the transmission delay information acquired online, generating a target trajectory of the at least one moving object based on surrounding information of the at least one moving object, generating a control amount of the at least one moving object based on a control gain set based on the transmission delay distribution information, the target trajectory, and moving object information of the at least one moving object, The at least one moving object is controlled according to the control amount.

Images