Information processing device, information processing method, and program

The control system addresses inefficient remote control due to communication delays by adjusting speed commands and displaying intended vs. actual speeds, ensuring efficient operation of remotely controlled robots.

JP7823528B2Active Publication Date: 2026-03-04OKI ELECTRIC INDUSTRY CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2022139370
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2022-09-01
Publication Date
2026-03-04
Estimated Expiration
2042-09-01

AI Technical Summary

Technical Problem

Existing remote control technologies struggle with communication delays, leading to inefficient movement of remotely controlled robots due to speed command limitations, which hinder efficient operation.

Method used

A control system that includes a command value generation unit and a correction unit to adjust speed commands based on the difference between the intended and actual speed of the moving body, correcting the command to account for communication delays, and displaying the intended and actual speeds for operator awareness.

Benefits of technology

This system allows for more efficient movement of remotely controlled objects by minimizing excessive movement caused by communication delays, enhancing operational efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007823528000002
    Figure 0007823528000002
  • Figure 0007823528000003
    Figure 0007823528000003
  • Figure 0007823528000004
    Figure 0007823528000004
Patent Text Reader

Abstract

To allow for a more efficient movement of a remotely controlled moving body.SOLUTION: An information processing device includes: a command value generation unit that generates a speed command for movement of a moving body existing in a remote location, based on an input to a control unit by an operator; and a command value correction unit that corrects the speed command based on a difference between a speed indicated by the speed command and a speed of the moving body that is received from the moving body, to generate a corrected speed command to be transmitted to the moving body.SELECTED DRAWING: Figure 5
Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] 2. Description of the Related Art In recent years, with the development of communication networks, it has become more common to remotely control devices such as robots or manipulators located in remote locations.

[0003] However, communication networks cause communication delays, so it is important to take communication delays into consideration in remote control, where real-time performance is important. Therefore, various technologies are being studied to prevent the deterioration of controllability due to communication delays in remote control.

[0004] For example, Patent Document 1 listed below discloses that in a remote control device that remotely controls a work machine via a communication network, an upper limit is set for the speed command value of a robot according to the measured communication delay time. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2019 / 225118 Summary of the Invention [Problem to be solved by the invention]

[0006] However, with the technology disclosed in Patent Document 1, when a communication delay occurs, the speed command value is limited, making it difficult to move the robot efficiently.

[0007] Therefore, the present invention has been made in consideration of the above problems, and an object of the present invention is to provide a new and improved information processing device, information processing method, and program that are capable of moving a remotely controlled moving object more efficiently. [Means for solving the problem]

[0008] In order to solve the above problem, according to one aspect of the present invention, a control system is provided which includes: a command value generation unit that generates a speed command for movement of a moving body located at a remote location based on an input to a control unit by an operator; and a command value correction unit that corrects the speed command based on a difference between a speed indicated by the speed command and a speed of the moving body received from the moving body, thereby generating a corrected speed command to be transmitted to the moving body. the corrected speed command is transmitted to the moving body via a network in which a communication delay occurs in transmission and reception, and the speed of the moving body received from the moving body is the speed instructed in the corrected speed command transmitted to the moving body immediately before. An information processing device is provided.

[0012] The command value correction unit may correct the speed command so that the speed commanded by the speed command becomes slower as the difference between the speed commanded by the speed command and the speed of the moving body received from the moving body becomes larger.

[0014] The operator may be presented with an image captured by an imaging device mounted on the moving body and transmitted via the network.

[0015] The operator may be presented with an image showing, in parallel, the speed indicated by the speed command generated based on input from the control unit and the speed indicated by the corrected speed command.

[0016] In order to solve the above problem, according to another aspect of the present invention, the method includes the steps of: generating, by a calculation processing device, a speed command for movement of a moving body located at a remote location based on an input to a control unit by an operator; and generating a corrected speed command to be transmitted to the moving body by correcting the speed command based on a difference between a speed indicated by the speed command and a speed of the moving body received from the moving body. the corrected speed command is transmitted to the moving body via a network in which a communication delay occurs in transmission and reception, and the speed of the moving body received from the moving body is the speed instructed in the corrected speed command transmitted to the moving body immediately before. A method for processing information is provided.

[0017] In order to solve the above problem, according to another aspect of the present invention, a computer is caused to function as a command value generating unit that generates a speed command for movement of a moving body located at a remote location based on an input to a control unit by an operator, and a command value correcting unit that corrects the speed command based on a difference between a speed indicated by the speed command and a speed of the moving body received from the moving body, thereby generating a corrected speed command to be transmitted to the moving body. the corrected speed command is transmitted to the moving body via a network in which a communication delay occurs in transmission and reception, and the speed of the moving body received from the moving body is the speed instructed in the corrected speed command transmitted to the moving body immediately before. Programs are offered. [Effects of the Invention]

[0018] As described above, according to the present invention, it is possible to move a remotely controlled moving object more efficiently. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a schematic diagram showing the overall configuration of a remote control system according to an embodiment of the present invention; [Figure 2] FIG. 2 is a schematic diagram showing a specific configuration of a moving body. [Figure 3] FIG. 2 is a schematic diagram showing a specific configuration of a moving body. [Figure 4] FIG. 2 is a block diagram showing the functional configuration of a mobile object and an information processing device. [Figure 5] FIG. 2 is a block diagram showing processing executed by a command value generating unit and a command value correcting unit of the information processing device. [Figure 6] 10 is a graph showing the forward movement amount of a moving body when a speed command is corrected due to a communication delay. FIG. [Figure 7] 7 is a graph showing a comparison between Vref and Vcmd in the communication delay shown in FIG. 6. FIG. [Figure 8] 10 is a graph showing the forward movement amount of a moving body when a speed command is not corrected due to a communication delay. FIG. [Figure 9] FIG. 10 is an explanatory diagram showing an example of a display image generated by an information processing device according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant explanations will be omitted.

[0021] <1. Remote control system> First, the overall configuration of a remote control system according to one embodiment of the present invention will be described with reference to Figures 1 to 3. Figure 1 is a schematic diagram showing the overall configuration of a remote control system according to this embodiment. Figures 2 and 3 are schematic diagrams showing the specific configuration of a moving object 1.

[0022] 1, the remote control system includes a mobile object 1, an information processing device 20, a display device 21, and a control device 30. The mobile object 1 and the information processing device 20 are located in remote locations and are communicably connected to each other via a network 40. An operator (not shown) can thereby remotely control the mobile object 1 located in a remote location by operating the control device 30 while visually checking the images and information displayed on the display device 21.

[0023] The moving body 1 is a robotic device that includes a movement mechanism and moves based on inputs input to a control device 30. The moving body 1 may be connected to a network 40 via, for example, wireless communication. For example, as shown in FIGS. 2 and 3, the moving body 1 may be a robotic device that includes a movement mechanism 11 and imaging devices 12a, 12b, 12c, and 12d.

[0024] The moving mechanism 11 is a mechanism that is driven by power and can move the moving body 1 to any location. The moving mechanism 11 may be two or more wheels, two or more legs, two or more leg-wheels, caterpillar tracks, or air levitation. Furthermore, the moving mechanism 11 may be a mechanism that can fly in the air, such as a fixed-wing or rotary-wing mechanism.

[0025] The imaging devices 12a, 12b, 12c, and 12d are mounted on the front, rear, left, and right sides of the moving object 1 and are cameras or the like that capture images of the environment around the moving object 1. The imaging devices 12a, 12b, 12c, and 12d may be wide-angle fisheye cameras with a viewing angle of nearly 180°. The images captured by the imaging devices 12a, 12b, 12c, and 12d are transmitted to the information processing device 20 via the network 40 and displayed on the display device 21. In this way, by visually checking the images displayed on the display device 21, the operator can grasp the state of the environment around the moving object 1 and monitor the state of the moving object 1.

[0026] The information processing device 20 controls the input and output of information between the moving body 1 and the operator by controlling the input and output of the display device 21 and the control device 30. Specifically, the information processing device 20 can present the images and information received from the moving body 1 to the operator by generating images to be displayed on the display device 21 based on images and information received from the moving body 1 via the network 40. In addition, the information processing device 20 can transmit remote control from the operator to the moving body 1 by generating movement commands for the moving body 1 based on input to the control device 30 from the operator.

[0027] The display device 21 is an image display device that displays an image generated by the information processing device 20. The display device 21 may be, for example, a CRT (Cathode Ray Tube) display device, a Liquid Crystal Display (LCD) device, or an OLED (Organic Light Emitting Diode) device. The display device 21 displays an image generated based on the image and information received from the moving object 1.

[0028] The control device 30 is an input device into which an operator inputs movement instructions to the moving body 1. The control device 30 may be composed of, for example, an input means such as a button, a switch, a lever, or a handle, and an input control circuit that generates a signal based on the input to the input means. The information processing device 20 can acquire movement instructions from the control device 30 to the moving body 1 given by the operator.

[0029] The network 40 is a wired or wirelessly connected communication network. For example, the network 40 may be the Internet communication network, a home LAN (Local Area Network), an in-house LAN, an infrared communication network, a radio wave communication network, a satellite communication network, or the like.

[0030] The moving body 1 and the information processing device 20 transmit and receive data to and from each other via a network 40. Specifically, the moving body 1 can transmit, to the information processing device 20, via the network 40, images captured by the imaging devices 12a, 12b, 12c, and 12d mounted on the moving body 1, information related to the movement of the moving body 1, information related to the power source of the moving body 1, and the like. In addition, the information processing device 20 can transmit, to the moving body 1, movement commands generated based on input from an operator to the control device 30, via the network 40.

[0031] However, when transmitting and receiving data via the network 40, communication delays occur depending on the amount of data being transmitted and received and the line speed of the network 40. As a result, a time lag occurs between the actual movement of the moving object 1 and the input to the control device 30 by the operator viewing the display device 21. In the remote control system according to this embodiment, when a difference occurs between the actual movement speed of the moving object 1 and the movement speed instructed by the operator due to the time lag, the movement speed instructed by the operator can be corrected to prevent excessive movement. As a result, the remote control system according to this embodiment can move the moving object 1 more efficiently than when the movement speed of the moving object 1 is constantly limited.

[0032] <2. Configuration of information processing device> Next, a specific functional configuration of the above-mentioned moving object 1 and information processing device 20 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the functional configuration of the moving object 1 and information processing device 20.

[0033] (Mobile unit 1) As shown in FIG. 4, the moving object 1 includes a calculation unit 110, an input / output unit 120, a communication unit 130, a storage unit 140, and a power supply unit 150.

[0034] The input / output unit 120 is a connection interface that inputs and outputs various data between the imaging device 12 (i.e., the imaging devices 12a, 12b, 12c, and 12d), the moving mechanism 11, the encoder unit 13, and the calculation unit 110. The input / output unit 120 may be, for example, a Universal Serial Bus (USB) port, an IEEE 1394 port, a Small Computer System Interface (SCSI) port, an RS-232C port, an optical audio terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) port, or the like.

[0035] The communication unit 130 is a communication interface configured with a communication device or the like for connecting to the network 40. The communication unit 130 may be, for example, a communication card for a wireless LAN (Local Area Network), Wi-Fi (registered trademark), Bluetooth (registered trademark), or WUSB (Wireless USB).

[0036] The storage unit 140 is a device for storing data of the mobile object 1. The storage unit 140 may include, for example, a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded on the storage medium. The storage unit 140 may store map data of a specific base acquired from the information processing device 20, or may store parameters related to the movement of the mobile object 1.

[0037] The power supply unit 150 is a power source for the moving object 1 and supplies power to each part of the moving object 1. The power supply unit 150 may include a battery that stores power. The power supply unit 150 may include, for example, a lithium ion secondary battery.

[0038] The calculation unit 110 performs all data processing in the moving body 1 and controls all operations of the moving body 1. The calculation unit 110 may perform data processing and operation control of the moving body 1 through cooperation of a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory). The CPU is an arithmetic processing device that performs calculations and control according to various programs. The ROM stores programs used by the CPU, calculation parameters, etc. The RAM temporarily stores programs used in the execution of the CPU, and parameters that change as appropriate during program execution, etc.

[0039] Specifically, the calculation unit 110 includes a drive control unit 111 , a communication processing unit 113 , a video encoding unit 114 , and a battery monitoring unit 115 .

[0040] The drive control unit 111 drives the moving mechanism 11 to execute the movement command received from the information processing device 20. Specifically, the drive control unit 111 may drive the moving mechanism 11 to move in the direction and at the speed instructed by the movement command, based on information relating to the driving status of the moving mechanism 11 measured by the encoder unit 13.

[0041] The video encoding unit 114 compresses the captured images of the surrounding environment of the moving object 1 acquired from the imaging device 12. The compressed captured images are transmitted to the information processing device 20 via the network 40. The video encoding unit 114 can reduce the amount of data of the captured images through compression, thereby reducing communication delays when transmitting the captured images.

[0042] The battery monitoring unit 115 monitors the state of the power supply unit 150. Specifically, the battery monitoring unit 115 monitors the state of the power supply unit 150 by acquiring information about the battery included in the power supply unit 150 from the power supply unit 150. For example, the battery monitoring unit 115 may acquire from the power supply unit 150 information about the remaining capacity of the battery included in the power supply unit 150, information about error notifications, and the like.

[0043] The communication processing unit 113 controls communication between the moving object 1 and the information processing device 20. Specifically, the communication processing unit 113 may transmit to the information processing device 20 a compressed image of the surrounding environment of the moving object 1, information indicating the moving state of the moving object 1, information regarding the remaining charge of the battery included in the power supply unit 150, and the like. The communication processing unit 113 may also receive a movement command from the information processing device 20 instructing the moving object 1 to move.

[0044] (information processing device 20) The information processing device 20 includes a calculation unit 210 , an input / output unit 220 , a communication unit 230 , and a storage unit 240 .

[0045] The input / output unit 220 is a connection interface that inputs and outputs various data between the display device 21 and the control device 30 and the calculation unit 210. The input / output unit 120 may be, for example, a USB port, an IEEE1394 port, or a SCSI port, or may be an RS-232C port, an optical audio terminal, or an HDMI (registered trademark) port.

[0046] The communication unit 230 is a communication interface configured with a communication device or the like for connecting to the network 40. The communication unit 230 may be, for example, a wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), or a communication card for WUSB.

[0047] The storage unit 240 is a device for storing data of the information processing device 20. The storage unit 240 may include, for example, a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded on the storage medium. The storage unit 240 may store map data of multiple bases, and may also store a log of movement commands executed by the mobile body 1. Note that the map data of a specific base stored in the storage unit 240 may be transmitted to the mobile body 1 according to the base at which the mobile body 1 is located.

[0048] The calculation unit 210 performs all data processing in the information processing device 20 and controls all operations of the information processing device 20. The calculation unit 210 may perform data processing and operation control of the information processing device 20 through cooperation of a CPU, a ROM, and a RAM. The CPU is an arithmetic processing device that performs calculations and control according to various programs. The ROM stores programs used by the CPU, calculation parameters, etc. The RAM temporarily stores programs used in the execution of the CPU, and parameters that change as appropriate during program execution, etc.

[0049] Specifically, the calculation unit 210 includes a command value generation unit 211 , a command value correction unit 212 , a communication processing unit 213 , and a video decoding unit 214 .

[0050] The command value generation unit 211 generates a movement command for the moving body 1 based on an input by the operator to the control device 30. Specifically, the command value generation unit 211 may generate a movement command for the moving body 1 based on an input by the operator to the control device 30. The movement command generated by the command value generation unit 211 includes a direction command that instructs the movement direction of the moving body 1 and a speed command that instructs the movement speed of the moving body 1.

[0051] The command value corrector 212 corrects the speed command to be sent to the moving body 1 based on the difference between the moving speed indicated by the speed command generated by the command value generator 211 and the moving speed of the moving body 1 received from the moving body 1. Specifically, the command value corrector 212 may correct the speed command so that the moving speed indicated by the speed command becomes slower the greater the difference between the moving speed indicated by the speed command generated by the command value generator 211 and the moving speed of the moving body 1 received from the moving body 1. The moving speed of the moving body 1 received from the moving body 1 is, in other words, the actual moving speed of the moving body 1. Specifically, the moving speed of the moving body 1 received from the moving body 1 may be the moving speed actually measured by the moving body 1, or may be the moving speed indicated by a speed command transmitted to the moving body 1 at an earlier point in time.

[0052] A large difference between the movement speed instructed by the speed command and the actual movement speed of the moving body 1 means that the movement speed instructed by the operator and the actual movement speed of the moving body 1 are separated due to a time lag caused by communication delays in the network 40. Therefore, the command value corrector 212 corrects the speed command so that the movement speed instructed by the operator to the moving body 1 becomes slower, in order to prevent the moving body 1 from moving excessively. In this way, the information processing device 20 can prevent the moving body 1 from moving excessively by controlling the moving body 1 so that the movement amount is smaller than the movement command generated based on the operator's input.

[0053] On the other hand, even if a communication delay occurs, if the difference between the movement speed commanded by the speed command and the actual movement speed of the moving body 1 is small, the command value corrector 212 reduces the amount of correction to the movement speed commanded by the speed command generated by the command value generator 211. In this way, the information processing device 20 can move the moving body 1 more efficiently by correcting the movement speed commanded by the speed command only when the difference between the movement speed commanded by the speed command and the actual movement speed of the moving body 1 is large and there is a high possibility that the moving body 1 will move excessively.

[0054] The video decoding unit 214 decompresses the compressed image of the surrounding environment of the moving object 1 received from the moving object 1, and performs image processing on the decompressed captured image of the surrounding environment of the moving object 1, thereby generating an image to be displayed on the display device 21. Specifically, the video decoding unit 214 may generate a bird's-eye image of the moving object 1 from above by performing image processing on the captured image of the surrounding environment of the moving object 1. Furthermore, the video decoding unit 214 may change the line of sight direction of the generated bird's-eye image to any direction based on input from an operator.

[0055] The communication processing unit 213 controls communication between the information processing device 20 and the mobile object 1. Specifically, the communication processing unit 213 may transmit a movement command to the mobile object 1 to instruct the mobile object 1 to move. The communication processing unit 213 may also receive from the mobile object 1 an image captured by the imaging device 12, information indicating the movement state of the mobile object 1, information regarding the remaining charge of the battery included in the power supply unit 150, and the like.

[0056] According to the above configuration, the information processing device 20 according to this embodiment can suppress the movement of the moving body 1 when there is a discrepancy between the movement speed indicated by the speed command generated based on the operator's input and the actual movement speed of the moving body 1. Therefore, the information processing device 20 according to this embodiment can suppress the moving body 1 from moving excessively without setting an upper limit on the speed command to the moving body 1, and can move the moving body 1 more efficiently.

[0057] <3. Operation of the information processing device> Next, a specific flow of operations of the information processing device 20 will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the processes executed by the command value generating unit 211 and the command value correcting unit 212 of the information processing device 20.

[0058] As shown in FIG. 5, the command value generator 211 generates a speed command V ref Generates a speed command V ref is a movement command that instructs the moving speed of the moving body 1 when it moves forward, backward, turns, or moves in translation to the left or right.

[0059] Here, it is assumed that the network 40 connecting the information processing device 20 and the mobile body 1 causes a delay of T1 seconds in data transmission from the information processing device 20 to the mobile body 1, and a delay of T2 seconds in data transmission from the mobile body 1 to the information processing device 20. Therefore, in such a network 40, a delay of (T1+T2) seconds occurs from the time an operator inputs data to the mobile body 1 until a response to the input is returned from the mobile body 1.

[0060] Therefore, the command value corrector 212 corrects the speed command V ref and the reply speed V received from mobile unit 1. msg Based on the difference between ref By correcting the velocity command V cmd The reply speed V received from mobile station 1 is generated. msg is the speed command V received by the moving object 1. cmd , and the speed command V generated by the command value generating unit 211 ref For (T1+T2) seconds ago, the speed command is 。

[0061] For example, the command value corrector 212 calculates the speed command V to be transmitted to the moving object 1 by using the following equation (1): cmd In equation (1), C1 is a feedback coefficient and the delay difference (V ref -V msg ) is a coefficient that determines the magnitude of the influence of the fractional term (|V ref | / V max ) is a delayed differential (V ref -V msg ) is a variable gain that suppresses the influence of max is the maximum speed command generated by the operator's input).

[0062]

number

[0063] The speed command V corrected by the command value corrector 212 cmd is transmitted to the moving object 1 via the network 40. cmd The moving object 1 receives the speed command V cmd The moving mechanism 11 can be controlled based on the velocity command V ref The received speed command V cmd Reply speed V msg The moving body 1 returns the odometry information Odom relating to the position and posture of the moving mechanism 11 to the information processing device 20. msg , and the captured image of the imaging device 12 may be transmitted to the information processing device 20.

[0064] This enables the information processing device 20 to correct the speed command sent to the moving body 1 in a direction that suppresses the movement of the moving body 1 based on the difference between the actual moving speed caused by communication delay and the instructed moving speed.

[0065] The function of correcting the speed command caused by the communication delay may be turned on or off by an operator. Also, the value of the feedback coefficient C1 may be arbitrarily controlled by an operator.

[0066] <4. Action and Effects> Next, the operation and effect of the information processing device 20 according to this embodiment will be described with reference to Figs. 6 to 8. Fig. 6 is a graph showing the forward movement amount of the moving body 1 when the speed command is corrected due to the communication delay. Fig. 7 is a graph showing the V ref and V cmd 8 is a graph showing a comparison between the amount of forward movement of the moving body 1 when the speed command is not corrected due to communication delay.

[0067] Speed ​​command V input by operator refFor example, the speed command V is a trapezoidal waveform pattern command that accelerates to 0.4 m / s in 0.5 seconds after the start, maintains the speed for 1 second, and then decelerates and stops in 0.5 seconds. Therefore, the speed command V ref If the moving object 1 moves as shown, the moving object 1 will move 0.6 m, which is equivalent to the area of ​​the trapezoidal waveform.

[0068] For example, as shown in Figure 6, when the speed command is corrected due to communication delay, the speed command V ref The corrected speed command V cmd is transmitted to the moving object 1. In this case, the actual moving speed of the moving object 1 and the speed command V ref Since the moving speed indicated by V is different from the moving speed indicated by V, the corrected speed command V is cmd The speed commanded by the delay time decreases, thereby suppressing the forward movement amount of the moving body 1. In other words, the movement speed of the moving body 1 decreases as the delay time increases, and therefore the forward movement amount of the moving body 1 decreases.

[0069] On the other hand, as shown in Fig. 8, if the speed command is not corrected due to communication delay, the speed command V ref is transmitted to the moving body 1, the forward movement amount of the moving body 1 will be the same regardless of the delay time due to communication delay.

[0070] Therefore, when an operator remotely controls the moving body 1 while checking the moving position of the moving body 1 on the display device 21, the moving body 1 will move too far relative to the target moving position as the delay time increases. The information processing device 20 according to this embodiment can correct the speed command due to communication delays and suppress the forward movement amount and movement speed of the moving body 1, thereby suppressing excessive movement of the moving body 1. Therefore, the information processing device 20 according to this embodiment can move the moving body 1 more efficiently.

[0071] <5. Variations> Furthermore, a modified example of the information processing device 20 according to the present embodiment will be described with reference to Fig. 9. Fig. 9 is an explanatory diagram showing an example of a display image generated by the information processing device 20 according to the modified example.

[0072] A modified example of the information processing device 20 may generate a display image PV shown in Fig. 9. The display image PV shown in Fig. 9 is displayed on the display device 21, for example, and is presented to the operator.

[0073] The display image PV shown in FIG. 9 includes a captured image display area PVa, an overhead image display area PVb, and a maneuvering delay display area PVc.

[0074] The captured image display area PVa is an area where an image captured by the imaging device 12 mounted on the moving body 1 is displayed. For example, the captured image display area PVa may display an image of the environment in the traveling direction of the moving body 1. The overhead image display area PVb is an area where an overhead image generated by further image processing the image captured by the imaging device 12 is displayed. Note that the viewpoints of the images displayed in the captured image display area PVa and the overhead image display area PVb may be changed as appropriate based on input from the operator.

[0075] The steering delay display area PVc is an area that displays, in comparison, the moving speed indicated by the moving command generated based on the input to the control device 30 by the operator and the moving speed indicated by the moving command transmitted to the moving body 1. For example, the steering delay display area PVc displays the moving speed indicated by the speed command V generated based on the input to the control device 30 by the operator for each of the forward / backward direction, left / right direction, and turning direction of the moving body 1. ref and the speed command V sent to the moving object 1. cmd The moving speed indicated by the arrow may be displayed in parallel with the moving speed indicated by the arrow.

[0076] In the display image PV shown in FIG. 9, the speed command V ref The moving speed commanded by V is displayed as bar-shaped images FBi, LRi, and RTi that expand and contract in the horizontal direction for the forward / backward, left / right, and turning directions of the moving body 1. Similarly, the speed command Vcmd The moving speed instructed by is displayed by bar-shaped images FBc, LRc, and RTc that expand and contract laterally for the forward / backward, left / right, and turning directions of the moving body 1. These images may be displayed at an update cycle synchronized with the input to the control device 30 by the operator.

[0077] In a modified example of the information processing device 20, an operator who views the display image PV can determine the velocity command V generated based on an input to the control device 30. ref and the speed command V sent to the moving object 1. cmd Therefore, the operator can check the moving speed by comparing it with the speed command V ref and the speed command V cmd It is possible to recognize the presence and degree of communication delay from the difference between the moving speed commanded by the speed command V ref The speed command V cmd When the moving speed commanded by V is decreasing, the operator must consider the difference between the actual moving speed of the moving object 1 and the speed command V due to the communication delay. ref Therefore, the modified example of the information processing device 20 can prevent the operator from making excessive inputs to the control device 30.

[0078] Although the preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person skilled in the art to which the present invention pertains can conceive of various modifications and alterations within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present invention.

[0079] Furthermore, the series of processes performed by the information processing device described in this specification may be realized using software, hardware, or a combination of software and hardware. The programs constituting the software are stored in advance, for example, in a storage medium (non-transitory medium) provided inside or outside each device. Then, each program is loaded into RAM when executed by a computer, for example, and executed by a processor such as a CPU. The storage medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, etc. Furthermore, the computer program may be distributed, for example, via a network, without using a storage medium. [Explanation of symbols]

[0080] 1. Mobile 11 Moving mechanism 12 Imaging device 13 Encoder section 20 Information processing equipment 21 Display device 30 Controls 40 Network 110 Arithmetic section 111 Drive control unit 113 Communication processing unit 114 Video Encoding Unit 115 Battery monitoring unit 120 Input / output section 130 Communications Department 140 Storage section 150 Power supply section 210 Arithmetic section 211 Command value generation unit 212 Command value correction unit 213 Communication processing unit 214 Video Decoder 220 Input / output section 230 Communications Department 240 Storage section

Claims

1. a command value generating unit that generates a speed command for the movement of a moving object located at a remote location based on an input to the control unit by an operator; a command value correcting unit that corrects the speed command based on a difference between a speed indicated by the speed command and a speed of the moving body received from the moving body, thereby generating a corrected speed command to be transmitted to the moving body; Equipped with the corrected speed command is transmitted to the moving body via a network in which a communication delay occurs in transmission and reception; An information processing device wherein the speed of the moving body received from the moving body is the speed instructed in the corrected speed command transmitted to the moving body immediately before.

2. 2. The information processing device according to claim 1, wherein the command value correction unit corrects the speed command so that the speed commanded by the speed command becomes slower as the difference between the speed commanded by the speed command and the speed of the moving body received from the moving body increases.

3. The information processing apparatus according to claim 1 , wherein the operator is presented with an image captured by an imaging device mounted on the moving object and transmitted via the network.

4. The information processing device according to claim 3 , wherein the operator is presented with an image showing, in parallel, the speed indicated by the speed command generated based on the input from the control unit and the speed indicated by the corrected speed command.

5. By the processing unit, generating a speed command for the movement of the mobile object located at a remote location based on an input to the control unit by an operator; generating a corrected speed command to be transmitted to the moving body by correcting the speed command based on a difference between a speed indicated by the speed command and a speed of the moving body received from the moving body; Including, the corrected speed command is transmitted to the moving body via a network in which a communication delay occurs in transmission and reception; An information processing method, wherein the speed of the moving body received from the moving body is the speed instructed in the corrected speed command transmitted to the moving body immediately before.

6. Computer, a command value generating unit that generates a speed command for the movement of a moving object located at a remote location based on an input to the control unit by an operator; a command value correcting unit that corrects the speed command based on a difference between a speed indicated by the speed command and a speed of the moving body received from the moving body, thereby generating a corrected speed command to be transmitted to the moving body; It functions as the corrected speed command is transmitted to the moving body via a network in which a communication delay occurs in transmission and reception; The program, wherein the speed of the moving body received from the moving body is the speed instructed in the corrected speed command transmitted to the moving body immediately before.

Citation Information

Patent Citations

  • Operating method and operation device

    JP2009282720A

  • Remote control device, remote control system, remote control method, and remote control program

    JP2018107568A

  • Remote operation device, remote operation method, non-temporary computer-readable medium, and remote operation system

    WO2019225118A1