Control device

Abstract

This invention provides a control device that improves the operability of a mobile object during remote operation. The control device includes: a communication unit for communicating with the mobile object; and a control unit for outputting information to overlay a predetermined path of the mobile object onto an image captured in front of the mobile object's direction of travel. When the mobile object's travel speed is a first travel speed, the length of the predetermined path is a length corresponding to the first travel speed; when the mobile object's travel speed is a second travel speed less than the first travel speed, the length of the predetermined path is a predetermined length.

Description

Technical Field

[0001] This disclosure relates to control devices. Background Technology

[0002] Various technologies for remotely operating mobile bodies such as walking robots have been proposed. For example, Patent Document 1 discloses a technology in which a marker indicating the target position toward which the mobile body is being operated is displayed in a remote operating device.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2014-106576

[0004] When the difficulty of remote operation varies depending on the surrounding environment and speed of the mobile object's movement path, there is room to improve the operability of remote operation of the mobile object. Summary of the Invention

[0005] The following discloses a control device that improves operability in remote operation of a mobile body.

[0006] The control device disclosed herein includes: a communication unit for communicating with a mobile body; and a control unit for outputting information to overlay a predetermined path of the mobile body onto an image captured in front of the mobile body in its direction of travel. When the mobile body's speed is a first speed, the length of the predetermined path is a length corresponding to the first speed. When the mobile body's speed is a second speed, which is less than the first speed, the length of the predetermined path is a predetermined length.

[0007] According to the control device of this disclosure, the operability of remote operation of a mobile body can be improved. Attached Figure Description

[0008] Figure 1 This is a diagram illustrating an example of the structure of a remote operating system.

[0009] Figure 2 It is a sequence diagram representing the action steps of a remote operating system.

[0010] Figure 3 This is a diagram showing an example of a screen used for operation.

[0011] Figure 4 This is a diagram illustrating the predetermined path length.

[0012] Explanation of reference numerals in the attached figures

[0013] 1…Remote operating system; 10…Control device; 12…Mobile body; 101…Communication unit; 102…Storage unit; 103…Control unit; 105…Input unit; 106…Output unit. Detailed Implementation

[0014] The implementation method will be described below.

[0015] Figure 1 This diagram illustrates an example of the configuration of a remote operating system in one embodiment. The remote operating system 1 includes a mobile body 12 and a control device 10 for remotely operating the mobile body 12. The mobile body 12 and the control device 10 are connected via a network 11 and mediated by a server device 13, enabling them to communicate with each other. The mobile body 12 is, for example, a walking robot used for transporting goods in warehouses, commercial facilities, etc. The control device 10 is an information processing device with communication capabilities and an interface for user operation, such as a PC (personal computer), a smartphone terminal, a tablet terminal, or an operating terminal for the mobile body 12. The network 11 is, for example, a network, but it can also be an ad hoc network, a LAN (Local Area Network), a MAN (Metropolitan Area Network), or other networks or any combination thereof. The server device 13 is, for example, a server computer belonging to a cloud computing system or other computing system, equipped with various functions.

[0016] In the remote operating system 1 of this embodiment, the control device 10 includes: a communication unit 101 for communicating with the mobile body 12; and a control unit 103 for outputting information for overlaying a predetermined path of the mobile body 12 onto a captured image (hereinafter referred to as the forward image) in the direction of travel of the mobile body 12. Here, when the moving speed of the mobile body 12 is a first moving speed, the length of the displayed predetermined path is a length corresponding to the first moving speed; when the moving speed of the mobile body 12 is a second moving speed less than the first moving speed, the length of the displayed predetermined path is a predetermined length. When the control device 10 displays the forward image of the mobile body 12 toward the user and accepts remote operation of the mobile body 12 by the user, by displaying the predetermined path of travel in the forward image, the user can pre-confirm the presence or absence of obstacles around the predetermined path of travel of the mobile body 12. At this point, the greater the moving speed of the mobile body 12, the longer the length of the predetermined path will be displayed; conversely, the smaller the moving speed, the shorter the length of the predetermined path will be displayed. Thus, regardless of the moving speed of the mobile body 12, the user can always confirm the predetermined path that the mobile body 12 will reach in a constant time, such as 1 to a few seconds. However, when the speed of the mobile body 12 is low enough, for example, when careful operation is required, the length of the predetermined path will be displayed as a fixed length longer than the length corresponding to the moving speed of the mobile body 12. This ensures the predictability of the predetermined path for the user. Therefore, the operability of remote operation of the mobile body 12 can be improved.

[0017] like Figure 1 As shown, the control device 10 includes a communication unit 101, a storage unit 102, a control unit 103, an input unit 105, and an output unit 106.

[0018] The communication unit 101 is equipped with modules corresponding to communication standards such as wired LAN, wireless LAN, LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation), and is connected to the network 11 via a router or base station. Through the communication unit 101, the control device 10 communicates with the mobile body 12 via the server device 13 through the network 11.

[0019] Storage unit 102 includes semiconductor memory, magnetic memory, or optical memory. Semiconductor memory includes RAM (Random Access Memory) or ROM (Read Only Memory). RAM includes SRAM (Static RAM) or DRAM (Dynamic RAM), and ROM includes EEPROM (Electrically Erasable Programmable ROM). Storage unit 102 stores the control program of control device 10, information required for operation, or operation results.

[0020] The control unit 103 includes processors such as CPU (Central Processing Unit) and GPU (Graphics Processing Unit), or dedicated circuits such as FPGA (Field-Programmable Gate Array) and ASIC (Application-Specific Integrated Circuit), and performs information processing of the control device 10.

[0021] The input unit 105 is equipped with a microphone, electrostatic capacitive keys, a touch screen, an indicator device, or physical keys and joysticks, etc., and accepts user operations and sends the input information corresponding to the operation to the control unit 103.

[0022] The output unit 106 includes a speaker or a display. The display may be an LCD (Liquid Crystal Display) or an organic EL (Organic Electroluminescence), and the output unit 103 displays the processing results.

[0023] The mobile body 12 is a walking robot equipped with a mechanism for transporting goods and a walking mechanism including wheels. It has a communication unit 121, a control unit 123, a sensor unit 127, and a drive mechanism 128. The communication unit 121 is compatible with communication standards such as wired LAN, wireless LAN, LTE, 4G, and 5G, and communicates with the control device 10. The control unit 123 includes a processor such as a CPU or a dedicated circuit such as an ASIC. While communicating with the control device 10 through the communication unit 121, it uniformly controls the movement of the mobile body 12. The sensor unit 127 includes an accelerometer, a gyroscope, a distance sensor, a camera, and a GPS (Global Positioning System) module. It acquires detection results related to the surrounding environment, detection results of the movement state of the mobile body 12, captured images of the surrounding environment, and information such as the current position of the mobile body 12, and sends them to the control unit 103. The camera of the sensor unit 127 is positioned so that the environment in front of the mobile body 12 is within its field of view at least in the direction of movement when the mobile body 12 moves. The control unit 103 transmits captured images and other data to the control device 10 via the communication unit 121, and receives instructions from the control device 10 to control the operation of the drive mechanism 128. The drive mechanism 128 includes wheels for walking, a motor for driving the wheels, and an actuator, and realizes the forward, backward, and rotational movements of the mobile body 12 based on instructions from the control unit 123. Power is supplied to the above-mentioned components from a battery mounted on the mobile body 12.

[0024] Figure 2 (A) is a sequence diagram used to illustrate the action steps of remote operating system 1. Figure 2 (A) represents an example of the steps involved in the coordinated action of the moving body 12 and the control device 10, which are executed at any period, such as tens of microseconds, when the moving body 12 moves.

[0025] The mobile body 12 detects and captures images of its state in the walking environment (S20), and sends information including speed information and a forward image to the control device 10 (S21). Within the mobile body 12, based on the control of the control unit 123, the sensor unit 127 detects the acceleration, speed, and current position of the mobile body 12, and captures images of the surrounding area in front of it, including the direction of travel. The control unit 123 transmits speed information indicating the speed of the mobile body 12 and information including a forward image via the communication unit 121.

[0026] Next, the control device 10 derives a predetermined travel path for the mobile body 12 based on the information obtained from the mobile body 12 (S22), generates an operation image (S23), and displays the operation image to the user (S24). In the control device 10, the control unit 103 performs noise removal, distortion correction, and other processing on the image in front of the mobile body 12, as well as recognition processing for various objects. Furthermore, the control unit 103 derives the predetermined travel path, generates a marker image representing the predetermined travel path, and overlays the marker image on the image in front to generate the operation image. Moreover, the control unit 103 displays the operation image on the display of the output unit 106. Figure 3 (A) shows a rough example of an image used for manipulation. Figure 3 (A) is an example of an operational image obtained by overlaying a marker image 31 representing the predetermined path of travel of the mobile body 12 onto a forward image 30 of the mobile body 12. This forward image 30, for example, includes an image of the interior environment of an office taken by the mobile body 12 moving within the office. The predetermined path of travel is shown as a marker image 31 extending forward in the direction of travel of the mobile body 12 moving in the passageway.

[0027] In this embodiment, when generating the marker image 31, the control unit 103 varies the predetermined path length, i.e., the distance L1 up to the front end of the marker image 31, according to the speed of the moving body 12. For example, as Figure 4 As shown in (A), the relationship between the moving speed and the predetermined path length is such that when the moving speed is greater than any reference speed Vr (e.g., 0.5 to 1 meter per second), the control unit 103 dynamically and linearly changes the predetermined path length L1 according to the moving speed V1. That is, the position of the beginning of the predetermined path length L1 corresponds to the position reached by the moving body 12 after any time, for example, 1 second, when moving at the moving speed V1. The control unit 103 can derive the predetermined path length corresponding to the moving speed using any mathematical formula or table data. On the other hand, when the moving speed is less than or equal to the reference speed Vr, the control unit 103 fixes the predetermined path length to Lf. Here, the predetermined path length Lf corresponds, for example, to the position reached by the moving body 12 after 1 second when moving at the reference speed Vr. As a variation, such as Figure 4 As shown in (B), the control unit 103 can also change the predetermined path length in such a way that, as the moving speed decreases below the reference speed Vr, the predetermined path length is gradually reduced. For example, corresponding to the moving speed V2, the predetermined path length L2 is gradually reduced towards the intercept of the predetermined path length L3. In this way, for example, a predetermined path length can be generated and displayed. Figure 3 (B) Such operations use images. Figure 3(B) shows the relationship with Figure 4 The pattern of the change in the predetermined path length shown in (A) corresponds to an example where the marked image 31 of the predetermined path length Lf is superimposed in the forward image 30 when the moving body 12 decelerates.

[0028] return Figure 2 (A) The control device 10 receives the operation performed by the user based on the operation image (S25) and sends the action instruction corresponding to the operation to the mobile body 12 (S26). The control unit 103 generates an action instruction corresponding to the user's operation to the input unit 105 and sends the action instruction through the communication unit 121. The action instruction is, for example, an instruction for acceleration, deceleration, or stopping, rotation, etc. Moreover, the mobile body 12 executes the action corresponding to the action instruction, that is, acceleration, deceleration, or stopping, rotation, etc. (S27).

[0029] Through the steps described above, the control device 10 displays an operation image to the user, allowing the user to remotely operate the mobile body 12 while simultaneously monitoring its movement. However, assuming that the predetermined path length varies linearly across the entire range of the mobile body 12's movement speed, and that the mobile body 12 moves at low speed, for example, in a congested passage, or alternately stops and moves, the marked image of the predetermined path shown in the operation image may be extremely short or absent, potentially preventing the user from predicting the mobile body 12's predetermined path. Therefore, as in this embodiment, even when the mobile body 12 moves at a relatively low speed or stops, a predetermined path of a certain length is displayed to ensure the predictability of the predetermined path during user operation.

[0030] Furthermore, when the control device 10 generates a marker image 31 representing a predetermined path, the control unit 103 can make the lateral width of the marker image 31 correspond to the width occupied by the moving body 12 when it moves. For example, the storage unit 102 stores information in advance for determining the area occupied by the moving body 12 when it moves in the captured image based on the installation position and viewing angle of the camera of the moving body 12. Using this information, the control unit 103 can generate the marker image 31 with a width corresponding to the lateral width of the moving body 12. This further improves operability for the user.

[0031] Furthermore, as the moving speed of the moving body 12 changes, from Figure 3 The operation of (A) is used to transfer images to... Figure 3 When the operation of (B) uses an image, the change in the predetermined path length stops, so the control unit 103 can, for example, display or output a sound such as "driving at low speed", thereby reducing user confusion.

[0032] In a modified example, if an obstacle is identified at a position that interferes with the marker image 31 on the predetermined path of travel, the length of the marker image 31 can be set to extend up to the position of the obstacle, and the operational image can include information notifying of the presence of the obstacle. For example, it can also be as follows: Figure 3 As shown in the example of the operation image (C), when the front image 30 contains an obstacle 32, the control unit 103 generates and overlays a marker image 31 up to the position of the obstacle 32, and overlays a message 33, such as "path not displayed," onto the operation image. The message 33 can also be output by sound.

[0033] In other variations, such as Figure 2 As shown in (B), the control unit 123 of the mobile body 12 performs the predetermined travel path derivation (S22) and the operation image generation (S23), and sends the operation image from the mobile body 12 to the control device 10 (S23'). Furthermore, the control unit 103 of the control device 10 outputs the received operation image to the output unit 106 and displays it on the output unit 106. Alternatively, the travel path can be derived in the mobile body 12, and the forward image and its result can be sent to the control device 10, while the control unit 103 of the control device 10 generates and outputs the operation image. This allows for the distribution of the burden of image processing, etc.

[0034] In the foregoing description, embodiments of the present disclosure have been described based on the accompanying drawings. However, it is worth noting that those skilled in the art can easily make various changes and modifications in accordance with the spirit of the present disclosure. Therefore, these changes and modifications should be considered to be included within the scope of the present disclosure. For example, the functions included in each functional unit and step can be reconfigured without logical conflict, or multiple units or steps can be combined into a single unit or step, or a single unit or step can be split.

Claims

1. A control device, wherein, have: The communications department communicates with the mobile device. as well as The control unit outputs information to overlay the predetermined path of the moving object onto an image captured in front of the moving object's direction of travel. When the moving speed of the aforementioned moving body is a first moving speed, the length of the predetermined travel path is the length corresponding to the first moving speed. When the moving speed of the aforementioned moving body is a second moving speed that is less than the first moving speed, the length of the predetermined travel path is a predetermined length.

2. The control device according to claim 1, wherein, The aforementioned predetermined path of travel has a width corresponding to the width of the aforementioned moving body in the aforementioned direction of travel.

3. The control device according to claim 1, wherein, If the captured image contains an object that interferes with the predetermined travel path, the control unit will output a specified notification.

4. The control device according to claim 1, wherein, It also has an input section that receives instructions for operating the aforementioned moving body. The control unit sends information for controlling the movement of the mobile body according to the instructions to the mobile body.

Images