Information processing method, program, and information processing device

Abstract

An information processing method according to one embodiment of the present disclosure comprises: acquiring a rotation amount of a handle that controls the movement of a moving body (S10); and causing a display device to display, on the basis of the acquired rotation amount, a handle image indicating a rotation state of the handle in a display mode corresponding to the rotation speed of the handle (S20).

Description

[Technical Field]

[0001] The present disclosure relates to an information processing method, a program, and an information processing device. [Background technology]

[0002] Conventionally, there are devices that, from the viewpoint of safety, present the operation amount of a steering wheel provided in a vehicle on a UI (User Interface). Patent Document 1 discloses a vehicle display device that displays the operation amount of an operation unit that adjusts and controls a specific device on a display unit arranged on an instrument panel or dashboard. In this vehicle display device, the operation unit is made up of a rotary operation unit that is rotated by manual operation by the driver. Furthermore, the display unit rotates in response to the rotation operation of the rotary operation unit, and forms and displays a display image that indicates the operation amount. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 9-292265 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides an information processing method that makes it easy to understand the rotation state of a steering wheel. [Means for solving the problem]

[0005] An information processing method according to one aspect of the present disclosure acquires the amount of rotation of a handle that controls the movement of a moving body, and displays a handle image indicating the rotation state of the handle on a display device based on the acquired amount of rotation, so that the display mode corresponds to the number of rotations of the handle.

[0006] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0007] According to the information processing method according to one aspect of the present disclosure, the rotation state of the steering wheel can be made easy to understand. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a configuration of a terminal according to an embodiment. [Figure 2] FIG. 2 is a diagram showing a display image displayed on the output device according to the embodiment. [Figure 3] FIG. 3 is a flowchart illustrating a processing procedure of the management device according to the first example. [Figure 4] FIG. 4 is a diagram showing a handle icon according to a first example. [Figure 5] FIG. 5 is a flowchart illustrating a processing procedure of a management device according to the second example. [Figure 6] FIG. 6 is a diagram showing a steering wheel icon according to the second example. [Figure 7] FIG. 7 is a diagram showing a handle icon according to a third example. [Figure 8] FIG. 8 is a flowchart illustrating a processing procedure of a management device according to the fourth example. [Figure 9] FIG. 9 is a flowchart illustrating a processing procedure of a management device according to the fifth example. [Figure 10] FIG. 10 is a diagram showing a handle icon according to the fifth example. [Figure 11] FIG. 11 is a flowchart illustrating a processing procedure of a management device according to the sixth example. [Figure 12] FIG. 12 is a diagram showing a handle icon according to the sixth example. [Figure 13] FIG. 13 is a flowchart illustrating a processing procedure of a management device according to the seventh example. [Figure 14] FIG. 14 is a diagram showing a handle icon according to the seventh example. [Figure 15] FIG. 15 is a flowchart illustrating a processing procedure of a management device according to an eighth example. [Figure 16] FIG. 16 is a diagram showing a handle icon according to the eighth example. [Figure 17] FIG. 17 is a flowchart illustrating a processing procedure of a management device according to a ninth example. [Figure 18] FIG. 18 is a diagram showing a handle icon according to the ninth example. [Figure 19] FIG. 19 is a flowchart illustrating a processing procedure of a management device according to a tenth example. [Figure 20] FIG. 20 is a flowchart showing the processing procedure of a management device according to the eleventh embodiment. [Figure 21] FIG. 21 is a diagram for explaining a method for calculating a forecasted trajectory. [Figure 22] FIG. 22 is a diagram showing a display image displayed on an output device according to the eleventh example. [Figure 23] FIG. 23 is a flowchart illustrating a processing procedure of a management device according to a twelfth example. [Figure 24] FIG. 24 is a diagram showing a display image displayed on an output device according to the twelfth example. [Figure 25] FIG. 25 is a flowchart showing an information processing method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] An information processing method according to a first aspect of the present disclosure acquires the amount of rotation of a handle that controls the movement of a moving body, and displays a handle image indicating the rotation state of the handle on a display device based on the acquired amount of rotation, so that the display mode corresponds to the number of rotations of the handle.

[0010] This allows the user to easily grasp the number of rotations of the steering wheel simply by checking the steering wheel image. Therefore, the information processing method according to the first aspect of the present disclosure makes it easy to understand the rotation state of the steering wheel.

[0011] Also, for example, in an information processing method according to a second aspect of the present disclosure, in the first aspect, in the process of displaying the steering wheel image on the display device, the steering wheel image is displayed on the display device in a display mode corresponding to the direction in which the steering wheel is being rotated.

[0012] This makes it easy to tell whether the steering wheel is turning right or left.

[0013] Also, for example, in an information processing method according to a third aspect of the present disclosure, in the first or second aspect, in the process of displaying the handle image on the display device, the handle image is displayed on the display device so that the display mode is different when the handle is being rotated and when the handle is not being rotated.

[0014] This makes it easy to see whether the steering wheel is being turned or not.

[0015] Also, for example, in an information processing method according to a fourth aspect of the present disclosure, in any one of the first to third aspects, the handle is an operating device connected to an information processing device that communicates with the mobile body and is operated by an operator, and in the process of displaying the handle image on the display device, the handle image is displayed on the display device so that it has a different display mode when the operating device and the information processing device are not connected or when the information processing device and the mobile body cannot communicate, and when the operating device and the information processing device are connected and the information processing device and the mobile body can communicate.

[0016] When the operation device and the information processing device are not connected, or when the information processing device and the moving body cannot communicate with each other, it is considered that the moving body is not being properly controlled by the operation of the operation device (steering wheel) by the operator. Therefore, this makes it easy to understand whether the moving body is being properly controlled by the operation of the operation device by the operator.

[0017] Also, for example, in an information processing method according to a fifth aspect of the present disclosure, in any one of the first to fourth aspects, in the process of displaying the handle image on the display device, the handle image including a numerical value indicating the amount of rotation is displayed on the display device.

[0018] This makes it easier to understand the amount of steering wheel rotation.

[0019] Also, for example, in an information processing method according to a sixth aspect of the present disclosure, in any one of the first to fifth aspects, in the process of displaying the handle image on the display device, the handle image is displayed on the display device so that the display mode differs between when the rotation amount is at the limit value and when the rotation amount is not at the limit value.

[0020] There is a limit to how far the steering wheel can be turned to the right or left. Therefore, by displaying a steering wheel image on the display device in a different display mode when the amount of steering wheel rotation is at the limit and when the amount of steering wheel rotation is not at the limit, it becomes easy to understand when the limit of steering wheel rotation has been reached.

[0021] Also, for example, in an information processing method according to a seventh aspect of the present disclosure, in any one of the first to sixth aspects, the process of displaying the steering wheel image on the display device causes the steering wheel image to be displayed on the display device, the steering wheel image including a steering angle image indicating the steering angle of the moving body.

[0022] This makes it easier to understand the steering angle of the moving object.

[0023] Also, for example, in an information processing method according to an eighth aspect of the present disclosure, in any one of the first to seventh aspects, the process of estimating the travel path of the moving body based on the steering angle of the moving body and displaying the handle image on the display device further causes the display device to display a predicted trajectory image showing the estimated travel path.

[0024] This makes it easier to understand how the moving object moves.

[0025] Furthermore, for example, a program according to a ninth aspect of the present disclosure causes a computer to execute the information processing method according to any one of the first to eighth aspects.

[0026] This provides the same effects as the information processing method according to at least one of the first to eighth aspects of the present disclosure.

[0027] Also, for example, an information processing device according to a tenth aspect of the present disclosure includes an acquisition unit that acquires the amount of rotation of a handle that controls the movement of a moving body, and a control unit that displays a handle image indicating the rotation state of the handle on a display device based on the acquired amount of rotation so that the display mode corresponds to the number of rotations of the handle.

[0028] This provides the same effects as the information processing method according to the first aspect of the present disclosure.

[0029] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0030] The embodiments described below are all comprehensive or specific examples, and the numerical values, shapes, materials, components, arrangement and connection of the components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure.

[0031] Furthermore, each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, for example, the scales in each figure do not necessarily match. Furthermore, explanations of substantially the same configurations or the same processes may be omitted or simplified when the same reference numerals are used.

[0032] Furthermore, in this specification, when a statement is made in comparison with, for example, "greater than a threshold" or "equal to or less than a threshold," it means that the distinction is made on the basis of the threshold, and may mean "greater than or equal to the threshold" or "less than the threshold," respectively.

[0033] Furthermore, in this specification, ordinal numbers such as "first" and "second" do not refer to the number or order of components unless otherwise specified, but are used for the purpose of avoiding confusion and distinguishing between components of the same type.

[0034] (Embodiment) [composition] FIG. 1 is a block diagram showing the configuration of a terminal 10 according to an embodiment.

[0035] The terminal 10 is a device for outputting information related to the vehicle 20 .

[0036] Vehicle 20 is a vehicle such as an automobile. In this embodiment, vehicle 20 is a vehicle that can be remotely controlled by terminal 10. Specifically, vehicle 20 is remotely monitored by an operator (user) via a remote control system. In some cases, vehicle 20 can be moved by remote control by the operator via the remote control system.

[0037] The vehicle 20 may be a manually operated vehicle that can be manually driven by a driver, or may be an autonomous vehicle.

[0038] Here, the remote control system is a system that operates in a remote control center or server device where one or more operators are stationed, and allows each operator to remotely monitor and, in some cases, remotely operate one or more vehicles 20. Each operator uses a terminal 10 such as a personal computer to monitor the status of each vehicle 20, which is output (e.g., displayed) on an output device 300. Furthermore, each operator remotely operates the vehicle 20, for example, when a remote operation request is received from the vehicle 20, or when the operator determines that the vehicle 20 is in a state where it should be remotely operated.

[0039] The vehicle 20 includes, for example, a wireless communication interface for communicating with the terminal 10 via a network such as the Internet, a positioning system such as a global positioning system (GPS) for detecting the position of the vehicle 20, various sensors such as a speed sensor, an acceleration sensor, an angular velocity sensor, a distance sensor, a camera, a light detection and ranging (LiDAR), and a radar, and a processing unit that serves as the main controller of the vehicle 20. The wireless communication interface is realized, for example, by a wireless communication circuit. The processing unit is realized, for example, by a processor such as a central processing unit (CPU) and a memory that stores a control program executed by the processor. The memory is, for example, a read-only memory (ROM) and a random access memory (RAM), and can store the program executed by the processor.

[0040] The terminal 10 is, for example, a smartphone, a tablet terminal, or a personal computer. In the following description, the terminal 10 is assumed to be a personal computer. Although only one terminal 10 is shown in FIG. 1, if there are multiple operators, there will be multiple terminals 10 used by each of the multiple operators.

[0041] The terminal 10 includes a management device 100, an input device 200, an output device 300, and a communication device 400.

[0042] The management device 100 is a computer that performs various processes. The management device 100 manages and controls the entire terminal 10 by cooperating with each of the input device 200, the output device 300, and the communication device 400. The management device 100 is realized by, for example, a memory, a processor that executes programs stored in the memory, and a storage device that stores various information. The memory is, for example, a ROM and a RAM, and can store programs executed by the processor. The management device 100 is an example of an information processing device.

[0043] The management device 100 includes an acquisition unit 110, a control unit 120, and a storage unit .

[0044] The acquisition unit 110 is a processing unit that acquires various types of information. For example, the acquisition unit 110 acquires information input by an operator to the input device 200, acquires information about the vehicle 20 from the vehicle 20 via the communication device 400, or acquires information stored in the storage unit 130. For example, the acquisition unit 110 acquires an operation amount of a steering wheel that controls the movement of the vehicle 20 (specifically, operation information indicating the operation amount). For example, the steering wheel that controls the movement of the vehicle 20 is the input device 200. The acquisition unit 110 acquires, from the input device 200, for example, an operation amount input (operated) by the operator to the input device 200. Note that the acquisition unit 110 may also acquire, from the vehicle 20, an operation amount of a steering wheel provided in the vehicle 20. In other words, the acquisition unit 110 may acquire the operation amount from the input device 200, or may acquire an actual operation amount of the vehicle 20 based on the operation amount.

[0045] The control unit 120 is a processing unit that performs various processes. For example, the control unit 120 controls information output by the output device 300 and transmits information to the vehicle 20 via the communication device 400. In this embodiment, the output device 300 is a display that displays images. The control unit 120 controls the images that the output device 300 displays. For example, the control unit 120 causes the output device 300 to display a steering wheel image that indicates the rotation state of the steering wheel, so that the display mode corresponds to the number of rotations of the steering wheel, based on the acquired operation amount.

[0046] Each processing unit such as the acquisition unit 110 and the control unit 120 is realized by, for example, a processor and a memory that stores a control program executed by the processor.

[0047] The storage unit 130 is a storage device that stores various types of information. The storage unit 130 stores, for example, information such as vehicle type information and a conversion table, which will be described later. The storage unit 130 is realized, for example, by a hard disk drive (HDD) or a solid state drive (SSD).

[0048] The input device 200 is an input interface that accepts input from an operator. The input device 200 may include, for example, a steering wheel for controlling the movement of the vehicle 20. The input device 200 may also include components for controlling the movement of the vehicle 20, such as an accelerator pedal and a brake pedal. The input device 200 may also include, for example, a pointing device such as a keyboard or a mouse. The input device 200 accepts, for example, input of driving operations for the vehicle 20 by the operator.

[0049] The output device 300 is a device that outputs information processed by the management device 100. The output device 300 is an example of a display device. The output device 300 is realized by a display such as a liquid crystal display.

[0050] FIG. 2 is a diagram showing a display image P1 displayed on the output device 300 according to the embodiment.

[0051] The output device 300 displays, for example, an image such as a display image P1 transmitted from the management device 100. Specifically, the output device 300 displays an image of the area in front of the vehicle 20 (image P11), an image of the area to the left of the vehicle 20 (image P13), an image of the area to the right of the vehicle 20 (image P14), and an image of the area behind the vehicle 20 (image P12). Furthermore, the output device 300 displays, for example, information based on information transmitted from the management device 100, such as the speed and angular velocity of the vehicle 20, the status of a system installed in the vehicle 20, and a route R1 along which the vehicle 20 is scheduled to travel (for example, a route from the current location of the vehicle 20 to the destination).

[0052] In addition, the output device 300 displays a handle image.

[0053] The steering wheel image is an image (icon) that indicates the rotation state of a steering wheel provided on the vehicle 20. The acquisition unit 110 acquires the amount of steering wheel operation (specifically, the amount of rotation that indicates the amount of the steering wheel rotation angle when the steering wheel is rotated) from the input device 200. Based on the amount of rotation acquired by the acquisition unit 110, the control unit 120 causes the output device 300 to display a steering wheel image that indicates the rotation state of the steering wheel so that the display mode corresponds to the number of rotations of the steering wheel. The steering wheel icon 500 shown in FIG. 2 is an example of a steering wheel image.

[0054] In addition to the display, the output device 300 may also include devices that output information as sound, such as an amplifier and a speaker.

[0055] The communication device 400 is a wireless communication interface for communicating with the vehicle 20 via a network. The wireless communication interface is realized by, for example, a wireless communication circuit. The communication device 400 receives information about the vehicle 20 transmitted from the vehicle 20.

[0056] For example, when the input device 200 receives an input for operating the steering wheel of the vehicle 20 (e.g., an input indicating the amount of steering wheel operation) from an operator, the management device 100 transmits a signal corresponding to the input for operating the steering wheel (e.g., information indicating the amount of operation) to the vehicle 20 via the communication device 400. The management device 100 also causes the output device 300 to display a steering wheel image corresponding to the amount of operation. The vehicle 20 also operates the steering wheel and other devices provided on the vehicle 20 in accordance with the received signal.

[0057] For example, the vehicle 20 may transmit operation information indicating the amount of actual operation of a steering wheel provided in the vehicle 20 to the management device 100. Upon receiving the operation information, the management device 100 may cause the output device 300 to display a steering wheel image corresponding to the amount of operation indicated by the operation information.

[0058] [Operation] The operation of the management device 100 according to the embodiment will be described below.

[0059] <Example 1> FIG. 3 is a flowchart showing the processing procedure of the management device 100 according to the first example.

[0060] First, the acquisition unit 110 acquires operation information including the amount of steering wheel operation (S100). For example, the acquisition unit 110 acquires operation information indicating the amount of steering wheel operation associated with the time when the steering wheel was actually operated.

[0061] Next, the control unit 120 determines whether or not the operation amount indicated by the operation information acquired by the acquisition unit 110 is 0 (S200). That is, the control unit 120 determines whether or not the steering wheel is not being operated (in other words, is not turning).

[0062] When the control unit 120 determines that the operation amount is 0 (Yes in S200), it determines the display mode of the handle image to be the first mode (S500).

[0063] On the other hand, when it is determined that the operation amount is not 0 (No in S200), the control unit 120 determines whether the operation direction (that is, the rotation direction) of the steering wheel is a clockwise rotation (S300).

[0064] When the control unit 120 determines that the steering wheel is being operated in a clockwise direction (Yes in S300), it determines whether the amount of operation is one rotation or more (S400). That is, the control unit 120 determines whether the steering wheel has been rotated one rotation or more to the right. For example, when the amount of operation is expressed as a rotation angle, with a clockwise rotation direction being positive, the control unit 120 determines whether the amount of operation is 360 degrees or more.

[0065] When the control unit 120 determines that the operation amount is one rotation or more (Yes in S400), it determines the display mode of the handle image to be the second mode (S510).

[0066] On the other hand, when the control unit 120 determines that the operation amount is not one rotation or more (No in S400), it determines the display mode of the handle image to be the third mode (S520).

[0067] Furthermore, when the control unit 120 determines that the steering wheel is not being operated to the right (No in S300), that is, when the control unit 120 determines that the steering wheel is being operated to the left, it determines whether the amount of operation is one rotation or more (S410). That is, the control unit 120 determines whether the steering wheel has been rotated to the left one rotation or more. For example, when the amount of operation is expressed as a rotation angle with the direction of right rotation being positive, the control unit 120 determines whether the amount of operation is -360 degrees or less.

[0068] When the control unit 120 determines that the operation amount is one rotation or more (Yes in S410), it determines the display mode of the handle image to be the fourth mode (S530).

[0069] On the other hand, when the control unit 120 determines that the operation amount is not one rotation or more (No in S410), it determines the display mode of the handle image to be the fifth mode (S540).

[0070] The control unit 120 rotates the steering wheel image in the display mode determined in steps S500 to S540 by the amount of operation (i.e., by an angle corresponding to the amount of operation) (S600). The control unit 120 causes the output device 300 to display the rotated steering wheel image. For example, the control unit 120 causes the steering wheel image to be superimposed on the display image P1 being displayed by the output device 300. Also, for example, if a steering wheel image is already being displayed on the output device 300, the control unit 120 changes the displayed steering wheel image to the determined display mode and rotates the steering wheel image by the amount of operation.

[0071] FIG. 4 is a diagram showing steering wheel icons 500 to 540 according to a first example. The steering wheel icons 500 to 540 are examples of steering wheel images. The steering wheel icon 500 shown in FIG. 4(a) is an example of a steering wheel image in a first mode. The steering wheel icon 510 shown in FIG. 4(b) is an example of a steering wheel image in a second mode. The steering wheel icon 520 shown in FIG. 4(c) is an example of a steering wheel image in a third mode. The steering wheel icon 530 shown in FIG. 4(d) is an example of a steering wheel image in a fourth mode. The steering wheel icon 540 shown in FIG. 4(e) is an example of a steering wheel image in a fifth mode.

[0072] The steering wheel image includes a rotating portion 600, as shown in steering wheel icon 500, for example.

[0073] The rotating portion 600 is an image that is rotated according to the amount of operation. The rotating portion 600 includes, for example, a central portion 610 and a peripheral portion 620.

[0074] The center portion 610 is an image shaped like a steering wheel, and is, for example, a circle with a cutout (specifically, a C-shape). For example, the cutout portion of the center portion 610 is displayed to be located at the top when the steering wheel is not being operated (rotated) relative to a reference position such that the vehicle 20 moves forward. When the steering wheel is being operated, for example, as in the steering wheel icons 510 to 540, the center portion 610 is displayed rotated in accordance with the amount of steering wheel operation. Furthermore, for example, in the steering wheel icons 500, 520, and 540 indicating that the steering wheel has not rotated more than once, the center portion 610 is displayed in black. On the other hand, in the steering wheel icons 510 and 530 indicating that the steering wheel has rotated more than once, the center portion 610 is displayed in white.

[0075] The peripheral portion 620 is an image that indicates the direction of rotation of the steering wheel. For example, when the steering wheel is not being operated, the peripheral portion 620 is displayed on the output device 300 as an arc with no arrows on either side of the central portion 610. When the steering wheel is being operated, for example, as in the steering wheel icons 510 to 540, the peripheral portion 620 is displayed with an arrow indicating the direction of rotation of the steering wheel. Furthermore, when the steering wheel is being rotated clockwise, for example, as in the steering wheel icons 510 and 520, the peripheral portion 620 is displayed with an arrow indicating that the steering wheel is rotating right. On the other hand, when the steering wheel is being rotated counterclockwise, for example, as in the steering wheel icons 530 and 540, the peripheral portion 620 is displayed with an arrow indicating that the steering wheel is rotating left.

[0076] As described above, when the steering wheel has rotated one or more times, the steering wheel image is displayed in a different display mode from when it has not rotated one or more times. That is, the management device 100 acquires the amount of operation of the steering wheel that controls the movement of the vehicle 20 (specifically, operation information indicating the amount of operation), and causes the output device 300 to display a steering wheel image (e.g., one of the steering wheel icons 500 to 540) indicating the rotation state of the steering wheel so that the display mode corresponds to the number of rotations of the steering wheel based on the acquired amount of operation. For example, in the process of displaying the steering wheel image on the output device 300, the management device 100 changes the display mode of the steering wheel image when the number of rotations of the steering wheel changes. Also, for example, in the process of displaying the steering wheel image on the output device 300, the management device 100 causes the output device 300 to display the steering wheel image so that the display mode differs between when the steering wheel has rotated a predetermined number of times or more and when the steering wheel has not rotated the predetermined number of times or more.

[0077] This allows the user to easily grasp the number of rotations of the steering wheel simply by checking the steering wheel image, making it easier for the operator to understand the steering wheel rotation state.

[0078] Furthermore, for example, when the steering wheel is turning right, the steering wheel image is displayed in a different display mode from when the steering wheel is turning left. That is, for example, in the process of displaying the steering wheel image on the output device 300, the management device 100 causes the output device 300 to display the steering wheel image in a display mode according to the direction in which the steering wheel is being turned. Specifically, in the process of displaying the steering wheel image on the output device 300, the management device 100 causes the output device 300 to display the steering wheel image in a different display mode when the steering wheel is turning right and when the steering wheel is turning left.

[0079] This allows the operator to easily tell whether the steering wheel is turning right or left.

[0080] Furthermore, for example, when the steering wheel is not being operated (specifically, when the steering wheel is not being rotated), the steering wheel image is displayed in a display mode that is different from when the steering wheel is being operated. That is, for example, in the process of displaying the steering wheel image on the output device 300, the management device 100 displays the steering wheel image on the output device 300 so that the display mode is different when the steering wheel is being rotated and when the steering wheel is not being rotated.

[0081] This allows the operator to easily see whether the handle is being rotated or not.

[0082] In the above example, different colors and the presence or absence of an arrow are shown as examples of different display modes, but the display mode of the handle image to be determined is not limited to these. The display mode of the handle image may be determined by any combination of display modes such as color, hatching, size, brightness, shape, and / or symbols.

[0083] The predetermined number of times may be one, two, three or more, and is not particularly limited. For example, the steering wheel image may be displayed on the output device 300 in a different display mode depending on whether the steering wheel has rotated once or twice.

[0084] <Example 2> Next, a second example will be described. In the second example, the management device 100 determines whether the steering wheel has been rotated two or more times in addition to the processing in the first example.

[0085] FIG. 5 is a flowchart showing the processing procedure of the management device 100 according to the second example.

[0086] In the second example, when the control unit 120 determines that the steering wheel is being operated in a right direction (Yes in S300) and that the amount of operation is one rotation or more (Yes in S400), it further determines whether the amount of operation is two rotations or more (S420).

[0087] When the control unit 120 determines that the operation amount is two or more rotations (Yes in S420), it determines the display mode of the handle image to be the sixth mode (S550).

[0088] On the other hand, if the control unit 120 determines that the operation amount is not two or more rotations (No in S420), that is, if the operation amount is one or more rotations but less than two rotations, it determines the display mode of the handle image to be the second mode (S510).

[0089] In the second example, when the control unit 120 determines that the steering direction is a left rotation (No in S300) and that the amount of operation is one rotation or more (Yes in S410), it further determines whether the amount of operation is two rotations or more (S430).

[0090] When the control unit 120 determines that the operation amount is two or more rotations (Yes in S430), it determines the display mode of the handle image to be the seventh mode (S560).

[0091] On the other hand, if the control unit 120 determines that the operation amount is not two or more rotations (No in S430), that is, if the operation amount is one or more rotations but less than two rotations, it determines the display mode of the handle image to be the fourth mode (S530).

[0092] FIG. 6 is a diagram showing steering wheel icons 500 to 560 according to a second example. The steering wheel icons 500 to 560 are examples of steering wheel images. The steering wheel icons 500 to 540 shown in (a) to (e) of FIG. 6 are examples of steering wheel images of the first to fifth modes, similar to the steering wheel icons 500 to 540 shown in (a) to (e) of FIG. 4. The steering wheel icon 550 shown in (f) of FIG. 6 is an example of a steering wheel image of the sixth mode. The steering wheel icon 560 shown in (g) of FIG. 6 is an example of a steering wheel image of the seventh mode.

[0093] As shown in (f) and (g) of FIG. 6, in the handle icons 550 and 560 that indicate a state in which the handle has been rotated two or more times, the center portion 610 is displayed with hatching.

[0094] As described above, in the second example, when the steering wheel has rotated two or more times, the steering wheel image is displayed in a different display mode from when the steering wheel has not rotated two or more times. In this way, the steering wheel image is displayed in a display mode corresponding to each number of rotations of the steering wheel. Of course, even when the steering wheel has rotated three or more times, the steering wheel image may be displayed in a display mode corresponding to each number of rotations.

[0095] <Example 3> Next, a third example will be described. In the third example, the display mode of the center of the steering wheel image is different from the first and second examples.

[0096] 7 is a diagram showing a steering wheel icon 590 according to a third example. The steering wheel icon 590 is an example of a steering wheel image.

[0097] The steering wheel icon 590 is a steering wheel image showing a state in which the steering wheel has been rotated one or more times to the left. The steering wheel icon 590 includes a rotating portion 601.

[0098] The rotating portion 601 is an image that is rotated according to the amount of operation. The rotating portion 601 includes, for example, a central portion 611 and a peripheral portion 620.

[0099] The central portion 611 is an image shaped like a steering wheel, and is, for example, a circle with a cutout (specifically, a C-shape). In this example, the central portion 611 is a double arc (two arcs). In this example, the number of arcs in the central portion 611 increases according to the number of rotations of the steering wheel. For example, the control unit 120 determines the display mode of the steering wheel image so that when the number of rotations of the steering wheel is less than one rotation, the number of arcs in the central portion 611 is one; when the number of rotations of the steering wheel is one rotation or more but less than two rotations, the number of arcs in the central portion 611 is two; and when the number of rotations of the steering wheel is two rotations or more, the number of arcs in the central portion 611 is three.

[0100] In this way, the number of arcs in the central portion 611 may be changed depending on the number of turns of the steering wheel.

[0101] <Example 4> Next, a fourth example will be described. In the fourth example, the management device 100 acquires the operation amount taking into consideration the communication time between the terminal 10 and the vehicle 20.

[0102] FIG. 8 is a flowchart showing the processing procedure of the management device 100 according to the fourth example.

[0103] First, the acquisition unit 110 acquires a communication delay time, which is a delay time required for communication between the terminal 10 and the vehicle 20 (S101).

[0104] The communication delay time is, for example, the time it takes from when an operator operates the steering wheel using the input device 200 until the terminal 10 transmits information indicating the operation to the vehicle 20 and the vehicle 20 is actually controlled in accordance with the information. The information indicating the communication delay time may be stored in advance in the storage unit 130, or may be measured by the terminal 10 and the vehicle 20 actually communicating with each other.

[0105] Next, the acquiring unit 110 acquires operation information indicating the amount of operation before the communication delay time (S102). For example, when an operator operates a steering wheel, operation information is periodically and repeatedly transmitted to the management device 100. The acquiring unit 110 acquires operation information indicating the amount of operation at a time before the current time (the time when step S102 is performed) by the communication delay time from among the multiple pieces of operation information repeatedly transmitted.

[0106] The control unit 120 performs the processes from step S200 onward shown in the first or second example based on the operation information acquired in step S102.

[0107] In this way, the management device 100 executes steps S101 and S102 instead of step S100 described above, for example.

[0108] For example, there is a delay of the above-mentioned communication delay time from when the operator operates the steering wheel until the vehicle 20 is controlled and the tires of the vehicle 20 actually move. Therefore, for example, the management device 100 rotates the icon on the UI, that is, the steering wheel image displayed on the output device 300, taking this delay into consideration.

[0109] This reduces the difference between the timing at which the steering wheel image displayed on the output device 300 is rotated and the timing at which the operator actually controls the vehicle 20 through the steering wheel operation. This makes it easier for the operator to remotely control the vehicle 20 while checking the image displayed on the output device 300.

[0110] The management device 100 may include a clock unit such as an RTC (Real Time Clock).

[0111] <Example 5> Next, a fifth example will be described. In the fifth example, the handle image includes a rotating portion 600 and a stationary portion 630.

[0112] FIG. 9 is a flowchart showing the processing procedure of the management device 100 according to the fifth example.

[0113] The management device 100 performs the processes of steps S100 to S560 in the same manner as the processes according to the second example shown in FIG.

[0114] After the display mode of the steering wheel image is determined by the processing of any one of steps S500 to S560, the control unit 120 rotates the rotating portion of the steering wheel image by the amount of operation (S601). Specifically, of a rotating portion 600 and a stationary portion 630 (see FIG. 10) which are included in the steering wheel image and will be described later, the control unit 120 rotates only the rotating portion 600 by an angle corresponding to the amount of operation. In other words, the control unit 120 does not rotate the stationary portion 630 regardless of the amount of operation.

[0115] FIG. 10 is a diagram showing handle icons 501, 511, 521, 531, 541, 551, and 561 according to the fifth example.

[0116] Steering wheel icons 501, 511, 521, 531, 541, 551, and 561 are examples of steering wheel images. The steering wheel icon 501 shown in FIG. 10(a) is an example of a steering wheel image of a first aspect. The steering wheel icon 511 shown in FIG. 10(b) is an example of a steering wheel image of a second aspect. The steering wheel icon 521 shown in FIG. 10(c) is an example of a steering wheel image of a third aspect. The steering wheel icon 531 shown in FIG. 10(d) is an example of a steering wheel image of a fourth aspect. The steering wheel icon 541 shown in FIG. 10(e) is an example of a steering wheel image of a fifth aspect. The steering wheel icon 551 shown in FIG. 10(f) is an example of a steering wheel image of a sixth aspect. The steering wheel icon 561 shown in FIG. 10(g) is an example of a steering wheel image of a seventh aspect.

[0117] Handle icons 501, 511, 521, 531, 541, 551, and 561 shown in (a) to (g) of FIG. 10 are modified versions of handle icons 500, 510, 520, 530, 540, 550, and 560 shown in (a) to (g) of FIG.

[0118] The handle image according to the fifth example includes a rotating portion 600 and a stationary portion 630, for example, like a handle icon 501 shown in FIG. 10(a).

[0119] Unlike the rotating part 600, the stationary part 630 is an image that does not rotate regardless of the amount of operation. In other words, the stationary part 630 is a part of the handle image that does not rotate. By displaying the stationary part 630 together with the rotating part 600, the stationary part 630 serves as a guide for the amount of rotation of the rotating part 600, making it easy for the operator to understand how much the rotating part 600 has been rotated.

[0120] As with the rotating part 600, the display mode of the stationary part 630 may or may not change depending on the number of rotations of the handle.

[0121] <Example 6> Next, a sixth example will be described. A steering wheel for controlling the movement of the vehicle 20 is, for example, an input device 200 connected to a management device 100 that communicates with the vehicle 20. Therefore, the vehicle 20 is operated by an operator via the input device 200. In the sixth example, when the operator inputs to the input device 200 to operate the steering wheel, the management device 100 determines whether the steering wheel is operated appropriately in accordance with the input.

[0122] FIG. 11 is a flowchart showing the processing procedure of the management device 100 according to the sixth example.

[0123] First, the control unit 120 determines whether the steering wheel is connected (S700). Specifically, the control unit 120 determines whether the steering wheel operation by the operator (i.e., input to the input device 200) is being properly transmitted to the vehicle 20. For example, the control unit 120 determines whether the input device 200 and the management device 100 are connected to each other so as to be able to communicate with each other. Alternatively, for example, the control unit 120 determines whether the management device 100 (more specifically, the communication device 400 to which the management device 100 is connected) and the vehicle 20 are able to communicate with each other.

[0124] If the control unit 120 determines that the steering wheel is connected (Yes in S700), the process proceeds to step S100 (or step S101) described above. For example, if the management device 100 determines that the input device 200 and the management device 100 are connected to be able to communicate with each other and / or if the management device 100 determines that the management device 100 and the vehicle 20 are able to communicate with each other, the management device 100 executes the processes from step S100 (or step S101) described above onward.

[0125] On the other hand, if the control unit 120 determines that the steering wheel is not connected (No in S700), it determines the display mode of the steering wheel image to be the eighth mode (S570). For example, if the management device 100 determines that the input device 200 and the management device 100 are not connected to be able to communicate with each other and / or if it determines that the management device 100 and the vehicle 20 are not able to communicate with each other, it determines the display mode of the steering wheel image to be the eighth mode and causes the output device 300 to display the steering wheel image in the determined display mode.

[0126] 12 is a diagram showing a steering wheel icon 591 according to a sixth example. The steering wheel icon 591 is an example of a steering wheel image. Specifically, the steering wheel icon 591 is an example of an eighth aspect of the steering wheel image.

[0127] The handle icon 591 includes a center portion 610 and a connection information icon 640 .

[0128] The connection information icon 640 is information indicating that the steering wheel will not be operated even if the operator inputs to the input device 200 to operate the steering wheel.

[0129] It should be noted that the steering wheel icon 591 does not necessarily have to include the center portion 610. In this example, the connection information icon 640 is an exclamation mark, but any display mode may be adopted as long as it is displayed in a different manner depending on whether the steering wheel image is connected to a steering wheel or not.

[0130] As described above, for example, a steering wheel for controlling the movement of the vehicle 20 is the input device 200 that is connected to the management device 100 that communicates with the vehicle 20 and is operated by an operator. Also, for example, in the process of displaying a steering wheel image on the output device 300, the management device 100 displays the steering wheel image on the output device 300 in a manner that differs depending on whether the input device 200 and the management device 100 are not connected or cannot communicate with the vehicle 20, or whether the input device 200 and the management device 100 are connected and can communicate with the vehicle 20.

[0131] This makes it easy for the operator to know whether the vehicle 20 is being appropriately controlled by the operator's operation of the operating device.

[0132] <Example 7> Next, a seventh example will be described. In the seventh example, the management device 100 causes the output device 300 to display the manipulated variable as a number.

[0133] FIG. 13 is a flowchart showing the processing procedure of the management device 100 according to the seventh example.

[0134] For example, after the above step S600, the control unit 120 causes the output device 300 to display a number indicating the operation amount (S800). Specifically, the control unit 120 causes the output device 300 to display a handle image including a numerical value indicating the operation amount.

[0135] FIG. 14 is a diagram showing handle icons 502, 512, 522, 532, and 542 according to the seventh example.

[0136] Steering wheel icons 502, 512, 522, 532, and 542 are examples of steering wheel images. The steering wheel icon 502 shown in FIG. 14(a) is an example of a steering wheel image of a first aspect. The steering wheel icon 512 shown in FIG. 14(b) is an example of a steering wheel image of a second aspect. The steering wheel icon 522 shown in FIG. 14(c) is an example of a steering wheel image of a third aspect. The steering wheel icon 532 shown in FIG. 14(d) is an example of a steering wheel image of a fourth aspect. The steering wheel icon 542 shown in FIG. 14(e) is an example of a steering wheel image of a fifth aspect.

[0137] Handle icons 502, 512, 522, 532, and 542 shown in (a) to (e) of FIG. 14 are modified versions of the handle icons 500, 510, 520, 530, and 540 shown in (a) to (e) of FIG.

[0138] The handle image according to the seventh example includes a rotation portion 600 and a numerical portion 650, for example, like the handle icon 502 shown in FIG.

[0139] The numerical value section 650 is a number indicating the amount of steering wheel operation. For example, when the steering wheel is rotated 45 degrees to the right, the numerical value section 650 indicating the amount of steering wheel operation is displayed on the output device 300 as "45" in (c) of FIG. 14. Also, when the steering wheel is rotated 45 degrees to the left, the numerical value section 650 indicating the amount of steering wheel operation is displayed on the output device 300 as "-45" in (e) of FIG. 14.

[0140] As described above, for example, in the process of displaying a steering wheel image on the output device 300, the management device 100 causes the output device 300 to display a steering wheel image including a numerical value indicating the amount of steering wheel operation.

[0141] This makes it easier to understand the amount of steering wheel operation.

[0142] <Example 8> Next, an eighth example will be described. In the eighth example, when the operation amount (rotation amount) of the steering wheel is rotated to a limit value (maximum value), in other words, when the steering wheel is rotated to the maximum number of rotations (the number of rotations up to the limit position of rotation in the left and right directions of the steering wheel), the management device 100 causes the output device 300 to display a steering wheel image in a display mode different from that when the steering wheel is not rotated to the maximum number of rotations.

[0143] FIG. 15 is a flowchart showing the processing procedure of the management device 100 according to the eighth example.

[0144] In the eighth example, when the steering wheel operation direction is clockwise (Yes in S300), the control unit 120 further determines whether the operation amount is maximum (S310). Specifically, the control unit 120 determines whether the operation amount is the maximum number of rotations of the steering wheel (limit value). Information indicating the limit value is stored in advance in, for example, the storage unit 130. Information indicating the limit value may be acquired from an external device such as the vehicle 20. The limit value may be determined, for example, by a limit value of a steering wheel actually provided in the vehicle 20. Furthermore, the limit value may be determined, for example, by a limit value included in the input device 200 and of the operation amount of the steering wheel operated by the operator.

[0145] When it is determined that the operation amount is maximum (Yes in S310), the control unit 120 determines the display mode of the handle image to be the ninth mode (S580).

[0146] On the other hand, if the control unit 120 determines that the operation amount is not the maximum (No in S310), the control unit 120 proceeds to step S400.

[0147] In the eighth example, when the steering wheel is operated counterclockwise (No in S300), the control unit 120 further determines whether the amount of operation is maximum (S320).

[0148] When it is determined that the operation amount is maximum (Yes in S320), the control unit 120 determines the display mode of the handle image to be the tenth mode (S590).

[0149] On the other hand, if the control unit 120 determines that the operation amount is not the maximum (No in S320), the process proceeds to step S410.

[0150] FIG. 16 is a diagram showing steering wheel icons 500, 510, 520, 530, 540, 570, and 580 according to an eighth example. The steering wheel icons 500, 510, 520, 530, 540, 570, and 580 are examples of steering wheel images. The steering wheel icons 500 to 540 shown in (a) to (e) of FIG. 16 are examples of steering wheel images of the first to fifth aspects, similar to the steering wheel icons 500 to 540 shown in (a) to (e) of FIG. 4. The steering wheel icon 570 shown in (f) of FIG. 16 is an example of a steering wheel image of the ninth aspect. The steering wheel icon 580 shown in (g) of FIG. 16 is an example of a steering wheel image of the tenth aspect.

[0151] 16(f) and 16(g), in handle icons 570 and 580 that indicate a state in which the handle is rotated to the maximum rotational speed, the center portion 610 is displayed with hatching. For example, handle icons 570 and 580 are displayed with hatching that is different from that of handle icons 550 and 560.

[0152] As described above, for example, in the process of displaying a steering wheel image on the output device 300, the management device 100 displays the steering wheel image on the output device 300 so that the display mode differs depending on whether the amount of steering wheel rotation is at the limit value or not.

[0153] When the vehicle 20 is operated (controlled) by remote control by an operator, the limit value of the amount of rotation that can be operated may differ between the input device 200 used by the operator and the steering wheel that the vehicle 20 actually has. In such cases, it is particularly difficult for the operator to know whether the steering wheel is at a limit where it cannot be rotated to control the vehicle 20. Therefore, by displaying a steering wheel image on the output device 300 in a different display mode when the amount of rotation of the steering wheel is at its limit and when the amount of rotation of the steering wheel is not at its limit, it is possible to make it easier for the operator to know when the limit of steering wheel rotation has been reached.

[0154] Note that, when the amount of steering wheel operation is at the limit value, the management device 100 may cause the output device 300 to execute an animation in which the steering wheel icons 570 and 580 vibrate for a predetermined period of time. For example, when displaying the steering wheel icon 570 or 580 on the output device 300, the management device 100 causes the output device 300 to display the steering wheel icon 570 or 580 in a display mode in which the steering wheel icon 570 or 580 vibrates left and right and / or up and down for a predetermined period of time.

[0155] The predetermined time may be determined arbitrarily and is not particularly limited.

[0156] <Example 9> Next, a ninth example will be described. In the ninth example, the management device 100 causes the output device 300 to display an icon indicating the steering angle of the vehicle 20 (the direction of the tires relative to the front of the vehicle 20).

[0157] FIG. 17 is a flowchart showing the processing procedure of the management device 100 according to the ninth example.

[0158] The acquisition unit 110 acquires vehicle type information (S900), for example, after step S600.

[0159] The vehicle type information is information that indicates the type of the vehicle 20. The vehicle type information may be acquired from the vehicle 20, or may be stored in the storage unit 130 in advance.

[0160] Next, the control unit 120 converts the acquired operation amount into a steering angle of the vehicle 20 based on the acquired vehicle type information (S910). For example, a conversion table including parameters for calculating the steering angle associated with the vehicle type is stored in advance in the storage unit 130. For example, the control unit 120 converts the operation amount into a steering angle using the conversion table.

[0161] The conversion table includes, for example, the ratio between the steering wheel operation amount and the steering angle (specifically, information indicating (steering wheel operation amount) / (average steering angle), hereinafter simply referred to as the conversion ratio) associated with the vehicle type. For example, the conversion table includes information such as the conversion ratio being "2", the vehicle width being "2 m", and the wheelbase length being "6 m" when the vehicle type is "truck A". Such information corresponding to the vehicle type is included in the conversion table for multiple vehicle types. For example, the conversion table includes information such as the conversion ratio being "1.5", the vehicle width being "0.6 m", and the wheelbase length being "1.0 m" when the vehicle type is "robot B". Furthermore, for example, the conversion table includes information such as the conversion ratio being "1.2", the vehicle width being "1.8 m", and the wheelbase length being "5 m" when the vehicle type is "vehicle C".

[0162] For example, when the acquired vehicle type information indicates "Truck A," that is, when the type of vehicle 20 is "Truck A," the control unit 120 calculates the steering angle using the acquired operation amount and the conversion ratio "2."

[0163] Next, the control unit 120 rotates the steering angle image, which is included in the steering wheel image and indicates the steering angle, by the calculated steering angle (i.e., by an angle corresponding to the calculated steering angle) (S920). The control unit 120 causes the output device 300 to display a steering wheel image that includes the rotated steering angle image and has the determined display mode.

[0164] Note that steps S900 to S920 may be executed before step S600.

[0165] 18 is a diagram showing a steering wheel icon 592 according to Example 9. The steering wheel icon 592 is an example of a steering wheel image.

[0166] The steering wheel icon 592 is a steering wheel image that indicates a state in which the steering wheel has not been rotated more than once and has been rotated 90 degrees to the right.

[0167] The steering wheel icon 592 includes a rotating portion 600 and a steering icon 660. The steering icon 660 is an example of a steering angle image.

[0168] The steering icon 660 is an image that rotates according to the steering angle. In this example, the steering icon 660 is in the shape of an isosceles triangle with its height longer than its base, and is displayed on the output device 300 with its apex pointing upward, for example, when the steering wheel is not being turned (i.e., when the steering angle is 0). In this example, the steering wheel icon 592 indicates a state in which the steering wheel has been turned 90 degrees to the right as described above, and the steering icon 660 is shown to be rotated by an angle corresponding to the steering angle calculated from the operation amount and the conversion table.

[0169] Note that, like the rotating portion 600, the display mode of the steering icon 660 may or may not change depending on the number of rotations of the steering wheel.

[0170] Furthermore, the shape of the steer icon 660 does not have to be an isosceles triangle and may be any shape.

[0171] As described above, for example, in the process of displaying a steering wheel image on the output device 300, the management device 100 causes the output device 300 to display a steering wheel image including a steering angle image indicating the steering angle of the vehicle 20.

[0172] This makes it easier for the operator to understand the steering angle of the vehicle 20.

[0173] <Example 10> Next, a tenth example will be described. In the tenth example, the management device 100 rotates a steering icon based on steering angle information indicating the steering angle of the vehicle 20 and causes the output device 300 to display the rotated steering icon.

[0174] For example, after step S600, the acquisition unit 110 acquires steering angle information indicating the steering angle of the vehicle 20 (S901). The acquisition unit 110 acquires the steering angle information from the vehicle 20 via the communication device 400, for example.

[0175] Next, the control unit 120 rotates the steering angle image, which is included in the steering wheel image and indicates the steering angle, by the steering angle indicated by the acquired steering angle information (S920).

[0176] According to this process, the process of calculating the steering angle becomes unnecessary. When the steering angle image is displayed and rotated by this process, the conversion table does not need to be stored in the storage unit 130.

[0177] Note that steps S901 and S920 may be executed before step S600.

[0178] <Example 11> Next, an eleventh example will be described. In the eleventh example, the management device 100 causes the output device 300 to display the forecasted trajectory of the vehicle 20.

[0179] FIG. 20 is a flowchart showing the processing procedure of the management device 100 according to the eleventh embodiment.

[0180] The acquisition unit 110 acquires vehicle type information (S900), for example, after step S600.

[0181] Next, the control unit 120 converts the acquired operation amount into a steering angle of the vehicle 20 based on the acquired vehicle type information (S910).

[0182] Next, the control unit 120 estimates a travel path of the vehicle 20 (for example, a path along which the vehicle 20 will travel in the near future due to an operation by the operator) based on the calculated steering angle, and calculates a predicted trajectory indicating the estimated travel path (S930). The control unit 120 calculates the predicted trajectory using, for example, the conversion table described above.

[0183] Fig. 21 is a diagram for explaining a method for calculating a forecasted trajectory. Specifically, Fig. 21 shows a vehicle model that models vehicle 20, that is, a model that simply represents vehicle 20. More specifically, Fig. 21 shows the vehicle model of vehicle 20 when viewed from above, with the upper side of the paper being the front of the vehicle model.

[0184] Here, c indicates the center of rotation of the vehicle model. Specifically, c indicates the center of the trajectory (rotation trajectory) of the travel path of the vehicle 20 when the vehicle 20 travels while turning, such as turning right or left. C_left indicates the rotation trajectory of the left front wheel. C_right indicates the rotation trajectory of the right front wheel. R indicates the rotation radius of the body center of the vehicle model (the center of the two front wheels). R_left indicates the rotation radius of the left front wheel. R_right indicates the rotation radius of the right front wheel. L indicates the wheelbase length of the vehicle model. W indicates the vehicle width of the vehicle model. θ indicates the average steering angle (for example, the average value of the steering angles of the left and right front wheels of the vehicle 20).

[0185] For example, if the vehicle model is turning right, R_left=sqrt((Rcos(θ)+W / 2)^2+L^2) R_right=sqrt((Rcos(θ)-W / 2)^2+L^2) Also, when the vehicle model turns left, R_left=sqrt((Rcos(θ)-W / 2)^2+L^2) R_right=sqrt((Rcos(θ)+W / 2)^2+L^2) The calculated R_left and R_right become the predicted trajectory of the vehicle 20.

[0186] Next, the control unit 120 converts the calculated coordinates of the predicted trajectory of the vehicle 20 into a camera image coordinate system (a coordinate system used when displaying the predicted trajectory on the output device 300), and displays the converted predicted trajectory on the output device 300 (S940).

[0187] FIG. 22 is a diagram showing a display image P1A displayed on the output device 300 according to the eleventh example.

[0188] Display image P1A includes, for example, an image of the area ahead of vehicle 20 (image P11A), an image of the area behind vehicle 20 (image P12), an image of the area to the left of vehicle 20 (image P13), an image of the area to the right of vehicle 20 (image P14), and a steering wheel image such as steering wheel icon 500. Image P11A is an image in which a calculated forecasted trajectory 700 is superimposed on image P11 shown in Fig. 2. Forecasted trajectory 700 is an example of a forecasted trajectory image.

[0189] The control unit 120, for example, uses the calculated R_left and R_right and the vehicle width W to determine the size and shape of the forecasted trajectory 700 to be superimposed on the image P11A, and causes the output device 300 to display the image P11A including the determined forecasted trajectory 700. For example, the control unit 120 causes the output device 300 to display the forecasted trajectory 700 up to a predetermined distance from the vehicle 20. In this example, for example, a 1-m line, a 2-m line, and a 3-m line forward from the vehicle 20 are indicated by horizontal lines in a ladder shape on the forecasted trajectory 700. Furthermore, the vertical line on the right side of the forecasted trajectory 700 indicates the estimated trajectory of the right tire of the vehicle 20, and the vertical line on the left side of the forecasted trajectory 700 indicates the estimated trajectory of the left tire of the vehicle 20. Note that the predetermined distance may be determined arbitrarily and is not particularly limited.

[0190] As described above, for example, the management device 100 estimates the driving route of the vehicle 20 based on the steering angle of the vehicle 20, and in the process of displaying a steering wheel image on the output device 300, further causes the output device 300 to display a predicted trajectory image showing the estimated driving route.

[0191] This makes it easier to understand how the vehicle 20 will move in the near future.

[0192] Note that steps S900, S910, S920, S930, and S940 may be executed before step S600.

[0193] <Example 12> Next, a description will be given of Example 12. In Example 12, the management device 100 causes the output device 300 to display the predicted trajectory of the vehicle 20, taking into consideration the communication delay time and the speed of the vehicle 20.

[0194] FIG. 23 is a flowchart showing the processing procedure of the management device 100 according to the twelfth example.

[0195] For example, after step S600, the management device 100 performs the processes of steps S900 to S930 in the same manner as in the flowchart shown in FIG.

[0196] Next, the acquisition unit 110 acquires information indicating the communication delay time and information indicating the speed of the vehicle 20 (S931). For example, the acquisition unit 110 acquires information indicating the speed of the vehicle 20 from the vehicle 20 via the communication device 400. The acquisition unit 110 may acquire, from the input device 200, information indicating the speed of the vehicle 20 input to the input device 200 by an operator.

[0197] Next, the control unit 120 determines the display position of the forecasted trajectory based on the communication delay time and the speed of the vehicle 20 acquired by the acquisition unit 110 (S932).

[0198] Next, the control unit 120 transforms the coordinates of the forecasted trajectory into the camera image coordinate system, and causes the output device 300 to display the forecasted trajectory of the vehicle 20 so that the transformed forecasted trajectory is displayed at the determined display position (S940).

[0199] FIG. 24 is a diagram showing a display image P1B displayed on the output device 300 according to the twelfth example.

[0200] Display image P1B includes, for example, an image of the area ahead of vehicle 20 (image P11B), an image of the area behind vehicle 20 (image P12), an image of the area to the left of vehicle 20 (image P13), an image of the area to the right of vehicle 20 (image P14), and a steering wheel image such as steering wheel icon 500. Image P11B is an image in which a calculated forecasted trajectory 710 is superimposed on image P11 shown in Fig. 2. Forecasted trajectory 710 is an example of a forecasted trajectory image.

[0201] The forecasted trajectory 710 is calculated, for example, by a method similar to the calculation method of the forecasted trajectory 700 described above. In this example, the control unit 120 further calculates the display position of the forecasted trajectory 710. Specifically, the control unit 120 determines the display position of the start position 711 of the forecasted trajectory 710 (the underline of the ladder shape shown as the forecasted trajectory 710 in FIG. 24). The forecasted trajectory 700 shown in FIG. 22 is displayed, for example, so as to extend upward from the bottom end of the image P11A. In contrast, the control unit 120 causes the output device 300 to display the forecasted trajectory 710 not from the bottom end of the image P11B, but shifted upward in accordance with the communication delay time and the speed of the vehicle 20. In this example, the control unit 120 calculates this shift amount and determines the display position of the forecasted trajectory 710 based on the calculated shift amount. Specifically, the control unit 120 calculates this shift amount in step S932. If this shift amount is D, D can be expressed, for example, as follows: D=t_delay×s It is calculated as follows.

[0202] Here, t_dray is the communication delay time, and s is the speed of the vehicle 20.

[0203] In step S940, the control unit 120 shifts the forecasted trajectory 710 upward from the bottom end of the image P11B by a length corresponding to the calculated D, and displays the forecasted trajectory 710 on the output device 300. Since the output device 300 displays an image delayed by the communication delay time, the current actual position of the vehicle 20 becomes the start position 711 of the forecasted trajectory 710.

[0204] Note that steps S900, S910, S920, S931, S932, and S940 may be executed before step S600.

[0205] [summary] Fig. 25 is a flowchart showing an information processing method according to an embodiment. For example, an information processing device includes a processor and a memory, and the processor uses the memory to execute the information processing method shown in Fig. 25. The information processing device is, for example, management device 100.

[0206] First, the information processing device acquires the amount of rotation of the handle that controls the movement of the moving object (S10).

[0207] Next, the information processing device causes the display device to display a steering wheel image showing the rotation state of the steering wheel in a display mode according to the number of rotations of the steering wheel based on the acquired rotation amount (S20).

[0208] The moving body is, for example, a vehicle 20. The rotation amount is, for example, the amount of operation indicated by the operation information. The rotation count is, for example, an integer calculated from the rotation amount (for example, a numerical value obtained by rounding down the decimal point of the value obtained by converting the rotation amount to the rotation count). The display device is, for example, an output device 300. For example, in the process of displaying a steering wheel image on the display device, the information processing device changes the display mode of the steering wheel image when the rotation count of the steering wheel changes. Also, for example, in the process of displaying a steering wheel image on the display device, the information processing device displays the steering wheel image on the display device so that the display mode differs between when the steering wheel has been rotated a predetermined number of times or more and when the steering wheel has not been rotated a predetermined number of times or more.

[0209] This allows the user to easily grasp the number of rotations of the steering wheel. From the viewpoint of functional safety, if a system used by an operator (e.g., a remote control system including the terminal 10) malfunctions, the operator must immediately stop remote operation to reduce the risk of runaway of a moving object such as the vehicle 20. To achieve this, it is necessary to display the amount of rotation of the steering wheel on a UI so that the operator can determine whether the system used by the operator is correctly recognizing the steering wheel operation. Therefore, by displaying a steering wheel image that rotates by the same amount as the amount of rotation of the steering wheel, it is possible to visualize the steering wheel operation. However, simply rotating a steering wheel image (steering wheel icon) indicating the steering wheel rotation state makes it difficult for the operator to determine from the steering wheel image whether the steering wheel has rotated one or more revolutions, for example, when the steering wheel has been rotated one or more revolutions. In particular, when a steering wheel image is displayed for an operator who remotely operates a moving object such as the vehicle 20, the operator will operate the steering wheel of the moving object using an operating device located at hand from outside the moving object, rather than using a steering wheel provided on the moving object. Therefore, in such a case, it is particularly difficult to determine from the steering wheel image whether the steering wheel has rotated one or more revolutions. Therefore, in the present disclosure, for example, a steering wheel image is displayed on a display device in a manner that differs depending on whether the steering wheel has been rotated a predetermined number of times or more and whether the steering wheel has not been rotated a predetermined number of times or more. For example, if the steering wheel has been rotated one or more times, the color or shape of the steering wheel image is changed. This makes it easier for the operator to intuitively grasp how many times the steering wheel has rotated. As described above, the information processing method according to the first aspect of the present disclosure makes it easier to understand the steering wheel rotation state.

[0210] Furthermore, if the steering wheel image showing the steering wheel rotation state is simply rotated, it is difficult for the operator to tell whether the steering wheel is being rotated right or left, for example, when the steering wheel is rotated more than one revolution. Alternatively, for example, the color or shape of the steering wheel image can be changed to indicate different rotations depending on whether the steering wheel is rotated right or left. This makes it easier for the operator to intuitively understand how many rotations the steering wheel has made to the left or right.

[0211] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0212] (Other embodiments) While the information processing device according to one or more aspects of the present disclosure has been described above based on the embodiments, the present disclosure is not limited to the embodiments. As long as it does not deviate from the spirit of the present disclosure, various modifications conceivable by a person skilled in the art to the embodiments and configurations constructed by combining components of different embodiments and examples may also be included within the scope of one or more aspects of the present disclosure.

[0213] For example, the mobile object may include a ship or an airplane in addition to the vehicle 20.

[0214] Also, for example, the display device may be disposed in a mobile body, the information processing device may be disposed in a mobile body, and the mobile body may be operated (driven) by a driver on board the mobile body without being remotely controlled.

[0215] For example, the present disclosure can be realized as a program for causing a processor to execute steps included in an information processing method. Furthermore, the present disclosure can be realized as a non-transitory computer-readable recording medium, such as a CD-ROM, on which the program is recorded.

[0216] Furthermore, for example, when the present disclosure is realized as a program (software), each step is performed by running the program using hardware resources such as a computer's CPU, memory, input / output circuits, etc. In other words, each step is performed by the CPU acquiring data from memory or input / output circuits, etc., performing calculations, and outputting the calculation results to memory or input / output circuits, etc.

[0217] In the above embodiment, each component included in the terminal 10 may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may also be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0218] Some or all of the functions of the management device 100 according to the above-described embodiment are typically realized as an LSI, which is an integrated circuit. These may be individually integrated into single chips, or some or all of them may be integrated into a single chip. Furthermore, the integrated circuit is not limited to an LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array), which can be programmed after LSI manufacturing, or a reconfigurable processor, which can reconfigure the connections and settings of circuit cells within an LSI, may also be used.

[0219] Furthermore, various modifications made to the embodiments of the present disclosure within the scope that would occur to a person skilled in the art are also included in the present disclosure, as long as they do not deviate from the gist of the present disclosure. [Industrial Applicability]

[0220] The present disclosure can be applied to a system that assists an operator in remotely controlling a moving object. [Explanation of symbols]

[0221] 10 devices 20 vehicles 100 Management device 110 Acquisition Department 120 control section 130 Storage section 200 Input Device 300 Output Device 400 Communication equipment 500, 501, 502, 510, 511, 512, 520, 521, 522, 530, 531, 532, 540, 541, 542, 550, 551, 560, 561, 570, 580, 590, 591, 592 Handlebar Icons 600, 601 Rotating part 610, 611 central part 620 Periphery 630 Stationary part 640 connection information icon 650 Numerical part 660 Steer Icon 700, 710 predicted trajectory 711 Starting position P1, P1A, P1B display image P11, P11A, P11B, P12, P13, P14 images R1 Route

Claims

1. Obtain the rotation amount of the handle that controls the movement of the moving object, displaying a steering wheel image showing a rotation state of the steering wheel on a display device in a display mode corresponding to the number of rotations of the steering wheel based on the acquired rotation amount; In the process of displaying the steering wheel image on the display device, the steering wheel image is displayed on the display device so that the number of arcs included in a steering wheel icon that resembles the steering wheel included in the steering wheel image corresponds to the number of rotations of the steering wheel. Information processing methods.

2. A computer-implemented method for processing information according to claim 1, program.

3. an acquisition unit that acquires a rotation amount of a handle that controls the movement of the moving object; a control unit that displays a steering wheel image indicating a rotation state of the steering wheel on a display device in a display mode corresponding to the number of rotations of the steering wheel based on the acquired rotation amount, the control unit causes the display device to display the steering wheel image so that the number of arcs included in the steering wheel icon resembling the steering wheel included in the steering wheel image corresponds to the number of rotations of the steering wheel. Information processing device.

Images