Information processing method, information processing apparatus, and storage medium

Abstract

To avoid a situation where a plurality of moving bodies each transmitting video data for remote control interfere with each other's communication.SOLUTION: An information processing method according to the present disclosure is executed by an information processing apparatus that supports remote control of a plurality of mobile bodies by an operator, the plurality of mobile bodies each being configured to autonomously move and execute a predetermined task, the remote control including remote monitoring of the plurality of mobile bodies by the operator based on image data transmitted from each of the plurality of mobile bodies, and remote operation of a first mobile body that is a remote operation target among the plurality of mobile bodies by the operator based on image data transmitted from the first mobile body. The information processing method includes determining whether it is possible to stop transmission of video data regarding a second mobile body that may interfere with communication for transmitting the video data from a first mobile body during a remote operation by an operator, and when it is possible to stop the transmission of the video data, taking an action to stop the transmission of the video data from the second mobile body.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing method, an information processing device, and a program. [Background technology]

[0002] In recent years, various types of services using autonomous mobile vehicles have been put into practical use, and remote control systems that can remotely monitor or operate these mobile vehicles are being developed. In these remote control systems, operators remotely monitor and operate the mobile vehicles based on video transmitted from the mobile vehicles. Therefore, if the communication quality when transmitting video data from the mobile vehicles deteriorates, it may be impossible to obtain video sufficient for remote control, which could cause problems with remote control.

[0003] For example, Patent Document 1 discloses a technology for determining the importance of multiple images captured at different positions and adjusting the quality of each image according to the results of the importance determination, in order to obtain images that allow the remote side to accurately grasp the vehicle's status even in a limited communication environment.Furthermore, for example, Patent Document 2 discloses a technology for collecting vehicle information, determining priority from the work status, and calculating and notifying an upper limit on transmission data so that the total data volume is kept below a predetermined value, in order to prevent interference with the remote control of each work machine and surrounding monitoring, even when multiple work machines transmit camera video data to a remote location. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2021 / 199350 [Patent Document 2] Japanese Patent Application Publication No. 2023-142240 Summary of the Invention [Problem to be solved by the invention]

[0005] However, for example, in a situation where a mobile object that is the target of remote control by an operator, such as monitoring or operating, is surrounded by other mobile objects that are the target of remote control by the same or another operator, there is a risk that the mobile objects will interfere with each other's communications, thereby degrading the communication quality when transmitting video data from each mobile object.Under such circumstances, there is room for improvement in how to avoid a situation where multiple mobile objects that each transmit video data for remote control interfere with each other's communications.

[0006] One of the objects of the present disclosure is to avoid a situation in which multiple mobile objects, each transmitting video data for remote control, interfere with each other's communications. [Means for solving the problem]

[0007] An information processing method according to the present disclosure is executed by an information processing device that supports remote control of a plurality of moving objects by an operator, the method including remote monitoring by an operator of a plurality of moving objects configured to move autonomously and perform a predetermined task based on video data transmitted from each of the plurality of moving objects, and remote operation by the operator of a first moving object that is a remote operation target among the plurality of moving objects based on video data transmitted from the first moving object. The information processing method includes determining whether transmission of video data from a second moving object that may interfere with communication transmitting the video data from the first moving object during the remote operation by the operator is possible, and taking action to stop transmission of the video data from the second moving object if transmission of the video data is possible. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to avoid a situation in which multiple mobile objects, each transmitting video data for remote control, interfere with each other's communications. Note that the effects described herein are not necessarily limited to those described herein, and may be any of the effects described in this specification. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of a remote control system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of each device included in the remote control system according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of a control device that realizes the functions of each device included in the remote control system according to the embodiment. [Figure 4] FIG. 4 is a flowchart showing an example of the flow of the remote control support process executed in the server device according to the embodiment. [Figure 5] FIG. 5 is a flowchart showing an example of the flow of a process for determining whether the plurality of vehicles (moving bodies) in FIG. 4 are in a state where they interfere with communication with each other. [Figure 6] FIG. 6 is a flowchart showing an example of the flow of a process for determining whether the second moving object in FIG. 4 can stop video transmission. [Figure 7] FIG. 7 is a sequence diagram showing an example of the flow of processing executed in the remote control system according to the embodiment. [Figure 8] FIG. 8 is a diagram for explaining an application example of the remote control system according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a display on the operator terminal according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating another example of a display on the operator terminal according to the embodiment. [Figure 11] FIG. 11 is a diagram illustrating another example of a notification in the operator terminal according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Below, with reference to the attached drawings, embodiments of the remote control support method (information processing method), remote control system (information processing system), server device (information processing device), operator terminal (information processing device), vehicle (mobile body), program, and storage medium according to the present disclosure will be described in detail.

[0011] In the description of the present disclosure, components having the same or substantially the same functions as those described above with respect to the previously-mentioned drawings may be given the same reference numerals, and descriptions thereof may be omitted as appropriate. Furthermore, even when the same or substantially the same parts are shown, the dimensions and proportions may be different depending on the drawing. Furthermore, for example, in order to ensure the visibility of the drawings, reference numerals may be given to only the main components in the description of each drawing, and reference numerals may not be given to components having the same or substantially the same functions as those described above with respect to the previously-mentioned drawings.

[0012] (Example of remote control system configuration) Fig. 1 is a diagram illustrating an example of a schematic configuration of a remote control system 1 according to an embodiment. As shown in Fig. 1, the remote control system 1 includes a server device 10, a plurality of operator terminals 20a, 20b to 20m (m is any integer), and a plurality of vehicles 30a, 30b to 30n (n is any integer).

[0013] 1, the server device 10 and each of the plurality of vehicles 30a, 30b to 30n are connected to each other so as to be able to communicate with each other via a network N such as the Internet. Also, the server device 10 and each of the plurality of operator terminals 20a, 20b to 20m are connected to each other so as to be able to communicate with each other via any telecommunications line such as a LAN (Local Area Network). Note that communication between the server device 10 and each of the plurality of operator terminals 20a, 20b to 20m may be realized via the network N.

[0014] In the description of the present disclosure, when the multiple operator terminals 20a, 20b to 20m are not distinguished from one another, they are simply referred to as "operator terminal 20." Similarly, when the multiple vehicles 30a, 30b to 30n are not distinguished from one another, they are simply referred to as "vehicle 30."

[0015] The number (m) of operator terminals 20 and the number (n) of vehicles 30 included in the remote control system 1 can be changed as desired depending on design conditions, etc. In addition, the server device 10 in the remote control system 1 may be realized by the cooperation of multiple server devices.

[0016] Here, the remote control system 1 according to the embodiment is an information processing system that provides various services such as delivery, security, cleaning, childcare, nursing care, sales, farm work, manufacturing, loading and unloading, transportation, and construction by remotely controlling an autonomously driven vehicle 30 (mobile body) by an operator. In this remote control system 1, an operator is assigned in response to a request for assistance (remote request) from a vehicle 30 requiring remote control, and the assigned operator intervenes in the control of the vehicle 30, such as remote operation. The server device 10 according to the embodiment is an information processing device (remote control support device) that supports the remote monitoring and / or operation of the vehicle 30. The operator terminal 20 according to the embodiment is an information processing device (terminal device) operated by an operator in a remote control room installed in a control center or the like. The vehicle 30 according to the embodiment is an example of an autonomously driven mobile body, and is used to provide various services.

[0017] As an example, the remote control system 1 according to the embodiment can be constructed by applying edge computing. In this case, for example, a vehicle 30 is used as the network edge, but other devices may also be used as the edge.

[0018] As an example, the vehicle 30 is an example of a moving object that performs various tasks, including autonomous driving, related to various services provided by the remote control system 1, such as delivery, security, cleaning, childcare, nursing care, sales, farm work, manufacturing, loading and unloading, transportation, and construction. Hereinafter, in this disclosure, autonomous driving will be exemplified as a task of the vehicle 30.

[0019] As an example, the vehicle 30 is a mobile body configured to be able to move autonomously and execute predetermined tasks. As an example, the vehicle 30 is a mobile body configured to be able to move and execute predetermined tasks according to remote control by an operator who monitors the multiple vehicles 30.

[0020] The moving body is not limited to vehicle 30, and various moving bodies configured to be movable at least in response to remote operation by an operator can be used as appropriate. Vehicle 30 may be, for example, a four-wheeled vehicle or a two-wheeled vehicle. Furthermore, vehicle 30 may be, for example, an automatic guided vehicle (AGV), or various robots such as construction machinery, agricultural machinery, or drones. Furthermore, these moving bodies are not limited to those that transport people, but may also transport objects other than people, or may provide specific services other than transportation.

[0021] As an example, when the vehicle 30 is unable to drive autonomously, for example, when an obstacle such as a fallen object or a vehicle parked on the road is detected in its path, the vehicle 30 sends an assistance request (remote request) to the server device 10 requesting assistance through remote control, i.e., remote operation.

[0022] As an example, the server device 10 transmits an image captured by a camera mounted on the vehicle 30 to the corresponding operator terminal 20.

[0023] As an example, when the server device 10 receives a request for assistance from the vehicle 30, the server device 10 transmits to the operator terminal 20 a remote operation request requesting remote operation by an operator.

[0024] As an example, the operator terminal 20 displays a display screen including an image (video) captured by a camera (sensor 301) mounted on the vehicle 30. The display screen includes at least one image (video) of at least one vehicle 30 that is in charge of the operator operating the operator terminal 20. The operator monitors the status of each of the at least one vehicle 30 that he or she is in charge of while looking at the display screen of the operator terminal 20. This monitoring is an example of the operator's remote monitoring of multiple vehicles 30 based on video data transmitted from each of the multiple vehicles 30.

[0025] As an example, when the operator terminal 20 receives a remote operation request (remote request) from the server device 10, it notifies the operator operating the terminal that assistance via remote operation has been requested. The operator operates the operator terminal 20 while viewing the display screen, remotely controls the vehicle 30 that has made the assistance request, and provides assistance such as moving the vehicle 30. This assistance of the vehicle 30 via remote control is an example of the operator remotely operating a vehicle 30 that is a remotely operated target among the multiple vehicles 30, based on video data transmitted from the remotely operated target vehicle 30.

[0026] The display screen of the operator terminal 20 may be a screen or image generated by the operator terminal 20 based on display information from the server device 10, or may display an image (display information) generated in the server device 10.

[0027] In this way, the remote control system 1 of this embodiment is a remote operation in which an operator remotely operates a vehicle 30, and is configured to be able to execute an information processing method (remote control support method) that supports remote monitoring and / or operation (remote control) performed based on remote control images captured by a camera mounted on the vehicle 30.

[0028] Fig. 2 is a diagram showing an example of the configuration of each device (server device 10, operator terminal 20, and vehicle 30) included in the remote control system 1 according to the embodiment. Fig. 2 illustrates the configuration of one of the at least one operator terminal 20 and one of the at least one vehicle 30 included in the remote control system 1, but the other operator terminals 20 and vehicles 30 included in the remote control system 1 have the same configuration.

[0029] Hereinafter, the configurations of the server device 10, the operator terminal 20, and the vehicle 30 will be described with reference to FIG.

[0030] (Example of vehicle configuration) First, the configuration of the vehicle 30 will be described.

[0031] As shown in FIG. 2, the vehicle 30 includes a sensor 301, a remote control determination unit 302, a control information generation unit 303, a remote request information generation unit 304, a remote request information transmission unit 305, a control information acquisition unit 306, a control unit 307, and a drive unit 308.

[0032] The sensor 301 includes at least one sensor for acquiring information about the vehicle itself and its surroundings. The output (sensor information) of the sensor 301 is provided to the remote control determination unit 302 and the server device 10. The output of the sensor 301 may also be provided to the control unit 307.

[0033] As an example, the sensor 301 includes one or more cameras (image capturing devices) that capture images of the surroundings of the vehicle 30. As an example, the sensor 301 includes at least one sensor that detects at least one of the state of the vehicle 30, the speed, the acceleration, the current position, the presence or absence of surrounding objects, the number of surrounding objects, the type of surrounding objects, and the distance to the surrounding objects.

[0034] The remote control determination unit 302 determines, based on the sensor information from the sensor 301, whether or not remote control of the subject vehicle is required.

[0035] For example, the remote control determination unit 302 acquires information about the surrounding conditions of the vehicle 30 based on sensor information from the sensor 301 .

[0036] Here, the information on the surrounding conditions of the vehicle 30 may be, for example, information indicating the presence or absence, type, number, and location of surrounding objects. The information on the surrounding conditions of the vehicle 30 may be, for example, information indicating whether a surrounding object exists on the driving route of the vehicle, information indicating whether a surrounding object is approaching the vehicle, or information indicating whether there is a possibility of a collision between the vehicle and a surrounding object.

[0037] The surrounding objects are objects detected around the vehicle 30, such as surrounding obstacles such as fallen objects and walls, or surrounding traffic participants such as various moving objects such as pedestrians, bicycles, motorcycles, and automobiles. Note that surrounding traffic participants that are stopped, such as parked vehicles, may be treated as obstacles.

[0038] For example, the remote control determination unit 302 determines whether a situation has arisen in which remote control is necessary based on information about the surrounding conditions of the vehicle 30. The criteria for determining whether a situation has arisen in which remote control is necessary may be set as appropriate, for example, depending on the task of the vehicle 30. This determination criteria may be predetermined and stored, for example, in the control device 4 (see FIG. 3) mounted on the vehicle 30. As an example, if the information about the surrounding conditions of the vehicle 30 indicates "no surrounding objects," the remote control determination unit 302 determines that remote control is "not necessary." As an example, if the information about the surrounding conditions of the vehicle 30 indicates "there is a stopped vehicle (obstacle) on the driving route," the remote control determination unit 302 determines that remote control is "necessary." As an example, if the information about the surrounding conditions of the vehicle 30 indicates "there is a pedestrian nearby and no sign of an approaching pedestrian," the remote control determination unit 302 determines that remote control is "not necessary."

[0039] The control information generator 303 generates autonomous control information for autonomously driving the vehicle 30, such as moving the vehicle 30 along a planned route, based on, for example, sensor information from the sensor 301, information on the surrounding conditions of the vehicle 30, and the result of self-position estimation. This autonomous control information includes, for example, control information for steering, acceleration, deceleration, etc. for moving the vehicle 30.

[0040] The remote request information generation unit 304 generates remote request information when the remote control determination unit 302 determines that remote control is necessary. This remote request information includes information for notifying the operator that an event requiring remote control has occurred, such as avoiding a roadside parking situation when a parked vehicle is parked ahead and overlaps the vehicle's driving route. Furthermore, the remote request information generation unit 304 may generate information to be sent to the server device 10 to notify the operator that autonomous driving will be resumed when autonomous driving can be resumed, such as when the object that caused remote control to occur is no longer present.

[0041] The remote request information transmitting unit 305 transmits the information generated by the remote request information generating unit 309 to the server device 10 .

[0042] The control information acquisition unit 306 acquires the autonomous control information generated by the control information generation unit 303 and the remote control information transmitted from the operator terminal 20 .

[0043] The control unit 307 controls the operation of the vehicle 30 based on the control information acquired by the control information acquisition unit 306. As an example, the control unit 307 controls the operation of various tasks of the vehicle 30, including driving such as steering and acceleration / deceleration, and transmission of vehicle status information for remote control such as remote operation and / or monitoring. This vehicle status information for remote control includes sensor information acquired by the sensor 301 or information generated in the vehicle 30 based on the sensor information. For example, the vehicle status information for remote control includes camera image (video) data of the surroundings of the vehicle 30 and position information of the vehicle 30. Note that the function of the control unit 307 as a vehicle status information transmission unit that transmits the sensor information from the sensor 301 to the server device 10 may be provided independently of the control unit 307.

[0044] The driving unit 308 is driven under the control of the control unit 307. The driving unit 308 is configured to include a mechanism capable of performing operations for executing various tasks of the vehicle 30, such as a mechanism that can drive the vehicle 30 in a predetermined direction for a predetermined distance at a predetermined speed.

[0045] (Server device configuration example) Next, the configuration of the server device 10 will be described.

[0046] As shown in FIG. 2, the server device 10 includes a remote monitoring control unit 101, a remote request information acquisition unit 102, a situation determination unit 103, a response determination unit 104, and a remote request notification unit 105.

[0047] The remote monitoring control unit 101 acquires information about the vehicle status for remote control from each vehicle 30. In addition, the remote monitoring control unit 101 outputs the information about the vehicle status for remote control from each vehicle 30 to the operator terminal 20 of the operator in charge of the remote control.

[0048] The remote request information acquisition unit 102 acquires information related to the remote request transmitted from the remote request information transmission unit 305 of the vehicle 30 .

[0049] The situation determination unit 103 determines that the vehicle 30 that issued the remote request is a "first moving body."

[0050] Furthermore, the situation determination unit 103 determines that a vehicle 30 that may interfere with communication transmitting video data from the "first moving body" during remote control operation by the operator is a "second moving body." Specifically, the situation determination unit 103 determines whether or not a situation exists in which multiple vehicles 30 (moving bodies) are interfering with each other's communication. Furthermore, if a situation exists in which multiple vehicles 30 (moving bodies) are interfering with each other's communication, the situation determination unit 103 determines that another vehicle 30 (moving body) of the "first moving body" is a "second moving body" to be controlled. On the other hand, if a situation exists in which multiple vehicles 30 (moving bodies) are interfering with each other's communication, the situation determination unit 103 determines that another vehicle 30 (moving body) of the "first moving body" is a "third moving body." Note that the determination threshold is, for example, predetermined and stored inside the control device 4.

[0051] As an example, the situation determination unit 103 determines whether or not the distance between each vehicle 30 (moving body) and the "first moving body" is less than a predetermined threshold. In other words, the situation determination unit 103 determines whether or not each vehicle 30 (moving body) is within a predetermined distance range 609 (see FIG. 8) from the "first moving body." For example, if the distance between the vehicle 30 (moving body) and the "first moving body" is less than a predetermined threshold, the situation determination unit 103 may determine that the vehicle 30 (moving body) is a "second moving body" to be controlled.

[0052] As an example, the situation determination unit 103 may determine whether the vehicle 30 (moving body) is within a predetermined distance range 609 (see FIG. 8) from the "first moving body" depending on whether video transmission is being performed via the same base station as the "first moving body." For example, if video transmission is being performed via the same base station as the "first moving body," the situation determination unit 103 may determine that the vehicle 30 (moving body) is a "second moving body" to be controlled.

[0053] As an example, the situation determination unit 103 may determine the stability of communication related to the transmission of camera images (video) for remote control from the "first moving object" to the server device 10. For example, the situation determination unit 103 may determine whether the maximum communication delay exceeds a predetermined threshold. For example, the situation determination unit 103 may identify a "second moving object" when the maximum communication delay exceeds a predetermined threshold. For example, the situation determination unit 103 may determine that a situation exists in which multiple vehicles 30 (moving objects) are interfering with each other's communication when the maximum communication delay exceeds a predetermined threshold and the second moving object is located within a predetermined distance range 609 (see FIG. 8) from the "first moving object" that issued the remote request. Alternatively, the situation determination unit 103 may determine that all of the other vehicles 30 are "third moving objects" when the maximum communication delay is equal to or less than a predetermined threshold.

[0054] As an example, the situation determination unit 103 may determine whether the minimum received power when the camera image (video) from the "first moving body" is received is less than a predetermined threshold to determine the stability of communication related to the transmission of the camera image (video) for remote control from the "first moving body" to the server device 10. For example, the situation determination unit 103 may identify a "second moving body" when the minimum received power is less than the predetermined threshold. For example, the situation determination unit 103 may determine that a situation exists in which multiple vehicles 30 (moving bodies) are interfering with each other's communication when the minimum received power is less than the predetermined threshold and the second moving body is located within a predetermined distance range 609 (see FIG. 8 ) from the "first moving body." Alternatively, the situation determination unit 103 may determine that all of the other vehicles 30 are "third moving bodies" when the minimum received power exceeds the predetermined threshold.

[0055] The above-mentioned criteria are merely examples, and other criteria may be used, or a combination of multiple criteria may be used.

[0056] Furthermore, the situation judgment unit 103 determines whether or not it is possible to stop the transmission of video data (stop video transmission) for each of at least one vehicle 30 that has been determined to be a "second moving body." Furthermore, the situation judgment unit 103 determines that a "second moving body" for which video transmission can be stopped is a "second moving body_A." On the other hand, the situation judgment unit 103 determines that a "second moving body" for which video transmission cannot be stopped, i.e., a "second moving body" for which video transmission cannot be stopped, is a "second moving body_B." The judgment threshold is, for example, predetermined and stored inside the control device 4.

[0057] As an example, the situation determination unit 103 determines whether the road width of the current location exceeds a predetermined threshold. Furthermore, if the road width exceeds the predetermined threshold, the situation determination unit 103 determines that the vehicle 30 (moving body) is a "second moving body_A" for which video transmission can be stopped. Here, the determination of whether the road width exceeds the predetermined threshold can also be expressed as a determination of whether stopping the operation of the vehicle 30 may cause a hindrance to other traffic participants, such as pedestrians or other moving bodies, for example, when video transmission is stopped from a safety perspective.

[0058] As an example, the situation determination unit 103 determines whether the distance to the surrounding moving object exceeds a predetermined threshold. Furthermore, if the distance to the surrounding moving object exceeds the predetermined threshold, the situation determination unit 103 determines that the vehicle 30 (moving object) is a "second moving object_A" for which video transmission can be stopped. Here, the determination of whether the distance to the surrounding moving object exceeds the predetermined threshold can also be expressed as a determination of whether there is a risk of collision with a surrounding traffic participant.

[0059] As an example, the situation determination unit 103 may determine whether the video transmission of the "second moving body" can be stopped based on whether the "second moving body" is captured in a camera image (video) of another vehicle 30 (moving body). For example, the situation determination unit 103 may determine that the "second moving body" captured in a camera image (video) of another vehicle 30 (moving body) is a "second moving body_A" for which video transmission can be stopped. Here, the determination of whether the "second moving body" is captured in a camera image (video) of another vehicle 30 (moving body) can be expressed as a determination of whether information about the surrounding situation of the "second moving body" can be grasped from the camera image (video) of the other vehicle 30 even if video transmission of the "second moving body" is stopped.

[0060] As an example, the situation determination unit 103 may determine whether the "second moving body" can stop video transmission depending on whether the "second moving body" is stopped. For example, the situation determination unit 103 may determine that the "second moving body" that is stopped is a "second moving body_A" whose video transmission can be stopped. Here, the case where the "second moving body" is stopped is an example of a case where the risk of collision with surrounding traffic participants is low.

[0061] In addition, the situation judgment unit 103 may exclude the vehicle 30 (mobile body) from the "second mobile body" if there is an object that the operator wants to keep an eye on, such as an approaching bicycle, if there is no time in the delivery schedule, or if video transmission cannot be stopped, such as when patrolling / guarding.

[0062] The above-mentioned criteria are merely examples, and other criteria may be used, or a combination of multiple criteria may be used.

[0063] Based on the results of the situation assessment of each vehicle 30 (mobile body) by the situation assessment unit 103, the response decision unit 104 decides on a response (mitigation control) to reduce the impact on the communication of the "first mobile body", such as stopping video transmission to the "second mobile body" to be controlled or controlling the vehicle, in other words, to improve the communication situation, and executes the decided response.

[0064] The response determining unit 104 causes the control device 4 to remotely instruct the "second moving body_A" to stop transmitting video data, thereby stopping the transmission of video data.

[0065] As an example, the response determination unit 104 remotely instructs the "second moving body_A" to stop transmitting video and performs vehicle control to stop the operation. Here, stopping the operation means putting the task on hold, not turning off the power.

[0066] Note that the response determination unit 104 may perform vehicle control that reduces other risks instead of remotely stopping the vehicle operation. Here, vehicle control that reduces risks refers to vehicle control that reduces the speed of the vehicle or reduces a threshold for a stop trigger, such as the distance to an obstacle.

[0067] As an example, the response determination unit 104 may remotely instruct the "second moving body_A" to stop video transmission and store the estimated bandwidth value at the time of the last video transmission for when the video transmission is resumed. In other words, the response determination unit 104 may remotely instruct the "second moving body_A" to stop video transmission and store the estimated bandwidth value before the video transmission was stopped. This stored estimated bandwidth value can be used as an initial value for the transmission volume (video quality) when the video transmission is resumed, because bandwidth estimation is not possible when the video transmission is stopped.

[0068] As an example, the response determining unit 104 may remotely control the vehicle to stop video transmission from the "second moving body_A" while continuing to notify sensor information such as object detection frames.

[0069] As an example, the response decision unit 104 may adjust the number and target range of "second moving body_A" for which video transmission is to be stopped based on the remote control work content of the "first moving body," such as the degree of impact on remote control of the "first moving body" if video transmission is stopped.

[0070] For example, if the "first moving body" is performing a remote operation that would be dangerous if it were to stop while crossing a crosswalk, for example, the determination condition may be set to "high," and the number and target range of "second moving body_A" may be increased. For example, if the impact of the "first moving body" stopping is large, such as when the road width is narrow or the distance to a moving body is close, the determination condition may be set to "medium." For example, if the impact of the "first moving body" stopping is small, such as when the road width is wide or the distance to a moving body is far, or there is no moving body, the determination condition may be set to "low," and the number and target range of "second moving body_A" may be reduced. As a result, video transmission is not stopped if the impact of stopping video transmission is minor, thereby suppressing a decrease in the operational efficiency of other moving bodies.

[0071] The threshold values ​​for the road width and the distance from the moving object regarding the "first moving object" are, for example, predetermined and stored inside the control device 4. These threshold values ​​may be the same as or different from the criteria for determining whether video transmission regarding the "second moving object" can be stopped.

[0072] The response determining unit 104 remotely instructs the "second moving body_B" to perform vehicle control that reduces the impact of the video transmission stop on the other "first moving body."

[0073] As an example, the response determination unit 104 may instruct the "second moving body_B" to reduce the bandwidth used for video transmission. The bandwidth used for video transmission may be reduced by reducing the image quality, such as the frame rate or image size, of the camera video to be transmitted, or by not transmitting some of the camera videos from the multiple cameras of the vehicle 30.

[0074] As an example, the response determining unit 104 may control the vehicle so that it is determined to be "second moving body_A," such as by rerouting (detouring) the "second moving body_B" to a location where it can stop.

[0075] As an example, the response decision unit 104 may control the vehicle so that the "second moving body_B" is judged to be a "third moving body," such as by rerouting the "second moving body_B" so that the distance to the "first moving body" is greater than or equal to a threshold value.

[0076] As an example, the response decision unit 104 may adjust the number (allowable number) of "second moving bodies_B" for which video transmission is to be stopped based on the remote control work content of the "first moving body," such as the degree of impact on remote control of the "first moving body" if video transmission is stopped.

[0077] For example, if the "first moving body" is performing remote control that would be dangerous if it were to stop while crossing a crosswalk, for example, the number of "second moving bodies_B" (allowable number) may be reduced. For example, if the impact of the "first moving body" stopping is large, such as when the road width is narrow or the distance to a moving body is close, the judgment condition may be set to "medium." For example, if the impact of the "first moving body" stopping is small, such as when the road width is wide or the distance to a moving body is far, or there is no moving body, the number of "second moving bodies_B" (allowable number) may be increased. This reduces the impact on the "first moving body" caused by the "second moving body_B" that cannot stop video transmission, allowing for safe and efficient remote control.

[0078] The threshold values ​​for the road width and distance from the moving object regarding the "first moving object" are, for example, predetermined and stored inside the control device 4. These threshold values ​​may be the same as or different from the criteria for determining whether video transmission regarding the "second moving object" can be stopped or the criteria for adjusting the number and target range of the "second moving object_A."

[0079] Furthermore, after taking action against the "second moving body", the action determining unit 104 causes the remote request notifying unit 105 to execute a request (remote control request) to an operator to remotely control the "first moving body".

[0080] As an example, the response decision unit 104 may execute a remote control request for the "first moving body" after the "second moving body_B" is determined to be the "second moving body_A" or the "third moving body" based on the vehicle control response for the "second moving body_B."

[0081] As an example, when the response decision unit 104 controls the vehicle so that the "second moving body_B" is judged to be the "second moving body_A" or the "third moving body" based on the vehicle control response to the "second moving body_B," if the movement is not completed within a specified time, it may cancel the remote request of the "first moving body" and instruct the operator to perform other work.

[0082] As an example, the response decision unit 104 may not execute a remote control request for the "first moving body" until the number of "second moving bodies_B" falls below an allowable number due to the vehicle control response for the "second moving body_B".

[0083] Furthermore, the response decision unit 104 terminates the response to improve the communication situation, such as stopping video transmission to the controlled "second moving object" or controlling the vehicle, after the operator has completed the remote operation of the "first moving object" that issued the remote request, or when the completion is expected. Examples of the case where the remote operation is completed but control according to the remote operation is being executed, or when the operator issues an instruction to resume, are cases where the remote operation is expected to be completed.

[0084] As an example, the response determining unit 104 remotely instructs the control device 4 to resume video transmission to the "second moving body_A."

[0085] As an example, the response decision unit 104 remotely instructs the "second moving body_A" to resume video transmission and performs vehicle control to resume operation. Note that, if the response decision unit 104 has performed vehicle control to continue the operation of the "second moving body_A" by performing an operation to reduce other risks that would cause the operation to be stopped, the response decision unit 104 performs vehicle control to restore the operation to reduce the risk.

[0086] As an example, the response determining unit 104 may cause the "second moving body_A" to resume video transmission using the estimated bandwidth value stored at the time of the last video transmission.

[0087] As an example, the response decision unit 104 may resume video transmission to the "second moving body_A" not only after the operator has completed remote control of the "first moving body" that issued the remote request, but also if the situation of the "second moving body_A" changes during remote control, such as when it becomes an obstruction to traffic, when a moving body approaches, or when the situation no longer obstructs communication.

[0088] As an example, the response determining unit 104 may temporarily restart video transmission to the "second moving body_A" at regular intervals.

[0089] As an example, the response determination unit 104 may cause the remote request notification unit 105 to check for a restart instruction from the operator at regular intervals. For example, the response determination unit 104 may notify at regular intervals that there is no change in the status of the "second moving body_A" for which video transmission has been stopped. This allows the operator to easily understand that it is OK to continue remote control of the "first moving body." Furthermore, when a restart instruction is received from the operator, the response determination unit 104 may temporarily restart video transmission to the "second moving body_A" or similarly to when remote control is completed.

[0090] As an example, the response determination unit 104 may sequentially resume video transmission for multiple "second moving bodies_A." For example, if multiple vehicles 30 (moving bodies) simultaneously resume video transmission, the communication bandwidth is likely to become congested, for example, because the data volume of the initial frame transmitted when video transmission resumes is larger than that of the subsequent differential frames. In such a situation, by sequentially resuming transmission, it is possible to prevent a situation in which multiple vehicles 30 (moving bodies) interfere with each other's communication when video transmission resumes.

[0091] As an example, the response determining unit 104 may resume transmission in ascending order of distance from the "first moving body." This makes it possible to suppress the impact on the remote control of the "first moving body" caused by the resumption of video transmission.

[0092] As an example, the response determination unit 104 may temporarily resume video transmission if the operator's remote control of the "first moving body" continues for a predetermined period of time or longer. This allows the operator to check the surrounding situation of the "second moving body_A" while minimizing the impact on the remote control of the "first moving body" caused by the resumption of video transmission.

[0093] As an example, the response determining unit 104 may instruct the "second moving body_B" to restore the bandwidth used for video transmission to the original bandwidth.

[0094] As an example, the response determining unit 104 may cause the "second moving body_B" to cancel vehicle control such as rerouting.

[0095] Furthermore, the response determination unit 104 may perform vehicle control to reroute or wait for the "third moving body," thereby maintaining a situation in which the "first moving body" is not affected, i.e., a situation in which multiple vehicles 30 (moving bodies) do not interfere with each other's communications. In this case, the response determination unit 104 may reroute again after the operator has completed remote operation of the "first moving body."

[0096] Furthermore, when it is determined that a situation exists in which multiple vehicles 30 (mobile bodies) are interfering with each other's communication, the response determination unit 104 may switch the communication line used for video transmission for a mobile body that has multiple available communication lines. As an example, when an operator remotely controls the "first mobile body," the response determination unit 104 may switch the communication line for the "first mobile body." As an example, the response determination unit 104 may switch the communication line for the "second mobile body" to reduce the impact on the operator's remote control of the "first mobile body." This makes it possible to reduce the impact on communication between mobile bodies while preventing video transmission from being stopped.

[0097] The remote request notification unit 105 transmits information for displaying a camera image, a notification (remote control request), and the like to the operator terminal 20 based on the information acquired by the remote request information acquisition unit 102.

[0098] (Example of operator terminal configuration) Next, the configuration of the operator terminal 20 will be described.

[0099] The operator terminal 20 includes a display unit 201 and a remote control information transmission unit 202 .

[0100] The display unit 201 displays, on a display device such as a liquid crystal display, information about the vehicle status for remote control, such as remote operation and / or monitoring, of at least one vehicle 30, transmitted from the remote monitoring control unit 101 of the server device 10. The information about the vehicle status of each vehicle 30 includes at least one image (video).

[0101] Furthermore, the display unit 201 displays information related to the remote request and various notifications to the operator on the display device based on the remote request information notified from the remote request notifying unit 105 of the server device 10.

[0102] The remote control information transmitting unit 202 acquires the result of an operator's operation on an input device such as a keyboard or touch panel, and transmits information indicating the operation result to the vehicle 30. For example, when a remote request is made, the operator inputs control information for the vehicle 30 using the input device. This control information is output to the corresponding vehicle 30 via the server device 10, for example.

[0103] 3 is a diagram showing an example of the hardware configuration of a control device 4 that realizes the functions of each device (server device 10, operator terminal 20, vehicle 30) included in the remote control system 1 according to the embodiment. The control device 4 is a computer that performs overall control of the operation of each device included in the remote control system 1.

[0104] The control device 4 that realizes each function of the vehicle 30 may be a computer such as an ECU (Electronic Control Unit) installed inside the vehicle 30, a DCU (Domain Control Unit) such as a CDC (Cockpit Domain Controller) that integrates multiple ECUs, or an OBU (On Board Unit).

[0105] As shown in FIG. 3, the control device 4 includes a processor 41, a ROM (Read Only Memory) 42, a RAM (Random Access Memory) 43, and a device I / F (Interface) unit 44.

[0106] The processor 41 is, for example, a CPU (Central Processing Unit), but in addition to or instead of a CPU, at least one of various processors such as a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array) can be used as appropriate.

[0107] As an example, in the control device 4 that realizes each function of the server device 10, the processor 41 comprehensively controls the operation of the control device 4, for example by executing a program, and realizes the functions of a remote monitoring control unit 101, a remote request information acquisition unit 102, a situation judgment unit 103, a response decision unit 104, and a remote request notification unit 105.

[0108] As an example, in the control device 4 that realizes each function of the operator terminal 20, the processor 41 comprehensively controls the operation of the control device 4, for example by executing a program, and realizes the functions of the display unit 201 and the remote control information transmission unit 202.

[0109] As an example, in the control device 4 that realizes each function of the vehicle 30, the processor 41 comprehensively controls the operation of the control device 4, for example by executing a program, and realizes the functions of a remote control judgment unit 302, a control information generation unit 303, a remote request information generation unit 304, a remote request information transmission unit 305, a control information acquisition unit 306, and a control unit 307.

[0110] 2 illustrates only the functions necessary for explaining the main parts of this embodiment, but the functions possessed by each device included in the remote control system 1 are not limited to these. In this embodiment, the processor 41 executes a program stored in the ROM 42 to realize the functions possessed by each device, including the functions of each part described above. However, this is not limiting, and some or all of these functions may be realized by dedicated hardware circuits.

[0111] The ROM 42 is a non-volatile memory that stores various information including programs executed by the processor 41. The memory of the control device 4 is not limited to the ROM 42, and various recording media and storage devices such as a hard disk drive (HDD), a solid state drive (SSD), and flash memory can be used as appropriate. The RAM 43 is a volatile memory that has a working area for the processor 41. The device I / F unit 44 is an interface for connecting the control device 4 to other devices in each device included in the remote control system 1, such as a communication device (not shown), a display device (not shown), and an input device (not shown).

[0112] Next, an example of the operation of the remote control system 1 according to the embodiment will be described with reference to the drawings. Note that the operation procedures and processing flows described below are merely examples, and the order of steps can be changed, some steps can be deleted, and other steps can be added as desired.

[0113] As an example, the flow of Figure 4 is initiated when an event occurs that requires the operator to remotely control the vehicle, such as crossing an intersection or avoiding an obstacle, i.e., when a remote request issued from any vehicle 30 is received, and before a remote control request is sent to the operator terminal 20 to request remote control.

[0114] FIG. 4 is a flowchart showing an example of the flow of the remote control support process executed by the control device 4 in the server device 10 according to the embodiment.

[0115] The control device 4 determines whether a situation exists in which a plurality of vehicles 30 (moving bodies) are interfering with each other in communication (S101).

[0116] FIG. 5 is a flowchart showing an example of a process flow for determining whether the plurality of vehicles 30 (mobile bodies) in FIG. 4 are in a situation where they interfere with communication with each other. The control device 4 determines whether the distance between each vehicle 30 (mobile body) and a "first mobile body," which is the vehicle 30 that issued the remote request, is less than a predetermined threshold (S201). If the distance from the "first mobile body" is less than the predetermined threshold (S201: Yes), the control device 4 designates the vehicle 30 (mobile body) as a "second mobile body" to be controlled (S202). On the other hand, if the distance from the "first mobile body" is equal to or greater than the predetermined threshold (S201: No), the control device 4 designates the vehicle 30 (mobile body) as a "third mobile body" (S203). Thereafter, the flow in FIG. 5 ends, and the flow in FIG. 4 is resumed.

[0117] The control device 4 determines whether it is possible to stop video transmission for each of at least one vehicle 30 that has been determined to be a "second moving body" (S102).

[0118] FIG. 6 is a flowchart showing an example of a process flow for determining whether the "second moving body" in FIG. 4 can stop video transmission. For each of at least one vehicle 30 designated as the "second moving body," the control device 4 determines at least one of whether the road width exceeds a predetermined threshold and whether the distance to a surrounding moving body exceeds a predetermined threshold (S301). If the road width exceeds the predetermined threshold or if the distance to a surrounding moving body exceeds a predetermined threshold (S301: Yes), the control device 4 determines that the vehicle 30 (moving body) is a "second moving body_A" whose video transmission can be stopped (S302). On the other hand, if the road width does not exceed the predetermined threshold and the distance to a surrounding moving body does not exceed the predetermined threshold (S301: No), the control device 4 determines that the vehicle 30 (moving body) is a "second moving body_B" whose video transmission cannot be stopped (S303). Thereafter, the flow in FIG. 6 ends, and the process returns to the flow in FIG. 4.

[0119] Then, the control device 4 takes action such as stopping video transmission and / or controlling the vehicle for each of the at least one vehicle 30 determined to be a "second moving body" (S103).

[0120] After taking action against the "second moving body," the control device 4 executes assistance control for remote operation of the "first moving body" (S104). Note that the remote operation of the "first moving body" by the operator may be an operation to instruct the vehicle 30 to perform control, or an operation to instruct the vehicle 30 to move autonomously.

[0121] After the operator remotely controls the "first moving body," the control device 4 executes measures such as restarting video transmission and / or vehicle control for the "second moving body" (S105), after which the flow in FIG. 4 ends.

[0122] Fig. 7 is a sequence diagram showing an example of the flow of processing executed in the remote control system 1 according to the embodiment. Fig. 8 is a diagram for explaining an application example of the remote control system 1 according to the embodiment.

[0123] First, each vehicle 30 transmits vehicle status information for remote control, including camera images (video), to the server device 10 (401a to 401d). The server device 10 also receives vehicle status information for remote control from each vehicle 30, and transmits remote monitoring information to the operator terminal 20 to display various information and notifications for remote monitoring by the operator based on the received vehicle status information (S402).

[0124] Here, it is assumed that a remote request is issued from vehicle 30a (vehicle 30) that is the target of monitoring by an operator (S403), as shown in Fig. 8. Fig. 8 illustrates a case where, at an intersection 603 where roads 601a and 601b intersect, a remote request is issued from vehicle 30 traveling along road 601b to cross a crosswalk 605 provided on road 601a.

[0125] In response to receiving a remote request from vehicle 30a (first moving body), server device 10 starts the remote control support process of Fig. 4. In the example of Fig. 8, vehicle 30b is determined to be a "second moving body" to be controlled (S101) because it is present within a predetermined distance range 609 from vehicle 30a (first moving body). Thereafter, vehicle 30b is determined to be a "second moving body_A" capable of video transmission (S102).

[0126] The server device 10 executes measures to improve the communication situation, such as stopping video transmission and / or controlling the vehicle (S103). The server device 10 stops video transmission from the vehicle 30b (second moving body_A) and executes vehicle control to reduce risk as a measure to improve the communication situation (S404a). Furthermore, for the vehicle 30 determined to be the "second moving body_B," the server device 10 executes vehicle control to reduce the impact on the vehicle 30a (first moving body) as a measure to improve the communication situation (S404b). Furthermore, for the vehicle 30 determined to be the "third moving body," the server device 10 executes vehicle control to maintain a situation without impact on the vehicle 30a (first moving body) as a measure to improve the communication situation (S404c).

[0127] The server device 10 takes measures to improve the communication conditions, such as stopping video transmission and / or controlling the vehicle, and then transmits a remote control request to the operator terminal 20 (S405). The operator terminal 20, triggered by receiving the remote control request from the server device 10, accepts remote operation by the operator and transmits operation information indicating the operation content of the accepted remote operation to the server device 10 (S406). The server device 10 generates a remote operation command for remotely controlling the vehicle 30a (first moving object) in accordance with the operation information from the operator terminal 20, and transmits the command to the vehicle 30a (first moving object). The vehicle 30a (first moving object) operates in accordance with the remote operation command from the server device 10 (S407). Thereafter, the processes of S406 and S407 are repeated until the remote operation by the operator is completed.

[0128] When the remote operation by the operator is completed, the server device 10 transmits an autonomous restart instruction to the vehicle 30a (first moving body) to restart the autonomous operation. The vehicle 30a (first moving body) restarts the autonomous operation in accordance with the autonomous restart instruction from the server device 10 (S408).

[0129] After the operator has completed the remote operation, the server device 10 cancels the measures taken in the process of S404 to improve the communication situation (S409a to S409c).

[0130] Fig. 9 is a diagram illustrating an example of a display on the operator terminal 20 according to the embodiment. Fig. 9 illustrates a display screen 710 on the operator terminal 20 when an event requiring remote control occurs in the vehicle 30 corresponding to the image 801a of "Vehicle 1" and measures are taken to improve the communication situation in the remote control support process of Fig. 4.

[0131] The display screen 710 in Fig. 9 illustrates a case where four vehicles 30 are assigned to one operator who operates the operator terminal 20. As shown in Fig. 9, the display screen 710 includes images 801a to 801d for remote control (operation and monitoring) of the four vehicles 30.

[0132] On the display screen 710 in Fig. 9, images 801c and 801d of "Vehicle 3" and "Vehicle 4" correspond to vehicle 30 for which video transmission has been voluntarily stopped in response to measures to improve communication conditions in the remote control support process in Fig. 4. As shown in Fig. 9, on the display screen 710, a status display 803 such as "Video transmission stopped" may be displayed. This allows the operator to easily understand that this is different from an unintentional video stop caused by a communication delay.

[0133] 9, the image 801b of "vehicle 2" shows that the video transmission from vehicle 30 has been stopped due to a communication delay. As shown in FIG. 9, a status display 803 such as "delayed stop" may be displayed on display screen 710. This allows the operator to easily understand that the video stop is different from a voluntary video stop.

[0134] As shown in Fig. 9, an image 801a corresponding to a vehicle 30 that is the target of remote control, an image 801b corresponding to a vehicle 30 in which video transmission from the vehicle 30 has been stopped due to a communication delay, and images 801c and 801d corresponding to a vehicle 30 in which video transmission has been stopped voluntarily may be displayed in different display modes depending on the situation. Fig. 9 illustrates a case in which the display mode of the outer frame of each image 801 differs depending on the situation. The display modes for each situation are merely examples and can be set as appropriate.

[0135] Furthermore, as shown in images 801c and 801d of "vehicle 3" and "vehicle 4" in FIG. 9, display 805 of information regarding the resumption of video transmission may be displayed on display screen 710. For example, displays 805 such as "Scheduled to resume in 10 seconds" and "Scheduled to resume in 5 seconds" may be displayed so that the time until resumption is known. Note that display 805 of information regarding the resumption of video transmission may also be displayed so that the order in which video transmission will be resumed is known. Note that information regarding the resumption of video transmission is not limited to display 805 on images 801c and 801d, but may also be displayed by notification display 809. This allows the operator to easily know when the video suspension will be lifted, thereby alleviating anxiety for the operator.

[0136] Furthermore, as shown in images 801c and 801d of "vehicle 3" and "vehicle 4" in FIG. 9, an operation button 807 may be displayed on the display screen 710 to allow the operator to instruct the resumption of video transmission. This allows the operator to easily check the situation even if remote operation of the "first moving body" takes a long time or if the situation of the "second moving body_A" changes. Furthermore, the operation button 807 is displayed on each image 801c and 801d corresponding to each vehicle 30, allowing the operator to resume video transmission for only that specific vehicle 30. This makes it possible to suppress the impact of the resumption of video transmission on the video transmission from the "first moving body."

[0137] Fig. 10 is a diagram illustrating another example of a display on the operator terminal 20 according to the embodiment. Fig. 10 illustrates an example of a display screen 720 on the operator terminal 20 when an event requiring remote control occurs in the vehicle 30 corresponding to the image 801a of "Vehicle 1" and measures are taken to improve the communication situation in the remote control support process of Fig. 4.

[0138] As illustrated in FIG. 10 , the display screen 720 may display a display that indicates to the operator in advance vehicles 30 for which video transmission can be stopped. In the example of FIG. 10 , an operation button 811 is displayed for the operator to instruct the operator to stop video transmission. In this case, as illustrated in FIG. 10 , vehicles 30 for which video transmission can be stopped may be displayed by a notification display 813. Alternatively, as illustrated in FIG. 10 , images 801c and 801d of vehicles 30 for which video transmission can be stopped may be displayed in different display modes from images 801a and 801b of other vehicles 30. FIG. 10 illustrates a case where the display mode of the outer frame of each image 801 differs depending on whether video transmission can be stopped. Note that the display mode for each situation is merely an example and can be set as appropriate. This allows the operator to stop any video if an impact occurs during remote operation of the "first moving object." In other words, if the remote operation of the "first moving object" is not affected, the video display can continue as is.

[0139] 11 is a diagram showing another example of a notification in the operator terminal 20 according to the embodiment. For example, a notification display 815 asking whether to stop video transmission may be displayed on the display screen 720. In the example of FIG. 11, the notification display 815 displays an operation button 817 for allowing the operator to stop video transmission and an operation button 819 for not allowing it. In this way, the remote control system 1 may propose stopping video transmission and determine whether to stop video transmission based on an instruction from the operator.

[0140] In the remote control system 1 according to the above embodiment, the remote control instruction to the operator may be configured to be given after the video transmission of the "second moving body" is stopped. Alternatively, the remote control instruction may be configured to prevent the operator from remotely controlling the "second moving body" until the video transmission of the "second moving body" is stopped. This prevents the operator from remotely controlling the "second moving body" before the video transmission is stopped, which could cause an impact on the remote control.

[0141] The remote control system 1 according to the above embodiment may be configured so that the operator can specify the "second moving body_B." For example, similar to FIG. 10, an operation button may be displayed to allow the operator to specify the vehicle 30 (second moving body_B) for which video transmission should not be stopped. This allows the vehicle 30 (moving body) that the operator is watching to be excluded from the targets for which video transmission should be stopped.

[0142] In this way, in the remote control system 1 according to this embodiment, the server device 10 determines whether it is possible to stop video transmission for a "second mobile body" that may interfere with communication of a "first mobile body" that is being remotely controlled while the operator is operating it, and if it is possible to stop it, stops the video transmission.

[0143] This configuration allows for a communication situation equivalent to that when the "second moving body" is not present. In other words, the above configuration makes it possible to avoid a situation in which multiple vehicles 30 (moving bodies) interfere with each other's communication while the operator is operating the "first moving body." Therefore, even when remote control of the "first moving body" is being performed, which would have a large impact if the "first moving body" is forced to make an emergency stop, the remote control of the "first moving body" can be smoothly performed without communication being interfered with.

[0144] Furthermore, according to the above configuration, depending on whether or not the transmission of video data can be stopped, the operator can appropriately separate the control regarding the "second mobile body" that may interfere with communication of the "first mobile body" that is the remote control target while operating it.

[0145] For example, if the route to avoid a communication disruption is long, operational efficiency will decrease. Furthermore, for example, if an operator is monitoring multiple mobile objects and is operating one of the mobile objects, if a communication disruption occurs in the multiple mobile objects, the other mobile objects may also be stopped, resulting in decreased operational efficiency. Furthermore, in an environment where bandwidth cannot be guaranteed, such as a public telecommunications network, it is difficult to grasp the upper limit of the bandwidth, and therefore, it may not be possible to avoid a communication disruption situation simply by controlling the total transmission bandwidth to be below the upper limit.

[0146] In this situation, the remote control system 1 according to this embodiment can appropriately separate control depending on whether or not transmission of video data can be stopped, thereby realizing safe and efficient remote control.

[0147] The configurations of the above-described embodiments can be combined in any manner.

[0148] Note that some or all of the functions of each device in the remote control system 1 according to the above-described embodiment may be realized by other devices in the remote control system 1. For example, some of the functions of the server device 10 according to the above-described embodiment may be realized in at least one of the operator terminal 20 and the vehicle 30. Alternatively, in the remote control system 1 according to the above-described embodiment, the server device 10 and the operator terminal 20 may be configured as an integrated unit. Alternatively, in the remote control system 1 according to the above-described embodiment, any one of the multiple vehicles 30 and the server device 10 may be configured as an integrated unit.

[0149] In the remote control system 1 according to the above embodiment, the plurality of vehicles 30 remotely controlled by one operator via one operator terminal 20 includes at least one vehicle 30 that can be controlled differently depending on whether video transmission can be stopped, as described above. On the other hand, the plurality of vehicles (moving bodies) remotely controlled by one operator via one operator terminal 20 may include a mixture of vehicles 30 according to the above embodiment and vehicles that are not controlled differently depending on whether video transmission can be stopped.

[0150] In the above-described embodiment, the determination of "whether it is A or not" may be realized by determining only that it is A, or by determining only that it is not A, or by determining both of these.

[0151] In the above embodiment, "any of A" means "at least one of A."

[0152] In addition, the programs executed by each device of the remote control system 1 according to the above-described embodiment may be provided by being recorded in an installable or executable file format on a computer-readable recording medium (Computer Program Product) such as a CD-ROM, FD, CD-R, or DVD.

[0153] The programs executed by the devices in the remote control system 1 according to the above-described embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The programs executed by the devices in the remote control system 1 according to the above-described embodiment may be provided or distributed via a network such as the Internet.

[0154] Furthermore, the programs executed by the devices of the remote control system 1 according to the above-described embodiment may be provided by being pre-installed in a ROM or the like.

[0155] Although the embodiments of the present disclosure have been described above, the above-described embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These novel embodiments and modifications thereof are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0156] 1. Remote control system 10 Server device (information processing device) 101 Remote monitoring control unit 102 Remote Request Information Acquisition Department 103 Situation Assessment Department 104 Response Decision Department 105 Remote request notification section 20 Operator terminal 201 Display section 202 Remote control information transmission unit 30 Vehicles (moving objects) 301 Sensor 302 Remote Control Judgment Unit 303 Control Information Generation Unit 304 Remote Request Information Generation Unit 305 Remote Request Information Transmission Department 306 Control Information Acquisition Unit 307 Control Unit 308 Drive Unit 4. Control device 41 processors 42 ROM 43 RAM 44 Device I / F section N Network

Claims

1. An information processing method executed by an information processing device that supports remote control of a plurality of moving objects by an operator, the method including: remote monitoring by an operator of a plurality of moving objects configured to be able to move autonomously and execute a predetermined task, based on video data transmitted from each of the plurality of moving objects; and remote operation by the operator of a first moving object to be remotely operated among the plurality of moving objects, based on video data transmitted from the first moving object, determining whether it is possible to stop transmission of the video data from a second moving object that may interfere with communication transmitting the video data from the first moving object during the remote control operation by the operator; If it is possible to stop the transmission of the video data, a response is taken to stop the transmission of the video data from the second moving object. Information processing methods.

2. If it is not possible to stop the transmission of the video data, the second moving body is caused to execute control to reduce the influence of the communication transmitting the video data from the second moving body on the communication transmitting the video data from the first moving body. The information processing method according to claim 1 .

3. determining that at least one of a moving body of the plurality of moving bodies whose distance from the first moving body is less than a predetermined threshold and a moving body that transmits the video data via the same base station as the first moving body is the second moving body; 3. The information processing method according to claim 1.

4. Identifying the second moving body when at least one of a maximum communication delay in communication in which the first moving body transmits the video data exceeds a predetermined threshold and a minimum reception power when the video data is received from the first moving body is less than a predetermined threshold.

3. The information processing method according to claim 1.

5. determining that transmission of the video data can be stopped when the road width of the current location of the second moving object exceeds a predetermined threshold; 3. The information processing method according to claim 1.

6. determining that transmission of the video data can be stopped when the distance between the second moving object and a surrounding moving object exceeds a predetermined threshold; 3. The information processing method according to claim 1.

7. determining that transmission of the video data can be stopped when the second moving object is captured in the video data from another moving object other than the second moving object among the plurality of moving objects; 3. The information processing method according to claim 1.

8. determining that transmission of the video data can be stopped when the second moving object is stopped; 3. The information processing method according to claim 1.

9. The control method includes at least one of the following controls: a control to stop the operation of the second moving body when it is possible to stop the transmission of the video data; a control to operate the second moving body while reducing the risk; a control to store an estimated bandwidth value before the transmission of the video data is stopped in the second moving body; and a control to adjust the number of the second moving bodies that will stop the transmission of the video data based on the work content of the remote operation by the operator on the first moving body. The information processing method according to claim 1 .

10. The control to reduce the impact includes at least one of the following controls: control to reduce the bandwidth used for transmitting the video data; control to detour the second moving body to a location where it can stop traveling; control to detour the second moving body so that the distance to the first moving body is equal to or greater than a predetermined threshold; control to detour the second moving body so that the video data is transmitted via a base station different from that used for the first moving body; and control to adjust the allowable number of the second moving bodies that will stop transmitting the video data based on the content of the remote operation work by the operator on the first moving body. The information processing method according to claim 2 .

11. After executing the action on the second moving object, request the operator to perform the remote operation on the first moving object.

3. The information processing method according to claim 1.

12. a request is made to the operator to perform the remote control of the first moving body in at least one of the following cases: when the second moving body is able to stop transmitting the video data as a result of the response; when there is no longer a possibility that communication transmitting the video data from the first moving body will be interrupted during the remote control operation by the operator; and when the number of the second moving bodies for which transmission of the video data is to be stopped falls to or below an allowable number adjusted based on the work content of the remote control of the first moving body by the operator. The information processing method according to claim 2 .

13. An information processing device operated by an operator that performs remote control of a plurality of moving bodies, each of which is configured to move autonomously and execute a predetermined task, including remote monitoring by the operator based on video data transmitted from each of the plurality of moving bodies, and remote operation by the operator of a first moving body that is a remote operation target among the plurality of moving bodies based on video data transmitted from the first moving body, displaying the video data from each of the plurality of moving bodies; stopping the display of the video data from a second moving object that is determined to have the potential to interfere with communication transmitting the video data from the first moving object during the remote control operation by the operator and that is capable of stopping the transmission of the video data; at least one processor configured to notify the operator that the transmission of the video data is voluntarily stopped; Information processing device.

14. A computer that realizes an information processing device that supports remote control of a plurality of moving objects by an operator, the computer including remote monitoring of a plurality of moving objects, each configured to move autonomously and execute a predetermined task, by an operator based on video data transmitted from each of the plurality of moving objects, and remote operation of a first moving object to be remotely operated among the plurality of moving objects by the operator based on video data transmitted from the first moving object, determining whether transmission of the video data from the first moving object can be stopped with respect to a second moving object that may interfere with the communication for transmitting the video data from the first moving object during the remote control operation by the operator; If it is possible to stop the transmission of the video data, stop the transmission of the video data from the second moving object. program.

Images