Remote control system, communication system, and remote control method

Abstract

The purpose of the present disclosure is to improve the safety of automatic movement of a moving body under remote monitoring.　For this purpose, the disclosed remote control system, which remotely controls the movement of an automatically movable moving body on the basis of a video transmitted from the moving body, comprises: a video quality measurement device that measures the video quality of video information obtained as a result of image capture by the moving body, and thereby generates video quality information indicating the video quality; and a control determination device that acquires the video quality information from the video quality measurement device, determines degradation of the video quality indicated by the video quality information, and gives a moving body control instruction to the moving body to accommodate the degradation.

Description

[Technical Field]

[0001] The present disclosure relates to a technology for remotely controlling a mobile object capable of autonomous movement. [Background technology]

[0002] In recent years, the development of self-driving cars has progressed. For example, self-driving cars of level 3 and level 4 defined by the Ministry of Land, Infrastructure, Transport and Tourism are capable of autonomous driving by utilizing sensors, cameras, etc. (Non-Patent Documents 1 and 2).

[0003] Furthermore, as one of the means to ensure safety when autonomous driving is difficult due to system malfunctions or false detections, it is being considered to have an operator in a remote location monitor footage from an onboard camera sent via a communication network and manually control the autonomous vehicle remotely in an emergency (Non-Patent Document 2).

[0004] However, current communication networks can experience fluctuations or interruptions in communication quality, which can cause disruptions or stoppages in remote monitoring footage, making it difficult to constantly remotely monitor an autonomous vehicle. If autonomous driving continues without remote monitoring, it will be impossible to respond to emergencies, which could reduce the safety of the autonomous driving system. For this reason, Non-Patent Document 3 discloses technology that automatically stops an autonomous vehicle from driving if the network is interrupted during remote monitoring. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Ministry of Land, Infrastructure, Transport and Tourism, "Guidelines for Safety Technology of Autonomous Driving Vehicles (Summary)"<http: / / www.mlit.go.jp / common / 001253666.pdf> [Non-patent document 2] Strategic Headquarters for the Promotion of an Advanced Information and Telecommunications Network Society, Strategic Conference for the Promotion of Public and Private Sector Data Utilization, "Public-Private ITS Initiative Roadmap 2019," June 7, 2019, < https: / / cio.go.jp / sites / default / files / uploads / documents / its_roadmap_2019.pdf> [Non-patent document 3] Ministry of Land, Infrastructure, Transport and Tourism, Tohoku Regional Development Bureau, "Autonomous Driving Vehicle (Level 3 Compatible Basic Vehicle)", Autonomous Driving Service Based at Roadside Station "Takahata" 1st Regional Experimental Conference (November 7, 2017), (2) Overview of Demonstration Experiment and Explanation of Experimental Vehicle, [Document-2] Autonomous Driving Vehicle<https: / / www.thr.mlit.go.jp / road / koutsu / Michi-no-Eki / jidou-unten / kisya-index / index_171107.html> , <https: / / www.thr.mlit.go.jp / road / koutsu / Michi-no-Eki / jidou-unten / kisya / pdf / 171107 / 03.pdff> Summary of the Invention [Problem to be solved by the invention]

[0006] However, even when communication between a vehicle or other moving object and a remote monitoring center is working properly, depending on the communication quality of the communication network (delays, etc.), this can lead to a decrease in resolution or the occurrence of block noise, making it difficult for the center operator to visually inspect the vehicle or for remote monitoring using systems such as AI (Artificial Intelligence).If the vehicle is allowed to drive autonomously in this state, it will be unable to respond to emergencies and safety will be reduced, which is an issue.

[0007] The present invention has been made in view of the above circumstances, and has as its object to improve the safety of automatic movement of a mobile object under remote monitoring. [Means for solving the problem]

[0008] In order to achieve the above object, the invention of claim 1 is a remote control system for remotely controlling the movement of a mobile object capable of autonomous movement based on video transmitted from the mobile object, the remote control system comprising: a video quality measurement device that measures the video quality of video information obtained by shooting the video information by the mobile object, and generates video quality information indicating the video quality; and a control determination device that acquires the video quality information from the video quality measurement device, determines degradation of video quality related to the video quality information, and issues a mobile object control instruction to the mobile object in response to the degradation; The control determination device has a video quality degradation determination unit that determines a degradation level of the video quality, a mobile object control instruction unit that transmits to the mobile object predetermined mobile object control instruction information indicating the mobile object control instruction that differs according to the degradation level, and a mobile object information acquisition unit that acquires from the mobile object mobile object information including a moving state of the mobile object, and the video quality degradation determination unit does not determine the degradation level of the video quality according to the moving state. It is a remote control system. [Effects of the Invention]

[0009] As described above, the present invention has the effect of improving the safety of automatic movement of a mobile object under remote monitoring. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is an overall configuration diagram of a communication system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an electrical hardware configuration of each device according to the embodiment. [Figure 3] FIG. 2 is a functional configuration diagram of the information collection device. [Figure 4] FIG. 2 is a functional configuration diagram of a video quality measurement device. [Figure 5] FIG. 2 is a functional configuration diagram of a control determination device. [Figure 6] FIG. 2 is a functional configuration diagram of a movement control device. [Figure 7] FIG. 10 is a diagram showing an example of a video information table 51 (for each video). [Figure 8] 10A and 10B are diagrams showing examples of video quality measurement information tables 52 and 72 (for each video). [Figure 9A] 10 is a diagram showing an example of a portion of the mobile object information table 33, 73, 93. FIG. [Figure 9B] 10 is a diagram showing an example of a portion of the mobile object information table 33, 73, 93. FIG. [Figure 9C]10 is a diagram showing an example of a portion of the mobile object information table 33, 73, 93. FIG. [Figure 10] FIG. 7 is a diagram showing an example of a detection target camera image setting condition table 74. [Figure 11] 10 is a diagram showing an example of a detection target camera image setting result table 75. FIG. [Figure 12] FIG. 10 is a diagram showing an example of a quality degradation condition table 76 (for each video). [Figure 13] FIG. 10 is a diagram showing an example of a quality deterioration detection status table 77. [Figure 14] FIG. 10 is a diagram showing an example of a mobile object control instruction information table 78 according to deterioration level. [Figure 15] FIG. 10 is a diagram showing an example of a mobile object control instruction information table 79 during recovery. [Figure 16] FIG. 10 is a diagram showing an example of a mobile object control content correction information table 100 (when traveling is stopped). [Figure 17] FIG. 2 is a sequence diagram showing processing or operations of the communication system. [Figure 18] 10 is a flowchart showing a process of setting a camera as a deterioration detection target by a control determination device. [Figure 19] 4 is a flowchart showing a control determination process performed by the control determination device. [Figure 20] 4 is a flowchart showing a control determination process performed by the control determination device. [Figure 21] 4 is a flowchart showing a control determination process performed by the control determination device. [Figure 22] 10 is a flowchart showing a movement control process performed by the movement control device. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] [System configuration of the embodiment] The outline of the configuration of a communication system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the overall configuration of a communication system according to an embodiment.

[0013] As shown in FIG. 1, the communication system 1 includes a mobile communication system 1A established on the side of a moving object such as a vehicle, and a remote control system 1B established on the side of a remote monitoring center.

[0014] The mobile communication system 1A includes an information collection device 2, a mobile control device 8, a communication device 11, video transmission devices 12a and 12b, and cameras 13a and 13b. The number of video transmission devices 12a and 12b may be one or more than two, and they are collectively referred to as "video transmission device 12." The number of cameras 13a and 13b may be one or more than two, and they are collectively referred to as "camera 13."

[0015] The remote control system 1B has a video quality measuring device 4, a control determination device 6, a communication device 16, video receiving devices 17a and 17b, and a remote monitoring device 19. Note that the number of video receiving devices 17a and 17b may be one or more than two, and they are collectively referred to as "video receiving device 17."

[0016] The communication device 11 and the communication device 16 can communicate data via a communication network N. The communication network N is constructed using a mobile network, the Internet, and the like.

[0017] In this embodiment, the mobile body is assumed to be, for example, a vehicle (automobile) having a communication function and an automatic driving function. However, the mobile body is not limited to an automobile, and may be an agricultural machine such as a tractor having a communication function and an automatic driving function. Furthermore, the mobile body includes not only a vehicle but also a ship, an aircraft, etc.

[0018] The mobile object is equipped with one or more cameras 13, and the images captured by the cameras 13 (images of the surroundings of the mobile object) are sent to a remote monitoring device 19 via a video transmitting device 12, a communication device 11, a communication device 16, and a video receiving device 17, and a monitor monitors the images displayed on the display of the remote monitoring device 19. Note that the images sent to the remote monitoring device 19 do not necessarily have to be monitored by a person, and may be monitored by a system such as AI (Artificial Intelligence).

[0019] When a monitor watching the video or a system such as AI detects that the mobile object is in a dangerous state based on the conditions around the mobile object, the remote monitoring device 19 remotely controls the movement of the mobile object.

[0020] Camera 13a, video transmitter 12a, and video receiver 17a form a set of video communication routes. Camera 13b, video transmitter 12b, and video receiver 17b form a set of video communication routes.

[0021] The video transmitting device 12 is a device that compresses and encodes video and / or audio in any format such as H.265 and converts it into a format (packets) that can be transferred over the communication network N, and corresponds to an encoder.

[0022] The communication devices 11 and 16 are network devices such as routers that handle packet routing, layer 2 switches, etc., and network devices that handle connection to wireless networks such as mobile routers, etc. The communication devices 11 and 16 realize communication between various devices mounted on mobile objects (information collection device 2, mobile control device 8, etc.) and various devices present in the remote monitoring center (video quality measurement device 4, control decision device 6, etc.).

[0023] The video receiving device 17 is a device that restores the IP packets compressed and encoded by the encoder as video and / or audio, and corresponds to a decoder.

[0024] The information collection device 2, the video quality measurement device 4, the control decision device 6, and the movement control device 8 will be described in detail later.

[0025] [Hardware configuration] Next, the electrical hardware configuration of the information collection device 2, video quality measurement device 4, control determination device 6, and movement control device 8 will be described with reference to Fig. 2. Fig. 2 is a diagram showing the electrical hardware configuration of each device according to the embodiment.

[0026] As shown in FIG. 2, the information collection device 2 is a computer and includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an SSD (Solid State Drive) 104, an external device connection I / F (Interface) 105, a network I / F 106, a media I / F 109, and a bus line 110.

[0027] Of these, the CPU 101 controls the overall operation of the information collection device 2. The ROM 102 stores programs such as an IPL (Initial Program Loader) used to drive the CPU 101. The RAM 103 is used as a work area for the CPU 101.

[0028] The SSD 104 reads or writes various data under the control of the CPU 101. Note that instead of the SSD 104, an HDD (Hard Disk Drive) may be used.

[0029] The external device connection I / F 105 is an interface for connecting various external devices, such as a display, a speaker, a keyboard, a mouse, a USB (Universal Serial Bus) memory, and a printer.

[0030] The network I / F 106 is an interface for performing data communication via a communication network N or a LAN (Local Area Network). Note that the network I / F 106 may be replaced by a wireless communication device.

[0031] The media I / F 109 controls reading and writing (storing) of data from and to a recording medium 109m such as a flash memory, etc. The recording medium 109m includes a DVD (Digital Versatile Disc) and a Blu-ray Disc (registered trademark).

[0032] The bus line 110 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 101 shown in FIG.

[0033] The video quality measuring device 4, the control determining device 6, and the movement control device 8 have the same configuration as the information collecting device 2, and therefore a description thereof will be omitted.

[0034] [Functional configuration of information collection device] Next, each functional configuration of the information collection device 2 will be described with reference to FIGS.

[0035] Fig. 3 is a functional configuration diagram of the information collection device 2. As shown in Fig. 3, the information collection device 2 has a communication unit 20 and a processing unit 20p. These units have functions realized by commands from the CPU 101 in accordance with programs stored in the RAM 103 or the like. The information collection device 2 also has a storage unit 30 constructed by the RAM 103 and / or the SSD 104.

[0036] (Mobile information table) A mobile object information table 33 as shown in FIGS. 9A, 9B, and 9C is constructed in the storage unit 30. In the mobile object information table, mobile object status information, camera information, and environmental information are associated and managed as mobile object information. The mobile object information table 33 is merely divided into FIGS. 9A, 9B, and 9C due to space limitations, and may be one table or multiple tables. The mobile object information stores information required when changing the mobile object's travel control means depending on the mobile object's situation. In addition, when the control determination device 6 uses a function (described below) that automatically generates conditional expressions, etc. of a quality degradation condition table 76 described below and presents them to the user, information obtained from a decoder, etc. (such as the decoder model number, obtainable information, video resolution, video encoding method, buffer size, retransmission control method and its settings) is registered.

[0037] As shown in FIG. 9A, the mobile object information table 33 includes mobile object state information. The mobile object state information includes, for example, information indicating whether the mobile object is in autonomous driving mode, its moving state, its moving speed, whether there is a person inside the mobile object, and whether there is a following object. The moving state includes moving forward, turning right, turning left, and moving backward. If the mobile object is a vehicle, a following object is a vehicle traveling behind the vehicle. The mobile object state information may also include information other than the information shown in FIG. 9A.

[0038] 9B, the mobile object information table 33 also includes camera information. The camera information is information relating to the cameras 13a and 13b. The camera information includes, for example, information indicating the installation position of the camera on the mobile object, an identifier of the video transmitting device that transmits the camera video, an identifier of the video receiving device that receives the camera video, a media rate, and a codec to be used. The camera information may also include information other than the information shown in FIG. 9B.

[0039] Furthermore, as shown in Fig. 9C, the moving body information table 33 includes environmental information. The environmental information is information relating to the inside and outside of the moving body. The environmental information includes, for example, information indicating the time of day, weather, road surface conditions, and friction coefficient (friction coefficient of tires when the moving body is a vehicle). The environmental information may also include information other than the information shown in Fig. 9C.

[0040] <Each function configuration> Returning to Fig. 3, the communication unit 20 performs data communication with the mobile body system, various sensors of the mobile body, the control decision device 6, and the mobile control device 8. The mobile body system is a system using a computer or the like within the mobile body, and in the case of a vehicle, it indicates a system using a CAN (Controller Area Network). The various sensors are a temperature sensor, a humidity sensor, an illuminance sensor, etc.

[0041] The processing unit 20p includes a mobile object information acquisition unit 21 and a mobile object information transmission unit 29.

[0042] The mobile object information acquisition unit 21 collects mobile object information periodically, such as every second, or at any timing, such as when the traveling state of the mobile object changes, and stores the information in the mobile object information table 31. The method of acquiring the traveling state of the mobile object is arbitrary, but it is assumed that the information may be determined based on a combination of various sensor information, such as turn signal information and shift lever information, a traveling plan, the position of the mobile object based on a GPS (Global Positioning System), movement information, map information, etc.

[0043] The mobile object information transmitting unit 29 transmits mobile object information including the moving state of the mobile object periodically, such as every second, or at any timing, such as when collecting the mobile object information. Here, the entire contents of the mobile object information table 31 may be transmitted, or only the difference information from the previous transmission may be transmitted.

[0044] Furthermore, if it is desired to change the movement control means of a moving body depending on the situation of the moving body, such as whether or not a person is on board or whether or not there is a follower (vehicle), the moving body information acquisition unit 21 needs to collect the necessary information and store it in the moving body information table 33, and the moving body information transmission unit 29 needs to transmit that information to the movement control device 8. Furthermore, if the control determination device 6 uses a function (described later) of automatically generating conditional expressions, etc. of the quality degradation condition table 76 and presenting them to the user, the moving body information acquisition unit 21 needs to collect the necessary information (decoder model number, obtainable information, video resolution, video encoding method, buffer size, retransmission control method and its settings, etc.), store it in the moving body information table 33, and the moving body information transmission unit 29 needs to transmit that information to the control determination device 6.

[0045] [Functional configuration of video quality measurement device] Next, the functional configuration of the video quality measurement device 4 will be described with reference to FIGS.

[0046] Fig. 4 is a functional block diagram of a video quality measurement device. As shown in Fig. 4, the video quality measurement device 4 has a communication unit 40 and a processing unit 40p. These units each have functions realized by commands from the CPU 101 in accordance with programs stored in RAM 103 or the like. The video quality measurement device 4 also has a storage unit 50 constructed from the RAM 103 and / or SSD 104.

[0047] In this embodiment, we will explain the case where MDI is used as the (quality degradation) detection index. MDI is composed of two indexes, DF (Delay Factor) and MLR (Media Loss Rate), where DF represents the time it takes to drain the video buffer and MLR represents the number of video data losses.

[0048] It is also calculated as DF = [VB(max) - VB(min)] / MR. Here, MR represents the media rate during video transmission, and VB is defined as follows:

[0049] VB= sum (Sj) - MR * T; where j=1..i-1 Here, Sj represents the media payload size of the j-th packet, and T represents the time interval for aggregation.

[0050] It is also calculated as MLR = [P - Precv] / T. Here, P represents the number of expected received packets, Precv represents the number of actually received packets, and T represents the time interval for counting.

[0051] From these, when MDI is used as an index, the video information acquisition unit 42 (described later) acquires parameters required for calculating DF and MLR (media rate, media payload size of each packet, expected number of received packets, number of actually received packets) or information required for calculating these parameters from the video receiving device 17, and the video quality measurement information calculation unit 46 (described later) calculates DF and MLR at intervals of T seconds. Note that T is arbitrary, but in this embodiment, it is assumed that aggregation is performed at intervals of T=1 second. In addition, in this embodiment, the video quality measurement information (quality degradation detection index) used to determine video degradation is more specifically the MDI DF and MLR calculated at intervals of 1 second. Furthermore, in this embodiment, a case will be described in which video quality measurement information is transmitted at arbitrary timing (each time the MDI DF and MLR are calculated every second) and only updated information is transmitted every second.

[0052] In addition, if video receiving device 17 is unable to collect the information necessary to calculate the quality degradation detection index, or if a video quality degradation index that requires input of the video stream itself when calculating the value is used, as in ITU-T P.1204.3, the video receiving device 17a or communication device 16, etc., will be equipped with a function to copy the video stream and input it into video quality measuring device 4, which will then collect and calculate the necessary information.

[0053] (Video information table (for each video)) In the storage unit 30, a video information table 51 (for each video) as shown in FIG. 7 is created.

[0054] 7 is an example of a video information table 51 for each video (in the case of video (v1) from camera 13a), and a table can exist for each camera video (video identifier). In this embodiment, each time video receiving device 17a receives a packet, video information acquisition unit 42 acquires the media rate and payload size of the media packet, which are necessary for calculating the MDI, from video receiving device 17a along with the reception time, and registers them in video information table 51 for each video.

[0055] Because the media rate is updated less frequently than the packet reception interval, the video information acquisition unit 42 does not need to acquire and update the media rate each time a media packet is received. It may acquire and update the information only once during initial setup or when the rate is dynamically changed during operation. Calculating the MDI DF and MLR requires information on the expected number of received packets and the number of actually received packets. The expected number of received packets can be calculated from the aggregation period T, the media rate, packet size, etc. The number of actually received packets can be calculated by counting the number of packets (corresponding to the number of rows in the table) received during the aggregation period T based on the reception time information in the video information table 51 for each video. Some decoder products can aggregate the MDI DF and MLR at any time interval and then notify other systems. In this case, the received MDI DF and MLR may be stored in the video quality measurement information table 52 (described later).

[0056] The information registered in the video information table 51 for each video varies depending on the (quality degradation) detection index and conditional expression used for quality degradation detection, and may include, for example, the reception bit rate, delay, jitter, the state of the reception buffer, information used to calculate evaluation indexes for various video quality evaluation technologies such as MOS, VMAF, PSNR, SSIM, and MDI specified in ITU-T P.1204.3, and decoder error messages. If the video information acquisition unit 42 can acquire the quality degradation detection index itself from the video receiving device 17a, the same information as that registered in the video quality measurement information table 52 for each video, which will be described later, may be registered. There may also be information that is acquired periodically, such as at one-second intervals, and information that is notified at any time in an event-driven manner, such as information for each packet or error messages, and registered.

[0057] (Video quality measurement information table (per video)) In the storage unit 30, a video quality measurement information table 52 (for each video) as shown in Fig. 8 is created. Note that in this embodiment, a case will be described in which MDI is used as a quality degradation detection index.

[0058] FIG. 8 is an example of a video quality measurement information table 52 for each video (for video (v1) from camera 13a), and a table can exist for each camera video. Here, the results of calculating the time, MDI DF, and MDI MLR information at a counting interval T = 1 second are registered. For example, the time in row NO:n10 represents a value calculated based on the information of the video data received by the video information acquisition unit 42 from "2022-01-01 10:00:04.000000" to "2022-01-01 10:00:05.000000". The information registered in video quality measurement information table 52 varies depending on the index (quality degradation detection index) used for quality degradation detection and the conditional expression, and may include, for example, reception bit rate, delay, jitter, reception buffer status, evaluation indexes of various video quality evaluation technologies such as MOS, VMAF, PSNR, SSIM, and MDI in ITU-T P.1204.3, decoder error messages, etc. Also, there may be information that is calculated periodically, such as at one-second intervals, and information that is notified at any time in an event-driven manner, such as error messages, and registered.

[0059] <Each function configuration> Returning to FIG. 4, the communication unit 40 performs data communication with the video receiving device 17a, the video receiving device 17b, and the control determination device 6.

[0060] The processing unit 40p includes a video information acquisition unit 42, a video quality measurement information calculation unit 46, and a video quality measurement information transmission unit .

[0061] The video information acquisition unit 42 collects information about the video received by the video receiving device 17 from the video transmitting device 12 periodically, such as every second, or at any arbitrary timing, and stores the collected information in a video information table 51 for each video corresponding to the received video. The video information acquisition unit 42 also calls the video quality measurement information calculation unit 46 at the timing of acquiring the information. Here, it is assumed that the correspondence relationship between which video receiving device 17 is receiving which video from which camera 13 can be identified by the video receiving device 17 and the video information acquisition unit 42 based on, for example, information about the correspondence relationship between devices when the cameras and video receiving device 17 are installed, or by including a unique identifier (video identifier) ​​for each video in the information from the video receiving device 17. The same applies to the correspondence relationship between which camera's video is received on which video channel when one video receiving device 17 receives videos from multiple cameras 13. In this embodiment, a video identifier is embedded in various transmission and reception data.

[0062] The information collected by the video information acquisition unit 42 from the video receiving device 17 may vary depending on the detection index (quality degradation detection index) and conditional expression used to determine video quality degradation. Examples of quality degradation detection indexes that can be used include, but are not limited to, video quality evaluation indexes such as the MDI (Media Delivery Index) defined in RFC4445 and the MOS (Mean Opinion Score) defined in ITU-T P.1204.3, and error messages from decoders, etc. Error messages include those related to data loss, data order reversal, and decoding errors.

[0063] Video quality measurement information calculation unit 46 executes processing periodically, such as every second, or at any timing, such as when called by video information acquisition unit 49. By performing calculations based on the collected information, information used to determine video quality degradation (video quality measurement information) is calculated and stored in video quality measurement information table 52 for each video. Furthermore, video quality measurement information calculation unit 46 calls video quality measurement information transmission unit 48 when calculations are completed, etc.

[0064] The video quality measurement information transmitter 48 transmits the calculated video quality measurement information to the control determination device 6 periodically, such as every second, or at any timing, such as when called by the video quality measurement information calculator 46. The information transmitted at this time may be all the information in the video quality measurement information table 52, or may be only the difference from the previously transmitted information (such as information on videos whose information has been updated or limited to only the updated information). Furthermore, by including a video identifier in the transmitted information that can identify which video the information relates to, it becomes possible to link the video and the transmitted information.

[0065] [Functional configuration of the control decision-making device] Next, the functional configuration of the control decision device will be described with reference to FIGS. 5 and 10 to 15. FIG.

[0066] Fig. 5 is a functional configuration diagram of the control decision device 6. As shown in Fig. 5, the control decision device 6 has a communication unit 60 and a processing unit 60p. These units have functions realized by commands from the CPU 101 in accordance with programs stored in the RAM 103 or the like. The control decision device 6 also has a storage unit 70 constructed by the RAM 103 and / or the SSD 104.

[0067] A video quality measurement information table 72 (for each video) and a mobile object information table 73 are constructed in the storage unit 70. The video quality measurement information table 72 (for each video) is similar to the video quality measurement information table 52 (see FIG. 8), so a description thereof will be omitted. The mobile object information table 73 is similar to the mobile object information table 33 (see FIGS. 9A to 9C), so a description thereof will be omitted.

[0068] (Detection target camera image setting condition table) In the storage unit 70, a detection target camera image setting condition table 74 as shown in FIG. 10 is created.

[0069] 10 is an example of a detection target camera image setting condition table, in which the movement state of a moving object and the cameras and images to be detected for each movement state are registered. The camera to be targeted for each movement state may be set arbitrarily by the user, or a function may be provided to automatically collect information such as the installation location of the camera images and automatically present the target cameras. Furthermore, the information corresponding to which images correspond to which cameras may be set manually when the system is constructed, or may be set automatically by collecting information from sensors, etc.

[0070] (Detection target camera video setting result table) In the storage unit 70, a detection target camera image setting result table 75 as shown in FIG. 11 is created.

[0071] FIG. 11 is an example of the detection target camera image setting result table 75, in which detection target cameras and image identifiers corresponding to the cameras are registered.

[0072] (Quality degradation condition table (for each video)) In the storage unit 70, a quality degradation condition table 76 as shown in FIG. 12 is created.

[0073] FIG. 12 shows an example of a quality degradation condition table 76 for each video (for video (v1) from camera 13a), and a table may exist for each camera video. The quality degradation condition table 76 stores an index (video quality degradation detection index) used to detect video quality degradation, a conditional expression for determining video quality degradation for each degradation level, and a detection flag for determining whether each conditional expression matches. In this embodiment, the MDI DF is used as the video quality degradation detection index, and when the value exceeds a specific threshold (e.g., 200) a certain number of times (e.g., three times) in succession, a level 1 quality degradation is detected. However, the video quality degradation detection index and the conditional expression using the index are not limited to this. Furthermore, the quality degradation condition table 76 must be configured with one or more conditional expressions. However, the user may freely set the conditional expressions for each quality degradation level to be registered. Alternatively, the table may be provided with a function for automatically generating available video quality degradation detection indexes, detectable degradation levels, and conditional expressions based on information obtained from a decoder or the like, and presenting these to the user. The information obtained from the decoder, etc., includes the decoder model number, obtainable information, video resolution, video encoding method, buffer size, retransmission control method and its settings, etc. Furthermore, the optimal video quality degradation detection index, conditional expression, and threshold value within the conditional expression may vary depending on whether the video is monitored by a human or a system such as AI (Artificial Intelligence). In such cases, a quality degradation condition table for each video corresponding to each may be provided.

[0074] (Quality Deterioration Detection Status Table) In the storage unit 70, a quality degradation detection status table 77 as shown in FIG. 13 is created.

[0075] FIG. 13 is an example of the quality degradation detection status table 77, in which video identifiers and the latest past degradation levels of each video represented by the video identifiers are registered in association with each other.

[0076] (Table of mobile control instruction information according to deterioration level) In the storage unit 70, a mobile object control instruction information table 78 corresponding to the deterioration level as shown in FIG. 14 is constructed.

[0077] 14 is an example of a mobile object control instruction information table 78 according to deterioration level, in which the deterioration level and different action (mobile object control instruction) information according to the deterioration level are registered. The content of the mobile object control instruction information to be executed for each deterioration level is arbitrary, but at least one action corresponding to the deterioration level corresponding to one or more conditional expressions registered in the quality deterioration condition table 76 must be registered, either manually by the user or by automatically generating a recommended value from the system. By preparing multiple deterioration levels and specifying a deceleration instruction as the action for low-level deterioration and a stop instruction as the action for high-level deterioration, it becomes possible to perform step-by-step control, from decelerating to stopping the mobile object as the quality deterioration progresses.

[0078] (Mobile control instruction information table during recovery) In the storage unit 70, a mobile object control instruction information table 79 for recovery as shown in FIG. 15 is constructed.

[0079] 15 is an example of an action table at the time of recovery, in which the degradation level before recovery, the degradation level after recovery, and information on the action (mobile object control instruction) to be executed when there is a change from the degradation level before recovery to the degradation level after recovery are registered. The content of the action to be executed when each degradation level changes is arbitrary, but it is necessary that one or more actions to be executed when there is a change from a degradation level corresponding to one or more conditional expressions registered in the quality degradation condition table 76 to a lower degradation level are registered, either manually by the user or by automatically generating a recommended value from the system.

[0080] <Each function configuration> Returning to FIG. 5, the communication unit 60 performs data communication with the information collection device 2, the video quality measurement device 4, and the mobile control device 8.

[0081] The processing unit 60p includes a mobile object information acquisition unit 61, a degradation detection target camera image setting unit 62, an image quality information acquisition unit 64, an image quality degradation determination unit 66, and a mobile object control instruction unit 69.

[0082] The mobile object information acquisition unit 61 receives mobile object information from the information collection device 2 periodically, such as every second, or at any arbitrary timing, stores the received mobile object information in the mobile object information table 73, and calls the deterioration detection target camera image setting unit 62.

[0083] The degradation detection target camera image setting unit 62 executes processing by being called periodically, such as every second, or at any timing, such as when called by the mobile object information acquisition unit 61. The degradation detection target camera image setting unit 62 identifies cameras that should be targets for quality degradation detection based on the current movement state (forward, right turn, left turn, backward, etc.) registered in the mobile object information table 73 and the information of the detection target camera corresponding to the movement state in the detection target camera image setting condition table 74, and stores the results in the detection target camera image setting result table 75.

[0084] The video quality information acquisition unit 64 executes processing periodically, such as every second, or at an arbitrary timing. The video quality information acquisition unit 64 receives video quality measurement information from the video quality measurement device 4 and stores the received video quality measurement information in a video-specific video measurement information table 72 for the video corresponding to the information. Furthermore, the video quality information acquisition unit 64 references the detection target camera video setting result table 75. If the received video quality measurement information is for the detection target camera video, the video quality information acquisition unit 64 notifies the video quality degradation determination unit 66 of the received video quality measurement information (including the video identifier) ​​and calls the video quality degradation determination unit 66. Note that, in the configuration of this embodiment, one video quality measurement device 4 is connected to multiple video receiving devices 17. However, a separate video quality measurement device 4 and control determination device 6 may also be provided for each video receiving device 17. In this case, it is assumed that it is possible to distinguish which received information corresponds to which video by the correspondence between devices when the remote control system 1B is constructed or the IP address used for communication, even if the video identifier is not embedded in the transmission / reception information (the same applies to other descriptions regarding the video identifier). Also, while the description here assumes that the video quality measuring device 4 also measures the quality of camera images that are not currently being detected and transmits the information to the control decision device 6, it is also possible to assume that the video quality measuring device 4 is limited to measuring the quality and transmitting information only from cameras that are currently being detected.

[0085] The video quality degradation determination unit 66 executes processing by being called periodically, such as every second, or at an arbitrary timing, such as when called by the video quality information acquisition unit 64. Based on the video identifier and quality degradation detection indicator (e.g., MDI DF) included in the notified video quality measurement information, the video quality degradation determination unit 66 selects a conditional expression for each degradation level related to the quality degradation indicator from the quality degradation condition table 76 for that video, and evaluates each conditional expression using the received information and information in the video quality measurement information table 72 for each video, thereby determining (detecting) at what degradation level video quality degradation has occurred or whether the degradation has recovered. Furthermore, if a conditional expression is matched (video quality degradation has occurred), the video quality degradation determination unit 66 updates the detection flag corresponding to that conditional expression in the quality degradation condition table 76 to True. Furthermore, if the degradation detection flag no longer matches a conditional expression for which True is set, the video quality degradation determination unit 66 updates the degradation detection flag to False. Furthermore, the video quality degradation assessment unit 66 evaluates all the conditional expressions to be evaluated related to the quality degradation detection index of the evaluation target, and then extracts the maximum degradation level from among all conditional expressions for which the degradation detection flag is True, including conditional expressions other than the quality degradation detection index. (If all are False, the degradation level is set to 0, indicating no degradation.) Furthermore, if this value differs from the current degradation level of the video in the quality degradation detection status table 77, the video quality degradation assessment unit 66 notifies and calls the mobile control instruction unit 69 of the degradation level and video identifier. Here, the quality degradation level represents degradation of video quality in stages, enabling quality degradation detection and mobile control according to the stage. In this embodiment, the quality degradation levels are defined as four levels: Level 0: no degradation; Level 1: signs of degradation; Level 2: (Currently, degradation is occurring at a level that does not affect monitoring, but further deterioration is expected); and Level 3: Degradation is so severe that monitoring is impossible or video is stopped.

[0086] In this embodiment, the MDI DF and MLR are used individually as quality degradation detection indicators. If multiple detection indicators exist and the video quality information acquisition unit 64 acquires information including multiple detection indicators at once, the video quality degradation assessment unit 66 selects, for each detection indicator, a conditional expression for quality degradation detection that uses that detection indicator and performs matching in order. For example, the video quality degradation assessment unit 66 evaluates all conditional expressions using the MDI DF as a detection indicator, and then evaluates all conditional expressions using the MDI MLR as a detection indicator. If multiple conditional expressions are matched at once, the video quality degradation assessment unit 66 updates all detection flags corresponding to the matched conditional expressions to True and then extracts the highest (most severe) quality degradation level. If a conditional expression for a high degradation level includes a conditional expression for a low degradation level, the video quality degradation assessment unit 66 may evaluate the conditional expression for the high degradation level first and set the detection flag for that conditional expression to False without evaluating the conditional expression for the low degradation level, thereby shortening the time required to evaluate the conditional expressions. Furthermore, in this embodiment, when the conditional expression for detecting quality degradation is no longer matched, the video quality degradation determination unit 66 resets the detection flag to False (recovery from degradation), but a separate conditional expression for resetting the detection flag from True (degradation) to False (recovery from degradation) may be prepared, and the flag may be reset to False when the conditional expression is matched. This is intended for use in which the conditions for recovery from quality degradation are made stricter than the conditions for determining quality degradation, in order to reduce the risk of frequent control or inefficiency caused by the quality degraded state returning immediately after determining that recovery from quality degradation has occurred.

[0087] The indicators used for quality degradation detection, the conditional expressions used to express each quality degradation level, and the number of quality degradation levels are all arbitrary. For example, when the detection indicator used for quality degradation detection is the MDI DF, examples of conditional expressions include when the MDI DF value exceeds a specific threshold a certain number of times in succession, when the MDI DF value exceeds a specific threshold a certain number of times within a predetermined time t, when the MDI DF increase rate over t seconds exceeds a specific threshold, and when the predicted time required for the MDI DF to reach a specific value at the current rate of increase falls below a specific threshold. It is also possible to set a conditional expression that combines multiple indicators, such as when the MDI DF exceeds a specific threshold and the MDI MLR exceeds a specific threshold. Furthermore, in cases where a decoder error message or the like is used as a detection indicator, in addition to evaluation by matching a collected value with a specific threshold, it is also possible to determine quality degradation if a character string included in the error message matches a registered character string. Note that depending on the conditional expression, the video quality degradation determination unit 66 may need to aggregate information from the video quality measurement information table 72 and then match it with the conditional expression. FIG. 12 shows an example in which a level 1 quality degradation is detected when the MDI DF value exceeds a specific threshold (e.g., 200) a certain number of times (e.g., three times) in a row. In this case, the process of counting the number of times the threshold of 200 is exceeded is required using the latest unevaluated information and the two previous MDI DF values ​​(a total of three values). Note that the quality degradation condition table is assumed to have one or more conditional expressions pre-set. However, the conditional expressions for each registered quality degradation level may be freely set by the user, or a function may be provided to automatically generate usable video quality degradation detection indicators, detectable degradation levels, and conditional expressions based on information obtained from a decoder or the like, and present them to the user. Note that information obtained from a decoder or the like includes the decoder model number, available information, video resolution, video encoding method, buffer size, retransmission control method, and their settings.

[0088] The mobile object control instruction unit 69 executes processing by being called periodically, such as every second, or at an arbitrary timing, such as when called by the video quality degradation determination unit 66. The mobile object control instruction unit 69 compares the degradation level notified by the video quality degradation determination unit 66 with the current degradation level of the video of the video identifier (there may be multiple) set as the detection target in the detection target camera video setting result table 75 in the quality degradation detection status table 77, and if the notified degradation level is greater (the degradation has worsened), it updates the degradation level of the notified video identifier in the quality degradation detection status table 77 to the notified degradation level. In addition, the mobile object control instruction unit 69 executes a mobile object control instruction corresponding to the degradation level defined in the mobile object control instruction information table 78 according to the degradation level. Furthermore, the mobile control instruction unit 69 compares the notified degradation level with the current degradation level of the video of the video identifier (there may be multiple) set as the detection target in the detection target camera video setting result table 75 in the quality degradation detection status table 77, and if the notified degradation level is smaller (degradation has improved), it updates the degradation level of the notified video identifier in the quality degradation detection status table 77 to the notified degradation level.

[0089] Then, the mobile object control instruction unit 69 executes the mobile object control instruction defined in the mobile object control instruction information table 79 for recovery (corresponding to the deterioration level after the update from the deterioration level before the update). The mobile object control instructions include sending driving instructions to the mobile control device 8 such as stop driving, resume driving, decelerate, accelerate, or take shelter, or instructions to start or stop analysis using more advanced video quality degradation detection indicators or logic. In the case of deceleration or acceleration, speed information such as the speed to which the vehicle should decelerate or accelerate may also be included.

[0090] This function enables gradual processing, from deceleration to stopping, depending on the level of video quality degradation. The content of the mobile control instructions registered for each degradation level can be freely set by the user, but it is assumed that at least one mobile control instruction is registered. Furthermore, analysis using more advanced video quality degradation detection indicators or logic, for example, normally uses a method that can detect quality degradation in a relatively short time, such as in units of one second, using only packet-level information such as MDI. When signs of quality degradation are detected, the mobile decelerates to reduce the risk of accidents, while requiring more computational resources and analysis of several seconds of video streams, such as ITU-T P.1204.3. However, once the mobile control instruction unit 69 begins analysis using a high-precision method, it is expected that the decision to stop driving will be made based on the results, reducing false positives and preventing frequent stops from impairing the efficiency of automated driving. Furthermore, when starting or stopping additional analysis, the mobile object control instruction unit 69 transmits to the video quality measurement device 4 information on which index to use for analysis and an instruction to start or stop the additional analysis, and the video quality measurement device 4 then starts or stops the instructed analysis and starts or stops collecting the additional information necessary for the analysis. The mobile object control instruction unit 69 may also have a function to notify the remote monitoring device 39 of the status of video quality degradation detection (such as the current degradation level and recovery status) and the associated control of the mobile object's travel, or to notify the remote monitoring device 39 in advance by embedding the information in the monitoring video. In this case, the control determination device 6 must additionally have a function for communicating with the remote monitoring device 19 and the video transmission device 12.

[0091] [Functional configuration of the mobile control device] Next, each functional configuration of the mobile control device will be described with reference to FIGS.

[0092] Fig. 6 is a functional configuration diagram of the mobile control device. As shown in Fig. 6, the mobile control device 8 has a communication unit 80 and a processing unit 80p. These units have functions realized by commands from the CPU 101 in accordance with programs stored in the RAM 103 or the like. The mobile control device 8 also has a storage unit 90 constructed by the RAM 103 and / or the SSD 104.

[0093] A mobile object information table 93 is created in the storage unit 90. Since this is similar to the mobile object information table 33 (see FIGS. 9A to 9C), a description thereof will be omitted.

[0094] (Mobile object control content correction information table) In the storage unit 70, a moving object control content correction information table 100 (for the case of stopping travel) as shown in FIG. 16 is constructed.

[0095] FIG. 16 is an example of a mobile object control content correction information table 100 (here, for stopping travel), and there can be a table for each type of travel control (stop travel, resume travel, deceleration, acceleration, evacuation, etc., and speed in the case of acceleration / deceleration, etc.). The mobile object control content correction information table 100 stores conditions for correcting the content of a mobile object control instruction and correction contents for correcting the mobile object control instruction content when the conditions are met. The correction conditions and correction contents can be set arbitrarily by the user, or recommended settings, etc., can be automatically set by the system.

[0096] <Each function configuration> Returning to FIG. 6, the communication unit 80 performs data communication with the mobile system, various sensors, the information collection device 2, and the control decision device 6.

[0097] The processing unit 80p includes a mobile object information acquisition unit 81, a mobile object control instruction information reception unit 83, and a mobile object control execution unit 85.

[0098] The mobile entity information acquisition unit 81 receives mobile entity information from the information collection device 2 periodically, such as every second, or at an arbitrary timing, and stores the received information in the mobile entity information table 93.

[0099] The mobile body control instruction information receiving unit 83 receives mobile body control instructions (stopping, restarting, decelerating, accelerating, retreating, etc., and speed in the case of acceleration or deceleration, etc.) from the control decision device 6 periodically, such as every second, or at any timing, and notifies the mobile body control execution unit 85 of the received mobile body control instruction information and makes a call.

[0100] The mobile object control execution unit 85 controls the moving state of the mobile object in cooperation with the mobile object system in accordance with the notified mobile object control instruction information. For example, in the case of an instruction to stop moving, the mobile object control execution unit 85 stops the moving of the mobile object. Also, in the case of an instruction to decelerate to 20 km / h, the mobile object control execution unit 85 decelerates the moving speed of the mobile object to 20 km / h. Furthermore, the mobile object control execution unit 85 can also accept instructions such as an instruction to accelerate at an initial speed. In this case, the mobile object control execution unit 85 stores the speed (initial speed) before stopping or decelerating, and if the current state of the mobile object is stopped, resumes moving at the initial speed, or if moving, accelerates to the initial speed.

[0101] Furthermore, when the mobile object control content correction information table 100 corresponding to the notified mobile object control instruction content (e.g., stop traveling) is registered, the mobile object control execution unit 85 compares it with the information in the mobile object information table 93, and if the correction conditions are met, the mobile object control content may be changed to the content registered in the correction content and then the mobile object control may be performed. Here, before comparing it with the information in the mobile object information table 93, the mobile object control execution unit 85 may request the information collection device 2 to collect and transmit the latest mobile object information, and update the mobile object information to the latest before performing processing.

[0102] [Communication system processing or operation] Next, the processing or operation of each device in the communication system 1 will be described with reference to Fig. 17 to Fig. 22. Fig. 17 is a sequence diagram showing the processing or operation of the communication system.

[0103] S11: In the information collection device 2, the mobile object information acquisition unit 21 collects mobile object information including the traveling state of the mobile object.

[0104] S12: The mobile object information transmitting unit 29 transmits the mobile object information to the control decision device 6 via the communication unit 20.

[0105] S13: The control determination device 6 performs a process of setting the camera as the deterioration detection target.

[0106] The process of S13 will now be described in detail with reference to Fig. 18. Fig. 18 is a flowchart showing the process of setting the camera as a deterioration detection target by the control determination device.

[0107] S111: The mobile object information acquisition unit 61 receives the mobile object information via the communication unit 60, and stores the received mobile object information in the mobile object information table 73.

[0108] S112: The deterioration detection target camera image setting unit 62 identifies the corresponding detection target camera (image identifier) ​​by searching the detection target camera image setting condition table 74 based on the information on the traveling state included in the mobile object information received by the mobile object information acquisition unit 61. For example, if the traveling state is "forward," the front camera (image identifier v1) is identified.

[0109] S113: The degradation detection target camera video setting unit 62 sets the identification information (camera name, etc.) and video identifier of the detection target camera in the detection target camera video setting result table 75.

[0110] This concludes the detailed description of the process at S13.

[0111] S14: Returning to FIG. 17, in the video quality measurement device 4, the video quality measurement information calculation unit 46 measures the quality of the camera video (for example, MDI DF) every second.

[0112] S15: The video quality measurement information transmitter 48 transmits the video quality information indicating the video identifier and the video quality (for example, MDI DF) to the control decision device 6 via the communication unit 40.

[0113] S16: The control decision device 6 performs a control decision process.

[0114] The process of S16 will now be described in detail with reference to Figures 19 to 21. Figures 19 to 21 are flowcharts showing the control decision process performed by the control decision device.

[0115] S211: The video quality information acquisition unit 64 stores the video quality information received via the communication unit 60 in the video quality measurement information table 72 corresponding to the video identifier received via the communication unit 60.

[0116] S212: The video quality information acquisition unit 64 searches the detection target camera video setting result table 75 based on the received video identifier to search for (the identification information of) the camera that is the detection target for the corresponding video quality degradation.

[0117] S213: The video quality information acquisition unit 64 determines whether the video quality information received in process S15 is video quality information from the detection target camera, based on whether the information is managed in the detection target camera video setting result table 75 through the search in S212. If the video quality information received in process S15 is not video quality information from the detection target camera (S213; NO), process S16 ends.

[0118] S214: On the other hand, if the video quality information received in process S15 is video quality information from the camera that is the detection target (S213; YES), the video quality information acquisition unit 64 calls the video quality degradation determination unit 66.

[0119] S221: As shown in FIG. 20, the video quality degradation judgment unit 66 searches the quality degradation condition table 76 for a degradation level corresponding to a conditional expression that the video quality information satisfies based on a video quality degradation detection indicator (e.g., MDI DF) included in the video quality information, and registers the detection flag for that degradation level as True and registers the others as False.

[0120] S222: The video quality degradation determining unit 66 extracts, from all the conditional expressions in the quality degradation condition table 76, a predetermined degradation level for which the detection flag is True and which is the maximum.

[0121] S223: The video quality degradation determination unit 66 compares the current degradation level of the video quality information received in step S15 with the most recent degradation level of the video quality information with the same video identifier.

[0122] S224: The video quality degradation determination unit 66 determines whether the current degradation level is different from the most recent degradation level in the past. If they are not different, that is, if they are the same (S224; NO), the process S16 ends.

[0123] S225: On the other hand, if the current degradation level is different from the most recent degradation level (S224; YES), the video quality degradation determination unit 66 calls the mobile object control instruction unit 69.

[0124] S231: As shown in FIG. 21, the mobile object control instruction unit 69 changes the most recent degradation level in the past corresponding to the video identifier received in S15 to the current degradation level in the quality degradation detection status table 77.

[0125] S232: The mobile object control instruction unit 69 determines whether the current deterioration level has decreased below the most recent deterioration level in the past through the process of S232.

[0126] S233: If the current degradation level has dropped below the most recent degradation level in the past (S232; YES), the mobile object control instruction unit 69 extracts mobile object control instruction information (action information) corresponding to the degree of drop in the degradation level from the mobile object control instruction information table 79 at the time of recovery.

[0127] S234: On the other hand, if the current degradation level is not lower than the most recent degradation level in the past (S232; NO), the mobile object control instruction unit 69 extracts corresponding mobile object control instruction information (action information) based on the current degradation level.

[0128] This concludes the detailed description of the process in S16.

[0129] S17: Returning to FIG. 17, in the control decision device 6, the mobile object control instruction unit 69 transmits the mobile object control instruction information extracted in the process of S233 or S234 to the mobile control device 8 via the communication unit 60.

[0130] S18: The movement control device 8 performs movement control processing.

[0131] The processing of S18 will now be described in detail with reference to Fig. 22. Fig. 22 is a flowchart showing the movement control processing by the movement control device.

[0132] S311: When the mobile object control instruction information receiving unit 83 receives the mobile object control instruction information via the communication unit 80, it calls the mobile object control executing unit 85.

[0133] S312: The mobile object control execution unit 85 reads out the corresponding correction content information from the mobile object control content correction information table 100 based on the correction conditions related to the mobile object information acquired by the mobile object information acquisition unit 81.

[0134] S313: The moving object control execution unit 85 executes the control of the moving object based on the moving object control instruction information and the correction content. Note that S312 does not necessarily have to be executed. In this case, the moving object control execution unit 85 executes the control of the moving object without taking the correction content into consideration.

[0135] This completes the detailed description of the process in S18, and also completes the description of all the processes in FIG.

[0136] [Effects of the embodiment] As described above, according to this embodiment, the control decision device 6 detects deterioration in video quality and remotely issues mobile object control instructions to the mobile object before it becomes difficult for a center operator to visually inspect the image or for remote monitoring using a system such as AI, thereby achieving the effect of improving the safety of automatic movement of the mobile object under remote monitoring.

[0137] Furthermore, if a moving object such as a vehicle suddenly stops when communication is interrupted, it may be dangerous to following objects, or if people are riding in the moving object, it may cause discomfort to passengers or cause them to lose balance, resulting in a dangerous situation. Furthermore, even if there is some disturbance in the video, there may be situations where it does not pose a problem in terms of monitoring (safety), and if the vehicle is stopped immediately in this situation, the efficiency of autonomous driving will decrease. Therefore, the mobile control device 8 of this embodiment can issue remote mobile object control instructions that take safety into consideration by correcting the mobile object control instructions (see S312 and S313) in consideration of the moving state of the mobile object.

[0138] Furthermore, when remotely monitoring video from multiple cameras mounted on a moving object such as a vehicle, if the stopping control of the moving object when camera video is interrupted is always performed uniformly for all camera video, the stopping frequency increases, resulting in a problem of reduced efficiency of autonomous driving. Therefore, the video quality degradation determination unit 66 of this embodiment does not determine the degradation level of video quality depending on the moving state of the moving object (see S213; NO), thereby achieving the effect of preventing a decrease in efficiency of autonomous driving.

[0139] 〔supplement〕 As described above, the present invention is not limited to the above-described embodiment, and various modifications and applications are possible, for example, as shown below.

[0140] (1) Each device can be realized by a computer and a program, but this program can also be recorded on a (non-transitory) recording medium or provided via a communication network such as the Internet.

[0141] (2) The CPU 101 may be multiple, not just single.

[0142] (3) Each of the above tables may be a storage area such as the RAM 103 or a database. [Explanation of symbols]

[0143] 1. Communication Systems 1A Mobile communication system 1B Remote Control System 2. Information gathering device 4. Video quality measurement equipment 6. Control and decision device 8. Movement control device 11. Communications equipment 12, 12a, 12b Video transmission device 13, 13a, 13b Camera 16. Communications equipment 17, 17a, 17b Video receiving device 19 Remote monitoring equipment N Communication Network

Claims

1. A remote control system for remotely controlling movement of a moving object capable of moving automatically based on an image transmitted from the moving object, comprising: a video quality measuring device that measures video quality of video information captured by the moving object and generates video quality information indicating the video quality; a control determination device that acquires the video quality information from the video quality measurement device, determines degradation of video quality related to the video quality information, and issues a mobile object control instruction to the mobile object in response to the degradation; and The control determination device a video quality degradation determination unit that determines a degradation level of the video quality; a mobile object control instruction unit that transmits to the mobile object predetermined mobile object control instruction information indicating the mobile object control instruction that differs according to the deterioration level; a mobile object information acquisition unit that acquires mobile object information including a moving state of the mobile object from the mobile object; and the video quality degradation determination unit does not determine the degradation level of the video quality depending on the moving state. Remote control system.

2. the video quality degradation determination unit compares a most recent degradation level of the video quality with a current degradation level; When the current deterioration level is lower than the latest past deterioration level, the mobile object control instruction unit transmits to the mobile object the predetermined mobile object control instruction information indicating a mobile object control instruction according to the degree of decrease in the deterioration level. The remote control system of claim 1 .

3. the video quality degradation determination unit compares a most recent degradation level of the video quality with a current degradation level; When the current deterioration level is equal to or higher than the most recent past deterioration level, the mobile object control instruction unit transmits, to the mobile object, the predetermined mobile object control instruction information indicating a mobile object control instruction corresponding to the current deterioration level. The remote control system of claim 1 .

4. A remote control system according to any one of claims 1 to 3; a movement control device provided on the moving body and configured to control the movement of the moving body based on a moving body control instruction from the remote control system; A communication system having: The mobile control device corrects the content of the mobile object control instruction according to the moving state of the mobile object, and controls the movement of the mobile object based on the corrected mobile object control instruction.

5. A remote control method executed by a remote control system that remotely controls movement of an automatically movable mobile object based on an image transmitted from the mobile object, comprising: a video quality measurement process for measuring video quality of video information obtained by capturing video from the moving object, and generating video quality information indicating the video quality; a control determination process for determining degradation of video quality related to the video quality information and issuing a mobile object control instruction to the mobile object in response to the degradation; Run The control determination process includes: a video quality degradation determination process for determining a degradation level of the video quality; a mobile object control instruction process for transmitting predetermined mobile object control instruction information to the mobile object, the predetermined mobile object control instruction information indicating the mobile object control instruction that differs depending on the deterioration level; a mobile object information acquisition process for acquiring mobile object information including a moving state of the mobile object from the mobile object; and not executing the determination of the degradation level of the video quality by the video quality degradation determination process according to the moving state. Remote control method.

Images