COMMUNICATION SYSTEM, COMMUNICATION METHOD, PROGRAM, AND CONTROL DEVICE

Abstract

The purpose of the present disclosure is to preserve the quality of data such as image data requested under various conditions.　The present disclosure relates to a communication system constructed from a server and a communication terminal that perform transmission and reception of data, the communication system having: a means for estimating a future location of the communication terminal and the predicted time at which the communication terminal will arrive at the future location on the basis of route information, which indicates a route plan or a past route of the communication terminal, and current location / time information, which indicates the current location of the communication terminal and the current time; a means for managing network quality corresponding to a location; a means for predicting network quality at the future location on the basis of the future location and the network quality corresponding to the location; a means for performing streaming setup in a communication between the communication terminal and the server at the predicted time on the basis of the predicted network quality; and a means for performing streaming control on the basis of the streaming setup.

Description

[Technical Field]

[0001] The present disclosure relates to a communication system, a communication method, and a program. [Background technology]

[0002] Edge or cloud computing is used in a variety of fields, where information from the terminal side, obtained by sensors such as cameras and LiDAR (Light Detection and Ranging), is transmitted via a network to an information processing infrastructure on the edge or cloud side, where it is processed and adapted for users such as people or AI (Artificial Intelligence), and then signals are transmitted downstream from the information processing infrastructure (server) to control the communication terminal side as needed.

[0003] In the fields of smart agriculture and connected cars, use cases are being considered for remote monitoring or control based on information obtained from communication terminals, and progress is being made in practical application of technologies aimed at ensuring the safety of autonomous driving and creating dynamic maps.

[0004] Among sensor information, images have a wide range of uses for people and artificial intelligence (AI). While they are intuitively easy to understand as visual information, they also contain relatively large volumes of data. Therefore, real-time information transmission requires ingenuity and a certain degree of cost. For this reason, related technologies for controlling image streaming are being studied in various fields.

[0005] In remote monitoring or control, the quality of images, data, etc. received by communication terminals fluctuates due to the fluctuations and diversity of the environment, including the network, and the monitoring entity. Therefore, in order to maintain a certain level of quality, a method (combination of existing technologies) has been considered as a conventional technology to appropriately control and change the bit rate depending on the state of the network used for transmission and the entity receiving the image on the information processing infrastructure side (how the image is used) (Non-Patent Documents 1 and 2). [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Kimura et al., Video Bitrate Selection Method for Reducing Traffic Volume While Maintaining QoE, IEICE Technical Report, July 2017 [Non-patent document 2] Wakao et al., Quality Prediction Technology for Optimal Use of Multiple Wireless Access, NTT Technical Journal, April 2020 Summary of the Invention [Problem to be solved by the invention]

[0007] However, with these conventional technologies, when the network state changes discretely, such as when a communication terminal switches the network used for information transmission, a difference arises between the "network state prediction," which assumes continuous state changes assuming continued use of the same network, and the "actual network state" used for information transmission, making it impossible to make an appropriate selection and maintain the desired quality.

[0008] The present invention has been made in view of the above points, and has as its object to maintain the quality of data such as image data required in various situations. [Means for solving the problem]

[0009] In order to achieve the above object, the invention of claim 1 is a communication system constructed by a communication terminal and a server that transmit and receive data, comprising: a future position estimation means that estimates a future position of the communication terminal and a predicted time to arrive at the future position based on route information indicating a route plan or past routes of the communication terminal and current position time information indicating a current position of the communication terminal and a current time; and a network quality management means that manages network quality according to position; a network quality predicting means for predicting a network quality at the future position based on the future position and a network quality corresponding to the position; and a network quality predicting means for predicting a network quality at the future position based on the predicted network quality. from, The server Sending data to and a streaming control means for controlling streaming based on the streaming settings. [Effects of the Invention]

[0010] As described above, the present invention has the effect of being able to maintain the required data quality in various situations. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a conceptual diagram showing an overall configuration according to an embodiment of the present invention. [Figure 2] FIG. 2 is a hardware configuration diagram of a communication terminal according to an embodiment of the present invention. [Figure 3] FIG. 2 is a hardware configuration diagram of a server according to an embodiment of the present invention. [Figure 4] 1 is a functional block diagram showing the functional architecture of a communication system according to an embodiment of the present invention. [Figure 5] FIG. 10 is a conceptual diagram of a user request management DB. [Figure 6] FIG. 2 is a sequence diagram showing main processes in the communication system. DETAILED DESCRIPTION OF THE INVENTION

[0012] [Overall structure] Fig. 1 is a conceptual diagram showing the overall configuration of an embodiment of the present invention. Fig. 1 shows, as an example, a case in which a communication system 1 of this embodiment is used for remote control of an autonomously traveling agricultural machine (hereinafter, agricultural machine will be referred to as "agricultural machine"). The communication system 1 is constructed by a communication terminal 2 and a server 4.

[0013] Furthermore, the communication terminal 2 and the server 4 can communicate via one of multiple networks (local 5G networks 50a, 50b, carrier network 100). The local 5G networks 50a, 50b are dedicated 5G networks (fifth-generation mobile networks) built by companies or local governments that are not telecommunications carriers in certain areas or within buildings and premises. The carrier network 100 is a dedicated 5G network built by a telecommunications carrier (carrier) in a wide area across the country. Note that the three networks shown in FIG. 1 are examples, and other networks may also be used.

[0014] The communication terminal 2 is a small communication computer mounted on the agricultural machine 3.

[0015] Fig. 1 shows fields A and B, which are connected by road Y. Fig. 1 also shows the trajectory of the movement route of agricultural machine 3 equipped with communication terminal 2, showing that agricultural machine 3 moves sequentially as agricultural machines 3a and 3b within field A, then moves on road Y as agricultural machine 3c, and then moves sequentially as agricultural machines 3d and 3e within field B.

[0016] Furthermore, a reference point transceiver 8a for performing RTK-GPS (Real-Time Kinematic GPS) positioning is installed in field A. Similarly, a reference point transceiver 8b for performing RTK-GPS positioning is installed in field B. The communication terminal 2 can identify its current position with precision, with an error of about several centimeters, by bRTK-GPS positioning using a GPS (Global Positioning System) satellite 9 and the reference point transceivers 8a and 8b.

[0017] In this embodiment, in order to respond to the optimal communication quality depending on the location of the agricultural machine 3 (communication terminal 2), the network to be used is switched and streaming control (control of video codec, resolution, frame rate, and bit rate) is performed depending on the location of the agricultural machine 3 (communication terminal 2) and the user's request (desire).

[0018] FIG. 1 shows a case where the network to be used is switched. For example, when the agricultural machine 3 is located at agricultural machine 3a, the communication terminal 2 communicates with the server 4 using the carrier 5G network 100. When the agricultural machine 3 is located at agricultural machine 3b, the communication terminal 2 communicates with the server 4 using the local 5G network 50a. When the agricultural machine 3 is located at agricultural machine 3c, the communication terminal 2 communicates with the server 4 using the carrier 5G network 100. When the agricultural machine 3 is located at agricultural machine 3d, the communication terminal 2 communicates with the server 4 using the local 5G network 50d. When the agricultural machine 3 is located at agricultural machine 3e, the communication terminal 2 communicates with the server 4 using the carrier 5G network 100.

[0019] [Hardware configuration] <Hardware configuration of communication terminal> First, the hardware configuration of the communication terminal 2 will be described with reference to Fig. 2. Fig. 2 is a diagram showing the hardware configuration of the communication terminal.

[0020] 2, the communication terminal 2 includes a processor 201, a memory 202, an auxiliary storage device 203, a communication device 204, and a GPS (Global Positioning System) device 205. The communication terminal 2 also includes an audio input device 206, an audio output device 207, a display device 208, an imaging device 209, a connection device 210, and a short-range wireless communication device 211. The hardware components constituting the communication terminal 2 are connected to each other via a bus 220.

[0021] The processor 201 serves as a control unit that controls the entire communication terminal 2, and includes various arithmetic devices such as a CPU (Central Processing Unit). The processor 201 reads and executes various programs on the memory 202. The processor 201 may include a GPGPU (General-purpose computing on graphics processing units).

[0022] The memory 202 has a main storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The processor 201 and the memory 202 form a so-called computer, and the processor 201 executes various programs read onto the memory 202, causing the computer to realize various functions.

[0023] The auxiliary storage device 203 stores various programs and various information used when the processor 201 executes the various programs.

[0024] The communication device 204 is a communication device for transmitting and receiving various information to and from other devices (servers). The GPS device 205 detects the location information of the communication terminal 2.

[0025] The audio input device 206 detects audio information such as the user's voice and surrounding sounds. The audio output device 207 is a device that outputs various types of information received from other devices (servers) as audio, for example.

[0026] The display device 208 is a device that displays, as an image, various types of information received from other devices (servers), for example.

[0027] The imaging device 209 captures images of the user and the surroundings and generates image information.

[0028] The connection device 210 is a connection device used when connecting various sensors, external memories, etc. to the communication terminal 2.

[0029] The short-range wireless communication device 211 is a wireless device for performing short-range wireless communication with another device (communication terminal) located near the communication terminal 2.

[0030] <Server hardware configuration> Next, the hardware configuration of the server 4 will be described with reference to Fig. 3. Fig. 3 is a diagram showing the hardware configuration of the server.

[0031] 3, the server 4 includes a processor 401, a memory 402, an auxiliary storage device 403, a connection device 404, a communication device 405, and a drive device 406. The hardware components constituting the server 4 are connected to each other via a bus 407.

[0032] The processor 401 serves as a control unit that controls the entire server 4, and includes various computing devices such as a CPU (Central Processing Unit). The processor 401 reads various programs into the memory 402 and executes them. The processor 401 may also include a GPGPU (General-purpose computing on graphics processing units).

[0033] The memory 402 has a main storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The processor 401 and the memory 402 form a so-called computer, and the processor 401 executes various programs read onto the memory 402, causing the computer to realize various functions.

[0034] The auxiliary storage device 403 stores various programs and various information used when the processor 401 executes the various programs.

[0035] The connection device 404 is a connection device that connects an external device (for example, a display device 410, an operation device 411) and the server 4.

[0036] The communication device 405 is a communication device for transmitting and receiving various types of information to and from other devices.

[0037] The drive device 406 is a device for loading a recording medium 430. The recording medium 430 here includes media that record information optically, electrically, or magnetically, such as a CD-ROM (Compact Disc Read-Only Memory), a flexible disk, a magneto-optical disk, etc. The recording medium 430 may also include semiconductor memory that records information electrically, such as a ROM (Read Only Memory), a flash memory, etc.

[0038] The various programs to be installed in the auxiliary storage device 403 are installed, for example, by setting the distributed recording medium 430 in the drive device 406 and reading the various programs recorded on the recording medium 430 by the drive device 406. Alternatively, the various programs to be installed in the auxiliary storage device 403 may be installed by being downloaded from a network via the communication device 405.

[0039] [Functional configuration of communication system] Next, the functional configuration of the communication system will be described with reference to Fig. 4. Fig. 4 is a functional block diagram showing the functional architecture of the communication system in an embodiment of the present invention. In Fig. 4, the communication terminal 2 and the server 4 each have various units (functions), but this is just an example, and the communication terminal 2 and the server 4 may have any units (functions).

[0040] 4, the communication terminal 2 has a location and time information acquisition unit 11, a location and time information distribution unit 12, a network quality measurement unit 24, a streaming control unit 26, a network switching implementation unit 27, and a streaming unit 28. Each of these units is a function that the processor 201 causes the communication terminal 2 to realize using one or more programs installed in the communication terminal 2.

[0041] 4, the server 4 includes a terminal location transition management unit 10, a future location estimation unit 13, a network quality management unit 20, a network quality prediction unit 21, a network switching determination unit 22, a streaming setting unit 23, a network switching notification unit 25, and a streaming control result management unit 30. Each of these units is a function that the processor 401 causes the server 4 to realize using one or more programs installed in the server 4. Furthermore, a user request management DB (Data Base) 40 is built in the auxiliary storage device 403 of the server 4. The user request management DB 40 may also be built in the auxiliary storage device 203 of the communication terminal 2.

[0042] <User request management DB> Next, the user request management DB 40 will be described with reference to Fig. 5. Fig. 5 is a conceptual diagram of the user request management DB. The user request management DB 40 is configured by a user request management table as shown in Fig. 5, and manages user requests (priority details) that have been agreed upon in advance between the user of the communication terminal 2 and the operator of the server 4.

[0043] As shown in Figure 5, the user request management table manages user IDs, preferred networks, fixed video codecs, minimum resolutions (dpi), minimum frame rates (fps), and minimum bit rates (bps) in association with each other.

[0044] Of these, the user ID is an example of user identification information for identifying the user of the communication terminal 2.

[0045] The network in use is information indicating the type of network that the user prefers to use. For example, in FIG. 1, when agricultural machine 3 is located in field A, both local 5G network 51a and carrier network 100 are available. Considering network quality alone, it may be better to use carrier 5G network 100 when agricultural machine 3 moves to the location of agricultural machine 3a, and to use carrier 5G network 50a when agricultural machine 3 moves to the location of agricultural machine 3b. However, some users may prefer to use the local network even if it temporarily degrades network quality due to reasons such as a business relationship between the user and the local local network operator or relatively low communication fees. The "Preferred Network" column is set in this way to meet the requirements (demands) of each user.

[0046] The preferred network may be information indicating a specific network rather than a type of network. For example, in FIG. 1, even though they are the same local 5G network, the local 5G network 50a and the local 5G network 50b are managed as separate networks, and management is possible in which the user preferentially uses the local 5G network 50a but preferentially does not use the local 5G network 50b.

[0047] The fixed video codec is information indicating the fixed video codec used in communication between the communication terminal 2 and the server 4. For example, it is set to meet the request of a user who wants to use only H.264. Note that if the setting is "none," the user is not particular about the video codec, and the video codec is subject to automatic control of network quality.

[0048] The minimum resolution is information indicating the minimum resolution of video data used in communication between the communication terminal 2 and the server 4. For example, it is set to meet a user's request for a resolution of FHD (Full High Definition) or higher.

[0049] The minimum frame rate is information indicating the minimum frame rate of video data used in communication between the communication terminal 2 and the server 4. For example, it is set to meet a user's request for a frame rate of 15 fps or higher.

[0050] The minimum bit rate is information indicating the minimum bit rate of video data used in communication between the communication terminal 2 and the server 4. For example, it is set to meet a user's request for a bit rate of 100 kbps or more.

[0051] <Functional configuration of the communication system> Next, each functional unit will be explained by returning to Fig. 4. Note that hereinafter, "output" and "input" can be expressed as "transmission" and "reception", respectively, when they are via a network.

[0052] The terminal position transition management unit 10 manages route information indicating the route plan of the communication terminal 2 (agricultural machine 3) and the past routes of the communication terminal 2 (agricultural machine 3). For example, in Fig. 1, this route information is information regarding a future route plan in which the agricultural machine 3 moves in the order of agricultural machines 3a, 3b, 3c, 3d, and 3e, or a past route (for example, last year).

[0053] The position time information acquisition unit 11 uses the GPS 9 and the reference point transmitters and receivers 8a and 8b, and acquires position time information indicating the current position and current time of the communication terminal 2 (agricultural machine 3) by the GPS device 205.

[0054] The position and time information distribution unit 12 outputs the current position and time information input from the position and time information acquisition unit 11 to the future position estimation unit 13 .

[0055] The future position estimation unit 13 estimates the future position of the communication terminal 2 (agricultural machine 3) based on the route information input from the terminal position transition management unit 10 and the current position time information input from the position time information distribution unit 12, and also estimates the predicted time of arrival at this future position. For example, in FIG. 1, if the agricultural machine 3 is located at agricultural machine 3a as of 9:00, the future position estimation unit 13 estimates that it will move to the position of agricultural machine 3b 30 minutes later at 9:30. Note that the future position may be expressed not only as an absolute position on Earth, but also in a format that includes the transition range of movement, the possibility of movement, etc. The future position estimation unit 13 outputs estimation result information (future movement position, predicted time) indicating the estimation result to the network quality prediction unit 21.

[0056] The network quality management unit 20 outputs network quality heat map information to the network quality prediction unit 21 based on a request for network quality heat map information from the network quality prediction unit 21. The network quality heat map information is information that indicates, by using shades of gray, network quality that differs depending on the location. This network quality heat map information indicates the network quality that has been actually measured in advance at the site, such as field A.

[0057] The network quality prediction unit 21 receives the estimation result information output by the future position estimation unit 13 and the network quality heat map information from the network quality management unit 20, and thereby predicts the network quality at the future position based on the future position and the network quality corresponding to the above-mentioned position. For example, in FIG. 1, even if the agricultural machine 3 (communication terminal 2) is located at agricultural machine 3a, the network quality prediction unit 21 can predict the network quality when the agricultural machine 3 is located at agricultural machine 3b at the predicted time. Then, the network quality prediction unit 21 outputs network quality prediction information indicating the predicted network quality to the network switching determination unit 22 and the streaming setting unit 23. The network quality prediction information is information indicating the predicted communication quality of all networks that the communication terminal 2 may use for data transmission.

[0058] The network switching determination unit 22 determines the specific network to switch to and the time to switch based on the network quality prediction information from the network quality prediction unit 21. In this case, the network switching determination unit 22 further takes into consideration the user request management DB 40 to ultimately determine the specific network to switch to. For example, in FIG. 5, if the preferred network is set to "local", the network switching determination unit 22 determines that the local 5G network 50a (50b) will be used preferentially, even if it determines that the carrier 5G network 100 would have better network quality based on the network quality prediction information.

[0059] Then, the network switching determination unit 22 outputs network switching determination information indicating the final switching determination result to the streaming setting unit 23 and the network switching notification unit 25. Note that the network switching determination information output to the network switching notification unit 25 may include, in addition to the content indicating the specific network to be switched to, output timing information indicating the timing at which the network switching notification unit 25 outputs a network switching instruction to the network switching implementation unit 27 t1 seconds after receiving the network switching determination information from the network switching determination unit 22 (after waiting t1 seconds) (second method). Note that t1 is 5 seconds.

[0060] The streaming setting unit 23 performs streaming settings (bit rate estimate, control timing, etc.) for communication between the communication terminal 2 and the server 4 at the predicted time based on the network quality prediction information (predicted network quality) from the network quality prediction unit 21, the network switching instruction from the network switching determination unit 22, the network quality measurement information from the network quality measurement unit 24, and the streaming control result feedback information from the streaming control result management unit 30.

[0061] Furthermore, the streaming setting unit 23 performs final streaming setting in consideration of the user instruction management DB 40. For example, in Fig. 5, if the minimum bit rate is set to "10 Mbps," even if the streaming setting unit 23 determines to set the bit rate to 1 Mbps based on the network quality measurement information, etc., it cannot lower the bit rate below 10 Mbps, so it changes other streaming parameters (video codec, resolution, frame rate) within the range set by the user.

[0062] Furthermore, in this embodiment, the streaming setting unit 23 not only inputs network quality measurement information from the network quality measurement unit 24, but also inputs network judgment information from the network switching judgment unit 22, thereby interlocking network switching control and streaming control. For example, in FIG. 1, when the agricultural machine 3 (communication terminal 2) moves from the position of agricultural machine 3a to the position of agricultural machine 3b, the network to be used is switched from the local 5G network 50a to the carrier 5G network 100, and streaming (bit rate, etc.) control is performed at the timing of this switch. However, interlocking is not necessarily required. In FIG. 4, the functional units (21, 23, 25, 26, 40) surrounded by dashed lines are parts involved in interlocking streaming setting and network switching.

[0063] Then, the streaming setting unit 23 outputs streaming setting information indicating the streaming setting contents to the streaming control unit 26. In this case, the streaming setting information includes the contents of the streaming setting parameters (video codec, resolution, frame rate, and bit rate) (first method). Note that, in addition to the streaming setting parameters, the streaming setting information may also include output timing information indicating the timing at which the streaming control unit 26 outputs a streaming setting signal to the streaming unit 28 t2 seconds after (waiting t2 seconds) after receiving the streaming setting information from the streaming setting unit 23 (second method). Note that the above-mentioned t1 and t2 may be the same time.

[0064] The network quality measurement unit 24 constantly measures the communication quality of each network.

[0065] The network switching notification unit 25 outputs a switching instruction to switch to the switching destination determined by the network switching determination unit 22 to the network switching execution unit 27 that executes the network switching.

[0066] Based on the streaming setting information from the streaming setting unit 23, the streaming control unit 26 outputs streaming setting signals (control signals for the video codec, resolution, frame rate, and bit rate) to the streaming unit 28.

[0067] The network switching execution unit 27 executes network switching based on the network switching instruction output from the network switching notification unit 25 .

[0068] The streaming unit 28 executes streaming setting (switching) based on the streaming setting signals (control signals for the video codec, resolution, frame rate, and bit rate) output from the streaming control unit 26.

[0069] The streaming control result management unit 30 manages the streaming control result, and feeds back the streaming control result to the streaming setting unit 23 .

[0070] [Processing or Operation of the Embodiment] Next, main processes in the communication system 1 will be described with reference to Fig. 6. Fig. 6 is a sequence diagram showing main processes in the communication system. Note that, before the processes shown in Fig. 6 are performed, the network quality predictor 21 receives the estimation result information output by the future position estimator 13.

[0071] First, the network quality prediction unit 21 outputs network quality prediction information to the network switching determination unit 22 (S11). Similarly, the network quality prediction unit 21 outputs the network quality prediction information to the streaming setting unit 23 (S12). As a result, the network switching determination unit 22 and the streaming setting unit 23 each receive the network quality prediction information.

[0072] Next, the network quality measurement unit 24 outputs the network quality measurement information to the streaming setting unit 23 (S13). As a result, the streaming setting unit 23 receives the network quality measurement information as an input.

[0073] Next, the network switching determination unit 22 outputs network switching determination information to the network switching notification unit 25 based on the network quality prediction information and the user request management information managed in the user request management DB 40 (S14). Similarly, the network switching determination unit 22 outputs network switching determination information to the streaming setting unit 23 based on the network quality prediction information and the user request management information managed in the user request management DB (S15).

[0074] Next, the streaming setting unit 23 outputs streaming setting information (bit rate, etc.) to the streaming control unit 26 (S16). As a result, the streaming control unit 26 inputs the streaming setting information.

[0075] Furthermore, the network switching notifying unit 25 outputs a network switching instruction to the network switching implementing unit 27 (S17). As a result, the network switching implementing unit 27 inputs the network switching instruction.

[0076] Meanwhile, the network switching notifier 25 outputs a network switching instruction to the network switching implementer 27 (S17). As a result, the network switching implementer 27 receives the network switching instruction and implements network switching based on the network switching instruction. For example, in FIG. 1, when the agricultural machine 3 (communication terminal 2) moves from the position of agricultural machine 3a to the position of agricultural machine 3b, the network switching implementer 27 switches from the carrier 5G network 100 to the local 5G network 50a.

[0077] Furthermore, the streaming control unit 26 outputs a streaming setting signal (bit rate control signal, etc.) to the streaming unit 28 (S18). As a result, the streaming unit 28 receives the streaming setting signal and changes the streaming setting. For example, when the agricultural machine 3 (communication terminal 2) moves from the position of the agricultural machine 3a to the position of the agricultural machine 3b, the streaming unit 28 switches the bit rate of the image data from 10 Mbps to 1 Mbps.

[0078] [Major Effects of the Embodiments] As described above, by having the streaming setting unit 23 know the timing of network switching in advance, optimal control can be continued through control (bit rate control, etc.) based on the predicted quality of the network to which the switching will be made. Furthermore, even if the streaming setting unit 23 cannot know the timing of network switching, if the predicted qualities of all networks that may be used for transmission are known, communication control (bit rate control, etc.) can be performed according to the worst value that takes into account the predicted quality of the network currently in use as well as the predicted quality of candidate networks to which the switching will be made, thereby realizing continuous real-time reception of images of a certain level of quality or higher, except in cases where the network quality is absolutely insufficient.

[0079] Furthermore, as described above, in addition to always considering the worst value, by detecting signs of network switching from the absolute value of the predicted quality of the known network or its trend, and then controlling communication (control of bit rate, etc.) according to the detected signs, taking into account not only the predicted quality of the network currently in use but also the predicted quality of candidate networks to switch to, it becomes possible to achieve more appropriate control than when always considering the worst value.

[0080] There may be a plurality of information sources (for example, the network quality prediction unit 21 and the network switching determination unit 22) and a plurality of streaming setting units 23. In this case, even if there are two types of streaming setting units 23, one for visual observation by a user (person) and the other for AI learning and image recognition, as control targets that should be linked with network switching, it is possible to extract the control targets that need to be linked and then control streaming simultaneously or in linkage based on the proposed method.

[0081] <Control that takes into account the time required for network switching> If the time from the network switching instruction to the actual switching to a specific network is t (e.g., t = 5 seconds), then the streaming settings (bit rate, etc.) must also be selected and controlled taking time t into consideration. In this case, if the streaming settings (bit rate, etc.) are selected and controlled on the assumption that the network switching is completed when the network switching instruction is output, the control (bit rate, etc.) will be advanced by the time t, which may result in a discrepancy between the set bit rate, etc. and the bit rate, etc. that should be set. Therefore, the streaming setting unit 23 measures and grasps in advance the time t from the network switching determination information to the actual switching of the network, and delays the application of the selected settings (bit rate, etc.) by the time t when a network switch is involved. This makes it possible to avoid the time t advance in the control of the bit rate, etc., and therefore prevents a discrepancy between the settings (the bit rate, etc. that is set and the bit rate, etc. that should be set).

[0082] <Control using feedback loops> The streaming control result management unit 30 applies feedback control to the streaming setting unit 23 based on the network quality prediction results, network quality measurement results, streaming settings (bit rate estimates, etc.), control timing, and streaming (bit rate, etc.) control results (such as the subjective quality of actual users (people) and the AI ​​image recognition rate), thereby making it possible to improve the accuracy of streaming (bit rate) control.

[0083] 〔supplement〕 The present invention is not limited to the above-described embodiment, and may have the following configurations or processes (operations).

[0084] The communication terminal 2 and the server 4 can be realized by a computer and a program, but this program can also be recorded on a recording medium or provided via a network such as the Internet.

[0085] In the above embodiment, the communication target is described as video data, but the communication target is not limited to this. For example, the communication target may be data of the five senses (touch data, smell data, etc.), sound data (uncompressed live sound source data, music data, etc.), game data, data related to rendering for xR (x Reality), etc.

[0086] Furthermore, in the above embodiment, the mobile object is described as the movement of the agricultural machine 3, but the present invention is not limited to this. For example, the mobile object may be an airplane, a ship, an automobile, a bicycle, or the like.

[0087] Furthermore, in the above embodiment, the network switching determination unit 22 determines the destination of the network used by the communication terminal 2 and the server 4 based on the network quality predicted by the network quality prediction unit 21, but this process is merely an example. Furthermore, the network switching determination unit 22 is merely an example of a network control determination unit. For example, the network control determination unit may determine whether to increase or decrease the network bandwidth or change the priority of network priority control (see FIG. 5). In this case, a network control notification unit, an example of which is the network switching notification unit 25, outputs an instruction for the control determined to be performed by the network control determination unit 22 to a network control execution unit, an example of which is the network switching execution unit 27. [Explanation of symbols]

[0088] 1. Communication Systems 2. Communication terminals 4 Server 10 Terminal location shift management unit (an example of terminal location shift management means) 11 Position and time information acquisition unit (an example of a position and time information acquisition means) 12 Location and time information distribution unit (an example of location and time information distribution means) 13 Future position estimation unit (an example of a future position estimation means) 20 Network Quality Control Unit (an example of a network quality control method) 21 Network quality prediction unit (an example of a network quality prediction means) 22 Network switching determination unit (an example of network switching determination means, an example of network control determination means) 23 Streaming setting unit (an example of streaming setting means) 24 Network quality measurement unit (an example of a network quality measurement means) 25 Network switching notification unit (an example of a network switching notification means, an example of a network control notification means) 26 Streaming control unit (an example of streaming control means) 27 Network switching execution unit (an example of a network switching execution means, an example of a network control execution means) 28 Streaming unit (an example of streaming means) 30 Streaming control result management unit 40 User request management DB (an example of a user request management means)

Claims

1. A communication system constructed by a communication terminal and a server that transmit and receive data, a future position estimation means for estimating a future position of the communication terminal and a predicted time at which the communication terminal will arrive at the future position, based on route information indicating a route plan or a past route of the communication terminal, and current position and time information indicating a current position and current time of the communication terminal; a network quality management means for managing network quality according to location; a network quality prediction means for predicting a network quality at the future location based on the future location and a network quality corresponding to the location; a streaming setting means for performing streaming setting for data transmission from the communication terminal to the server at the predicted time based on the predicted network quality; a streaming control means for controlling streaming based on the streaming setting; A communication system having:

2. The communication system according to claim 1 , wherein the parameters of the streaming settings are a codec, a resolution, a frame rate, or a bit rate of the data.

3. 3. A communication system according to claim 1 or 2, the communication terminal and the server transmit and receive data via one of a plurality of networks; a network control determination means for determining whether to control the network used by the communication terminal and the server based on the network quality predicted by the network quality prediction means; a network control notification means for outputting an instruction for the control determined by the network control determination means to a network control implementation means for implementing the control of the network; A communication system having:

4. The determination by the network control determination means as to whether to control the network is to be determined as a destination of the network switching to one of the plurality of networks, The control instruction by the network control notification means is to output a switching instruction to the network control implementation means to switch to the switching destination determined by the network control determination means. The communication system according to claim 3 .

5. 4. The communication system according to claim 3, wherein the streaming setting means performs the streaming setting in conjunction with the control of the network when the network control determining means determines that the network control should be performed.

6. 6. The communication system according to claim 5, a user request management means for managing requests from users of the communication terminals regarding parameters of the streaming control; The streaming setting means performs the streaming setting by giving priority to the user's request.

7. the user request management means manages requests of the user of the communication terminal regarding the specific network to which the network is switched; 7. The communication system according to claim 6, wherein said network control determining means determines the destination of said network by giving priority to a request from said user regarding said specific network.

8. 2. The communication system according to claim 1, further comprising a network quality measuring means for measuring the network quality, wherein the streaming setting means performs streaming setting based on information indicating the network quality received from the network quality measuring means.

9. 2. The communication system according to claim 1, further comprising a streaming control result management means for managing a streaming control result, wherein said streaming setting means performs streaming setting based on the streaming control result fed back from said streaming control result management means.

10. A communication method executed by a communication system constructed by a communication terminal and a server that transmit and receive data, the communication system includes a network quality management means for managing network quality according to location, The communication system includes: estimating a future position of the communication terminal and a predicted time at which the communication terminal will arrive at the future position based on route information indicating a route plan or a past route of the communication terminal, and current position and time information indicating a current position and current time of the communication terminal; predicting a network quality at the future location based on the future location and a network quality according to the location; performing streaming settings for data transmission from the communication terminal to the server at the predicted time based on the predicted network quality; Streaming control is performed based on the streaming settings. Communication method.

11. A program causing a computer to execute the method according to claim 10.

12. a future position estimation means for estimating a future position of the communication terminal and a predicted time at which the communication terminal will arrive at the future position based on route information indicating a route plan or past routes of the communication terminal that transmits and receives data to and from a server, and current position and time information indicating a current position and current time of the communication terminal; a network quality management means for managing network quality according to location; a network quality prediction means for predicting a network quality at the future location based on the future location and a network quality corresponding to the location; a streaming setting means for performing streaming setting for the communication terminal in data transmission from the communication terminal to the server at the predicted time based on the predicted network quality; A control device having:

13. The control device according to claim 12 , wherein the parameters of the streaming settings are a codec, a resolution, a frame rate, or a bit rate of the data.

14. the communication terminal and the server transmit and receive data via one of a plurality of networks, and a network control determination means determines whether to switch the network used by the communication terminal and the server based on the network quality predicted by the network quality prediction means; 14. The control device according to claim 12 or 13, comprising:

15. The control device according to claim 14, wherein the streaming setting means performs the streaming setting in conjunction with the control of the network when the network control determining means determines that the network control should be performed.

16. a user request management means for managing requests from the user of the communication terminal regarding parameters of streaming control performed based on the streaming setting; The control device according to claim 14, wherein the streaming setting means performs the streaming setting by giving priority to the request of the user.

17. the user request management means manages requests of the user of the communication terminal regarding the specific network to which the network is switched; The control device according to claim 16, wherein the network control determining means determines the destination of the network to be switched to by giving priority to the user's request regarding the specific network.

18. 13. The control device according to claim 12, further comprising a streaming control result management means for managing a streaming control result, wherein the streaming setting means performs streaming setting based on the streaming control result fed back from the streaming control result management means.

19. The communication terminal and the server transmit and receive data via one of a plurality of networks, a future position estimation means for estimating a future position of the communication terminal and a predicted time at which the communication terminal will arrive at the future position based on route information indicating a route plan or a past route of the communication terminal that transmits and receives data to and from the server, and current position and time information indicating a current position and current time of the communication terminal; a network quality management means for managing network quality according to location; a network quality prediction means for predicting a network quality at the future location based on the future location and a network quality corresponding to the location; a network control determination means for determining whether to switch the network used by the communication terminal and the server based on the predicted network quality; a user request management means for managing requests from the user of the communication terminal regarding a specific network to which the network is switched; and The network control determination means determines the destination of the network to be switched to, giving priority to the user's request regarding the specific network.

20. A communication method executed by a communication terminal that transmits and receives data to and from a server and a control device that transmits and receives data, the communication terminal and the server transmit and receive data via one of a plurality of networks; the control device has a network quality management means for managing network quality according to a location, The control device estimating a future position of the communication terminal and a predicted time at which the communication terminal will arrive at the future position based on route information indicating a route plan or a past route of the communication terminal, and current position and time information indicating a current position and current time of the communication terminal; predicting a network quality at the future location based on the future location and a network quality according to the location; A communication method for performing streaming settings for communication between the communication terminal and the server at the predicted time based on the predicted network quality.

21. A communication method executed by a communication terminal that transmits and receives data to and from a server and a control device that transmits and receives data, the communication terminal and the server transmit and receive data via one of a plurality of networks; the control device has a network quality management means for managing network quality according to a location, and a user request management means for managing a request from a user of the communication terminal regarding a specific network to which the network is switched; The control device estimating a future position of the communication terminal and a predicted time at which the communication terminal will arrive at the future position based on route information indicating a route plan or a past route of the communication terminal, and current position and time information indicating a current position and current time of the communication terminal; predicting a network quality at the future location based on the future location and a network quality according to the location; determining whether to switch the network used by the communication terminal and the server based on the predicted network quality; determining a destination of the network to be switched to by giving priority to the user's request regarding the specific network; Communication method.

Images