COMMUNICATION SYSTEM, CLIENT TERMINAL, COMMUNICATION METHOD, AND COMMUNICATION PROGRAM

Abstract

To improve the quality of moving image data and audio data stored in a communication system that transmits and receives the moving image data and audio data.SOLUTION: A communication system includes a client terminal and a server device. A control unit of the client terminal transmits image data or audio data acquired by the client terminal to the server device in real time, and stores the image data or audio data acquired by the client terminal in a first storage unit as first image data or first audio data. A control unit of the server device identifies a range of the first image data or first audio data stored in the first storage unit that the server device needs to acquire, and requests the client terminal to transmit the first image data or first audio data in the identified range.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a communication system, a client terminal, a communication method, and a communication program. [Background technology]

[0002] There is known a two-way communication system that enables remote communication by bidirectionally transmitting and receiving video data and audio data. In the two-way communication system, for example, by using an archive function, the video data and audio data during communication can be stored in a server device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-107978 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-described communication system, for example, if the communication quality during communication deteriorates, the quality of the video data and audio data stored in the server device also deteriorates.

[0005] The present disclosure improves the quality of video and audio data stored in a communication system that transmits and receives video and audio data. [Means for solving the problem]

[0006] A first aspect of the present disclosure is A communication system having a client terminal and a server device, The control unit of the client terminal transmitting the image data or audio data acquired by the client terminal to the server device in real time; storing the image data or audio data acquired by the client terminal in a first storage unit as first image data or first audio data; The control unit of the server device Identifying a range of the first image data or the first audio data stored in the first storage unit that needs to be acquired by the server device; A request is made to the client terminal to transmit the first image data or first audio data in the specified range.

[0007] According to the first aspect of the present disclosure, it is possible to improve the quality of moving image data and audio data stored in a communication system that transmits and receives moving image data and audio data.

[0008] A second aspect of the present disclosure is a communication system according to the first aspect, The control unit of the server device specifies a time range that the server device needs to acquire.

[0009] A third aspect of the present disclosure is a communication system according to the second aspect, The control unit of the server device storing the image data or audio data transmitted in real time in a second storage unit as second image data or second audio data; The first image data or first audio data of the identified time range transmitted from the client terminal is used to replace the corresponding time range of the second image data or second audio data.

[0010] A fourth aspect of the present disclosure is the communication system according to the third aspect, The control unit of the server device performing at least one of a quality inspection and an attribute inspection on the image data or voice data transmitted in real time; Based on the result of the check, a time range that needs to be acquired by the server device is identified.

[0011] A fifth aspect of the present disclosure is a communication system according to the fourth aspect, The control unit of the server device The quality inspection is performed based on one or more of information regarding the communication quality when the image data or audio data is transmitted in real time, or information regarding the data quality of the image data or audio data transmitted in real time.

[0012] A sixth aspect of the present disclosure is a communication system according to the fourth aspect, The control unit of the server device Based on the result of the quality inspection, a time range that needs to be acquired by the server device is identified, and based on the result of the attribute inspection, the identified time range is narrowed down.

[0013] A seventh aspect of the present disclosure is a communication system according to the fourth aspect, The control unit of the server device The attribute inspection is performed based on information regarding changes between frames of image data that meets a predetermined condition, among the image data transmitted in real time.

[0014] An eighth aspect of the present disclosure is a communication system according to any one of the fourth to seventh aspects, The control unit of the server device A judgment parameter to be used when performing one or more of the quality inspection and the attribute inspection is acquired from the client terminal.

[0015] A ninth aspect of the present disclosure is the communication system according to the first aspect, A time synchronization process is performed between the control unit of the client terminal and the control unit of the server device at predetermined intervals.

[0016] A tenth aspect of the present disclosure is the communication system according to the first aspect, The first storage unit When the image data or audio data acquired by the client terminal is stored as the first image data or first audio data, a timestamp is stored.

[0017] An eleventh aspect of the present disclosure is a communication system according to the third aspect, The control unit of the client terminal The first image data or the first audio data in the specified time range is transmitted to the server device.

[0018] A twelfth aspect of the present disclosure is a communication system according to the eleventh aspect, The control unit of the client terminal performing at least one of a quality inspection and an attribute inspection on the image data or voice data transmitted in real time; Based on the result of the inspection, it is determined whether or not the first image data or first audio data in the specified time range is to be transmitted to the server device.

[0019] A thirteenth aspect of the present disclosure is a communication system according to the twelfth aspect, The control unit of the client terminal An attribute check is performed on the first image data or first audio data stored in the first storage unit.

[0020] A fourteenth aspect of the present disclosure is a communication system according to the twelfth aspect, The control unit of the client terminal Using sensor information from a sensor attached to the client terminal, one or more of the quality inspection and attribute inspection are performed.

[0021] A fifteenth aspect of the present disclosure is a communication system according to the twelfth aspect, The control unit of the client terminal Even if transmission of the specified time range is requested, if it is determined not to transmit based on the results of the inspection, the first image data or first audio data for the specified time range will not be transmitted to the server device.

[0022] A sixteenth aspect of the present disclosure is a communication system according to the twelfth aspect, The control unit of the client terminal When a request for transmission of the specified time range is made and a determination is made as to whether or not to transmit the request based on the result of the inspection, a determination result is output; If a transmission instruction is input in response to outputting the determination result, the first image data or first audio data in the specified time range is transmitted to the server device.

[0023] Furthermore, a seventeenth aspect of the present disclosure is A client terminal having a control unit and communicating with a server device, The control unit transmitting the image data or audio data acquired by the client terminal to the server device in real time; storing the image data or audio data acquired by the client terminal in a storage unit; performing at least one of a quality inspection and an attribute inspection on the image data or voice data transmitted in real time; Based on the result of the inspection, the image data or audio data stored in the storage unit is transmitted to the server device.

[0024] An eighteenth aspect of the present disclosure is a client terminal according to the seventeenth aspect, The control unit When the image data or audio data is stored in the storage unit, a time stamp is stored.

[0025] A nineteenth aspect of the present disclosure is a client terminal according to the seventeenth aspect, Based on the result of the inspection, it is determined whether or not the image data or audio data stored in the storage unit should be transmitted to the server device.

[0026] A twentieth aspect of the present disclosure is a client terminal according to the seventeenth aspect, The control unit An attribute check is performed on the image data or audio data stored in the storage unit.

[0027] A twenty-first aspect of the present disclosure is a client terminal according to the seventeenth aspect, The control unit Using sensor information from a sensor attached to the client terminal, one or more of the quality inspection and attribute inspection are performed.

[0028] A twenty-second aspect of the present disclosure is a client terminal according to the nineteenth aspect, The control unit If it is determined based on the result of the inspection that the image data or audio data should not be transmitted to the server device, the image data or audio data is not transmitted to the server device.

[0029] A twenty-third aspect of the present disclosure is a client terminal according to the nineteenth aspect, The control unit When it is determined whether or not to transmit based on the result of the inspection, the determination result is output; If a transmission instruction is input in response to outputting the determination result, the image data or audio data is transmitted to the server device.

[0030] Furthermore, a 24th aspect of the present disclosure is A communication method in a communication system having a client terminal and a server device, a communication step of transmitting the image data or audio data acquired by the client terminal to the server device in real time; a storage step of storing the image data or audio data acquired by the client terminal; a specifying step of specifying a range of the image data or audio data stored in the storing step that needs to be acquired by the server device; and a request step of requesting the client terminal to transmit the image data or audio data in the range specified in the specifying step.

[0031] Furthermore, a 25th aspect of the present disclosure is A communication method in a client terminal that communicates with a server device, comprising: a first communication step of transmitting image data or audio data acquired by the client terminal to the server device in real time; a storage step of storing the image data or audio data acquired by the client terminal; an inspection step of performing at least one of a quality inspection and an attribute inspection on the image data or voice data transmitted in the first communication step; and a second communication step of transmitting the image data or audio data stored in the storage step to the server device based on the result of the inspection in the inspection step.

[0032] A 26th aspect of the present disclosure is a communication program, A control unit of a client terminal that communicates with the server device, a first communication step of transmitting image data or audio data acquired by the client terminal to the server device in real time; a storage step of storing the image data or audio data acquired by the client terminal; an inspection step of performing at least one of a quality inspection and an attribute inspection on the image data or voice data transmitted in the first communication step; A second communication step is executed in which the image data or audio data stored in the storage step is transmitted to the server device based on the result of the inspection in the inspection step. [Brief explanation of the drawings]

[0033] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration of a two-way communication system. [Figure 2A] FIG. 2 illustrates an example of a hardware configuration of a server device. [Figure 2B] FIG. 2 is a diagram illustrating an example of a hardware configuration of a worker terminal and a supporter terminal. [Figure 3] FIG. 2 illustrates an example of a functional configuration of a two-way communication unit of the server device. [Figure 4] FIG. 2 illustrates an example of a functional configuration of a data management unit of the server device. [Figure 5] 10A and 10B are diagrams illustrating a list of complement range candidates and non-complement ranges identified by each functional unit of the data management unit. [Figure 6] FIG. 2 illustrates an example of a functional configuration of a worker terminal. [Figure 7] 10 is a diagram showing a list of complement range candidates and non-complement ranges identified by each functional unit of the worker terminal; FIG. [Figure 8] FIG. 10 is a sequence diagram showing a processing flow in a two-way communication system. [Figure 9] 10 is a flowchart showing the flow of a complement range correction process. [Figure 10] 10 is a flowchart showing the flow of a completion request process. DETAILED DESCRIPTION OF THE INVENTION

[0034] Hereinafter, each embodiment will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configuration are designated by the same reference numerals, and redundant description will be omitted.

[0035] [First embodiment] <System configuration of two-way communication system> First, the system configuration of the two-way communication system according to the first embodiment will be described. Fig. 1 is a diagram showing an example of the system configuration of the two-way communication system. As shown in Fig. 1, the two-way communication system 100 includes a server device 110 and client terminals (a worker terminal 120 and a supporter terminal 130).

[0036] In this embodiment, A worker 160 working at a work site; A supporter 170 who supports the worker 160 by remotely giving various work instructions to the worker 160; The following describes a case where communication is performed using the two-way communication system 100. For this reason, as described above, the client terminals constituting the two-way communication system 100 in this embodiment include the worker terminal 120 used by the worker 160 and the supporter terminal 130 used by the supporter 170. However, the client terminals constituting the two-way communication system 100 are not limited to the combination of the worker terminal 120 and the supporter terminal 130.

[0037] The server device 110 is a device that provides a cloud service (two-way communication service) via the network 140. Specifically, a two-way communication program is installed in the server device 110, and by executing the program, the server device 110 functions as a two-way communication unit 111 and a data management unit 112. In this way, the server device 110 provides the two-way communication service to the worker 160 and the supporter 170.

[0038] In the server device 110, the two-way communication unit 111 is connected to the worker terminal 120 and the supporter terminal 130 for communication, and transmits video data and audio data received from the worker terminal 120 to the supporter terminal 130. The two-way communication unit 111 also transmits audio data received from the supporter terminal 130 to the worker terminal 120.

[0039] In the server device 110, the data management unit 112 stores, in real time, moving image data transmitted unidirectionally between the worker terminal 120 and the supporter terminal 130 and audio data transmitted bidirectionally.

[0040] Furthermore, in the server device 110, the data management unit 112 identifies a range of stored video data and audio data where quality has deteriorated (a range that the server device 110 needs to acquire). The data management unit 112 also requests video data and audio data in the identified range, thereby receiving the video data and audio data in the identified range from the worker terminal 120 or the supporter terminal 130, and replacing the video data and audio data already stored therewith.

[0041] As a result, according to the server device 110 of the first embodiment, it is possible to improve the quality of the video data and audio data in a specified range among the video data and audio data stored in the server device 110 in the two-way communication system 100.

[0042] The worker terminal 120 is, for example, a wearable terminal worn by a worker 160 at a work site. In this embodiment, it is assumed that the work site is an environment where the communication quality of two-way communication is likely to deteriorate.

[0043] The worker terminal 120 has a function to capture video of the work site, a function to detect audio (e.g., the speech of the worker 160), and a function to store these. The worker terminal 120 also has a function to transmit video data of the work site captured and audio data of the speech of the worker 160 to the supporter 170 via the network 140. This allows the supporter 170 to obtain data necessary to instruct the worker 160.

[0044] Furthermore, the worker terminal 120 has a function to output instructions (audio data) from the support staff 170, and a function to transmit stored video data and audio data to the data management unit 112 in response to a request from the data management unit 112.

[0045] The supporter terminal 130 is, for example, a terminal installed in an office where the supporter 170 is stationed, and is operated by the supporter 170.

[0046] The supporter terminal 130 has a function to output moving image data and audio data transmitted from the worker terminal 120, a function to detect audio (e.g., the speech of the supporter 170), and a function to store the detected audio data. The supporter terminal 130 also has a function to transmit audio data obtained by detecting the speech of the supporter 170 to the worker 160 via the network 140. This allows the worker 160 to perform work under the instructions of the supporter 170.

[0047] Furthermore, the supporter terminal 130 has a function of transmitting stored voice data to the data management unit 112 in response to a request from the data management unit 112 .

[0048] <Hardware configuration of each device> Next, the hardware configuration of each device (server device 110, worker terminal 120, supporter terminal 130) that constitutes the two-way communication system 100 will be described.

[0049] 2A is a diagram showing an example of the hardware configuration of server device 110. As shown in Fig. 2A, server device 110 includes processor 201, memory 202, auxiliary storage device 203, operation device 204, display device 205, communication device 206, and drive device 207. The hardware components of server device 110 are connected to each other via bus 208.

[0050] The processor 201 has various arithmetic devices such as a CPU (Central Processing Unit), etc. The processor 201 reads various programs (for example, a two-way communication program) onto the memory 202 and executes them.

[0051] The memory 202 has a main storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The processor 201 and the memory 202 form a so-called computer (also called a "control unit"), and the processor 201 executes various programs read onto the memory 202, causing the computer to realize various functions.

[0052] The auxiliary storage device 203 stores various programs and various data used when the various programs are executed by the processor 201. Note that a data storage unit 470, which will be described later, is realized by the auxiliary storage device 203.

[0053] The operation device 204 is an operation device for the administrator of the server device 110 to perform various operations. The display device 205 is a display device for displaying the results of various processes executed by the server device 110.

[0054] The communication device 206 is a communication device for communicating with external devices (for example, the worker terminal 120 and the supporter terminal 130) via the network 140.

[0055] The drive device 207 is a device for loading a storage medium 210. The storage medium 210 here includes media that store information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk. The storage medium 210 may also include semiconductor memory that stores information electrically, such as a ROM or flash memory.

[0056] The various programs to be installed in the auxiliary storage device 203 are installed, for example, by setting the distributed storage medium 210 in the drive device 207 and reading out the various programs stored in the storage medium 210 by the drive device 207. Alternatively, the various programs to be installed in the auxiliary storage device 203 may be installed by being downloaded from the network 140 via the communication device 206.

[0057] 2B is a diagram showing an example of the hardware configuration of the worker terminal 120 and the supporter terminal 130. The worker terminal 120 and the supporter terminal 130 have roughly the same hardware configuration, so they will be described together here.

[0058] 2B , the worker terminal 120 and the supporter terminal 130 each have a processor 221, a memory 222, an auxiliary storage device 223, and a communication device 224. The worker terminal 120 also has a GPS (Global Positioning System) device 225, an acceleration sensor 226, and an imaging device 229. The worker terminal 120 and the supporter terminal 130 also have a voice input device 227, a voice output device 228, and an operation device 230. The supporter terminal 130 also has a display device 231 and a drive device 232. The hardware components of the worker terminal 120 and the supporter terminal 130 are connected to each other via a bus 233.

[0059] The processor 221 has various arithmetic devices such as a CPU (Central Processing Unit), etc. The processor 221 reads various programs (for example, a client program, which will be described later) onto the memory 222 and executes them.

[0060] The memory 222 has a main storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The processor 221 and the memory 222 form a so-called computer (also called a "control unit"), and the processor 221 executes various programs read onto the memory 222, causing the computer to realize various functions.

[0061] The auxiliary storage device 223 stores various programs and various information used when the various programs are executed by the processor 221. A data storage unit 690, which will be described later, is realized by the auxiliary storage device 223.

[0062] The communication device 224 is a communication device for transmitting and receiving various data (e.g., video data, audio data, etc.) to and from the server device 110. The GPS device 225 detects the position of the worker terminal 120 and outputs the position data. The acceleration sensor 226 detects the movement of the worker 160 wearing the worker terminal 120 and outputs the acceleration data.

[0063] The voice input device 227 detects the voices of the worker 160 and the assistant 170, the surrounding sounds at the work site, the surrounding sounds in the office, and the like, and outputs voice data.

[0064] The audio output device 228 is, for example, a device that outputs various data received from the server device 110 as audio to the worker 160 wearing the worker terminal 120 or the supporter 170 operating the supporter terminal 130.

[0065] The imaging device 229 captures images of the surroundings (work site) of the worker 160 wearing the worker terminal 120, and generates moving image data.

[0066] The operation device 230 accepts simple operations of the worker 160 on the worker terminal 120, such as the power ON / OFF switch of the worker terminal 120. Alternatively, the operation device 230 accepts various operations of the supporter 170 on the supporter terminal .

[0067] The display device 231 is a device that displays the video data received from the server device 110 to the supporter 170 of the supporter terminal 130.

[0068] The drive device 232 is a device for loading a storage medium 240. The storage medium 240 here is the same as the storage medium 210 described above.

[0069] In the case of the supporter terminal 130, the various programs to be installed in the auxiliary storage device 223 are installed, for example, by setting the distributed storage medium 240 in the drive device 232 and reading the programs by the drive device 232. Alternatively, the various programs to be installed in the auxiliary storage device 223 may be installed by being downloaded from the network 140 via the communication device 224.

[0070] <Details of the functional configuration of the bidirectional communication unit of the server device> Next, a detailed description will be given of the functional configuration of the two-way communication unit 111 of the server device 110. Fig. 3 is a diagram showing an example of the functional configuration of the two-way communication unit of the server device. As shown in Fig. 3, the two-way communication unit 111 has a two-way communication control unit 310, a real-time data storage unit 320, a communication quality inspection unit 330, and an inspection result notification unit 340.

[0071] The two-way communication control unit 310 controls two-way communication between the worker terminal 120 and the supporter terminal 130. Specifically, the two-way communication control unit 310 receives video data and audio data transmitted from the worker terminal 120 as real-time data and transmits it to the supporter terminal 130. The two-way communication control unit 310 also receives audio data transmitted from the supporter terminal 130 as real-time data and transmits it to the worker terminal 120.

[0072] The real-time data storage unit 320 notifies the data management unit 112 of the real-time data transmitted and received by the two-way communication control unit 310 .

[0073] The communication quality inspection unit 330 inspects the communication quality of the two-way communication controlled by the two-way communication control unit 310 using information related to the communication quality. Specifically, when communication with the worker terminal 120 or the supporter terminal 130 is disconnected, the communication quality inspection unit 330 detects this and notifies the inspection result notification unit 340. Furthermore, when a loss occurs in some frames in the video data transmitted from the worker terminal 120, the communication quality inspection unit 330 detects this and notifies the inspection result notification unit 340.

[0074] For example, when the two-way communication control unit 310 transmits and receives video data and audio data based on a standard such as WebRTC (Real Time Communication), the communication quality inspection unit 330: ·Generic Nack, ·PLI(Picture Loss Indication), By logging the frequency of occurrence (information on communication quality), communication interruptions and frame loss can be detected.

[0075] The inspection result notification unit 340 notifies the data management unit 112 of the communication disconnection or frame loss notified by the communication quality inspection unit 330 as communication quality information.

[0076] <Details of the functional configuration of the data management unit of the server device> Next, the functional configuration of the data management unit 112 of the server device 110 will be described in detail. Fig. 4 is a diagram showing an example of the functional configuration of the data management unit of the server device. As shown in Fig. 4, the data management unit 112 has a real-time data acquisition unit 410, a communication quality information acquisition unit 420, and a complement request unit 430. The data management unit 112 also has a video quality inspection unit 441, a video attribute inspection unit 442, an audio quality inspection unit 443, an audio attribute inspection unit 444, a complement data receiving unit 451, a complement unit 452, and a data control unit 460.

[0077] The real-time data acquisition unit 410 acquires the real-time data notified by the server device 110 and notifies the data control unit 460. Note that the moving image data (an example of second image data) and audio data (an example of second audio data) included in the real-time data notified to the data control unit 460 are stored in the data storage unit 470 (an example of a second storage unit).

[0078] The communication quality information acquisition unit 420 notifies the complement request unit 430 of the communication quality information notified by the two-way communication unit 111 .

[0079] The video quality inspection unit 441 inspects the data quality of video data read by the data control unit 460 from the real-time data stored in the data storage unit 470, based on information about the video data quality. The video quality inspection unit 441 also notifies the complement request unit 430 of the inspection result of the data quality.

[0080] The video attribute inspection unit 442 inspects the attributes of video data that is read by the data control unit 460 from the real-time data stored in the data storage unit 470 and that satisfies predetermined conditions, and notifies the inspection result to the complement request unit 430. Note that the video data that satisfies the predetermined conditions here refers to video data that is included in the complement range candidates identified by the complement request unit 430, which will be described later.

[0081] The audio quality inspection unit 443 inspects the data quality of the audio data read by the data control unit 460 from the real-time data stored in the data storage unit 470, based on information related to the audio data quality. The audio quality inspection unit 443 also notifies the complement request unit 430 of the inspection result of the data quality.

[0082] The voice attribute inspection unit 444 inspects the attributes of voice data that is read by the data control unit 460 from the real-time data stored in the data storage unit 470 and that satisfies predetermined conditions, and notifies the inspection result to the complement request unit 430. Note that the voice data that satisfies the predetermined conditions here refers to voice data that is included in the complement range candidates identified by the complement request unit 430, which will be described later.

[0083] The complement request unit 430 transmits a complement request to the worker terminal 120 or the supporter terminal 130. The complement request includes the complement range identified by the complement request unit 430 (the range that the data management unit 112 needs to acquire).

[0084] Specifically, the complement request unit 430 extracts the time range in which communication was interrupted or the time range in which frame loss occurred from the communication quality information notified by the communication quality information acquisition unit 420. Furthermore, the complement request unit 430 identifies the extracted time range as a candidate complement range for complementing video data and audio data.

[0085] In addition, the complement request unit 430 extracts the time range in which the video data quality has deteriorated from the inspection results notified by the video quality inspection unit 441, and identifies the extracted time range as a candidate complement range for complementing the video data.

[0086] Furthermore, based on the inspection result notified by the video attribute inspection unit 442, the complement request unit 430 specifies a time range (non-complement range) that should be excluded from the complement range for complementing the video data.

[0087] In addition, the complement request unit 430 extracts the time range in which the quality of the audio data has deteriorated from the inspection results notified by the audio quality inspection unit 443, and identifies the extracted time range as a candidate complement range for complementing the audio data.

[0088] Furthermore, based on the inspection result notified by the audio attribute inspection unit 444, the complement request unit 430 specifies a time range (non-complement range) that should be excluded from the complement range in which audio data is complemented.

[0089] Furthermore, the complement request unit 430 specifies, as the complement range for complementing the video data, a time range narrowed down by excluding non-complement ranges of the video data from the complement range candidates for complementing the video data.

[0090] Furthermore, the complement request unit 430 specifies a time range narrowed down by excluding non-complementary ranges of audio data from the candidates for the complementation range for complementing the audio data as the complementation range for complementing the audio data.

[0091] Furthermore, the complement request unit 430 generates a complement request including the complement range of the moving image data and the complement range of the audio data, and transmits it to the worker terminal 120 or the supporter terminal 130. This allows the data management unit 112 to request the worker terminal 120 or the supporter terminal 130 for the range that needs to be acquired.

[0092] In response to transmitting the complement request, the complement data receiving unit 451 notifies the complement unit 452 of the complement data (moving image data, audio data) received from the worker terminal 120 or the supporter terminal 130.

[0093] The complementing unit 452 replaces the time range corresponding to the moving image data read by the data control unit 460 from the real-time data stored in the data storage unit 470 with the complementing data (moving image data) notified by the complementing data receiving unit 451. The complementing unit 452 also replaces the time range corresponding to the audio data read by the data control unit 460 from the real-time data stored in the data storage unit 470 with the complementing data (audio data) notified by the complementing data receiving unit 451.

[0094] Furthermore, the complementing unit 452 notifies the data control unit 460 of the video data and audio data replaced with the complemented data, which allows the data control unit 460 to update the real-time data stored in the data storage unit 470 with the replaced real-time data.

[0095] <List of candidate complementary ranges and non-complementary ranges> Next, we will explain the candidate complement ranges identified based on the inspection results of each functional unit (communication quality information acquisition unit 420, video quality inspection unit 441, audio quality inspection unit 443) of the data management unit 112. We will also explain the non-complement ranges that should be excluded from the identified complement ranges based on the inspection results of each functional unit (video attribute inspection unit 442, audio attribute inspection unit 444) of the data management unit 112.

[0096] FIG. 5 is a diagram showing a list of complement range candidates and non-complement ranges identified by each functional unit of the data management unit.

[0097] As shown in FIG. 5, the communication quality information acquisition unit 420 identifies a time range in which a communication disconnection or a frame loss is detected as a candidate complement range for complementing video data and audio data.

[0098] 5, the video quality inspection unit 441 identifies a time range in which the frame rate (information about data quality) is equal to or lower than a predetermined threshold as a candidate complement range for complementing video data. Specifically, the video quality inspection unit 441 counts the number of frames per unit time to identify a time range in which the frame rate is equal to or lower than the predetermined threshold. Note that the frame rate may decrease, for example, when communication traffic occurs between the worker terminal 120 and the server device 110. Meanwhile, the worker terminal 120 stores video data in which the frame rate is not decreased within the corresponding time range. Therefore, video data in which the frame rate is equal to or lower than the predetermined threshold can be complemented.

[0099] 5, the video attribute inspection unit 442 identifies a time range in which a blackout occurs or a time range in which the inter-frame change is equal to or less than a predetermined threshold as a non-complement range that should be excluded from the complement range in which video data is complemented. Specifically, the video attribute inspection unit 442 determines whether a blackout has occurred by inspecting the video data frame by frame. The video attribute inspection unit 442 also determines whether the inter-frame change is equal to or less than a predetermined threshold by calculating the difference between each frame of the video data and the immediately preceding frame.

[0100] Note that, because a blackout is an event that occurs on the worker terminal 120, video data in which a blackout does not occur during the corresponding time range is not stored on the worker terminal 120 either. For this reason, the time range in which a blackout occurs is excluded from the complement range. Furthermore, when the inter-frame change is equal to or less than a predetermined threshold, this means that the worker 160 is stationary, and is not important as video data. For this reason, the time range in which the inter-frame change is equal to or less than a predetermined threshold is excluded from the complement range.

[0101] 5, the audio quality inspection unit 443 identifies a time range in which the audio bit rate (information about data quality) is equal to or less than a predetermined threshold as a candidate complement range for complementing audio data. Specifically, the audio quality inspection unit 443 counts the number of audio bits per unit time to identify a time range in which the audio bit rate is equal to or less than the predetermined threshold. Note that the audio bit rate decreases when, for example, communication traffic occurs. Meanwhile, the worker terminal 120 stores audio data in which the audio bit rate has not decreased within the corresponding time range. Therefore, audio data in which the audio bit rate is equal to or less than the predetermined threshold can be complemented.

[0102] 5, the voice attribute inspection unit 444 identifies a time range in which the voice signal strength is equal to or less than a predetermined threshold as a non-complement range that should be excluded from the complement range for complementing voice data. A voice signal strength equal to or less than the predetermined threshold indicates a state in which the worker 160 is not speaking, and is not important as voice data. For this reason, a time range in which the voice signal strength is equal to or less than the predetermined threshold is excluded from the complement range.

[0103] <Functional configuration of worker terminal> Next, the functional configuration of the worker terminal 120 will be described. FIG. 6 is a diagram showing an example of the functional configuration of the worker terminal. As described above, a client program is installed in the worker terminal 120. When the program is executed, the worker terminal 120 functions as a video data acquisition unit 611, a voice data acquisition unit 612, and a voice data output unit 613. When the program is executed, the worker terminal 120 functions as a data control unit 630, a communication control unit 640, a communication quality inspection unit 650, and an interpolation range correction unit 680. When the program is executed, the worker terminal 120 functions as an acceleration data acquisition unit 661, a GPS data acquisition unit 662, a radio wave intensity data acquisition unit 663, and an attribute inspection unit 670.

[0104] The video data acquisition unit 611 acquires video data captured by the imaging device 229 and notifies the data control unit 630 of the data.

[0105] The voice data acquisition unit 612 acquires the voice data detected by the voice input device 227 and notifies the data control unit 630 of the data.

[0106] The voice data output unit 613 outputs the voice data notified by the data control unit 630 to the worker 160 via the voice output device 228 .

[0107] The data control unit 630 notifies the communication control unit 640 of the moving image data notified by the moving image data acquisition unit 611 and the audio data notified by the audio data acquisition unit 612 as real-time data. The data control unit 630 also stores the moving image data (an example of first image data) notified by the moving image data acquisition unit 611 and the audio data (an example of first audio data) notified by the audio data acquisition unit 612 in the data storage unit 690 (an example of a first storage unit).

[0108] Furthermore, the data control unit 630 notifies the audio data output unit 613 of the audio data notified by the communication control unit 640 .

[0109] Furthermore, the data control unit 630 reads out the video data and audio data stored in the data storage unit 690, and notifies the communication control unit 640 of the video data and audio data in the corrected complement range notified by the complement range correction unit 680 as complement data.

[0110] Communication control unit 640 transmits the real-time data (video data, audio data) notified by data control unit 630 to server device 110, and also receives the real-time data (audio data) transmitted from server device 110.

[0111] Furthermore, the communication control unit 640 receives a complement request sent from the server device 110 and notifies the complement range correction unit 680 of the request.

[0112] The communication quality inspection unit 650 inspects the communication quality of the real-time data transmitted and received by the communication control unit 640. Specifically, when communication with the server device 110 is cut off, the communication quality inspection unit 650 detects this and notifies the complement range correction unit 680 of the cut off as a complement range candidate. Furthermore, when a frame loss occurs in video data included in the real-time data, the communication quality inspection unit 650 detects this and notifies the complement range correction unit 680 of the cut off as a complement range candidate. Note that the communication quality inspection unit 650 detects a communication cut off using, for example, the same detection method as the communication quality inspection unit 330 of the server device 110. Furthermore, the communication quality inspection unit 650 detects a frame loss using, for example, ·RTCInboundRTPStreamStats.fractionLost This will be done using an API (Application Programming Interface) that calculates statistical information such as:

[0113] The acceleration data acquisition unit 661 acquires acceleration data measured by the acceleration sensor 226 (an example of sensor information of a sensor attached to the worker terminal 120) and notifies the attribute inspection unit 670 of the data.

[0114] The GPS data acquisition unit 662 acquires the position data of the worker terminal 120 measured by the GPS device 225 (an example of sensor information of a sensor attached to the worker terminal 120) and notifies the attribute inspection unit 670 of the data.

[0115] The radio wave intensity data acquisition unit 663 acquires radio wave intensity data measured by the communication device 224 and notifies the attribute inspection unit 670 of the data.

[0116] The attribute inspection unit 670 performs a quality inspection or an attribute inspection using sensor information from a sensor attached to the worker terminal 120. Specifically, the attribute inspection unit 670 determines whether the worker 160 is stationary or moving based on the acceleration data notified by the acceleration data acquisition unit 661, and if it determines that the worker 160 is stationary, it identifies that time range as a non-complementary range. This is because the state in which the worker 160 is stationary is not important for video image data.

[0117] Furthermore, the attribute inspection unit 670 determines whether the worker 160 is in a predetermined area based on the location data notified by the GPS data acquisition unit 662, and if it determines that the worker 160 is in the predetermined area, it identifies that time range as a non-complementary range. For example, if the worker 160 is on the way to the work site (if the worker 160 is in an area other than the work site), the video data captured during that time range is not important.

[0118] In addition, the attribute inspection unit 670 determines whether the communication environment (communication quality) has deteriorated based on the radio wave intensity data notified by the radio wave intensity data acquisition unit 663, and if it determines that the communication environment has deteriorated, it identifies that time range as a candidate for the complementary range.

[0119] The complement range correction unit 680 corrects the complement range included in the complement request notified by the communication control unit 640, using the complement range candidate notified by the communication quality inspection unit 650 and the non-complement range or complement range candidate identified by the attribute inspection unit 670. In this way, the complement range correction unit 680 identifies the corrected complement range and notifies the data control unit 630 of the identified corrected complement range.

[0120] <List of candidate complementary ranges and non-complementary ranges> Next, we will explain the candidate complementary ranges identified based on the inspection results of the communication quality inspection unit 650 of the worker terminal 120, and the non-complementary ranges and candidate complementary ranges to be excluded from the complementary ranges identified based on the inspection results of the attribute inspection unit 670.

[0121] FIG. 7 is a diagram showing a list of complement range candidates and non-complement ranges identified by the respective functional units of the worker terminal.

[0122] As shown in the list 700 of complement range candidates and non-complement ranges in FIG. 7, the communication quality inspection unit 650 identifies the time range in which communication disconnection or frame loss is detected as a complement range candidate for complementing video data and audio data.

[0123] Furthermore, as shown in the list 700 of candidate complement ranges and non-complement ranges in FIG. 7, the attribute inspection unit 670 identifies a time range in which the signal strength of the acceleration data is below a predetermined threshold as a non-complement range that should be excluded from the complement range that complements the video data.

[0124] Furthermore, as shown in the list 700 of candidate complement ranges and non-complement ranges in FIG. 7, the attribute inspection unit 670 identifies a time range in which the position identified by the GPS data is within a predetermined area as a non-complement range that should be excluded from the complement range that complements the video data.

[0125] Furthermore, as shown in the list 700 of candidate complement ranges and non-complement ranges in FIG. 7, the attribute inspection unit 670 identifies a time range in which the radio wave intensity measured by the communication device 224 is below a predetermined threshold as a candidate complement range for complementing the video data.

[0126] <Processing flow in a two-way communication system> Next, a description will be given of the flow of processing in the two-way communication system 100. Fig. 8 is a sequence diagram showing the flow of processing in the two-way communication system.

[0127] In step S801, the worker terminal 120 starts two-way communication with the supporter terminal 130, and starts acquiring and storing video data and audio data.

[0128] In step S802, the worker terminal 120 transmits the acquired video data and audio data to the server device 110 as real-time data, and also receives audio data from the server device 110 as real-time data.

[0129] Similarly, in step S831, the supporter terminal 130 starts two-way communication with the worker terminal 120, and starts acquiring and storing voice data.

[0130] In step S832, the supporter terminal 130 transmits the acquired voice data to the server device 110 as real-time data, and receives video data and voice data from the server device 110 as real-time data.

[0131] In step S811, the two-way communication unit 111 of the server device 110 transmits the received real-time data to the worker terminal 120 or the supporter terminal 130, and also notifies the data management unit 112 of the same.

[0132] In step S821, the data management unit 112 stores the real-time data transmitted and received by the two-way communication unit 111.

[0133] In step S812, the two-way communication unit 111 checks the communication quality of the two-way communication between the worker terminal 120 and the supporter terminal .

[0134] In step S813, the two-way communication unit 111 determines whether communication with the worker terminal 120 or the supporter terminal 130 has been disconnected, or whether a frame loss has occurred in the video data received from the worker terminal 120.

[0135] If it is determined in step S813 that communication has been disconnected or that a frame loss has occurred (YES in step S813), the process proceeds to step S814.

[0136] In step S814, the two-way communication unit 111 notifies the data management unit 112 of the fact that communication has been disconnected or that a frame loss has occurred as communication quality information.

[0137] On the other hand, if it is not determined that the communication has been disconnected and that a frame loss has not occurred (NO in step S813), the communication quality information is not notified.

[0138] In step S822, the data management unit 112 performs a complement request process, the details of which will be described later.

[0139] In step S823, the data management unit 112 determines whether to send a complement request based on the result of the complement request processing. If it is determined in step S823 that a complement request is to be sent (YES in step S823), the process proceeds to step S824.

[0140] In step S824, the data management unit 112 transmits a complement request including the identified complement range to the worker terminal 120.

[0141] In step S803, the worker terminal 120 performs an interpolation range correction process to correct the interpolation range based on the interpolation request received from the data management unit 112. Details of the interpolation range correction process will be described later.

[0142] In step S804, the worker terminal 120 transmits the moving image data and audio data in the complement range corrected in the complement range correction process to the data management unit 112 as complement data.

[0143] In step S825, the data management unit 112 updates the real-time data by replacing the time range of the stored real-time data that corresponds to the corrected complement range with the complement data.

[0144] <Details of the interpolation range correction process> Next, the complement range correction process (step S803) will be described in detail. Fig. 9 is a flowchart showing the flow of the complement range correction process.

[0145] In step S901, the worker terminal 120 determines whether a communication disconnection or a frame loss has been detected.

[0146] If it is determined in step S901 that a communication disconnection or frame loss has been detected (YES in step S901), the process proceeds to step S902.

[0147] In step S902, the worker terminal 120 identifies the detected time range as a candidate for a complementary range, and proceeds to step S903.

[0148] On the other hand, if it is determined in step S901 that a communication disconnection or frame loss has not been detected (NO in step S901), the process proceeds directly to step S903.

[0149] In step S903, the worker terminal 120 determines whether a time range to be excluded from the complement range or a time range to be complemented has been detected based on the acceleration data, the position data, and the radio wave intensity data.

[0150] In step S903, if it is determined that a time range to be excluded from the complement range or a time range to be complemented has not been detected (NO in step S903), the process proceeds to step S905.

[0151] On the other hand, if it is determined in step S903 that a time range to be excluded from the complement range or a time range to be complemented has been detected (YES in step S903), the process proceeds to step S904.

[0152] In step S904, the worker terminal 120 identifies a time range to be excluded from the complement range as a non-complement range, and also identifies a time range to be complemented as a complement range candidate.

[0153] In step S905, the worker terminal 120 determines whether or not a complement request has been received from the server device 110. If it is determined in step S905 that a complement request has been received (YES in step S905), the process proceeds to step S906.

[0154] In step S906, the worker terminal 120 corrects the complement range included in the complement request using the complement range candidate identified in step S902 and the non-complement range and complement range candidate identified in step S904, and ends the complement range correction process.

[0155] On the other hand, if it is determined in step S905 that a complement request has not been received (NO in step S905), the complement range correction process ends.

[0156] <Details of Completion Request Processing> Next, the complement request process (step S822) will be described in detail with reference to the flowchart of FIG.

[0157] In step S1001, data management unit 112 determines whether or not communication quality information has been acquired from bidirectional communication unit 111. If it is determined in step S1001 that communication quality information has been acquired (YES in step S1001), the process proceeds to step S1002.

[0158] In step S1002, the data management unit 112 identifies complement range candidates based on the communication quality information, and the process proceeds to step S1003.

[0159] On the other hand, if it is determined in step S1001 that communication quality information has not been acquired (NO in step S1001), the process proceeds directly to step S1003.

[0160] In step S1003, the data management unit 112 checks the data quality of the video data and audio data for the real-time data and determines whether there is a time range in which the data quality has deteriorated. If it is determined in step S1003 that there is a time range in which the data quality has deteriorated (YES in step S1003), the process proceeds to step S1004.

[0161] In step S1004, the data management unit 112 identifies the time range in which the data quality has deteriorated as a candidate for a complement range, and the process proceeds to step S1005.

[0162] On the other hand, if it is determined in step S1003 that there is no time range in which the data quality has deteriorated, the process proceeds directly to step S1005.

[0163] In step S1005, the data management unit 112 checks the attributes of the video data and audio data that satisfy a predetermined condition among the real-time data, and determines whether a time range that should be excluded from the complement range has been detected. If it is determined in step S1005 that a time range that should be excluded from the complement range has been detected (YES in step S1005), the process proceeds to step S1006.

[0164] In step S1006, the data management unit 112 identifies the time range to be excluded from the complementation range as a non-complementation range, and proceeds to step S1007.

[0165] On the other hand, if data management section 112 determines in step S1005 that a time range to be excluded from the complement range has not been detected (NO in step S1005), the process proceeds directly to step S1007.

[0166] In step S1007, the data management unit 112 identifies the complement range based on the complement range candidate identified in step S1002, the complement range candidate identified in step S1004, and the non-complement range identified in step S1006.

[0167] In step S1008, the data management unit 112 generates a complement request that includes the identified complement range, and ends the complement request process.

[0168] <Summary> As is clear from the above description, in the two-way communication system 100 having the client terminals (the worker terminal 120 and the supporter terminal 130) and the server device 110, The worker terminal 120 transmits the acquired video data or audio data to the server device 110 in real time. The worker terminal 120 stores the acquired video data or audio data in the data storage unit 690. The server device 110 identifies the complement range of the video data or audio data stored in the data storage unit 690 that the server device 110 needs to acquire. The server device 110 requests the worker terminal 120 to transmit the video data or audio data stored in the data storage unit 690 .

[0169] As a result, according to the first embodiment, it is possible to improve the quality of moving image data and audio data stored in a server device in a two-way communication system that transmits and receives moving image data and audio data.

[0170] [Second embodiment] In the above first embodiment, the result of the communication quality inspection by the communication quality inspection unit 650 of the worker terminal 120 and the result of the attribute inspection by the attribute inspection unit 670 are described as being used to correct the complement range. However, the result of the communication quality inspection and the result of the attribute inspection by the worker terminal 120 may be used, for example, to determine whether or not to transmit complement data.

[0171] In addition, when the information is used to determine whether or not to send complementary data, if it is determined that complementary data should not be sent, the complementary data may be configured not to be sent to the server device 110 even if a complementary request is received.

[0172] Alternatively, when used to determine whether or not to send complementary data, the result of the determination as to whether or not to send complementary data may be output to the worker 160, and when the worker 160 inputs a transmission instruction, the complementary data may be transmitted to the server device 110.

[0173] In this case, the complement data transmitted to server device 110 may be the complement range after correction or the complement range before correction.

[0174] [Third embodiment] In the first embodiment, the case where the data management unit 112 transmits the completion request to the worker terminal 120 has been described. However, the destination of the completion request is not limited to the worker terminal 120, and the completion request may be transmitted to the supporter terminal 130.

[0175] In the first embodiment, the moving image quality inspection unit 441 to the voice attribute inspection unit 444 of the data management unit 112 are described as performing predetermined inspections. However, the moving image quality inspection unit 441 to the voice attribute inspection unit 444 of the data management unit 112 may be configured to perform inspections according to judgment parameters transmitted from client terminals (the worker terminal 120, the supporter terminal 130), for example.

[0176] Furthermore, in the above-described first embodiment, no mention is made of time synchronization processing between the server device 110 and the client terminals (the worker terminal 120, the supporter terminal 130). However, the server device 110 and the client terminals (the worker terminal 120, the supporter terminal 130) may be configured to perform time synchronization processing at predetermined intervals. Specifically, the server device 110 and the client terminals (the worker terminal 120, the supporter terminal 130) may be configured to perform time synchronization processing at predetermined intervals using a constant connection service such as WebSocket.

[0177] Furthermore, in the above first embodiment, no details are mentioned about the process by which the data control unit 630 notifies the communication control unit 640 of video image data and audio data and the process by which the data control unit 630 stores the video image data and audio data in the data storage unit 690. However, the data control unit 630 may be configured to attach a timestamp when notifying the communication control unit 640 of the video image data and audio data and when storing the video image data and audio data in the data storage unit 690, for example.

[0178] In the first embodiment, only the functional configuration of the worker terminal 120 among the client terminals has been described. However, the supporter terminal 130 also has the same functional configuration as the worker terminal 120.

[0179] Furthermore, in the first embodiment, a case has been described in which a wearable terminal worn by a worker is used, but instead of a wearable terminal, a portable terminal carried by a worker may be used.

[0180] In addition, in the above first embodiment, the case where the client terminal is a combination of the worker terminal 120 and the supporter terminal 130 has been described, but the client terminal may also be a combination of multiple web terminals used for web meetings, etc.

[0181] In the first embodiment, a configuration has been described in which the client terminal transmits a specified range of video data or audio data to server device 110 in response to a request from server device 110 during two-way communication by the client terminal. However, the timing at which the client terminal transmits to server device 110 is not limited to this, and for example, the client terminal may be configured to transmit to server device 110 after two-way communication by the client terminal is completed. Furthermore, when transmitting after two-way communication by the client terminal is completed, the range to be transmitted is not limited to a specified range, and the client terminal may be configured to transmit all of the video data or audio data stored therein. Furthermore, the client terminal may be configured to transmit regardless of whether or not there is a request from server device 110.

[0182] In the first embodiment, the server device 110 is described as executing the two-way communication program by itself. However, if the server device 110 is configured with, for example, a plurality of computers (control units), the two-way communication program may be installed in each of the plurality of computers (control units) and executed in a distributed computing format.

[0183] Furthermore, in the first embodiment, a method of installing a client program by downloading it via a network was mentioned as an example of a method of installing the client program. At this time, no particular reference was made to the download source. However, when installing using this method, the download source may be, for example, a server device that stores the client program in an accessible manner. Furthermore, the server device may be a cloud-based device that accepts access from client terminals (worker terminal 120, supporter terminal 130) via a network and downloads the client program on the condition that a fee is charged. In other words, the server device may be a cloud-based device that provides a service of providing the client program.

[0184] Although the embodiments have been described above, it will be understood that various changes in form and details can be made without departing from the spirit and scope of the claims. [Explanation of symbols]

[0185] 100: Two-way communication system 110: Server device 111: Two-way communication unit 112: Data Management Department 120: Worker terminal 130: Supporter terminal 310: Two-way communication control unit 320: Real-time data storage section 330: Communication Quality Inspection Department 340: Test result notification department 410: Real-time data acquisition unit 420: Communication quality information acquisition unit 430: Completion request section 441: Video Quality Inspection Department 442: Video attribute inspection unit 443: Voice Quality Inspection Department 444: Voice Attribute Inspection Department 451: Complementary data receiving unit 452: Complementary part 460: Data control section 611: Video data acquisition unit 612 : Audio data acquisition unit 613: Audio data output unit 630: Data control section 640: Communication control unit 650: Communication Quality Inspection Department 661: Acceleration data acquisition unit 662: GPS data acquisition unit 663: Radio wave intensity data acquisition unit 670: Attribute Inspection Department 680: Complementary range correction section

Claims

1. A client terminal used by a worker; a server device that controls two-way communication between the client terminal and a supporter terminal used by a supporter who supports the worker, The control unit of the client terminal transmitting the image data or audio data acquired by the client terminal to the server device in real time; storing the image data or audio data acquired by the client terminal in a first storage unit as first image data or first audio data; The control unit of the server device storing the image data or audio data transmitted in real time in a second storage unit as second image data or second audio data; Identifying a range of the first image data or the first audio data stored in the first storage unit that needs to be acquired by the server device; requesting the client terminal to transmit the first image data or first audio data in the specified range; replacing a corresponding range of the second image data or the second audio data stored in the second storage unit with the first image data or the first audio data of the specified range transmitted from the client terminal; Communication system.

2. The communication system according to claim 1 , wherein the control unit of the server device specifies a time range in which the server device needs to acquire the information.

3. The control unit of the server device performing at least one of a quality inspection and an attribute inspection on the image data or voice data transmitted in real time; The communication system according to claim 2 , wherein a time range that needs to be acquired by the server device is specified based on the result of the check.

4. The control unit of the server device The communication system according to claim 3, wherein the quality inspection is performed based on one or more of information regarding the communication quality when the image data or audio data is transmitted in real time, or information regarding the data quality of the image data or audio data transmitted in real time.

5. The control unit of the server device The communication system according to claim 3 , wherein a time range that the server device needs to acquire is identified based on the result of the quality inspection, and the identified time range is narrowed down based on the result of the attribute inspection.

6. The control unit of the server device 4. The communication system according to claim 3, wherein the attribute inspection is performed based on information regarding changes between frames of image data that meet a predetermined condition, among the image data transmitted in real time.

7. The control unit of the server device The communication system according to claim 3 , wherein judgment parameters used when performing at least one of the quality inspection and the attribute inspection are acquired from the client terminal.

8. The communication system according to claim 1 , wherein time synchronization processing is performed between the control unit of the client terminal and the control unit of the server device at predetermined intervals.

9. The first storage unit 2. The communication system according to claim 1, wherein when the image data or audio data acquired by the client terminal is stored as the first image data or first audio data, a timestamp is stored.

10. The control unit of the client terminal The communication system according to claim 1 , wherein the first image data or the first audio data in the specified time range is transmitted to the server device.

11. The control unit of the client terminal performing at least one of a quality inspection and an attribute inspection on the image data or audio data transmitted in real time; determining whether or not to transmit the first image data or the first audio data within the specified time range to the server device based on a result of the inspection; The communication system of claim 10.

12. The control unit of the client terminal The communication system according to claim 11 , wherein an attribute check is performed on the first image data or the first audio data stored in the first storage unit.

13. The control unit of the client terminal The communication system according to claim 11 , wherein at least one of the quality inspection and the attribute inspection is performed using sensor information from a sensor attached to the client terminal.

14. The control unit of the client terminal 12. The communication system according to claim 11, wherein even if transmission of the specified time range is requested, if it is determined not to transmit based on the results of the inspection, the first image data or first audio data of the specified time range is not transmitted to the server device.

15. The control unit of the client terminal When a request for transmission of the specified time range is made and a determination is made as to whether or not to transmit the request based on the result of the inspection, a determination result is output; The communication system according to claim 11, wherein when a transmission instruction is input in response to outputting the determination result, the first image data or first audio data for the specified time range is transmitted to the server device.

16. A client terminal used by a worker receiving support from a supporter, which communicates with a server device that controls two-way communication between the client terminal and the supporter terminal used by the supporter, The control unit of the client terminal transmitting the image data or audio data acquired by the client terminal to the server device in real time; storing the image data or audio data acquired by the client terminal in a first storage unit; performing at least one of a quality inspection and an attribute inspection on the image data or audio data transmitted in real time; transmitting the image data or the audio data stored in the first storage unit to the server device based on the result of the inspection; the image data or audio data stored in the second storage unit of the server device as a result of the real-time transmission is replaced with the image data or audio data transmitted to the server device based on the result of the inspection; Client terminal.

17. The control unit The client terminal according to claim 16, wherein a timestamp is stored when the image data or audio data is stored in the first storage unit.

18. The client terminal according to claim 16, wherein the client terminal determines whether or not to transmit the image data or audio data stored in the first storage unit to the server device based on a result of the check.

19. The control unit The client terminal according to claim 16, wherein an attribute check is performed on the image data or audio data stored in the first storage unit.

20. The control unit The client terminal according to claim 16, wherein at least one of the quality inspection and the attribute inspection is performed using sensor information from a sensor attached to the client terminal.

21. The control unit The client terminal according to claim 18 , wherein, when it is determined based on the result of the inspection that the image data or the audio data should not be transmitted to the server device, the image data or the audio data is not transmitted to the server device.

22. The control unit When it is determined whether or not to transmit based on the result of the inspection, the determination result is output; 19. The client terminal according to claim 18, wherein, when a transmission instruction is input in response to outputting the determination result, the client terminal transmits the image data or audio data to the server device.

23. A client terminal used by a worker; a server device that controls two-way communication between the client terminal and a supporter terminal used by a supporter who supports the worker, a communication step of transmitting the image data or audio data acquired by the client terminal to the server device in real time; a first storage step of storing the image data or audio data acquired by the client terminal in a first storage unit as first image data or first audio data; a second storage step of storing the image data or audio data transmitted in real time in a second storage unit as second image data or second audio data; a specifying step of specifying a range of the image data or audio data stored in the first storing step that needs to be acquired by the server device; a requesting step of requesting the client terminal to transmit the image data or audio data within the range specified in the specifying step; a replacement step of replacing a corresponding range of the second image data or second audio data stored in the second storage unit with the first image data or first audio data of the specified range transmitted from the client terminal; A communication method comprising:

24. A communication method in a client terminal used by a worker receiving support from a supporter, by communicating with a server device that controls two-way communication between the client terminal and a supporter terminal used by the supporter, comprising: a first communication step of transmitting image data or audio data acquired by the client terminal to the server device in real time; a first storage step of storing the image data or audio data acquired by the client terminal in a first storage unit; an inspection step of performing at least one of a quality inspection and an attribute inspection on the image data or the voice data transmitted in the first communication step; a second communication step of transmitting the image data or audio data stored in the first storage step to the server device based on the result of the inspection in the inspection step, the image data or audio data stored in the second storage unit of the server device as a result of the real-time transmission is replaced with the image data or audio data transmitted to the server device based on the result of the inspection; Communication method.

25. By communicating with a server device that controls two-way communication between the supporter terminal used by the supporter and the supporter terminal used by the supporter, a control unit of the client terminal used by the worker receiving support from the supporter a first communication step of transmitting image data or audio data acquired by the client terminal to the server device in real time; a first storage step of storing the image data or audio data acquired by the client terminal in a first storage unit; an inspection step of performing at least one of a quality inspection and an attribute inspection on the image data or the voice data transmitted in the first communication step; a second communication step of transmitting the image data or audio data stored in the first storage step to the server device based on the result of the inspection in the inspection step; the image data or audio data stored in the second storage unit of the server device as a result of the real-time transmission is replaced with the image data or audio data transmitted to the server device based on the result of the inspection; Communications program.

Images