Relay system, information recording method, and information recording program

Abstract

To facilitate identification of a cause of a trouble that occurs when there is a communication delay.SOLUTION: A relay system that relays between a terminal and a processing apparatus that: performs processing in accordance with a request from the terminal and transmits a response to the terminal records an operation screen displayed on the terminal and accumulates first video recording data (normal video recording data) while the terminal and the processing apparatus are being connected; when a time difference between a time point of transmitting the request from the terminal and a time point of receiving the response corresponding to the request from the processing apparatus exceeds a reference value, replicates video recording data recorded at a predetermined time before the time point of transmission of the request or thereafter among the first video recording data (a double-headed arrow in Fig. 13(a)); generates second video recording data; and adds video recording data obtained by recording the operation screen while the time difference exceeds the reference value to the second video recording data (a dashed arrow in Fig. 13(b)).SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present invention relates to a relay system, an information recording method, and an information recording program. [Background technology]

[0002] In recent years, the use of teleworking and working from home has been increasing, and technology that uses web portals is known as a technology for issuing instructions from a client terminal operated by a user to an office PC, a cloud server, etc. (hereinafter referred to as various servers).

[0003] In this technology, the web portal relays communication between the client terminal and various servers via the Internet, and delivers operation screens sent from the various servers to the client terminal. The user operating the client terminal also views the operation screen on a browser and operates it using a keyboard and mouse.

[0004] In such systems, communication delays can occur between the web portal and various servers. When communication delays occur, a user performs an operation but the operation is not reflected, which can lead to problems such as the user performing the same operation again due to incorrect operation. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2018-195209 [Patent Document 2] Patent Publication No. 2021-56763 Summary of the Invention [Problem to be solved by the invention]

[0006] Usually, when a problem occurs, an administrator or the like often refers to logs generated by the operating programs of various servers in order to identify the cause of the problem.

[0007] However, when a problem occurs due to an incorrect operation caused by a communication delay as described above, the various servers normally process the corresponding operation, so information about the problem is often not recorded in the logs. As a result, it may not be possible to identify the cause of the problem from the logs.

[0008] In one aspect, the present invention aims to provide a relay system, an information recording method, and an information recording program that make it easier to identify the cause of a problem that occurs when there is a communication delay. [Means for solving the problem]

[0009] In one aspect, a relay system is a relay system that relays between a terminal and a processing device that executes processing in response to a request from the terminal and transmits a response to the terminal, and includes: a recording device that records an operation screen displayed on the terminal and accumulates first recording data while the terminal and the processing device are connected; and a transmission / reception device that, when receiving a request from the terminal to the processing device, transmits the request to the processing device, and, when receiving a response corresponding to the request from the processing device, transmits the response to the terminal; when the time difference between the time when the transmission / reception device sends a request and the time when it receives a response corresponding to the request exceeds a reference value, the recording device generates second recording data by duplicating recording data from the first recording data that was recorded after a predetermined time before the time when the request was sent, and adds recording data obtained by recording the operation screen while the time difference exceeds the reference value to the second recording data. [Effects of the Invention]

[0010] This makes it easier to identify the cause of problems that occur when communication delays occur. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating a schematic configuration of a business system according to an embodiment. [Figure 2] FIG. 2 illustrates an example of the hardware configuration of a web portal. [Figure 3] This is a functional block diagram of the web portal. [Figure 4] FIG. 10 is a diagram illustrating an example of a characteristic operation table. [Figure 5] FIG. 5(a) is a diagram showing an example of an operation information table, and FIG. 5(b) is a diagram showing an example of a screen information table. [Figure 6] FIG. 6 is a flowchart showing the processing of the user terminal. [Figure 7] 7 is a flowchart showing the processes of the web portal (screen providing unit), server, and recording server after the process of step S12 in FIG. 6 is executed. [Figure 8] 10 is a flowchart showing the processing of the screen providing unit, delay monitoring unit, and server when a user operates the keyboard or mouse after remote connection and operation information is sent from the user terminal to the web portal. [Figure 9] FIG. 10 is a diagram showing a state in which operation information data is registered in an operation information table. [Figure 10] 9 is a flowchart showing a process subsequent to the process of the delay monitoring unit in FIG. 8. [Figure 11] FIG. 11(a) is a diagram showing the state in which screen information data is registered in the screen information table, and FIG. 11(b) is a diagram showing the state in which the ID in FIG. 11(a) is registered as the response ID in the operation information table. [Figure 12] 10 is a flowchart showing the processing of a video extraction unit and a recording server. [Figure 13] 13(a) and 13(b) are diagrams for explaining the processing of the recording server. [Figure 14] 14(a) and 14(b) are diagrams for explaining problems caused by delays. DETAILED DESCRIPTION OF THE INVENTION

[0012] An embodiment of the business system will be described in detail below with reference to FIGS.

[0013] 1 shows a schematic configuration of a business system 100 according to an embodiment. The business system 100 includes a user terminal 10 as a terminal used by a user at home, at work, at a satellite office, etc., a server 70 as a processing device to which the user terminal 10 is remotely connected, and a relay system 50. The relay system 50 includes a web portal 20 as a transmitting / receiving device that mediates between the user terminal 10 and the server 70 via the Internet, and a recording server 60 as a recording device that performs processing in cooperation with the web portal 20.

[0014] The user terminal 10 is an information processing device such as a PC (Personal Computer) that executes various processes in response to user operations. The server 70 is an information processing device that executes processes in response to requests from the user terminal 10.

[0015] When a user requests the user terminal 10 to establish a remote connection with the server 70, the web portal 20 establishes a remote connection between the user terminal 10 and the server 70. In this remotely connected state, a screen created by the server 70 is displayed on the display unit of the user terminal 10, and when an operation is performed on the screen, the web portal 20 transmits operation information (operation request) corresponding to the operation to the server 70. Furthermore, upon receiving the operation request, the server 70 executes processing corresponding to the operation request and transmits information about the updated screen to the web portal 20 as a screen response. The web portal 20 then transmits the screen response to the user terminal 10 and displays the updated screen on the display unit of the user terminal 10 (performs a screen update).

[0016] The web portal 20 has a function of relaying between the user terminal 10 and the server 70, and includes one or more information processing devices. For simplicity of explanation, this embodiment will describe a case where the web portal 20 is realized by one information processing device.

[0017] FIG. 2 shows an example of the hardware configuration of the web portal 20. As shown in FIG. 2, the web portal 20 includes a central processing unit (CPU) 90, a read-only memory (ROM) 92, a random access memory (RAM) 94, a storage unit (here, a solid-state drive (SSD) or a hard disk drive (HDD)) 96, a network interface 97, and a portable storage medium drive 99. These components of the web portal 20 are connected to a bus 98. In the web portal 20, the CPU 90 executes a program (including an information recording program) stored in the ROM 92 or the storage unit 96, or a program read by the portable storage medium drive 99 from the portable storage medium 91, thereby realizing the functions of the components shown in FIG. 3. Note that the functions of the components shown in FIG. 3 may be realized by an integrated circuit such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0018] As shown in FIG. 3, the web portal 20 has the functions of a screen providing unit 22, a delay monitoring unit 24, and a video extracting unit .

[0019] The screen providing unit 22 transmits an operation request from the user terminal 10 to the server 70, and transmits a screen response from the server 70 to the user terminal 10 when the server 70 receives one.

[0020] The delay monitoring unit 24 monitors the time difference between sending an operation request and receiving a screen response, and determines that a delay has occurred if the time difference exceeds a predetermined threshold. In addition, the delay monitoring unit 24 requests the video extraction unit 26 to record the screen during the period in which the delay has occurred (delay period) according to the determination result.

[0021] The delay monitor 24 monitors the time difference when the operation related to the operation request is a predetermined "characteristic operation." A characteristic operation is an operation that can be used to determine whether a communication delay is occurring, such as an operation that requires a screen update (an operation that updates the screen). FIG. 4 shows an example of a characteristic operation table referenced by the delay monitor 24. The characteristic operation table in FIG. 4 associates the type of characteristic operation with a corresponding threshold value. As shown in FIG. 4, the characteristic operations include pressing a "character key on the keyboard," pressing the "Enter key on the keyboard," pressing a "modifier key such as Ctrl or Shift," and right-clicking or left-clicking a mouse. The threshold value for each operation is set based on 500 ms, taking into account the maximum delay that humans can tolerate during a remote connection (e.g., 250 ms) and the interrupt processing speed of the delay monitor 24. For example, in the case of pressing a "character key on the keyboard," a "right click," or a "left click," the processing on the side of the application running on the server 70 is mostly limited to processing to reflect the input content, so the threshold is set short. On the other hand, in the case of pressing the "Enter key on the keyboard," the threshold is set long because processing is often requested from the application on the server 70 along with the input. Also, in the case of pressing a "modifier key such as Ctrl or Shift," the threshold is set long because pressing the modifier key alone does not have any particular effect, but often requests processing from the server 70 in combination with other keys. Note that the processing corresponding to characteristic operations may differ depending on the application running on the server 70, so a characteristic operation table of FIG. 4 may be prepared for each application. For example, when a "left click" is performed on a dialog, a relatively high-load processing may be executed in the application on the server 70. In this case, the threshold is set long (e.g., 2000 ms).

[0022] For details on the maximum latency that humans can tolerate when connecting remotely (e.g., 250 ms), see 'Pedro Casas, Michael Seufert, Sebastian Egger, and Raimund Schatz, "Quality of Experience in Remote Virtual Desktop Services", https: / / ieeexplore.ieee.org / document / 6573191, (2021-12-21)'.

[0023] Furthermore, when monitoring the time difference, the delay monitor 24 generates an operation information table shown in FIG. 5(a) and a screen information table shown in FIG. 5(b) as appropriate.

[0024] Specifically, the delay monitoring unit 24 generates the operation information table shown in FIG. 5( a) when a characteristic operation is performed on the user terminal 10. The operation information table shown in FIG. 5( a) stores the following information: “ID,” “Connection ID,” “Transmission Time,” “Operator,” “Destination Server Information,” “Response ID,” and “Operation Content.” “ID” is identification information for identifying the operation information. “Connection ID” is identification information for the remote connection from which the operation information was generated. “Transmission Time” is the time the operation was recorded, “Operator” is the user name of the user who performed the operation, and “Destination Server Information” is the server name of the server to which the user is remotely connected. “Response ID” is the ID of the screen information table corresponding to the operation. Note that when a characteristic operation is performed by the user, the “Response ID” column is empty. However, when a screen response corresponding to the characteristic operation is transmitted from the server 70, data is entered into the “Response ID” column. “Operation Content” is information indicating how the user operated the keyboard or mouse. For example, in the case of a remote connection, the operation details are recorded in a comma-separated text format that conforms to X11 KeySym. For example, if you press the key identified by 'ff52', it will be recorded as 'key,ff52,1'. Also, if you left-click at coordinates (865,312) on the screen (pressing the button identified by 'fd11'), it will be recorded as 'mouse,865,312,fd11,1'.

[0025] Furthermore, the delay monitoring unit 24 generates the screen information table shown in FIG. 5(b) upon receiving a response containing updated screen information (screen update information) from the remotely connected server 70. The screen information table shown in FIG. 5(b) stores information such as "ID," "received time," and "screen update content." "ID" is identification information for identifying the screen update information. "received time" is the time when the screen update information was received. "screen update content" is the updated screen information, expressed in a comma-separated text file format. The updated screen information replaces the updated portion of the screen, and the previous screen information is replicated for non-updated portions. For example, when defining the layer size at startup, it is recorded as '4,size,1.0.4,1024,768', and when linking the image type and layer settings, it is recorded as '3,img,1.3.2,12, <meta> ,image / png:'. Also, when duplicating an unupdated part, it is recorded as '4,copy,2.-2,1.0.1.0.2,', and when replacing an updated part, it is recorded as '4,arc,1.3.2,12, <meta> '. In reality, it is recorded as '4,size,1.0.4,1024,768' or '3,img,1.3.2,12, <meta> ,image / png:' is recorded as a text file with the contents concatenated without line breaks.

[0026] Returning to FIG. 3, in response to a request from the delay monitor 24, the video extractor 26 instructs the recording server 60 to record the screen during the delay period.

[0027] The recording server 60 records the screen displayed on the display unit of the user terminal 10 while the user terminal 10 is remotely connected to the server 70, and generates normal recording data (first recording data). In addition, in response to an instruction from the video extraction unit 26, the recording server 60 records the screen displayed on the user terminal 10 during the delay period, in parallel with recording the normal recording data, and generates delay period recording data (second recording data).

[0028] (Regarding the processing of each device) The processing of each device will be described in detail below.

[0029] (Regarding the processing of the user terminal 10) First, a description will be given of the processing of the user terminal 10. The processing of the user terminal 10 is shown in a flowchart in FIG.

[0030] 6 starts, first, in step S10, the user terminal 10 waits until a remote connection request is input by the user. When the user inputs the remote connection request by specifying one of the servers 70, the process proceeds to step S12, and the user terminal 10 requests the web portal 20 for a remote connection.

[0031] Next, in step S14, the user terminal 10 waits until it receives screen change information from the web portal 20. The screen change information includes information about the screen transmitted from the server 70 after the remote connection is established, and information about the screen updated on the server 70 as a result of an operation performed on the screen transmitted from the server 70. When the screen change information is received from the web portal 20, the process proceeds to step S16.

[0032] In step S16, the user terminal 10 updates the screen based on the received screen change information.

[0033] Next, in step S18, the user terminal 10 waits until the user operates the keyboard or mouse. When the user operates the keyboard or mouse, the process proceeds to step S20, and the user terminal 10 transmits operation information to the web portal 20.

[0034] Next, in step S22, the user terminal 10 determines whether a request to terminate the remote connection has been input. That is, the user terminal 10 determines whether the operation in step S18 was an operation to input a request to terminate the remote connection. If the determination in step S22 is negative, the process returns to step S14, and the processes from step S14 onward described above are repeatedly executed. On the other hand, if the determination in step S22 is positive, all the processes in FIG. 6 are terminated.

[0035] (Processing when a remote connection request is received) Next, the processing of the web portal 20 (screen providing unit 22), server 70, and recording server 60 after the processing of step S12 in FIG. 6 is executed will be described with reference to the flowchart in FIG.

[0036] 7, the screen providing unit 22 of the web portal 20 waits until it receives a request to start a remote connection from the user terminal 10 (step S102). When the request is sent from the user terminal 10, the screen providing unit 22 proceeds to step S104, creates a request to start a remote connection, and sends the created request to the connection destination server 70 and the recording server 60 in step S106.

[0037] Meanwhile, the server 70 and the recording server 60 wait until they receive a request (steps S502 and S602). Therefore, when the screen provider 22 executes step S106, the server 70 proceeds to step S504, and the recording server 60 proceeds to step S604.

[0038] In step S604, the recording server 60 starts recording the screen displayed on the display unit of the user terminal 10, i.e., recording of normal recording data. Meanwhile, the server 70 generates screen change information (e.g., information on the initial screen after remote connection) and transmits it to the screen provider 22.

[0039] Returning to the explanation of the screen providing unit 22, after step S106, the screen providing unit 22 waits until it receives screen change information in step S108. Therefore, the screen providing unit 22 proceeds to step S110 when the server 70 executes step S504.

[0040] When the process proceeds to step S110, the screen providing unit 22 transmits the screen change information received from the server 70 to the user terminal 10. When the user terminal 10 receives the screen change information as described above (S14: Yes), it updates the screen with the received screen change information (S16).

[0041] (Processing when the user performs an operation) Next, the processing of the web portal 20 and the server 70 when the user operates the keyboard or mouse after remote connection (when operation information is sent to the web portal 20) will be described with reference to the flowcharts of FIGS.

[0042] 8, the screen providing unit 22 and delay monitoring unit 24 of the web portal 20 wait until they receive operation information from the user terminal 10 (steps S120 and S202). Therefore, when operation information is transmitted from the user terminal 10, the screen providing unit 22 proceeds to step S122 and creates an operation request. In addition, in step S124, the screen providing unit 22 transmits the created operation request to the connected server 70.

[0043] Furthermore, when operation information is transmitted from the user terminal 10, the delay monitoring unit 24 proceeds to step S204 and determines whether the user operation is a characteristic operation. At this time, the delay monitoring unit 24 refers to the characteristic operation table in Fig. 4 and determines whether the operation related to the received operation information matches the operation listed in the characteristic operation table. If the determination in step S204 is negative, the delay monitoring unit 24 ends the processing in Fig. 8, but if the determination in step S204 is positive, the delay monitoring unit 24 proceeds to step S206.

[0044] When the process proceeds to step S206, the delay monitoring unit 24 creates an operation information table (see FIG. 5(a)), and in step S208, registers operation information data in the operation information table. For example, data is registered in the operation information table as shown in FIG. 9. Note that the operation information table of FIG. 9 registers operation information that user "user-001" left-clicked the mouse at coordinates (865,312) on the screen provided by server "WorkServer-001" at 14:06:16:409 on Feb. 2, 2022. Note that at this stage, the "response ID" column in the operation information table is empty. After step S208, the delay monitoring unit 24 proceeds to the processing of FIG. 10.

[0045] Meanwhile, the server 70 waits until it receives an operation request (step S510). Therefore, when the screen providing unit 22 executes step S124, the server 70 proceeds to step S512 and executes processing in accordance with the operation request. In the next step S514, the server 70 transmits screen change information generated as a result of the processing of step S512 to the web portal 20. After step S514, the server 70 ends the processing of FIG. 8.

[0046] In response to this, after step S124, the screen providing unit 22 waits until it receives screen change information in step S126. Therefore, when the process of step S514 is executed in the server 70, the screen providing unit 22 proceeds to step S128 and transmits the received screen change information to the user terminal 10. As a result, a screen updated in response to the user's operation is displayed on the display unit of the user terminal 10 (step S16 in FIG. 6).

[0047] After the process of Fig. 8, the delay monitoring unit 24 waits until it receives screen change information in step S210 of Fig. 10. Therefore, when the process of step S514 is executed in the server 70, the delay monitoring unit 24 proceeds to step S212.

[0048] When the process proceeds to step S212, the delay monitoring unit 24 creates screen information data from the screen change information. Next, in step S214, the delay monitoring unit 24 registers the screen information data in the screen information table (FIG. 5(b)). As a result, the screen information table becomes as shown in FIG. 11(a). Note that FIG. 11(a) is a screen information table of the screen information data obtained from the screen change information received at 14:06:19:343 on Feb. 2, 2022.

[0049] Next, in step S216, the delay monitoring unit 24 updates the response ID in the operation information table. At this time, there may be multiple operation information tables with empty response IDs. In this case, the delay monitoring unit 24 stores the response ID registered in the screen information table in step S214 in the response ID column of the operation information table with the oldest transmission time among those operation information tables. For example, if the response ID in FIG. 11(a) is registered in the operation information table in FIG. 9, it will look like FIG. 11(b).

[0050] Next, in step S218, the delay monitoring unit 24 calculates the time difference between the timestamps. Specifically, the delay monitoring unit 24 calculates the time difference between the reception time of the screen information table (FIG. 11(a)) and the transmission time of the operation information table (FIG. 11(b)) corresponding to the screen information table (with a matching response ID). In the examples of FIGS. 11(a) and 11(b), the calculated time difference is 2934 ms.

[0051] Next, in step S220, the delay monitoring unit 24 determines whether the time difference exceeds a threshold value. In this case, the delay monitoring unit 24 refers to the characteristic operation table in Fig. 4 and determines whether the calculated time difference exceeds a threshold value (in this example, the threshold value corresponding to a left click (500 ms)).

[0052] If the determination in step S220 is negative, the delay monitoring unit 24 proceeds to step S230. In step S230, the delay monitoring unit 24 determines whether the delay flag is ON. The delay flag is a flag that indicates whether a processing delay is currently occurring between the web portal 20 and the server 70, and if it is ON, it means that a delay is occurring. In the initial stage, the delay flag is OFF. If the delay flag is OFF and the determination in step S232 is negative, all processing in FIG. 10 ends.

[0053] On the other hand, if the determination in step S220 is affirmative, the delay monitoring unit 24 proceeds to step S222. In step S222, the delay monitoring unit 24 determines whether the delay flag is ON. If the determination here is negative, that is, if it is determined that the current processing is delayed even though the processing has not been delayed up to now, the delay monitoring unit 24 proceeds to step S224.

[0054] In step S224, the delay monitoring unit 24 creates a request for video extraction for the delay period. Then, in step S226, the delay monitoring unit 24 transmits the created request for video extraction to the video extracting unit 26. Furthermore, in step S228, the delay monitoring unit 24 changes the delay flag from OFF to ON. After the processing of step S228 is performed, the processing of FIG. 10 ends.

[0055] On the other hand, if the time difference exceeds the threshold (S220: Yes) and the delay flag is ON, it means that the delay is continuing. In this case, since the determination in step S222 is Yes, the delay monitoring unit 24 does not perform any further processing and ends the processing in FIG. 10.

[0056] On the other hand, if the time difference does not exceed the threshold value (S220: No) and the delay flag is ON, it means that the delay that had been continuing until now has been resolved. In this case, the determination in step S230 is Yes, and the delay monitoring unit 24 proceeds to step S232.

[0057] In step S232, the delay monitoring unit 24 creates a request to end recording for the delay period. In step S234, the delay monitoring unit 24 transmits the created request to end recording for the delay period to the video extraction unit 26. In step S236, the delay monitoring unit 24 changes the delay flag from ON to OFF. After the processing of step S236 is performed, the processing of FIG. 10 ends.

[0058] (Processing of the video extraction unit 26 and the recording server 60) Next, the processing of the moving image extracting unit 26 and the recording server 60 will be described with reference to the flowchart of FIG.

[0059] 12 starts, the moving image extraction unit 26 first waits in step S302 until a request for extracting moving images during a delay period is received. That is, when the delay monitoring unit 24 executes the process of step S226 in FIG. 10, the moving image extraction unit 26 proceeds to step S304.

[0060] In step S304, the video extraction unit 26 calculates the time a predetermined time before (for example, three minutes before) the transmission time (timestamp) in the operation information table (FIG. 11(b)). In this embodiment, the time calculated here is set as the start of the delay period. The predetermined time may be any time other than three minutes as long as it is long enough to monitor the delay.

[0061] Next, in step S306, the video extracting unit 26 creates a copy request and a recording start request for the video recording data for the delay period. In step S308, the video extracting unit 26 sends the created copy request and recording request to the recording server 60.

[0062] In step S610, the recording server 60 waits until it receives a copy request and a recording start request. Therefore, once the video extractor 26 has executed step S308, the recording server 60 proceeds to step S612.

[0063] In step S612, the recording server 60 copies the data recorded during the delay period from the normal recording data. That is, the data recorded from the start of the delay period to the present, as indicated by the double-headed arrow in Figure 13(a), is copied from the normal recording data.

[0064] Next, in step S614, the recording server 60 starts recording for the delay period in parallel with the normal recording, that is, adding the newly recorded data to the copied recorded data (second recorded data).

[0065] On the other hand, after step S308, the moving image extraction unit 26 waits in step S310 until it receives a request to end recording for the delay period. Therefore, when the delay monitoring unit 24 executes step S234 in Fig. 10, the moving image extraction unit 26 proceeds to step S312.

[0066] When the process proceeds to step S312, the video extraction unit 26 generates a request to end recording. In step S314, the video extraction unit 26 transmits the request to end recording to the recording server 60. After step S314, the video extraction unit 26 ends the process of FIG. 12.

[0067] In response to this, after step S614, the recording server 60 waits in step S616 until it receives a request to end recording. Therefore, once the video extractor 26 executes step S314, the recording server 60 proceeds to step S618. In step S618, the recording server 60 ends and saves the recording for the delay period. As a result of this process, the recording server 60 is able to save the recorded data of the screen from the start of the delay period until it detects that the delay has ended, as indicated by the dashed arrow in Figure 13(b).

[0068] After the process of step S618, the recording server 60 ends the process of FIG.

[0069] 6, when a request to terminate the remote connection is input from the user to the user terminal 10 (S22: Yes), the request to terminate the remote connection is sent to the web portal 20. Upon receiving the request, the web portal 20 terminates the remote connection between the user terminal 10 and the server 70 and instructs the recording server 60 to end normal recording.

[0070] If a delay occurs when the user terminal 10 is remotely connected to the server 70, the reception of operations (inputs) from the user and the updating of the screen will stop. In such a case, the user cannot immediately determine whether the screen update is due to an operation error (such as a click error) on the user's part or a delay. In such a case, the user may end up repeating the same operation on the screen that is not being updated.

[0071] If the user repeats the same operation in this way, after the delay is resolved, the user terminal 10 will transmit all operations input up to that point to the server 70 at once. For example, suppose a delay occurs while the screen shown in FIG. 14(a) is displayed, and the user repeatedly left-clicks multiple times. Then, suppose the delay is subsequently resolved and the screen shown in FIG. 14(b) is displayed. In this case, operation information indicating that the Skip button in FIG. 14(b) has been clicked is transmitted from the user terminal 10 to the server 70, which may result in a problem because the processing intended by the user is not performed. In this embodiment, the screen during the period in which such a processing delay occurs (delay period) can be recorded. Therefore, by referring to the recorded data during the delay period, the cause of the problem can be easily identified. Furthermore, regardless of the click operation, similar problems may occur due to delays even when entering text or selecting a checkbox. Therefore, by recording the recorded data during the delay period, the cause of the problem can be easily identified.

[0072] As described above in detail, according to this embodiment, the business system 100 includes a recording server 60 that records the screen displayed on the user terminal 10 while the user terminal 10 and the server 70 are remotely connected. The business system 100 also includes a web portal 20 that relays the remote connection between the user terminal 10 and the server 70. The recording server 60 accumulates recording data for a delay period when the time difference between the transmission time of a request (operation information) and the reception time of a response (screen change information) exceeds a threshold. The recording data for the delay period includes data recorded a predetermined time (e.g., three minutes) before the transmission time of the request (operation information) and data recorded subsequent to the data recorded until the delay is resolved. By recording the recording data for the delay period in this way, if a problem occurs due to a communication delay between the server 70 and the web portal 20, the administrator can refer to the recording data for the delay period. This makes it easy to identify the cause of the problem. In addition, in this embodiment, the recording data before the delay is detected is included in the recording data for the delay period, so by checking the recording data, it is possible to trace back to before the delay was detected and investigate the cause of the problem.

[0073] In this embodiment, an operation information table (FIG. 9) is created when a user operation is one of the characteristic operations shown in FIG. 4. This allows the operation information table to be created by limiting it to operations that can be used to determine whether a communication delay has occurred, thereby preventing the creation of an excessively large operation information table. Furthermore, the amount of processing can be reduced compared to when determining whether a delay has occurred every time a user operation is performed.

[0074] In the above embodiment, as an example, the web portal 20 is described as being configured from one information processing device, but as mentioned above, it may include multiple information processing devices. In this case, each of the multiple information processing devices may have the screen providing unit 22, delay monitoring unit 24, and video extracting unit 26 of Fig. 3. Furthermore, the functions of the screen providing unit 22, delay monitoring unit 24, and video extracting unit 26 of Fig. 3 may be provided by separate information processing devices.

[0075] The server 70 in the above embodiment may be another information processing device such as an office PC to which the user terminal 10 can remotely connect.

[0076] The above processing functions can be realized by a computer. In this case, a program is provided that describes the processing contents of the functions that the processing device should have. By executing the program on a computer, the above processing functions are realized on the computer. The program that describes the processing contents can be recorded on a computer-readable storage medium (excluding carrier waves).

[0077] When distributing a program, it is sold in the form of a portable storage medium on which the program is recorded, such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory).The program can also be stored in the storage device of a server computer and transferred from the server computer to other computers via a network.

[0078] A computer that executes a program stores, for example, a program recorded on a portable storage medium or a program transferred from a server computer in its own storage device. The computer then reads the program from its own storage device and executes processing in accordance with the program. Note that the computer can also read the program directly from a portable storage medium and execute processing in accordance with that program. The computer can also execute processing in accordance with the program received each time a program is transferred from the server computer.

[0079] The above-described embodiment is a preferred example of the present invention, but the present invention is not limited to this and can be modified in various ways without departing from the spirit of the present invention.

[0080] In addition, the following supplementary notes are disclosed in relation to the above-described embodiments. (Supplementary Note 1) A relay system that relays between a terminal and a processing device that executes processing in response to a request from the terminal and transmits a response to the terminal, a recording device that records an operation screen displayed on the terminal while the terminal and the processing device are connected and stores first recorded data; a transmitting / receiving device that, when receiving a request from the terminal to the processing device, transmits the request to the processing device, and, when receiving a response corresponding to the request from the processing device, transmits the response to the terminal; The recording device is A relay system characterized in that, when the time difference between the time when the transmitting / receiving device sends a request and the time when the transmitting / receiving device receives a response corresponding to the request exceeds a reference value, second recorded data is generated by duplicating recorded data from the first recorded data that was recorded after a predetermined time before the time when the request was sent, and recorded data obtained by recording the operation screen while the time difference exceeds the reference value is added to the second recorded data. (Appendix 2) The relay system described in Appendix 1 is characterized in that the recording device generates the second recording data when the time difference between the time when a request of a predetermined type is sent and the time when a response corresponding to the request is received exceeds a reference value. (Supplementary Note 3) The relay system according to Supplementary Note 2, wherein the predetermined type of request is a request that causes a transition of the operation screen. (Supplementary Note 4) The relay system according to any one of Supplementary Notes 1 to 3, wherein the reference value is set for each content of the request. (Supplementary Note 5) A relay system that relays between a terminal and a processing device that executes processing in response to a request from the terminal and transmits a response to the terminal, While the terminal and the processing device are connected, an operation screen displayed on the terminal is recorded and first recorded data is accumulated; When the time difference between the time when the request is transmitted from the terminal and the time when a response corresponding to the request is received from the processing device exceeds a reference value, second recorded data is generated by duplicating the first recorded data recorded after a predetermined time before the time when the request is transmitted, and the second recorded data is added with recorded data obtained by recording the operation screen while the time difference exceeds the reference value. 2. An information recording method comprising: (Appendix 6) An information recording method as described in Appendix 5, characterized in that the second recording data is generated when the time difference between the time when a request of a predetermined type is sent and the time when a response corresponding to the request is received exceeds a reference value. (Supplementary Note 7) The information recording method according to Supplementary Note 6, wherein the predetermined type of request is a request that causes a transition of the operation screen. (Supplementary Note 8) The information recording method according to any one of Supplementary Notes 5 to 7, wherein the reference value is set for each content of the request. (Supplementary Note 9) A transmitting / receiving device that transmits and receives a request and a response between a terminal and a processing device that executes a process in response to a request from the terminal and transmits a response to the terminal, While the terminal and the processing device are connected, an instruction is issued to a recording device connected to the transmitting / receiving device, an operation screen displayed on the terminal is recorded, and first recording data is accumulated; when the time difference between the time when the request is transmitted from the terminal and the time when a response corresponding to the request is received from the processing device exceeds a reference value, an instruction is issued to the recording device to generate second recorded data by duplicating the first recorded data recorded after a predetermined time before the time when the request is transmitted, and to add to the second recorded data the recorded data obtained by recording the operation screen while the time difference exceeds the reference value; An information recording program that causes a process to be executed. [Explanation of symbols]

[0081] 10 User terminal (terminal) 20 web portal (transmitting / receiving device) 50 Relay System 60 Recording Server (Recording Device) 70 Server (processing device)

Claims

1. A relay system that relays between a terminal and a processing device that executes processing in response to a request from the terminal and transmits a response to the terminal, a recording device that records an operation screen displayed on the terminal and stores first recording data while the terminal and the processing device are connected; a transmitting / receiving device that, when receiving a request from the terminal to the processing device, transmits the request to the processing device, and, when receiving a response corresponding to the request from the processing device, transmits the response to the terminal; The recording device is A relay system characterized in that, when the time difference between the time when the transmitting / receiving device sends a request and the time when the transmitting / receiving device receives a response corresponding to the request exceeds a reference value, second recording data is generated by duplicating recording data from the first recording data that was recorded after a predetermined time before the time when the request was sent, and recording data obtained by recording the operation screen while the time difference exceeds the reference value is added to the second recording data.

2. 2. The relay system according to claim 1, wherein the recording device generates the second recording data when a time difference between a time when a request of a predetermined type is transmitted and a time when a response corresponding to the request is received exceeds a reference value.

3. 3. The relay system according to claim 2, wherein the predetermined type of request is a request that causes a transition of the operation screen.

4. 4. The relay system according to claim 1, wherein the reference value is set for each content of the request.

5. a relay system that relays between a terminal and a processing device that executes processing in response to a request from the terminal and transmits a response to the terminal; While the terminal and the processing device are connected, an operation screen displayed on the terminal is recorded and first recorded data is stored; When the time difference between the time when a request is transmitted from the terminal and the time when a response corresponding to the request is received from the processing device exceeds a reference value, second recorded data is generated by duplicating the first recorded data recorded after a predetermined time before the time when the request is transmitted, and the recorded data obtained by recording the operation screen while the time difference exceeds the reference value is added to the second recorded data.

2. An information recording method comprising:

6. a transmitting / receiving device that transmits and receives the request and the response between a terminal and a processing device that executes processing in response to a request from the terminal and transmits a response to the terminal; While the terminal and the processing device are connected, an instruction is issued to a recording device connected to the transmitting / receiving device, an operation screen displayed on the terminal is recorded, and first recording data is accumulated; when the time difference between the time when a request is transmitted from the terminal and the time when a response corresponding to the request is received from the processing device exceeds a reference value, an instruction is issued to the recording device to generate second recorded data by duplicating the first recorded data recorded after a predetermined time before the time when the request is transmitted, and to add to the second recorded data the recorded data obtained by recording the operation screen while the time difference exceeds the reference value; An information recording program that causes a process to be executed.

Images