Information processing device, screen sharing system, screen sharing method, and computer program

Abstract

To provide a screen sharing method capable of more accurately implementing web page screen sharing between browsers of two terminals.SOLUTION: A screen sharing method for displaying a web page, which is displayed on a browser of a second terminal, on a browser of a first terminal includes: storing, in a storage section, a header information ID associated with header information to be used when the browser of the second terminal displaying the web page performs communication with the outside; when request data including the header information ID and location information showing locations of asset files constituting the web page transmitted from the browser of the first terminal is received, acquiring the header information corresponding to the header information ID included in the request data from the storage section; acquiring the asset files from an external server using asset request data including the location information included in the received request data and the header information acquired from the storage section; and transmitting the acquired asset files to the first terminal.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a screen sharing system, a screen sharing method, and a computer program. [Background technology]

[0002] There is a screen sharing system that enables visual promotion of information communication between operators and end users by sharing a screen displayed on an end user browser via an operator browser (for example, Patent Document 1). The screen sharing system comprises an end user browser that displays a web page, an operator browser, and a relay server connected to each browser via a communication network. The end user browser and operator browser send and receive HTML structure data of the web page via the relay server, and the operator browser realizes screen sharing by reconstructing the web page displayed on the end user browser. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-177403 Summary of the Invention [Problem to be solved by the invention]

[0004] However, due to differences in the display environment of web pages, there is a problem in that simply transmitting and receiving and sharing the HTML structure data of a web page results in differences in the layout of the web page displayed on the shared screen of the operator terminal and the end user terminal.

[0005] An object of the present invention is to provide an information processing device, a screen sharing system, a screen sharing method, and a computer program that can realize more accurate screen sharing of a web page between browsers on two terminals. [Means for solving the problem]

[0006] An information processing device according to one embodiment of the present invention includes a communication unit that communicates between a browser of a first terminal and a browser of a second terminal; a processing unit that executes a process related to screen sharing, which displays a web page displayed on the browser of the second terminal on the browser of the first terminal; and a memory unit that stores header information used by the browser of the second terminal displaying the web page when communicating with an external device, in association with a header information ID. When the communication unit receives request data sent from the browser of the first terminal, the request data including location information indicating the location of an asset file constituting the web page and the header information ID, the processing unit retrieves the header information corresponding to the header information ID included in the request data from the memory unit, and generates asset request data including the location information included in the received request data and the header information retrieved from the memory unit. The communication unit uses the generated asset request data to request the asset file from an external server, receives the asset file sent from the external server in response to the request, and transmits the received asset file to the first terminal.

[0007] A screen sharing system according to one embodiment of the present invention is a screen sharing system comprising the information processing device and a first terminal and a second terminal connected to the information processing device via a communication network, wherein the second terminal comprises a second terminal processing unit that transmits data of the web page displayed in the browser of the second terminal to the first terminal via the information processing device, the first terminal comprises a first terminal processing unit that receives the data of the web page transmitted from the second terminal and reconstructs and displays the web page based on the received data, and when an element constituting the web page in the received data includes location information indicating the location of the asset file as an attribute value, the first terminal processing unit replaces the attribute value with a string including a domain name associated with the information processing device, the location information, and the header information ID.

[0008] A screen sharing method according to one embodiment of the present invention is a screen sharing method for displaying a web page displayed in a browser of a second terminal on a browser of a first terminal, the method including: storing in a memory unit, in association with header information used by the browser of the second terminal displaying the web page when communicating with the outside, a header information ID; upon receiving request data transmitted from the browser of the first terminal, the method including location information indicating the location of an asset file constituting the web page and the header information ID, the method obtains from the memory unit the header information corresponding to the header information ID contained in the request data; generating asset request data including the location information contained in the received request data and the header information obtained from the memory unit; using the generated asset request data, requesting the asset file from an external server; receiving the asset file transmitted from the external server in response to the request; and transmitting the received asset file to the first terminal.

[0009] A computer program according to one embodiment of the present invention is a computer program for causing a computer to execute a process related to screen sharing in which a web page displayed in a browser of a second terminal is displayed in a browser of a first terminal. The computer program stores in a memory unit header information used by the browser of the second terminal displaying the web page when communicating with the outside world, in association with a header information ID; when request data transmitted from the browser of the first terminal includes location information indicating the location of an asset file constituting the web page and the header information ID, the computer retrieves from the memory unit the header information corresponding to the header information ID included in the request data; generates asset request data including the location information included in the received request data and the header information retrieved from the memory unit; uses the generated asset request data to request the asset file from an external server; receives the asset file transmitted from the external server in response to the request; and transmits the received asset file to the first terminal. [Effects of the Invention]

[0010] Based on the above, it is possible to provide an information processing device, a screen sharing system, a screen sharing method, and a computer program that can realize more accurate screen sharing of a web page between browsers of two terminals. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of the configuration of a screen sharing system according to an embodiment. [Figure 2] 1 is a block diagram illustrating an example of the configuration of a screen sharing system according to an embodiment. [Figure 3] FIG. 1 is a functional block diagram of a screen sharing system according to an embodiment. [Figure 4] FIG. 10 is a sequence diagram illustrating a pre-processing procedure for starting a screen sharing process according to the embodiment. [Figure 5] FIG. 1 is a conceptual diagram illustrating a method for processing a document type declaration. [Figure 6] FIG. 10 is a sequence diagram showing a processing procedure for screen sharing according to the embodiment. [Figure 7] 10 is a flowchart showing a file processing procedure for an asset file. [Figure 8] 10 is a flowchart showing a file processing procedure for an asset file. [Figure 9] FIG. 10 is a conceptual diagram illustrating a file processing method for asset files. [Figure 10] FIG. 10 is an explanatory diagram showing a specific example of an attribute of a specific element. DETAILED DESCRIPTION OF THE INVENTION

[0012] An information processing device, a screen sharing system, a screen sharing method, and a computer program according to embodiments of the present disclosure will be described below with reference to the drawings. Note that the present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. Furthermore, at least some of the embodiments described below may be combined in any manner.

[0013] 1 is a schematic diagram illustrating an example of the configuration of a screen sharing system according to an embodiment. The screen sharing system according to this embodiment includes an information processing device 1, an agent terminal (first terminal) 2, a guest terminal (second terminal) 3, and a web server (external server) 4. The various terminals and servers are connected to the information processing device 1 via a communication network N.

[0014] A user (hereinafter referred to as an agent) who operates the agent terminal 2 is a person who guides a user (hereinafter referred to as a guest) who operates the guest terminal 3, for example, on how to operate a website. Note that the agent and the guest are examples of users of the information processing device 1 according to this embodiment, and the relationship between them is not particularly limited.

[0015] 2 is a block diagram showing an example of the configuration of a screen sharing system according to an embodiment. The information processing device 1 is a computer including a processing unit 11, a storage unit 12, and a communication unit 13. The information processing device 1 may be configured to perform distributed processing using multiple computers, may be realized by multiple virtual machines provided in a single server, may be realized using a cloud server, or may be partially configured by a quantum computer.

[0016] The processing unit 11 has one or more arithmetic processing devices such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), etc., and performs the screen sharing processing according to this embodiment by reading and executing a computer program P1 stored in the memory unit 12.

[0017] The storage unit 12 is a storage device such as a hard disk, an EEPROM (Electrically Erasable Programmable Read-Only Memory), or a flash memory. The storage unit 12 stores a computer program P1, an agent program P2, and a guest program P3 required for the processing unit 11 to share the contents of web pages displayed on the respective web browsers between the guest terminal 3 and the agent terminal 2 and the operation details. The agent program P2 and the guest program P3 are programs written in, for example, JavaScript (registered trademark), and the information processing device 1 transmits the agent program P2 to the agent terminal 2 and the guest program P3 to the guest terminal 3. The web browser running on the agent terminal 2 (hereinafter referred to as the agent browser B2) executes the agent program P2 to perform the processing required for screen sharing. The web browser running on the guest terminal 3 (hereinafter referred to as the guest browser B3) executes the guest program P3 to perform the processing required for screen sharing. The various programs may be recorded in a computer-readable manner on a recording medium 5. The storage unit 12 stores the computer program P1, the agent program P2, and the guest program P3 read from the recording medium 5 by a reading device (not shown). The recording medium 5 is a semiconductor memory such as a flash memory, an optical disk, a magnetic disk, a magneto-optical disk, or the like. Alternatively, the various programs according to this embodiment may be downloaded from an external server (not shown) connected to the communication network N and stored in the storage unit 12.

[0018] The communication unit 13 includes a processing circuit, a communication circuit, and the like for performing processing related to communication, and transmits and receives information between the agent terminal 2, the guest terminal 3, and the web server 4.

[0019] The agent terminal 2 is a computer equipped with a processing unit (first terminal processing unit) 21, a storage unit 22, a communication unit 23, an operation unit 24, and a display unit 25. The basic configurations of the processing unit 21, the storage unit 22, and the communication unit 23 are the same as those of the information processing device 1. The storage unit 22 stores at least a program related to an agent browser B2. The agent browser B2 implements DOM (Document Object Model), a program that references and controls elements that make up a web page, and can execute an agent program P2 transmitted from the information processing device 1. The operation unit 24 is a keyboard, a mouse, etc., and accepts operation and input of the agent. The display unit 25 is a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display, and displays information provided by the processing unit 21.

[0020] The guest terminal 3 is a computer equipped with a processing unit (second terminal processing unit) 31, a memory unit 32, a communication unit 33, an operation unit 34, and a display unit 35. The basic configurations of the processing unit 31, the memory unit 32, and the communication unit 33 are the same as those of the information processing device 1. The memory unit 32 stores at least a program related to the guest browser B3. The guest browser B3 implements DOM, which is a program that references and controls elements that make up a web page, and can execute the guest program P3 transmitted from the information processing device 1. The operation unit 34 is a keyboard, a mouse, etc., and accepts operations and inputs from the guest. The display unit 35 is a display device such as a liquid crystal display or an organic EL display, and displays information provided by the processing unit 31.

[0021] The web server 4 is a server that transmits a document file of a web page and an asset file that constitutes the web page to the guest browser B3 in response to at least a request from the guest browser B3. The web server 4 is also a server that transmits the asset file that constitutes the web page to the information processing device 1 in response to at least a request from the information processing device 1. In this embodiment, the guest browser B3 accesses the web server 4 and displays the web page.

[0022] 3 is a functional block diagram of a screen sharing system according to an embodiment. The agent terminal 2 has an agent browser B2 as a functional unit, and can execute an agent program P2 transmitted from the information processing device 1. The agent program P2 is transmitted from the information processing device 1 to the agent terminal 2 when the agent browser B2 accesses the information processing device 1 to read the management screen. The management screen is a screen for executing screen sharing processing of a web page.

[0023] The guest terminal 3 has a guest browser B3 as a functional unit, and can execute a guest program P3 transmitted from the information processing device 1. The guest program P3 is transmitted from the information processing device 1 to the guest terminal 3 when reading a web page to be shared on the screen.

[0024] The information processing device 1 includes an API processing unit 11a, a socket processing unit 11b, and a file processing unit 11c. The information processing device 1 also includes an account management database 12a, a screen sharing management database 12b, a screen sharing data relay store 12c, a program file storage 12d, and an asset file storage 12e.

[0025] The API processing unit 11a is a functional unit that acquires various setting data for the screen sharing system from the account management database 12a and executes processing such as settings related to screen sharing processing.

[0026] The socket processing unit 11b is a functional unit that establishes communication with the agent browser B2 and the guest browser B3 via WebSocket, enabling two-way communication. The socket processing unit 11b executes processes such as issuing a session ID (header information ID) for identifying a session of the screen sharing process, and transmitting and receiving various data between the agent browser B2 and the guest browser B3 during screen sharing. Note that WebSocket is an example of a communication connection method, and the socket processing unit 11b may be configured to realize pseudo two-way communication between the agent browser B2 and the guest browser B3 by long polling.

[0027] The file processing unit 11c is a functional unit that executes a process of requesting and acquiring asset files that constitute a web page to be screen-shared from the web server 4 and temporarily storing them. The asset files include, for example, CSS (Cascading Style Sheets) files, image files, font files, etc.

[0028] The account management database 12a is a database that stores various setting data for the screen sharing system. The setting data specifies items on a web page that cannot be shared, specifies the layout of various buttons displayed on the web page, and so on. The screen sharing management database 12b is a database that stores session IDs and the like for identifying sessions of screen sharing processing. The screen sharing data relay store 12c is a relay server that relays various data transmitted and received between the agent browser B2 and the guest browser B3 during screen sharing. The program file storage 12d is a storage for storing HTML files, CSS files, image files, the agent program P2, the guest program P3, and the like that constitute the management screen displayed on the agent browser B2. The asset file storage 12e is a storage that temporarily stores session information used for file processing of asset files that constitute a web page to be screen-shared, acquired asset files, and the like.

[0029] FIG. 4 is a sequence diagram showing a pre-processing procedure for starting a screen sharing process according to the embodiment, and FIG. 5 is a conceptual diagram showing a method for processing a document type declaration. The socket processing unit 11b of the information processing device 1 establishes communication between the agent terminal 2 and the guest terminal 3 via a WebSocket (step S11). The procedure for establishing communication is not particularly limited. The agent terminal 2 and the guest terminal 3 are able to perform two-way communication with the information processing device 1. Furthermore, the agent terminal 2 and the guest terminal 3 are able to send and receive data required for sharing a web page screen via the information processing device 1. The information processing device 1 issues a session ID when starting the screen sharing process and transmits the issued session ID to the agent terminal 2 and the guest terminal 3 at an appropriate timing. The agent terminal 2 and the guest terminal 3 receive and store the session ID. A series of processes related to screen sharing between the agent terminal 2 and the guest terminal 3 is managed by the session ID. Here, it is assumed that the guest browser B3 is accessing the web server 4 and displaying a web page. Note that steps S11 to S23 shown in Fig. 4 and steps S24 to S32 shown in Fig. 6 (described later) are repeatedly processed each time the guest browser B3 updates the web page being displayed or transitions to a different web page until the session of the screen sharing process ends.

[0030] The execution entity of the screen sharing process on the agent side can be the agent terminal 2, the processing unit 21, or the agent browser B2, but in the following explanation, it will be assumed that the agent browser B2 that executes the agent program P2 is the execution entity. Similarly, the execution entity of the screen sharing process on the guest side can be the guest terminal 3, the processing unit 31, or the guest browser B3, but in the following explanation, it will be assumed that the guest browser B3 that executes the guest program P3 is the execution entity.

[0031] <Memorizing session information> Next, the guest browser B3 transmits the session ID and the session information to the information processing device 1 (step S12). The session information includes request header information (header information), web page information, and the like. The request header information includes information such as the origin, referrer, and user agent used when the guest browser B3 communicates with the web server 4. The origin includes, for example, information indicating the scheme (protocol), host (domain), and port used in the URL of the web page displayed by the guest browser B3. The referrer includes, for example, information indicating the URL of the web page displayed by the guest browser B3. The user agent is, for example, information related to the guest browser B3. The web page information includes information such as document type declaration information, title, and URL of the web page displayed by the guest browser B3.

[0032] When the file processing unit 11c of the information processing device 1 receives the session ID and session information transmitted from the guest browser B3, the file processing unit 11c associates the received session ID with the session information and stores them in the asset file storage 12e (step S13). Note that in this embodiment, an example will be described in which session information including request header information is associated with the session ID, but the session information may be stored in association with any other ID. The request header information may also be stored in association with any ID. There are no particular limitations on the management and storage methods of the request header information and web page information. Next, the file processing unit 11c transmits session storage result information to the agent browser B2 (step S14). The session storage result information includes information such as the success or failure of the processing of step S13.

[0033] <Document Type Declaration> The agent browser B2 replaces the URL of the screen sharing display area on the management screen (step S15). The screen sharing display area is an element on the management screen for reconstructing and displaying the web page to be shared, such as an iframe element. The URL of the screen sharing display area converted in step S15 has the same domain name as the domain name of the file processing unit 11c of the information processing device 1, and includes a session ID as part of parameters such as a path and a query string.

[0034] Specifically, the agent browser B2 accesses the information processing device 1 using the URL of the screen sharing display area converted in step S15, and requests a screen sharing display template file (step S16). The screen sharing display template file is a document file, such as an HTML file, that serves as a template for the screen sharing display area.

[0035] The file processing unit 11c of the information processing device 1 acquires a session ID from a parameter of the URL requested by the agent browser B2 (step S17). The file processing unit 11c also acquires a screen sharing display template file from the program file storage 12d (step S18). The file processing unit 11c acquires session information from the asset file storage 12e using the session ID acquired in step S17 as a key, and determines whether document type declaration information exists in the web page information included in the session information (step S19). If it is determined that document type declaration information exists (step S19: YES), the file processing unit 11c acquires the document type declaration information whose existence has been confirmed (step S20), sets the acquired document type declaration information in the screen sharing display template file acquired in step S18 (step S21), and sends the screen sharing display template file with the document type declaration information added to the agent browser B2 as a response (step S22). If it is determined in step S19 that the document type declaration information does not exist (step S19: NO), the file processing unit 11c transmits the screen sharing display template file acquired in step S18 to the agent browser B2 as a response (step S22).

[0036] The agent browser B2 reads the shared screen display template file transmitted from the information processing device 1 and updates the shared screen display area (step S23).

[0037] <Screen sharing and operation sharing> The agent browser B2 and the guest browser B3 transmit and receive information for sharing the contents and operation contents of the web page to be screen-shared via the information processing device 1. For example, the agent browser B2 and the guest browser B3 transmit and receive page size data, HTML data, agent operation data, guest operation data, etc. via the information processing device 1. The agent browser B2 and the guest browser B3 may generate a common key for encrypting and decrypting information in advance, and use the generated common key to encrypt information before transmission and to decrypt information after reception.

[0038] The page size data is data that includes information indicating the height and width of the display area on the screen of the web page to be shared. The HTML data is information indicating the HTML structure of the webpage to be screen-shared, and is nested data generated in JSON (JavaScript (registered trademark) Object Notation) format. Specifically, it is information about the nodes that make up the webpage, including a unique identifier for identifying the node (hereinafter referred to as nodeID), the node type (hereinafter referred to as nodeType), the element name (hereinafter referred to as tagName), the property value (hereinafter referred to as property), and child node information (hereinafter referred to as childNodes). The nodeID is a unique identifier assigned to each node that makes up the webpage. The nodeType is an integer code that indicates the type of node, such as "1" for an element node (ELEMENT_NODE), "3" for a text node (TEXT_NODE), and "8" for a comment node (COMMENT_NODE). The tagName is the name of the node's element, and is present only when the nodeType is "1." For example, a tagName is represented as "P" for an element representing a paragraph, or "IMG" for an element representing an image. In addition to attributes defined in HTML such as id and class, property has different data depending on the type of tagName. For example, if tagName is "INPUT", there is the input value (value) and check status (checked). Therefore, it is necessary to predefine the property to be obtained depending on the value of tagName. childNodes is information about the nodes that are children of the node in question, and is repeated as long as there are child nodes. Agent operation data and guest operation data, i.e., operation data, are data containing information on operation events that can be realized through a web browser, such as operations using a mouse or touch screen (clicking, touching, hovering, swiping, scrolling, cursor movement, etc.) and operations using a keyboard (key input, etc.). The operation data consists of the name of the operation event and the content of the operation event. For example, in the case of a mouse cursor movement operation event, the content of the operation event includes information indicating the coordinates of the mouse cursor, and in the case of a scroll operation event, the content includes information indicating the amount of scrolling of the web page.

[0039] FIG. 6 is a sequence diagram showing a processing procedure for screen sharing according to an embodiment. The guest browser B3 creates initial screen sharing data based on the web page displayed on the guest browser B3 and transmits it to the agent browser B2 via the information processing device 1 (step S24). The initial screen sharing data includes page size data, HTML data, guest operation data, etc. The page size data included in the initial screen sharing data includes information indicating the height and width of the display area on the screen of the web page at the time of data creation. The HTML data included in the initial screen sharing data includes information (all) of the nodes constituting the web page at the time of data creation. The guest operation data included in the initial screen sharing data includes data similar to that when a mouse cursor movement operation event occurs, i.e., information indicating the coordinates of the guest's mouse cursor at the time of data creation, and also includes data similar to that when a scroll operation event occurs, i.e., information indicating the amount of scrolling of the web page at the time of data creation. When the agent browser B2 receives the initial screen sharing data transmitted from the guest browser B3, it analyzes the received initial screen sharing data, reconstructs a web page similar to the one displayed on the guest browser B3 in the screen sharing display area, and reproduces the operation contents of the guest (step S25). Specifically, the agent browser B2 sets the height and width of the screen sharing display area to values ​​indicated by the page size data. The agent browser B2 also creates and adds a node to the screen sharing display area based on the HTML data. Furthermore, the agent browser B2 sets the position of the mouse cursor element, which indicates the coordinates of the guest's mouse cursor and which has been added to the screen sharing display area, to a value indicating the coordinates of the mouse cursor included in the guest operation data. Furthermore, the agent browser B2 sets the scroll amount of the screen sharing display area to a value indicating the scroll amount of the web page included in the guest operation data.

[0040] After step S24, the guest browser B3 monitors the height and width of the display area on the screen of the web page, and whenever a change is detected, it creates page size data. The guest browser B3 also monitors the HTML structure of the web page, and whenever a change is detected, it creates HTML data (either the difference from before the change or the entire data). The guest browser B3 also monitors guest operations, and whenever an operation is detected, it creates guest operation data. Note that there are cases where the HTML structure of the web page is rewritten as a result of an operation such as a click by the guest, in which case both guest operation data and HTML data (the difference from before the operation) are created. The guest browser B3 transmits the created page size data, HTML data, guest operation data, etc. to the agent browser B2 via the information processing device 1 whenever (step S26). When the agent browser B2 receives each piece of data transmitted from the guest browser B3, it analyzes the received data, reflects the same changes as those in the web page displayed on the guest browser B3 in the screen sharing display area, and reproduces the operation contents of the guest (step S27). Specifically, the agent browser B2 sets the height and width of the screen sharing display area to the values ​​indicated by the page size data. The agent browser B2 also makes changes to the nodes on the screen sharing display area based on the HTML data. Furthermore, the agent browser B2 reproduces operation events similar to the guest operation in the screen sharing display area based on the guest operation data.

[0041] Similarly, from step S25 onwards, the agent browser B2 monitors the agent's operations, and whenever an operation is detected, it creates agent operation data. Note that there are cases where the HTML structure of the web page reconstructed in the screen sharing display area is rewritten as a result of the agent performing an operation such as a click, in which case both the agent operation data and HTML data (the difference from before the operation) are created. The agent browser B2 transmits the created HTML data, agent operation data, etc. to the guest browser B3 via the information processing device 1 each time (step S28). When the guest browser B3 receives each data transmitted from the agent browser B2, it analyzes the received data, reflects the same changes as those in the screen sharing display area on the web page, and reproduces the operation of the agent (step S29). Specifically, the guest browser B3 makes changes to the nodes on the web page based on the HTML data. Furthermore, the guest browser B3 reproduces operation events similar to the agent operation on the web page based on the agent operation data.

[0042] In steps S25 and S27, if the HTML data received by the agent browser B2 includes information on adding or changing a specific element node that includes an asset file URL as part of its attributes, or information on adding or changing a specific text node that includes an asset file URL as part of its text content (textContent), the agent browser B2 replaces the asset file URL included in part of the attribute or part of the text content with an asset file request URL. Text content is a property of a node. Note that if multiple asset file URLs are included in part of the attribute or part of the text content, all asset file URLs are replaced with the asset file request URL. When reflecting the contents of the converted HTML data in the screen sharing display area, the agent browser B2 accesses the information processing device 1 using the asset file request URL and requests an asset file (step S30). When the information processing device 1 receives a request for an asset file from the agent browser B2, it executes file processing of the asset file and transmits the acquired asset file to the agent browser B2 (step S31). The agent browser B2 expands the asset file transmitted from the information processing device 1 as part of the attributes of the element node on the screen sharing display area or as part of the text content of the text node (step S32). Details of the file processing of the asset file will be described later.

[0043] <File processing of asset files> 7 and 8 are flowcharts showing the file processing procedure for an asset file, and FIG. 9 is a conceptual diagram showing a file processing method for an asset file. The following describes the details of the processing in steps S30, S31, and S32 shown in FIG. 6. The agent browser B2 replaces the asset file URL included in part of the attributes of a specific element node with an asset file request URL (step S51). The specific element node is, for example, an audio element, an img element, an input element, a link element, a source element, a video element, or various elements that hold a style attribute. The attribute that includes the asset file URL is, for example, the src attribute or srcset attribute of an img element, or the href attribute of a link element. Similarly, the agent browser B2 replaces the asset file URL included in part of the text content of a specific text node with the asset file request URL (step S51). The specific text node is a text node that has a specific element node as its parent, and the specific element node is, for example, a style element. Furthermore, if the parent is a text node of a style element, the URL of the asset file included in the text content is, for example, a character string written as an argument of the url function, which is one of the CSS functions. The asset file request URL has the same domain name as the domain name of the file processing unit 11c of the information processing device 1, and includes a session ID and an original URL as part of parameters such as a path and a query string. The original URL is a URL included as part of the attribute of an element node of the HTML data sent from the guest browser B3, or a URL included as part of the text content of a text node of the HTML data sent from the guest browser B3. If the URL is relative, it is converted to an absolute URL before being used as a parameter. Note that instead of using the character string of the original URL as a parameter, the agent browser B2 may encode the character string of the original URL and use the resulting string as a replacement.

[0044] Then, the agent browser B2 accesses the information processing device 1 using the asset file request URL and requests the asset file (step S52).

[0045] The file processing unit 11c of the information processing device 1 acquires the session ID and the original URL from the parameters of the URL requested by the agent browser B2 (step S53). If the character string of the original URL is encoded, the character string of the original URL is decoded.

[0046] The file processing unit 11c of the information processing device 1 determines whether the asset file indicated by the acquired original URL has been saved, i.e., whether the asset file storage 12e stores the asset file (step S54). If it determines that the asset file has not been saved (step S54: NO), the file processing unit 11c acquires session information from the asset file storage 12e using the acquired session ID as a key, and acquires request header information included in the session information (step S55). Next, the file processing unit 11c creates asset file request data (asset request data) for acquiring the asset file, sets the original URL as the request URL constituting the asset file request data (step S56), and sets the request header information acquired in step S55 as the request header (step S57). Then, the file processing unit 11c accesses the web server 4 using the asset file request data and requests the asset file (step S58). The web server 4 transmits the target asset file corresponding to the asset file request data to the information processing device 1 as a response.

[0047] The file processing unit 11c of the information processing device 1 acquires the asset file transmitted from the web server 4 (step S59). Next, the file processing unit 11c determines whether the acquired asset file is a CSS file (step S60). If it is determined that the file is a CSS file (step S60: YES), the file processing unit 11c replaces the URL of the external asset file included in part of the text in the CSS file with an asset file request URL (step S61). The external asset file is an external CSS file, image file, font file, etc. that is read from the CSS file, and the URL of the external asset file is, for example, a character string described as an argument of the url function, which is one of the CSS functions. The asset file request URL here has the same domain name as the domain name of the file processing unit 11c of the information processing device 1, and includes a session ID and the URL of the external asset file as part of parameters such as a path and a query string. If the text in the CSS file contains multiple URLs of external asset files, all of the URLs of the external asset files are replaced with the asset file request URL. Note that, when replacing, if the URL of the external asset file is a relative URL, it is converted into an absolute URL before being used as a parameter. Note that, instead of using the character string of the URL of the external asset file as a parameter, the file processing unit 11c may encode the character string of the URL of the external asset file and use the replaced character string. The file processing unit 11c temporarily stores the CSS file, whose text has been converted in step S61, in the asset file storage 12e (step S62). The storage period is, for example, one day. If it is determined in step S60 that the file is not a CSS file (step S60: NO), the file processing unit 11c temporarily stores the asset file acquired in step S59 in the asset file storage 12e (step S62). The storage period is the same as for a CSS file. The file processing unit 11c stores the asset file in the asset file storage 12e in association with the original URL, for example.

[0048] The file processing unit 11c of the information processing device 1 transmits the asset file stored in the asset file storage 12e to the agent browser B2 (step S63). If it is determined in step S54 that the asset file has been stored (step S54: YES), the file processing unit 11c acquires the asset file corresponding to the original URL from the asset file storage 12e and transmits the acquired asset file to the agent browser B2 (step S63).

[0049] The agent browser B2 receives the asset file transmitted from the file processing unit 11c of the information processing device 1 (step S64). Furthermore, if the received asset file is a CSS file and part of the text in the CSS file contains an asset file request URL (step S65: YES), the agent browser B2 accesses the information processing device 1 using the asset file request URL, as in step S52, and requests the asset file (step S66). If the received asset file is not a CSS file, or if the asset file request URL is not included in the CSS file (step S65: NO), the file processing of the asset file is terminated.

[0050] Figure 10 is an explanatory diagram showing a specific example of the attributes of a specific element. Figure 10A is an explanatory diagram showing the HTML of a web page displayed by guest browser B3. The HTML includes an img element, and the original URL of the asset file is set as the src attribute.

[0051] 10B is an explanatory diagram showing HTML in the screen sharing display area when agent browser B2 executes screen sharing processing for the web page shown in FIG. 10A. The HTML includes an img element in which the URL of the src attribute has been replaced with the asset file request URL in the processing of step S51. As described above in step S51, the asset file request URL has the domain name of the file processing unit 11c of the information processing device 1 set as the domain name, and the session ID and original URL set as the query string. Note that reserved characters such as slash " / " and colon ":" in the string of the original URL have been percent-encoded.

[0052] 10C is an explanatory diagram showing HTML in the screen sharing display area when the agent browser B2 executes the screen sharing process for the web page shown in FIG. 10A, similar to FIG. 10B, but differs in that the URL of the src attribute of the img element is replaced using a different method from that of the present embodiment. Specifically, similar to FIG. 10B, the domain name is set to the domain name of the file processing unit 11c of the information processing device 1, and the original URL is set as the query string. However, similar to FIG. 10B, the query string contains request header information of the guest browser B3 instead of a session ID. When replaced using this method, the URL of the src attribute is significantly longer than the example shown in FIG. 10B. 10B and 10C, according to this embodiment, it is possible to shorten the URL required for file processing of the asset file. The technical effects will be described in detail later.

[0053] <Action and effect> In the prior art, which simply shares HTML structure data, the original URL of the asset file is accessed directly from the agent browser B2 without going through the information processing device 1. Therefore, for an asset file subject to cross-origin request control, the domain name of the URL of the asset file and the domain name of the URL of the screen-sharing display area are generally different, so this corresponds to a cross-origin request and the correct asset file is not responded to. Therefore, for example, if the correct CSS file is not responded to, the agent browser B2 cannot load the CSS file in the screen-sharing display area, resulting in a problem of differences in the layout of the web page displayed on the screen-sharing display by the agent browser B2 and the guest browser B3. According to the screen sharing system of this embodiment, the agent browser B2 is configured to convert the domain name of the URL of the asset file that constitutes the web page to be screen shared into the domain name of the URL of the screen sharing display area, which results in a same-origin request and eliminates problems caused by controlling cross-origin requests.

[0054] Furthermore, in the prior art, which simply shares HTML structure data, the original URL of the asset file is accessed directly from the agent browser B2 without going through the information processing device 1. Therefore, for asset files for which access restrictions are set based on the presence or value of certain request headers, the request headers used by the agent browser B2 when communicating with the web server 4 are generally different from the request headers used by the guest browser B3 when communicating with the web server 4. This means that the asset file may be subject to access restrictions and the correct asset file may not be returned. Therefore, for example, if the correct CSS file is not returned, the agent browser B2 cannot display the CSS file in the screen-sharing display area, resulting in a discrepancy in the layout of the web page displayed on the screen-sharing display between the agent browser B2 and the guest browser B3. Access restrictions may, for example, allow access only if a specific user agent is present in the request header. According to the screen sharing system of this embodiment, the information processing device 1 is configured to obtain an asset file using the request header information of the guest browser B3, so that the request header is similar to the request header used when the guest browser B3 communicates with the web server 4, thereby reducing the possibility of problems occurring due to access restrictions based on the presence or absence or value of certain request headers.

[0055] Furthermore, some legacy browsers have a character limit for loading a URL, so if a parameter in the asset file request URL used to request an asset file is included in the request header information of guest browser B3, depending on the length of the URL, the character limit for loading the URL may be exceeded, and the correct asset file may not be returned. Therefore, for example, if the correct CSS file is not returned, agent browser B2 cannot load the CSS file in the screen sharing display area, resulting in a problem of differences in the layout of the web page displayed during screen sharing between agent browser B2 and guest browser B3. According to the screen sharing system of this embodiment, the asset file request URL can be shortened by using a session ID (see Figures 10B and 10C), thereby reducing the possibility of problems occurring due to character limit when loading a URL.

[0056] Furthermore, since the interpretation of CSS by a web browser differs depending on the content of the document type declaration set in the HTML file, if the document type declaration of the web page displayed by guest browser B3 differs from the document type declaration of the web page reconstructed in agent browser B2, there is a problem that differences will occur in the layout of the web page displayed on the shared screen of agent browser B2 and guest browser B3. According to this embodiment, when loading a screen sharing display area, the document type declaration of the screen sharing display template file used in the screen sharing display area can be set to be the same as the document type declaration of the web page displayed by guest browser B3 (see Figure 5), thereby eliminating problems caused by differences in document type declarations. However, if the document type declaration is set later without loading the web page, the above problem will not be resolved. However, since the document type declaration is set when the screen sharing display area is loaded, the above problem can be resolved.

[0057] As described above, according to this embodiment, it is possible to resolve problems caused by controlling cross-origin requests and problems caused by differences in document type declarations, and also to reduce the possibility of problems caused by access restrictions based on the presence or absence or content of certain request headers and problems caused by character limit restrictions when loading a URL, thereby enabling more accurate screen sharing of web pages between agent browser B2 and guest browser B3.

[0058] In this embodiment, an example of sharing a web page between the agent terminal 2 and the guest terminal 3 has been described, but the functions and uses of each terminal, and the users and uses of each terminal are not particularly limited.

[0059] (Appendix 1) A screen sharing system including an information processing device, and a first terminal and a second terminal connected to the information processing device via a communication network, The information processing device includes: a communication unit that communicates between a browser of the first terminal and a browser of the second terminal; a processing unit that executes a process related to screen sharing for displaying a web page displayed on a browser of the second terminal on a browser of the first terminal; a storage unit that stores header information used when the browser of the second terminal displaying the web page communicates with an external device, in association with a header information ID; Equipped with The processing unit When the communication unit receives request data including location information indicating the location of an asset file constituting the web page and the header information ID, the request data is transmitted from the browser of the first terminal, and the header information corresponding to the header information ID included in the request data is acquired from the storage unit; generating asset request data including the location information included in the received request data and the header information acquired from the storage unit; The communication unit requesting the asset file from an external server using the generated asset request data; receiving the asset file transmitted from the external server in response to a request; The received asset file is transmitted to the first terminal. It is designed like this, The second terminal a second terminal processing unit that transmits data of the web page displayed on the browser of the second terminal to the first terminal via the information processing device; The first terminal a first terminal processing unit that receives data of the web page transmitted from the second terminal, and reconstructs and displays the web page based on the received data; The first terminal processing unit If an element constituting the web page of the received data includes first location information indicating the location of a first asset file as an attribute value, the attribute value is replaced with a character string including a domain name related to the information processing device, the first location information, and the header information ID; transmitting first request data including the first location information and the header information ID to the information processing device; when receiving a CSS file transmitted from the information processing device as a response to the first request data, if text of the CSS file includes second location information indicating the location of a second asset file, replacing a text portion of the second location information with a character string including a domain name related to the information processing device, the second location information, and the header information ID; transmitting second request data including the second location information and the header information ID to the information processing device; Screen sharing system. [Explanation of symbols]

[0060] 1. Information processing equipment 2 Agent terminal 3. Guest terminal 4 Web Server 5. Recording media 11a API processing section 11b Socket processing section 11c File Processing Unit 12a Account Management Database 12b Screen sharing management database 12c Screen Sharing Data Relay Store 12d Program File Storage 12e Asset File Storage B2 Agent Browser B3 Guest Browser

Claims

1. a communication unit that communicates between a browser of the first terminal and a browser of the second terminal; a processing unit that executes a process related to screen sharing for displaying a web page displayed on a browser of the second terminal on a browser of the first terminal; a storage unit that stores header information used when a browser of the second terminal displaying the web page communicates with an external device, in association with a header information ID; Equipped with The processing unit When the communication unit receives request data including location information indicating the location of an asset file constituting the web page and the header information ID, the request data is transmitted from the browser of the first terminal, and the header information corresponding to the header information ID included in the request data is acquired from the storage unit; generating asset request data including the location information included in the received request data and the header information acquired from the storage unit; The communication unit requesting the asset file from an external server using the generated asset request data; receiving the asset file transmitted from the external server in response to a request; Transmitting the received asset file to the first terminal Information processing device.

2. 10. A screen sharing system comprising: an information processing device according to claim 1; and a first terminal and a second terminal connected to the information processing device via a communication network, The second terminal a second terminal processing unit that transmits data of the web page displayed on the browser of the second terminal to the first terminal via the information processing device; The first terminal a first terminal processing unit that receives data of the web page transmitted from the second terminal and reconstructs and displays the web page based on the received data; The first terminal processing unit If an element constituting the web page of the received data includes the location information indicating the location of the asset file as an attribute value, the attribute value is replaced with a character string including a domain name related to the information processing device, the location information, and the header information ID. Screen sharing system.

3. The second terminal transmitting the header information and the header information ID used when a browser displaying the web page of the second terminal communicates with an external device to the information processing device; The information processing device includes: receiving the header information and the header information ID transmitted from the second terminal, and storing the received header information and the header information ID in the storage unit in association with each other; The screen sharing system according to claim 2 .

4. The domain name related to the information processing device is the same as a domain name set as an attribute value of a screen sharing display area element for reconstructing and displaying the web page displayed in the browser of the first terminal. The screen sharing system according to claim 2 or 3.

5. A screen sharing method in which an information processing device communicates between a browser on a first terminal and a browser on a second terminal, and a web page displayed on the browser on the second terminal is displayed on the browser on the first terminal, The information processing device includes: storing, in a storage unit, header information used when a browser of the second terminal displaying the web page communicates with an external device, and a header information ID in association with the header information; When request data including location information indicating the location of an asset file constituting the web page and the header information ID transmitted from the browser of the first terminal is received, the header information corresponding to the header information ID included in the request data is acquired from the storage unit; generating asset request data including the location information included in the received request data and the header information acquired from the storage unit; requesting the asset file from an external server using the generated asset request data; receiving the asset file transmitted from the external server in response to a request; Transmitting the received asset file to the first terminal Screen sharing method to perform the process.

6. A computer program for causing a computer to execute a process related to screen sharing in which a web page displayed on a browser of a second terminal is displayed on a browser of a first terminal, the computer program comprising: storing, in a storage unit, header information used when a browser of the second terminal displaying the web page communicates with an external device, and a header information ID in association with the header information; When request data including location information indicating the location of an asset file constituting the web page and the header information ID transmitted from the browser of the first terminal is received, the header information corresponding to the header information ID included in the request data is acquired from the storage unit; generating asset request data including the location information included in the received request data and the header information acquired from the storage unit; requesting the asset file from an external server using the generated asset request data; receiving the asset file transmitted from the external server in response to a request; Transmitting the received asset file to the first terminal A computer program for causing the computer to execute a process.

Images