Screen sharing methods, screen sharing systems, and computer programs

Abstract

To provide a screen sharing method that can promote visual information transmission even when there occur differences in the layout of a web page, in screen sharing methods that achieve screen sharing of web pages between the browsers of two terminals.SOLUTION: A screen sharing method displays a web page displayed on a browser of a second terminal on a browser of a first terminal by connecting the first terminal and the second terminal via a communication network and transmitting and receiving structural data including information on elements and text that make up the web page. The screen sharing method causes the first terminal to execute processes of receiving an operation on the web page elements or a selection operation on the text and transmitting selection information indicating the operated elements or a character string of the text that has undergone the selection operation to the second terminal, and causes the second terminal to execute processes of receiving the selection information transmitted from the first terminal and highlighting the operated elements or the character string of the text that has undergone the selection operation based on the received selection information.SELECTED DRAWING: Figure 8

Description

[[Technical Field]]

[0001] The present invention relates to a screen sharing method, a screen sharing system, and a computer program. [[Background Art]]

[0002] There is a screen sharing system that enables visual promotion of information transmission between both an operator and an end user by sharing the screen displayed on the end user's browser and mouse cursor operations via the operator's browser (see, for example, Patent Document 1). The screen sharing system includes an end user browser that displays a web page, an operator browser, and a relay server connected to each browser via a communication network. The end user browser and the operator browser transmit and receive HTML structure data of the web page via the relay server, and the operator browser realizes screen sharing by reconstructing the web page displayed on the end user browser. Further, the end user browser and the operator browser transmit and receive mouse cursor data indicating the coordinates of the mouse cursor, and share the content of the mouse operation by reproducing the mouse cursor displayed on the browser of the communication partner. [[Prior Art Documents]] [[Patent Documents]]

[0003] [[Patent Document 1]] Japanese Unexamined Patent Publication No. 2020-177403 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0004] However, differences in web page display environments may result in discrepancies in the layout of web pages displayed on the operator's terminal and the end-user's terminal during screen sharing. Such discrepancies in web page layouts may hinder communication between the operator and the end-user. For example, even if the operator attempts to communicate a specific element to the end-user using the mouse cursor, the position of that element and the mouse cursor may be misaligned in the end-user's browser, potentially hindering visual information transmission via screen sharing.

[0005] The object of the present invention is to provide a screen sharing method, a screen sharing system, and a computer program that enable screen sharing of web pages between browsers on two terminals, and that can facilitate visual information transmission even when there are differences in the layout of the web pages. [Means for solving the problem]

[0006] A screen sharing method according to one aspect of the present invention is a screen sharing method that connects a first terminal and a second terminal via a communication network and transmits and receives structural data including information on elements and text constituting a web page, thereby displaying a web page displayed in the browser of the second terminal on the browser of the first terminal, wherein the first terminal is instructed to accept an operation on an element of the web page or a selection operation on text, and to transmit selection information indicating the operated element or the selected text string to the second terminal, and the second terminal is instructed to receive the selection information transmitted from the first terminal and to perform a process of highlighting the operated element or the selected text string based on the received selection information.

[0007] A screen sharing system according to one aspect of the present invention comprises a first terminal and a second terminal connected via a communication network, wherein the first terminal and the second terminal transmit and receive structural data including information on elements and text constituting a web page, thereby displaying a web page shown in the browser of the second terminal on the browser of the first terminal, wherein the first terminal receives an operation on an element of the web page or a selection operation on text, transmits selection information indicating the operated element or the selected text string to the second terminal, and the second terminal receives the selection information transmitted from the first terminal and, based on the received selection information, highlights the operated element or the selected text string.

[0008] A computer program according to one aspect of the present invention is a computer program that connects a first terminal and a second terminal via a communication network and transmits and receives structural data including information on elements and text that constitute a web page, in order to perform screen sharing in which a web page displayed in the browser of the second terminal is displayed in the browser of the first terminal, the computer of the second terminal is made to execute a process to transmit structural data including information on elements and text that constitute a web page displayed in the browser to the first terminal, the computer of the second terminal is made to execute a process to highlight the operated element or selected text string based on the received selection information, the computer of the second terminal is made to execute a process to highlight the operated element or selected text string

[0009] A computer program according to one aspect of the present invention connects a first terminal and a second terminal via a communication network and transmits and receives structural data including information on elements and text constituting a web page, thereby enabling screen sharing to display a web page displayed in the browser of the second terminal on the browser of the first terminal. The computer program causes the computer of the first terminal to execute a process to receive structural data including information on elements and text constituting a web page displayed in the browser, and to display the web page based on the received structural data. The computer program also causes the computer of the first terminal to execute a process to accept operations on elements of the web page or selection operations on text, and to transmit selection information indicating the operated element or selected text string to the second terminal. [Effects of the Invention]

[0010] According to the above, a screen sharing method that enables screen sharing of web pages between two devices' browsers can facilitate visual information transmission even when there are differences in the layout of the web pages. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram illustrating an example configuration of a screen sharing system according to an embodiment. [Figure 2] This is a block diagram showing an example configuration of a screen sharing system according to an embodiment. [Figure 3] This is a functional block diagram of the screen sharing system according to the embodiment. [Figure 4] This is a sequence diagram showing the screen sharing process procedure according to the embodiment. [Figure 5] This is a flowchart showing the highlighting process. [Figure 6] This is a flowchart showing the highlighting process. [Figure 7] This is a flowchart showing the highlighting process. [Figure 8] This is an explanatory diagram showing how selected elements are highlighted. [Figure 9] This is an explanatory diagram showing how selected text is highlighted. [Modes for carrying out the invention]

[0012] A screen sharing method, screen sharing system, and computer program according to embodiments of this disclosure will be described below with reference to the drawings. This disclosure is not limited to these examples, but is intended to include all modifications within the meaning and scope of the claims, as indicated by the claims. Furthermore, at least some of the embodiments described below may be combined in any way.

[0013] Figure 1 is a schematic diagram illustrating an example configuration of a screen sharing system according to this embodiment. The screen sharing system according to this embodiment comprises an information processing device 1, an agent terminal (first terminal) 2, a guest terminal (second terminal) 3, and a web server (external server) 4. The various terminals and servers are connected to the information processing device 1 via a communication network N.

[0014] The user operating agent terminal 2 (hereinafter referred to as "agent") is, for example, the person who guides the user operating guest terminal 3 (hereinafter referred to as "guest") on how to use the website. Note that the agent and guest are examples of users of the information processing device 1 according to this embodiment, and their relationship is not particularly limited.

[0015] Figure 2 is a block diagram showing an example configuration of a screen sharing system according to an embodiment. The information processing device 1 is a computer equipped with a processing unit 11, a storage unit 12, and a communication unit 13. The information processing device 1 may be configured with multiple computers for distributed processing, or it may be implemented with multiple virtual machines located on a single server, or it may be implemented using a cloud server, or part of it may be made up of a quantum computer.

[0016] The processing unit 11 includes one or more arithmetic processing devices such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), and the like, and executes the screen sharing processing according to the present embodiment by reading and executing the computer program P1 stored in the storage unit 12.

[0017] The storage unit 12 is a storage device such as a hard disk, an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, or the like. The storage unit 12 stores the computer program P1, the agent program P2, and the guest program P3 that are required for the processing unit 11 to share the content of web pages displayed on respective web browsers and operation contents between the guest terminal 3 and the agent terminal 2. The agent program P2 and the guest program P3 are programs written, for example, in JavaScript (registered trademark). The information processing apparatus 1 transmits the agent program P2 to the agent terminal 2, and transmits the guest program P3 to the guest terminal 3. A web browser operating on the agent terminal 2 (hereinafter referred to as an agent browser B2) executes processing required for screen sharing by executing the agent program P2. A web browser operating on the guest terminal 3 (hereinafter referred to as a guest browser B3) executes processing required for screen sharing by executing the guest program P3. Note that various programs may be recorded in a computer-readable manner on the recording medium 5. The storage unit 12 stores the computer program P1, the agent program P2, and the guest program P3 read from the recording medium 5 by a reading device (not shown). The recording medium 5 is a semiconductor memory such as a flash memory, an optical disc, a magnetic disc, a magneto-optical disc, or the like. Alternatively, various programs according to the present embodiment may be downloaded from an external server (not shown) connected to the communication network N and stored in the storage unit 12.

[0018] The communication unit 13 includes a processing circuit, a communication circuit, and the like for performing processing related to communication, and transmits and receives information to and from the agent terminal 2, the guest terminal 3, and the web server 4.

[0019] The agent terminal 2 is a computer including a processing unit (first terminal processing unit) 21, a storage unit 22, a communication unit 23, an operation unit 24, and a display unit 25. The basic configuration of the processing unit 21, the storage unit 22, and the communication unit 23 is the same as that of the information processing apparatus 1. The storage unit 22 stores at least a program related to the agent browser B2. The agent browser B2 implements DOM (Document Object Model), which is a program for referencing and controlling elements constituting a web page, and can execute the agent program P2 transmitted from the information processing apparatus 1. The operation unit 24 is a keyboard, a mouse, or the like, and receives operations and inputs from an agent. The display unit 25 is a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display, and renders information provided from the processing unit 21.

[0020] The guest terminal 3 is a computer including a processing unit (second terminal processing unit) 31, a storage unit 32, a communication unit 33, an operation unit 34, and a display unit 35. The basic configuration of the processing unit 31, the storage unit 32, and the communication unit 33 is the same as that of the information processing apparatus 1. The storage unit 32 stores at least a program related to the guest browser B3. The guest browser B3 implements DOM, which is a program for referencing and controlling elements constituting a web page, and can execute the guest program P3 transmitted from the information processing apparatus 1. The operation unit 34 is a keyboard, a mouse, or the like, and receives operations and inputs from a guest. The display unit 35 is a display device such as a liquid crystal display or an organic EL display, and renders information provided from the processing unit 31.

[0021] The web server 4 is a server that sends the document file of the web page and the asset files that make up the web page to the guest browser B3 in response to a request from the guest browser B3. The web server 4 is also a server that sends the asset files that make up the web page to the information processing device 1 in response to a request from the information processing device 1. In this embodiment, the guest browser B3 accesses the web server 4 and displays the web page.

[0022] Figure 3 is a functional block diagram of the screen sharing system according to the embodiment. The agent terminal 2 has an agent browser B2 as a functional unit and can execute the agent program P2 transmitted from the information processing device 1. The agent program P2 is transmitted from the information processing device 1 to the agent terminal 2 when the agent browser B2 accesses the information processing device 1 and loads the management screen. The management screen is a screen for executing the screen sharing process of a web page.

[0023] The guest terminal 3 has a guest browser B3 as its functional unit and can execute the guest program P3 sent from the information processing device 1. The guest program P3 is sent from the information processing device 1 to the guest terminal 3 when loading the web page that is the target of screen sharing.

[0024] The information processing device 1 includes an API processing unit 11a, a socket processing unit 11b, and a file processing unit 11c. The information processing device 1 also includes an account management database 12a, a screen sharing management database 12b, a screen sharing data relay store 12c, a program file storage 12d, and an asset file storage 12e.

[0025] The API processing unit 11a is a functional unit that retrieves various setting data for the screen sharing system from the account management database 12a and executes processing such as setting up the screen sharing process.

[0026] The socket processing unit 11b is a functional unit that establishes communication with the agent browser B2 and the guest browser B3 via WebSockets, enabling bidirectional communication. The socket processing unit 11b performs processes such as issuing a session ID (header information ID) to identify the screen sharing session, and sending and receiving various data during screen sharing between the agent browser B2 and the guest browser B3. Note that WebSockets are just one example of a communication connection method, and the socket processing unit 11b may also be configured to achieve pseudo-bidirectional communication between the agent browser B2 and the guest browser B3 by using long polling.

[0027] The file processing unit 11c is a functional unit that requests asset files constituting the web page to be shared from the web server 4, retrieves them, and temporarily saves them. Asset files include, for example, CSS (Cascading Style Sheets) files, image files, font files, etc.

[0028] The account management database 12a is a database that stores various setting data for the screen sharing system. This setting data includes specifications for items on web pages that cannot be shared, and specifications for the layout of various buttons displayed on web pages. The screen sharing management database 12b is a database that stores session IDs and other information used to identify screen sharing sessions. The screen sharing data relay store 12c is a relay server that relays various data transmitted and received between the agent browser B2 and the guest browser B3 during screen sharing. Program file storage 12d is a storage facility that stores HTML files, CSS files, image files, agent program P2, guest program P3, etc., which make up the management screen displayed in agent browser B2. Asset file storage 12e is a storage device that temporarily stores session information used for processing asset files that make up the web page being shared on the screen, as well as acquired asset files.

[0029] Figure 4 is a sequence diagram showing the screen sharing process procedure according to the embodiment. The socket processing unit 11b of the information processing device 1 establishes communication between the agent terminal 2 and the guest terminal 3 via a WebSocket (step S11). The procedure for establishing communication is not particularly limited. The agent terminal 2 and the guest terminal 3 become capable of bidirectional communication with the information processing device 1. In addition, the agent terminal 2 and the guest terminal 3 become capable of sending and receiving data necessary for screen sharing of a web page via the information processing device 1. The information processing device 1 issues a session ID at the start of the screen sharing process and sends the issued session ID to the agent terminal 2 and the guest terminal 3 at an appropriate time. The agent terminal 2 and the guest terminal 3 receive and store the session ID. The series of processes related to screen sharing between the agent terminal 2 and the guest terminal 3 are managed by the session ID. Here, it is assumed that the guest browser B3 is accessing the web server 4 and displaying a web page. Steps S11 to S19 shown in Figure 4 are repeated each time the guest browser B3 refreshes the displayed web page or transitions to a different web page, until the screen sharing session ends.

[0030] The entity executing the screen sharing process on the agent side can be considered to be agent terminal 2, processing unit 21, or agent browser B2, but in the following explanation, agent browser B2, which executes agent program P2, will be considered the entity executing the process. Similarly, the entity executing the screen sharing process on the guest side can be considered to be guest terminal 3, processing unit 31, or guest browser B3, but in the following explanation, guest browser B3, which executes guest program P3, will be considered the entity executing the process.

[0031] <Screen sharing and operation sharing> Agent browser B2 and guest browser B3 send and receive information via information processing device 1 to share the content and operation details of the web page that is the target of screen sharing. For example, agent browser B2 and guest browser B3 send and receive page size data, HTML data, agent operation data, guest operation data, etc. via information processing device 1. Agent browser B2 and guest browser B3 may also generate a common key in advance for encrypting and decrypting information, and use the generated common key to perform encryption processing of information before transmission and decryption processing of information after reception.

[0032] Page size data is data that includes information indicating the height and width of the display area on the screen of the web page being shared. HTML data is information that shows the HTML structure of the web page being shared on the screen, and is nested data generated in JSON (JavaScript® Object Notation) format. Specifically, it is information about the nodes that make up the web page, and includes a unique identifier to identify the node (hereinafter referred to as nodeID), the type of node (hereinafter referred to as nodeType), the name of the element (hereinafter referred to as tagName), the value of the property (hereinafter referred to as property), and information about child nodes (hereinafter referred to as childNodes). nodeID is a unique identifier assigned to the nodes that make up the web page. nodeType is an integer code that represents the type of node, for example, "1" for an element node (ELEMENT_NODE), "3" for a text node (TEXT_NODE), and "8" for a comment node (COMMENT_NODE). tagName is the name of the element of the node, and exists only when nodeType is "1", for example, "P" for an element representing a paragraph, and "IMG" for an element representing an image. The `property` attribute contains attributes defined in HTML, such as `id` and `class`, as well as different data depending on the type of `tagName`. For example, if `tagName` is "INPUT", it may contain the input value and checked status. Therefore, it is necessary to pre-define which property to retrieve depending on the value of `tagName`. `childNodes` contains information about the nodes that are children of the node in question, and this is repeated as long as there are child nodes. Agent operation data and guest operation data, i.e., operation data, are data that contains information about operation events that can be performed through a web browser, such as operations using a mouse or touchscreen (click, touch, hover, swipe, scroll, cursor movement, etc.) and operations using a keyboard (key input, etc.). Operation data consists of the name of the operation event and the content of the operation event. For example, in the case of a mouse cursor movement operation event, the content of the operation event includes information indicating the coordinates of the mouse cursor (hereinafter referred to as mouse cursor data), and in the case of a scroll operation event, it includes information indicating the amount of scrolling of the web page (hereinafter referred to as scroll data). Specifically, the mouse cursor data is data indicating the coordinates of the mouse cursor on the web page displayed in guest browser B3, or data indicating the coordinates of the mouse cursor in the screen sharing display area 7 (see Figure 8) of agent browser B2. The scroll data is data indicating the current scroll amount of the web page displayed in guest browser B3, or data indicating the current scroll amount of the screen sharing display area 7 of agent browser B2, and includes data indicating both vertical and horizontal scroll amounts. Note that the present invention also includes a configuration in which the scroll data includes data indicating the vertical scroll amount but does not include data indicating the horizontal scroll amount.

[0033] As shown in Figure 4, guest browser B3 creates initial screen sharing data based on the web page displayed on guest browser B3 and sends it to agent browser B2 via information processing device 1 (step S12). The initial screen sharing data includes page size data, HTML data, guest operation data, etc. The page size data included in the initial screen sharing data includes information indicating the height and width of the display area of ​​the web page on the screen at the time the data is created. The HTML data included in the initial screen sharing data includes information (entire) of the nodes that make up the web page at the time the data is created. The guest operation data included in the initial screen sharing data includes data similar to that which occurs when a mouse cursor movement operation event occurs, i.e., mouse cursor data at the time the data is created, and also includes data similar to that which occurs when a scroll operation event occurs, i.e., scroll data at the time the data is created. When Agent Browser B2 receives the initial screen sharing data sent from Guest Browser B3, it analyzes the received data, reconstructs a webpage similar to the one displayed on Guest Browser B3 in the screen sharing display area 7, and reproduces the guest's actions (step S13). Specifically, Agent Browser B2 sets the height and width of the screen sharing display area 7 to the values ​​indicated by the page size data. Agent Browser B2 also creates and adds nodes on the screen sharing display area 7 based on the HTML data. Furthermore, Agent Browser B2 sets the position of the mouse cursor element 70 (see Figure 8 or Figure 9), which has been added to the screen sharing display area 7 and indicates the coordinates of the guest's mouse cursor, to the value indicated by the mouse cursor data included in the guest operation data. Finally, Agent Browser B2 sets the scroll amount of the screen sharing display area 7 to the value indicated by the scroll data included in the guest operation data.

[0034] From step S12 onward, guest browser B3 monitors the height and width of the display area on the web page and creates page size data whenever a change is detected. Guest browser B3 also monitors the HTML structure of the web page and creates HTML data (difference from before the change, or the entire HTML) whenever a change is detected. Furthermore, guest browser B3 monitors guest operations and creates guest operation data whenever an operation is detected. In some cases, the HTML structure of the web page may be rewritten as a result of an operation such as a click by the guest, in which case both guest operation data and HTML data (difference from before the operation) are created. Guest browser B3 sends the created page size data, HTML data, guest operation data, etc. to agent browser B2 via information processing device 1 each time (step S14). When Agent Browser B2 receives data sent from Guest Browser B3, it analyzes the received data and reflects the same changes as those made to the web page displayed in Guest Browser B3 in the screen sharing display area 7, thereby reproducing the guest's actions (step S15). Specifically, Agent Browser B2 sets the height and width of the screen sharing display area 7 to the values ​​indicated by the page size data. Agent Browser B2 also makes changes to the nodes on the screen sharing display area 7 based on the HTML data. Furthermore, Agent Browser B2 reproduces the same operation events as the guest's actions on the screen sharing display area 7 based on the guest operation data. Agent Browser B2 also creates a mouse cursor element 70 that indicates the coordinates of the mouse cursor on the web page displayed in Guest Browser B3 and displays it on the screen sharing display area 7, and changes the position of the mouse cursor element 70 in real time based on the value indicated by the mouse cursor data included in the guest operation data.

[0035] Similarly, from step S13 onward, agent browser B2 monitors the agent's operations and, when an operation is detected, creates agent operation data each time. In some cases, the HTML structure of the web page reconfigured in the screen sharing display area 7 may be rewritten as a result of an operation such as a click by the agent. In such cases, both agent operation data and HTML data (difference from before the operation) are created. Agent browser B2 sends the created HTML data, agent operation data, etc., to guest browser B3 via information processing device 1 each time (step S16). When guest browser B3 receives data sent from agent browser B2, it analyzes the received data and reflects the same changes on the web page as in the screen sharing display area 7, thereby reproducing the agent's operations (step S17). Specifically, guest browser B3 makes changes to nodes on the web page based on the HTML data. Guest browser B3 also reproduces operation events on the web page that are similar to the agent's operations based on the agent operation data. Guest browser B3 creates a mouse cursor element 80 (see Figure 8 or Figure 9) that indicates the coordinates of the mouse cursor in the screen sharing display area 7 of agent browser B2 and displays it on the web page. The position of the mouse cursor element 80 is changed in real time based on the value indicated by the mouse cursor data included in the agent operation data.

[0036] In steps S13 and S15, if the HTML data received by agent browser B2 contains information to add or modify a specific element node that includes the URL of an asset file as part of its attributes, or if it contains information to add or modify a specific text node that includes the URL of an asset file as part of its text content, agent browser B2 accesses information processing device 1 and requests the corresponding asset file when reflecting the contents of the HTML data in the screen sharing display area 7. Note that text content is a property of a node. When information processing device 1 receives a request for an asset file from agent browser B2, it performs file processing on the asset file and sends the asset file obtained from web server 4 to agent browser B2 (step S18). Agent browser B2 expands the asset file sent from information processing device 1 as part of the attributes of an element node on the screen sharing display area 7, or as part of the text content of a text node.

[0037] The agent browser B2, guest browser B3, and information processing device 1 execute the highlighting process according to this embodiment when a highlighting mode initiation operation is performed (step S19). An example of a highlighting mode initiation operation is when an agent presses a specific shortcut key set for initiating highlighting mode. Specifically, agent browser B2 sets the simultaneous pressing of the control key and the letter H key on the keyboard as the shortcut key for initiating highlighting mode, and starts executing the highlighting process when this shortcut key is pressed. Agent browser B2 monitors when an agent presses the shortcut key for initiating highlighting mode, and starts executing the highlighting process when this operation event occurs. Note that pressing a shortcut key is just one example of a highlighting mode initiation operation; the system may also be configured to enable highlighting mode by adding a button element for initiating highlighting mode to the management screen or screen sharing display area 7 when an agent performs a click operation such as a double-click on the screen sharing display area 7, and then enabling highlighting mode when an agent clicks on this button element.

[0038] <Highlighting> Figures 5 to 7 are flowcharts illustrating the highlighting procedure. The details of the process in step S19 shown in Figure 4 will be explained below. Although agent browser B2 and guest browser B3 send and receive data via information processing device 1, which functions as a relay device, the relay processing by information processing device 1 will be omitted from this explanation. In addition, Figures 5 to 7 illustrate the processing content in response to events such as element click operation (step S37), string range selection operation (step S44), HTML structure change detection (step S51), and highlight mode termination operation (step S55) in flowchart format for convenience, but this is to explain the concept of one aspect of the invention, and the processing is not necessarily executed in the order shown in Figures 5 to 7. Furthermore, even when highlighting is being performed, the processes from steps S14 to S18 shown in Figure 4 will continue to be executed, except for the sharing of some operation content which will be described later.

[0039] When Agent Browser B2 detects that the shortcut key for starting highlight mode has been pressed, it stops sharing some of the operation details (step S31). Specifically, Agent Browser B2 stops sending scroll data, mouse cursor data, etc., included in the agent operation data to Guest Browser B3.

[0040] Next, agent browser B2 hides the mouse cursor element 70 (step S32). The method of hiding it is not limited to, but may include, for example, deleting the mouse cursor element 70 or setting the value of the CSS display property applied to the mouse cursor element 70 to none. As a result of this process, agent browser B2 will no longer see the element indicating the guest's mouse cursor position.

[0041] Then, agent browser B2 sends notification data (hereinafter referred to as highlight mode start data) to guest browser B3 to notify it of the change to highlight mode (step S33).

[0042] Guest browser B3 receives highlight mode start data sent from agent browser B2 (step S34). Upon receiving the highlight mode start data, guest browser B3 stops sharing some operation details (step S35). Specifically, guest browser B3 stops sending scroll data, mouse cursor data, etc., included in the guest operation data to agent browser B2.

[0043] Next, guest browser B3 hides the mouse cursor element 80 (step S36). The method of hiding it is not limited to, but may include, for example, deleting the mouse cursor element 80 or setting the value of the CSS display property applied to the mouse cursor element 80 to none. As a result of this process, the element indicating the agent's mouse cursor position will no longer be displayed in guest browser B3.

[0044] Agent browser B2, in highlight mode, monitors the agent's operations on the web browser and determines whether a click operation has been performed on an element node within the screen sharing display area 7 (step S37). If it is determined that no click operation has been performed on an element node (step S37: NO), agent browser B2 executes the processes from step S44 onwards. If it is determined that a click operation has been performed on an element node (step S37: YES), agent browser B2 obtains information relating to the element node that was clicked and creates highlight target element data indicating that element node (step S38). Hereinafter, element nodes that have been clicked by the agent will be referred to as highlight target element nodes. The highlight target element data includes nodeID and other information that identifies the highlight target element node.

[0045] Then, agent browser B2 sends the created highlighted element data to guest browser B3 (step S39).

[0046] The guest browser B3 receives the highlight target element data sent from the agent browser B2 (step S40). Then, based on the information in the received highlight target element data, the guest browser B3 adjusts the amount of scrolling on the web page so that the highlight target element node is positioned in the center (predetermined position) of the display area on the web page (step S41). When the highlight target element node is displayed in the center of the display area, the agent and guest can also share the information displayed before and after the highlight target element node. Guest browser B3 may adjust the scroll amount of the web page so that the highlighted element node is positioned anywhere on the web page's display area (for example, at the top or bottom of the screen) instead of in the center. If there is a lot of information to be shared below the highlighted element node, it is preferable to display the highlighted element node at the top of the display area. If there is a lot of information to be shared above the highlighted element node, it is preferable to display the highlighted element node at the bottom of the display area. Agent browser B2 may be configured to accept the selection of the display position of the element node to be highlighted and send information indicating the accepted display position to guest browser B3. Guest browser B3 may receive the information indicating the display position and, based on the received information indicating the display position, may be configured to display the element node to be highlighted at the top, center, or bottom of the display area. The method by which agent browser B2 accepts the selection of the display position is not particularly limited. For example, agent browser B2 may be configured to accept the display position of the element node to be highlighted as a standard setting, or it may be configured to accept the display position each time an element node to be highlighted is selected. Agent browser B2 may also be configured to interpret and accept the position of the element node to be highlighted as it is displayed in guest browser B3.

[0047] For example, guest browser B3 detects the display position and vertical and horizontal dimensions of an element node having the same nodeID as the highlighted element node indicated by the information in the received highlighted element data, detects the current scroll amount of the web page, and detects the height and width of the display area on the web page screen (hereinafter referred to as the page size). Hereafter, element nodes having the same nodeID as described above in guest browser B3 will also be referred to as highlighted element nodes. Based on the detected display position and vertical and horizontal dimensions of the highlighted element node, the current scroll amount of the web page, and the page size, guest browser B3 calculates the difference between the center position of the highlighted element node and the center position of the display area on the web page screen. Guest browser B3 adjusts the scroll amount by adding or subtracting the calculated difference to the scroll amount so that the highlighted element node is positioned in the center of the display area on the web page screen. Guest browser B3 may adjust only the vertical scroll amount, but a configuration that adjusts both vertical and horizontal scroll amounts is desirable. Note that the above method of adjusting the scroll amount is just an example, and the adjustment method is not particularly limited.

[0048] Next, guest browser B3 creates an overlay element 8 to highlight the element or string specified by the agent, and displays the overlay element 8 so as to surround the highlighted element node by adjusting its position and dimensions to match the position and dimensions of the highlighted element node (step S42). In other words, guest browser B3 adds and displays an element node representing the overlay element 8 to the document of the web page it is displaying. The overlay element 8 is, for example, a rectangular div element. If an overlay element 8 has already been created on guest browser B3 at the start of step S42, the existing overlay element 8 is deleted before starting the process of step S42.

[0049] For example, guest browser B3 can detect the position and dimensions of the element node to be highlighted, and based on the detected position and dimensions data, it can set the attribute values ​​related to the display position and dimensions of the overlay element 8, as well as the values ​​of each property of the CSS to be applied, thereby displaying the overlay element 8 so as to surround the element node to be highlighted.

[0050] Additionally, guest browser B3 monitors the size of the highlighted element node, and if it detects a change in the size of the highlighted element node, it changes the display content of the overlay element 8 to match the changed position and width and height of the highlighted element node, similar to step S42 (step S43).

[0051] Next, Agent Browser B2 determines whether or not a range selection operation has been performed on the text node string within the screen sharing display area 7 (step S44). If it is determined that no range selection operation has been performed on the text node string (step S44: NO), Agent Browser B2 executes the processing from step S53 onwards, or the processing from step S55 onwards, depending on the event content. If it is determined that a range selection operation has been performed on the text node string (step S44: YES), Agent Browser B2 obtains information on the text node and the selected string, and creates highlight target string data that indicates the text node and the selected string (step S45). Hereinafter, a text node that has been selected by the agent will be referred to as a highlight target text node. Note that if a string is selected across multiple text nodes, all text nodes included in the selection will be referred to as highlight target text nodes. The highlight target string data includes data such as selection start position information and selection end position information. The selection start position information specifically includes a nodeID that identifies the text node to be highlighted corresponding to the selection start position (hereinafter referred to as the selection start text node) and the number of the character at the selection start position within that text node (hereinafter referred to as the selection start character number). For example, the selection start character number is "2" if the selection start position is the 2nd character, and "10" if it is the 10th character. The selection end position information specifically includes a nodeID that identifies the text node to be highlighted corresponding to the selection end position (hereinafter referred to as the selection end text node) and the number of the character at the selection end position within that text node (hereinafter referred to as the selection end character number). For example, the selection end character number is "2" if the selection end position is the 2nd character, and "10" if it is the 10th character.

[0052] Then, agent browser B2 sends the created highlighted string data to guest browser B3 (step S46).

[0053] The guest browser B3 receives the string data to be highlighted from the agent browser B2 (step S47). Next, the guest browser B3 adjusts the scroll amount of the web page to be displayed so that the line containing the character corresponding to the selection start character number in the selection start text node (hereinafter referred to as the selection start line) is located in the center of the display area on the web page, based on the selection start position information contained in the received string data to be highlighted (step S48). Alternatively, the guest browser B3 may adjust the scroll amount of the web page so that the selection start line is located anywhere on the display area on the web page (for example, at the top or bottom of the screen) instead of in the center. Or, instead of the selection start position information, the scroll amount of the web page to be displayed may be adjusted based on the selection end position information so that the line containing the character corresponding to the selection end character number in the selection end text node (hereinafter referred to as the selection end line) is located in the center of the display area on the web page.

[0054] For example, guest browser B3 detects the display position and vertical and horizontal dimensions of a text node having the same nodeID as the selection start text node indicated by the selection start position information contained in the received highlighted string data, and the line containing the character corresponding to the selection start character number within that text node (selection start line), detects the current scroll amount of the web page, and detects the page size. Hereinafter, the text node having the same nodeID on the guest browser B3 side will also be referred to as the selection start text node. Based on the detected display position and vertical and horizontal dimensions of the selection start line, the current scroll amount of the web page, and the page size, guest browser B3 calculates the difference between the center position of the selection start line and the center position of the display area on the web page screen. By adding or subtracting the calculated difference to the scroll amount, guest browser B3 adjusts the scroll amount so that the selection start line is positioned in the center of the display area on the web page screen. Guest browser B3 may adjust only the vertical scroll amount, but a configuration that adjusts both vertical and horizontal scroll amounts is desirable. Note that the above method of adjusting the scroll amount is just an example, and the adjustment method is not particularly limited.

[0055] Next, guest browser B3 creates an overlay element 8 to highlight the element or string specified by the agent, and displays the overlay element 8 so as to surround the string to be highlighted by adjusting its position and dimensions to match the position and dimensions of the string to be highlighted (step S49). In other words, guest browser B3 adds and displays an element node representing the overlay element 8 to the document of the web page being displayed. The overlay element 8 is, for example, a rectangular div element. Specifically, it detects the starting text node and starting character number based on the selection start position information of the string data to be highlighted. Similarly, it detects the ending text node and ending character number based on the selection end position information of the string data to be highlighted. Then, based on the selection start text node and starting character number and the selection end text node and ending character number, it detects the strings of the text nodes included in the range where the selection operation was performed on a line-by-line basis, and displays the overlay element 8 so as to surround each of the detected lines (hereinafter referred to as selected lines) by adjusting its position and dimensions to match the position and dimensions of each selected line. Furthermore, if the text of a text node included in the selected range spans multiple lines, an overlay element 8 is created for each selected line from the second line onward, and the overlay element 8 is displayed to surround each selected line. If the selected line is the starting line of the selection, the overlay element 8 is displayed to surround the character corresponding to the starting character number of the selection up to the end of the line or the character corresponding to the ending character number of the selection. If the selected line is the ending line of the selection, the overlay element 8 is displayed to surround the character corresponding to the beginning of the line or the character corresponding to the starting character number of the selection up to the character corresponding to the ending character number of the selection. If an overlay element 8 has already been created on the guest browser B3 at the start of step S49, the existing overlay element 8 is deleted before starting step S49.

[0056] For example, guest browser B3 can detect the position and dimensions of each selected line in the text node, and based on the detected position and dimensions data, it can set the attribute values ​​related to the display position and dimensions of the overlay element 8, as well as the values ​​of the applicable CSS properties, thereby displaying the overlay element 8 so as to surround each selected line in the text node.

[0057] Additionally, guest browser B3 monitors the size of each selected line in the text node, and if it detects a change in the size of a particular selected line in the text node, it changes the display content of the overlay element 8 by adjusting it to the position and width / height dimensions of the selected line after the change, similar to step S49 (step S50).

[0058] Meanwhile, as shown in step S14 in Figure 4 above, guest browser B3 continuously monitors the HTML structure of the displayed web page and detects any changes (step S51). If no change in the HTML structure is detected (step S51: NO), guest browser B3 executes the processes from step S59 onwards, from step S40 onwards, or from step S47 onwards, depending on the event.

[0059] If a change in the HTML structure is detected (Step S51: YES), the guest browser B3 creates HTML data showing the changes (the difference from before the change, or the entire data) and sends the created HTML data to the agent browser B2 (Step S52). For example, if overlay element 8 is created, HTML data including additional information about the overlay element 8 is sent. Also, if the displayed content of overlay element 8 is changed, HTML data including information about the changes to the overlay element 8 is sent.

[0060] Agent browser B2 receives HTML data sent from guest browser B3 (step S53) and makes changes to the nodes on the screen sharing display area 7 based on the received HTML data (step S54). For example, if it receives HTML data containing additional information for overlay element 8, it creates and adds the overlay element 8 on the screen sharing display area 7. Also, if it receives HTML data containing information about changes to overlay element 8, it makes changes to the overlay element 8 on the screen sharing display area 7. Specifically, if the content of the change information for overlay element 8 is information about changes to the display content of the overlay element 8, it displays the overlay element 8 so as to surround the highlighted element node or each selected row, similar to guest browser B3.

[0061] Agent Browser B2 determines whether or not an operation to exit highlight mode has been performed (step S55). An operation to exit highlight mode is, for example, when an agent presses a specific shortcut key set for exiting highlight mode. Specifically, Agent Browser B2 sets the simultaneous pressing of the Control key and the letter H key on the keyboard as the shortcut key for exiting highlight mode, and when this shortcut key is pressed, it executes the processes from step S56 onwards. Agent Browser B2 monitors when an agent presses the shortcut key for exiting highlight mode. Note that pressing a shortcut key is just one example of an operation to exit highlight mode; if an agent performs a click operation such as a double-click on the screen sharing display area 7, a button element for exiting highlight mode may be added to the management screen or screen sharing display area 7, and the operation to exit highlight mode may be realized when an agent clicks on this button element.

[0062] If the highlight mode termination operation has not been performed (step S55: NO), agent browser B2 returns to step S37 and continues the highlighting process. If the highlight mode termination operation has been performed (step S55: YES), agent browser B2 resumes sharing of some of the operations that were stopped (step S56) and displays the mouse cursor element 70 (step S57). Then, agent browser B2 sends notification data (hereinafter referred to as highlight mode termination data) to guest browser B3 to notify that the highlight mode has ended (step S58).

[0063] Guest browser B3 receives highlight mode termination data sent from agent browser B2 (step S59). Upon receiving the highlight mode termination data, guest browser B3 resumes sharing some of the operations that were stopped (step S60) and displays the mouse cursor element 80 (step S61). Guest browser B3 also deletes the overlay element 8 (step S62). After exiting highlight mode, agent browser B2 and guest browser B3 will share the content and operations of the web page being shared by executing steps S14 to S18 shown in Figure 4, just as they did before starting highlight mode.

[0064] Figure 8 is an explanatory diagram illustrating how a specified element is highlighted. As shown in Figure 8A, before the start of highlight mode, the web page is shared between agent browser B2 and guest browser B3, and the mouse cursor position is also shared by mouse cursor element 70 and mouse cursor element 80. When the agent presses the shortcut key to start highlight mode, the system transitions to highlight mode.

[0065] When the system switches to highlight mode, as shown in Figure 8B, the sharing of some operations between the agent browser B2 and the guest browser B3 is temporarily suspended. Specifically, the sharing of the mouse cursor position, the amount scrolled on the web page, etc., is temporarily suspended. In addition, mouse cursor elements 70 and 80 are hidden.

[0066] When the agent clicks on an element node, as shown in Figure 8C, the guest browser B3 adjusts the amount of scrolling on the web page so that the clicked element node is centered on the display area of ​​the web page. The guest browser B3 can also highlight the clicked element node by displaying an overlay element 8 around it. Since the overlay element 8 is shared with the agent browser B2, the overlay element 8 is also displayed in the agent browser B2.

[0067] When the agent presses the shortcut key to exit highlight mode, it transitions to normal mode, as shown in Figure 8D. Upon transitioning to normal mode, the sharing of some operations that had been suspended between agent browser B2 and guest browser B3 resumes. Specifically, the sharing of the mouse cursor position, web page scroll amount, etc., resumes. Additionally, mouse cursor elements 70 and 80 are displayed.

[0068] In this way, during screen sharing, the agent can use highlight mode and click on element nodes that they want to show to the guest, thereby highlighting those element nodes and displaying them in a visually clear location.

[0069] Figure 9 is an explanatory diagram illustrating how to highlight a specified string. As shown in Figures 9A and 9B, when the agent presses the shortcut key to start highlight mode, it transitions to highlight mode, and the sharing of some operations between the agent browser B2 and the guest browser B3 is temporarily suspended.

[0070] When the agent performs a range selection operation on the string of any text node, as shown in Figure 9C, the guest browser B3 adjusts the scroll amount so that the starting line of the selected text node is displayed in the center of the web page's display area. The guest browser B3 can also highlight the string of the selected text node by displaying an overlay element 8 surrounding each selected line of the selected text node. Since the overlay element 8 is shared with the agent browser B2, the overlay element 8 is also displayed in the agent browser B2.

[0071] When the agent presses the shortcut key to exit highlight mode, it transitions to normal mode, as shown in Figure 9D. Upon transitioning to normal mode, the sharing of some operations that had been suspended between agent browser B2 and guest browser B3 resumes. Specifically, the sharing of the mouse cursor position, web page scroll amount, etc., resumes. Additionally, mouse cursor elements 70 and 80 are displayed.

[0072] In this way, during screen sharing, the agent can use highlight mode to select a range of text in a text node that they want to show to the guest, thereby highlighting the text in that text node and displaying it in a visually clear location.

[0073] <Effects and Effects> In conventional technologies that simply share HTML structure data, for example, if the web browser used by the agent and the web browser used by the guest are different, the way CSS is processed differs for each web browser. This can lead to differences in the calculation results of each CSS property, potentially causing differences in the layout of the shared web page. Furthermore, if there are differences in the web page layout, it can lead to a discrepancy between the display area of ​​the web page viewed by the agent and the display area of ​​the web page viewed by the guest. This makes it difficult for the agent to visually communicate elements that they want the guest to interact with or text that they want the guest to confirm, hindering communication. Moreover, if there are differences in the web page layout, for example, even if the agent tries to communicate a specific element to the guest using the mouse cursor, the positions of that element and the mouse cursor element 80 in the guest browser B3 will be misaligned, making it impossible to resolve the aforementioned communication problem. According to the screen sharing system and the like of this embodiment, an agent can highlight specified elements or text strings by using highlight mode. Furthermore, since these elements or text strings can be displayed in a visually easy-to-understand position, even if there are differences in the layout of the web page, the possibility of communication problems can be reduced, and visual information transmission can be promoted more effectively.

[0074] Specifically, the agent can specify the element to be highlighted in the agent browser B2, thereby highlighting that element in the guest browser B3. Furthermore, by adjusting the scroll amount of the webpage in the guest browser B3, the element can be displayed in a visually clear position on the webpage's screen (for example, in the center), thereby more effectively promoting visual information transmission.

[0075] Similarly, the agent can specify the string to be highlighted in the agent browser B2, and then highlight that string in the guest browser B3. Furthermore, by adjusting the scroll amount of the webpage in the guest browser B3, the text can be displayed in a visually clear position on the webpage's display area (for example, in the center), thereby more effectively promoting visual information transmission.

[0076] Furthermore, if the system only focuses on highlighting and adjusting the display position of specified elements or text strings, sharing the mouse cursor coordinates, web page scroll amount, etc., may actually hinder visual information transmission. For example, if an element is highlighted and the agent attempts to point the mouse cursor to that element, in guest browser B3, the element may be highlighted while the mouse cursor element 80 points to a different element node. Also, if an element is highlighted and the agent scrolls the web page, in guest browser B3, that element may appear in a position unintended by the agent. According to the screen sharing system of this embodiment, the agent can stop the process of sharing some of the operations performed on agent browser B2 and some of the operations performed on guest browser B3 by using highlight mode. Specifically, the sharing of operations such as mouse cursor coordinates and web page scroll amount can be temporarily suspended, thereby eliminating obstacles and promoting more effective visual information transmission.

[0077] As described above, according to this embodiment, in a screen sharing method that enables screen sharing of a web page between agent browser B2 and guest browser B3, visual information transmission can be facilitated even if there are differences in the layout of the web page.

[0078] In this embodiment, an example of highlighting a specified element or string using a rectangular element has been described. However, the highlighting method is not particularly limited, and highlighting may also be achieved by displaying a callout element pointing to the element or string, or by changing the background color of the element or string.

[0079] Furthermore, while we have described an example where the sharing of some operation details is temporarily suspended when performing highlighting-related processing, the system may be configured to share such operation details if it does not hinder visual information transmission. Additionally, the system may be configured to allow users to choose whether or not to share some operation details while highlighting-related processing is being performed. [Explanation of Symbols]

[0080] 1. Information Processing Device 2 Agent terminals 3. Guest terminal 4 Web Server 5. Recording media 11a API Processing Unit 11b Socket Processing Unit 11c File Processing Unit 12a Account Management Database 12b Screen sharing management database 12c Screen Sharing Data Relay Store 12d Program File Storage 12e Asset File Storage B2 Agent Browser B3 Guest Browser

Claims

1. A screen sharing method in which an information processing device connects a first terminal and a second terminal via a communication network, and the first terminal and the second terminal send and receive structural data including information on elements and text that constitute a web page, thereby displaying a web page shown in the browser of the second terminal on the browser of the first terminal, The first terminal is, Accepts operations on web page elements or selection operations on text. The selection information, including the identifier of the manipulated element or the identifier of the selected text, is transmitted to the second terminal. The aforementioned second terminal is, Upon receiving the selection information transmitted from the first terminal, Based on the received selection information, an overlay element is added to the structure data to highlight the element or text indicated by the identifier. The structure data to which the overlay element has been added is transmitted to the first terminal. How to share your screen.

2. A screen sharing method in which an information processing device connects a first terminal and a second terminal via a communication network, and the first terminal and the second terminal send and receive structural data including information on elements and text that constitute a web page, thereby displaying a web page displayed in the browser of the second terminal on the browser of the first terminal, The first terminal is, Accepts operations on web page elements or selection operations on text. The second terminal receives selection information indicating the manipulated element or the selected text string. The aforementioned second terminal is, Upon receiving the selection information transmitted from the first terminal, Based on the received selection information, the manipulated element or the selected text string is highlighted. Based on the received selection information, the browser adjusts the scroll amount of the web page so that the manipulated element or the selected text string is displayed in a predetermined position within the browser's display area. How to share your screen.

3. A screen sharing method in which an information processing device connects a first terminal and a second terminal via a communication network, and the first terminal and the second terminal send and receive structural data including information on elements and text that constitute a web page, thereby displaying a web page shown in the browser of the second terminal on the browser of the first terminal, The first terminal is, Accepts operations on web page elements or selection operations on text. The second terminal receives selection information indicating the manipulated element or the selected text string. The aforementioned second terminal is, Upon receiving the selection information transmitted from the first terminal, Based on the received selection information, the manipulated element or the selected text string is highlighted. Furthermore, The first terminal transmits the operations performed on the browser of the first terminal to the second terminal. The second terminal transmits the operations performed on the browser of the second terminal to the first terminal. The first terminal displays the operations performed on the second terminal's browser on the first terminal's browser. The second terminal displays the operations performed on the browser of the first terminal on the second terminal's browser. When the first terminal and the second terminal perform the processing related to the highlighting, they stop the process of sending and receiving operation content to the browser. How to share your screen.

4. A screen sharing system comprising a first terminal and a second terminal connected via a communication network, wherein the first terminal and the second terminal transmit and receive structural data including information on elements and text that constitute a web page, thereby displaying a web page shown in the browser of the second terminal on the browser of the first terminal, The first terminal is, Accepts operations on web page elements or selection operations on text. The selection information, including the identifier of the manipulated element or the identifier of the selected text, is transmitted to the second terminal. The aforementioned second terminal is, Upon receiving the selection information transmitted from the first terminal, Based on the received selection information, an overlay element is added to the structure data to highlight the manipulated element or the selected text. The structure data to which the overlay element has been added is transmitted to the first terminal. Screen sharing system.

5. A screen sharing system comprising a first terminal and a second terminal connected via a communication network, wherein the first terminal and the second terminal transmit and receive structural data including information on elements and text constituting a web page, thereby displaying a web page shown in the browser of the second terminal on the browser of the first terminal, The first terminal is, Accepts operations on web page elements or selection operations on text. The second terminal receives selection information indicating the manipulated element or the selected text string. The aforementioned second terminal is, Upon receiving the selection information transmitted from the first terminal, Based on the received selection information, the manipulated element or the selected text string is highlighted. Based on the received selection information, the browser adjusts the scroll amount of the web page so that the manipulated element or the selected text string is displayed in a predetermined position within the browser's display area. Screen sharing system.

6. A screen sharing system comprising a first terminal and a second terminal connected via a communication network, wherein the first terminal and the second terminal transmit and receive structural data including information on elements and text constituting a web page, thereby displaying a web page displayed in the browser of the second terminal on the browser of the first terminal, The first terminal is, Accepts operations on web page elements or selection operations on text. The second terminal receives selection information indicating the manipulated element or the selected text string. The aforementioned second terminal is, Upon receiving the selection information transmitted from the first terminal, Based on the received selection information, the manipulated element or the selected text string is highlighted. Furthermore, The first terminal transmits the operations performed on the browser of the first terminal to the second terminal. The second terminal transmits the operations performed on the browser of the second terminal to the first terminal. The first terminal displays the operations performed on the second terminal's browser on the first terminal's browser. The second terminal displays the operations performed on the browser of the first terminal on the second terminal's browser. When the first terminal and the second terminal perform the processing related to the highlighting, they stop the process of sending and receiving operation content to the browser. Screen sharing system.

7. A computer program that causes the computer of the second terminal to execute a process to transmit to the first terminal structural data, which includes information about the elements and text of a web page displayed in the browser of the second terminal, in order to perform screen sharing in which a web page displayed in the browser of the second terminal is displayed in the browser of the first terminal, by which the information processing device connects the first terminal and the second terminal via a communication network, and the first terminal and the second terminal send and receive structural data including information about the elements and text of a web page, The first terminal receives selection information including the identifier of the element operated on or the identifier of the selected text. Based on the received selection information, an overlay element is added to the structure data to highlight the element indicated by the identifier or the selected text. The structure data to which the overlay element has been added is transmitted to the first terminal. A computer program that causes the computer of the second terminal to perform the processing.

8. A computer program that causes the computer of the second terminal to execute a process to transmit to the first terminal structural data, which includes information about the elements and text of a web page displayed in the browser of the second terminal, in order to perform screen sharing in which a web page displayed in the browser of the second terminal is displayed in the browser of the first terminal, by which the information processing device connects the first terminal and the second terminal via a communication network, and the first terminal and the second terminal send and receive structural data including information about the elements and text of a web page, The first terminal receives selection information indicating the element or text string that was operated on or selected. Based on the received selection information, the manipulated element or the selected text string is highlighted. Based on the received selection information, the browser adjusts the scroll amount of the web page so that the manipulated element or the selected text string is displayed in a predetermined position within the browser's display area. A computer program that causes the computer of the second terminal to perform the processing.

9. A computer program that causes the computer of the second terminal to execute a process to transmit to the first terminal structural data, which includes information about the elements and text of a web page displayed in the browser of the second terminal, in order to perform screen sharing in which a web page displayed in the browser of the second terminal is displayed in the browser of the first terminal, by which the information processing device connects the first terminal and the second terminal via a communication network, and the first terminal and the second terminal send and receive structural data including information about the elements and text of a web page, The operation details of the browser on the second terminal are sent to the first terminal. The system receives the operation details sent from the first terminal to the browser of the first terminal. The received operation details on the browser of the first terminal are displayed on the browser of the second terminal. In order to highlight, the first terminal receives selection information indicating the string of text that has been operated on or selected. Based on the received selection information, the manipulated element or the selected text string is highlighted. When performing the aforementioned highlighting process, the process of sending and receiving operation content to the browser is stopped. A computer program that causes the computer of the second terminal to perform the processing.

10. A computer program that causes the computer of the first terminal to perform screen sharing, in which an information processing device connects a first terminal and a second terminal via a communication network, and the first terminal and the second terminal send and receive structural data including information on elements and text that constitute a web page, so that a web page displayed in the browser of the second terminal can be displayed in the browser of the first terminal, by receiving structural data including information on elements and text that constitute a web page displayed in the browser, and displaying the web page based on the received structural data, Accepts operations on web page elements or selection operations on text. The selection information, including the identifier of the manipulated element or the identifier of the selected text, is transmitted to the second terminal. An overlay element is added to highlight the element or text indicated by the identifier included in the selection information, and the structure data transmitted from the second terminal is received. Based on the received structural data, the web page displayed in the browser of the second terminal is displayed in the browser of the first terminal. A computer program that causes the computer of the first terminal to perform a process.

11. A computer program for causing the computer of the first terminal to perform screen sharing, in which an information processing device connects a first terminal and a second terminal via a communication network, and the first terminal and the second terminal send and receive structural data including information on elements and text that constitute a web page, thereby displaying a web page displayed in the browser of the second terminal on the browser of the first terminal, the computer to receive structural data including information on elements and text that constitute a web page displayed in the browser, and to display a web page based on the received structural data, The operations performed on the browser of the first terminal are sent to the second terminal. The system receives the operation details sent from the second terminal to the browser of the second terminal, The received operation details on the browser of the second terminal are displayed on the browser of the first terminal. Accepts operations on web page elements or selection operations on text. Select information indicating the element or text that has been manipulated or selected for highlighting is transmitted to the second terminal. When performing the aforementioned highlighting process, the process of sending and receiving operation details to the previous browser is stopped. A computer program that causes the computer of the first terminal to perform a process.

Images