Image forming apparatus and image forming method

Abstract

The invention provides an image forming device and an image forming method, and the device comprises a display device which is used for displaying a video picture related to an image forming function; the embedded server is used for responding to a webpage request of a remote terminal and sending a webpage file to the remote terminal; the embedded server is also used for executing the following steps after detecting a remote control request initiated by the remote terminal through the webpage file: if the remote terminal is a transfer terminal configured by the embedded server, sending the video picture to the remote terminal, and if the remote terminal is a non-transfer terminal, sending the video picture to the remote terminal; and if yes, facilitating peer-to-peer connection between the non-transfer terminal and the transfer terminal, so that the non-transfer terminal directly acquires the video picture from the transfer terminal.

Description

Technical Field

[0001] This application relates to the field of image formation technology, and more specifically, to an image forming apparatus and an image forming method. Background Technology

[0002] With the gradual development of technology, intelligent technology is increasingly entering people's lives, and people are increasingly fond of various intelligent devices, such as intelligent printing equipment. Intelligent printing equipment is generally equipped with a control panel, which can be used to display device configuration information and operating parameters, and allows users to operate various functions on the panel, especially troubleshooting and recovery. To improve the convenience of intelligent printing equipment, remote operation can also be set up. In the case of multiple users remotely operating the printing equipment, the printing equipment needs to transmit real-time video feeds to multiple users simultaneously. However, due to the limited hardware processing power of some printing equipment, this can easily lead to the printing equipment consuming excessive processor power when providing remote feeds to multiple users simultaneously. This can cause lag and delays when users use other functions of the printing equipment, reducing its usability. Summary of the Invention

[0003] This application provides at least one image forming apparatus and one image forming method.

[0004] In a first aspect, embodiments of this application provide an image forming apparatus, including: A display device for displaying video images related to image forming functions; An embedded server is used to send web page files to a remote terminal in response to a web page request. The embedded server is further configured to perform the following steps after detecting a remote control request initiated by the remote terminal through the web page file: If the remote terminal is a relay terminal configured by the embedded server, then the video feed is sent to the remote terminal: If the remote terminal is a non-relay terminal, a peer-to-peer connection is established between the non-relay terminal and the relay terminal, so that the non-relay terminal can directly obtain the video image from the relay terminal.

[0005] In one optional implementation, the embedded server is further configured to: When no peer-to-peer connection is established between the non-relay terminal and the relay terminal, signaling messages are forwarded between the non-relay terminal and the relay terminal based on the remote control request initiated by the non-relay terminal, so as to facilitate a peer-to-peer connection between the relay terminal and the remote terminal.

[0006] In one optional implementation, the webpage file includes first code, which is used to send the video feed to a non-relay terminal that requests the video feed; wherein, The first code is used to enable the relay terminal to establish a peer-to-peer connection with the non-relay terminal based on the signaling message forwarded by the embedded server when there is no connection between the relay terminal and the non-relay terminal; it is also used to send the video image to the non-relay terminal that requested the video image after the peer-to-peer connection is established; or The first code is used to cause the relay terminal to add the non-relay terminal requesting the video image to the sending queue when a connection is established between the relay terminal and the non-relay terminal; it is also used to cause the relay terminal to send the video image to at least one non-relay terminal in the sending queue through the peer-to-peer connection when the relay terminal receives the video image.

[0007] In one optional implementation, the webpage file includes second code, which is used to receive the video feed from a relay terminal; wherein, The second code is used to enable the non-relay terminal to establish a peer-to-peer connection with the relay terminal based on the signaling messages forwarded by the embedded server when there is no connection between the non-relay terminal and the relay terminal; and The second code is used to request the video feed from the relay terminal only through the peer-to-peer connection after a peer-to-peer connection is established between the non-relay terminal and the relay terminal.

[0008] In one optional implementation, the embedded server is further configured to: The relay terminal is specified in at least one of the remote terminals that initiate the remote control request via the web page file, wherein the specification is based on the order in which the remote control request is initiated and / or the processing performance of the remote terminal.

[0009] In one optional implementation, the processing performance includes network response speed; the embedded server is further configured to: Configure the first remote terminal that initiates a remote control request via a webpage file as the relay terminal; or, Remote terminals whose network response speed meets a preset speed threshold are configured as relay terminals.

[0010] In one optional implementation, the at least one remote terminal initiating the remote control request via the webpage file includes a first remote terminal and a second remote terminal, wherein the first remote terminal has been designated as a relay terminal; the embedded server is further configured to: Based on the order initiation of the remote control requests and / or the processing performance of the remote terminal, determine whether the second remote terminal meets the specified conditions of the relay terminal; If it is determined that the specified conditions of the relay terminal are met, then the second remote terminal is used as a relay terminal instead of the first remote terminal, or the second remote terminal is designated as another relay terminal.

[0011] In one optional implementation, the embedded server is further configured to: The relay terminal receives trigger information from a non-relay terminal, the trigger information being generated by the non-relay terminal in response to the image forming function in the video frame received from the relay terminal; Based on the storage status of the video footage associated with the trigger information by the relay terminal, a method is determined to instruct the relay terminal to send the video footage to the non-relay terminal that generated the trigger information.

[0012] In one optional implementation, the embedded server is further configured to: When it is determined that the relay terminal does not have any video footage associated with the trigger information stored locally, a reconstructed video is generated based on the trigger information; The reconstructed image is sent to the relay terminal as a video image associated with the trigger information; The relay terminal is instructed to send the received reconstructed screen to the non-relay terminal that generated the trigger information.

[0013] In one optional implementation, the embedded server is further configured to: When it is determined that the relay terminal has video frames associated with the trigger information stored locally, the relay terminal is instructed to search for historical video frames associated with the trigger information in the locally stored video frames, and return the historical video frames to the non-relay terminal that generated the trigger information.

[0014] In one optional implementation, the trigger information carries a terminal identifier corresponding to the non-relay terminal, and the video frame associated with the trigger information transmitted by the relay terminal to each of the non-relay terminals carries the terminal identifier of the non-relay terminal.

[0015] In one optional implementation, the video feed includes a real-time view of the operation panel of the image forming device, and / or a function setting interface of the image forming device, the function setting interface including at least one function setting item associated with the image forming function.

[0016] In one optional implementation, the embedded server is further configured to: Receive the first trigger information generated by the non-relay terminal on the real-time screen and the second trigger information generated on the function setting interface, sent by the relay terminal; If the function setting effect corresponding to the second trigger information does not conflict with the function setting effect corresponding to the first trigger information, the relay terminal is instructed to send the corresponding video image to the non-relay terminal that generated the first trigger information and the non-relay terminal that generated the second trigger information.

[0017] In one optional implementation, the embedded server is further configured to perform the following steps: Send a first notification to the relay terminal; wherein the first notification is used to prompt the relay terminal to actively establish a peer-to-peer connection with the non-relay terminal; or, Send a second notification to the non-relay terminal; wherein the second notification is used to prompt the non-relay terminal to actively establish a peer-to-peer connection with the relay terminal.

[0018] In one optional implementation, the at least one remote terminal initiating the remote control request via the webpage file includes a first remote terminal and a second remote terminal, wherein the first remote terminal has been designated as a relay terminal; the embedded server is further configured to: If a disconnection of the remote control connection with the first remote terminal is detected, it is determined whether the second remote terminal meets the specified conditions of the relay terminal. If it is determined that the second remote terminal meets the specified conditions of the relay terminal, then the second remote terminal is designated as a relay terminal.

[0019] Secondly, embodiments of this application provide an image forming method, including: In response to a webpage request from a remote terminal, a webpage file is sent to the remote terminal; If a remote control request initiated by the remote terminal through the web page file is detected, the following steps are performed: If the remote terminal is a relay terminal configured by the embedded server, then send video footage to the remote terminal: If the remote terminal is not a relay terminal, a peer-to-peer connection is established between the non-relay terminal and the relay terminal, so that the non-relay terminal can directly obtain the video feed from the relay terminal.

[0020] This application provides an image forming apparatus and an image forming method. The image forming apparatus provided in this application includes a display device and an embedded server. The display device is used to display video images related to the image forming function. The embedded server is used to send a web page file to a remote terminal in response to a web page request. The embedded server is also used to perform the following steps after detecting a remote control request initiated by the remote terminal via a web page file: if the remote terminal is a relay terminal configured by the embedded server, then send video images to the remote terminal; if the remote terminal is not a relay terminal, then facilitate a peer-to-peer connection between the non-relay terminal and the relay terminal, so that the non-relay terminal can directly obtain video images from the relay terminal.

[0021] In the above embodiments, the embedded server in the image forming apparatus automatically sets up a relay terminal in the remote terminal that initiates the remote control request, and transmits video images to the non-relay terminal through the relay terminal. This processing method enables the non-relay terminal to avoid obtaining video images from the image forming apparatus, allowing direct communication between the non-relay terminal and the relay terminal via a peer-to-peer connection. The relay terminal forwards the video images obtained from the image forming apparatus to the non-relay terminal. When multiple terminals initiate remote control requests to the image forming apparatus, this processing method reduces the amount of data sent out by the image forming apparatus, thereby alleviating the technical problem of excessive processing burden caused by multiple terminals accessing the image forming apparatus.

[0022] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this application and, together with the specification, serve to explain the technical solutions of this application. It should be understood that the following drawings only show some embodiments of this application and should not be considered as limiting the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 A schematic diagram of an image forming apparatus provided in an embodiment of this application is shown; Figure 2 This illustration shows a schematic diagram of the interface effect of a web page file provided in an embodiment of this application; Figure 3 A flowchart of an image forming method provided in an embodiment of this application is shown; Figure 4 A flowchart of another image forming method provided by an embodiment of this application is shown; Figure 5 A flowchart of yet another image forming method provided by an embodiment of this application is shown; Figure 6 An interactive flowchart of an image forming method provided in an embodiment of this application is shown. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0026] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0027] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0028] With the gradual development of technology, intelligent technology is increasingly entering people's lives, and people are increasingly fond of various intelligent devices, such as intelligent printing equipment. Intelligent printing equipment is generally equipped with a control panel, which can be used to display device configuration information and operating parameters, and allows users to operate various functions on the panel, especially troubleshooting and recovery. To improve the convenience of intelligent printing equipment, remote operation can also be set up. In the case of multiple users remotely operating the printing equipment, the printing equipment needs to transmit real-time video feeds to multiple users simultaneously. However, due to the limited hardware processing power of some printing equipment, this can easily lead to the printing equipment consuming excessive processor power when providing remote feeds to multiple users simultaneously. This can cause lag and delays when users use other functions of the printing equipment, reducing its usability.

[0029] Based on the above research, this application provides an image forming apparatus and an image forming method. The image forming apparatus provided in this application includes a display device and an embedded server. The display device is used to display video images related to the image forming function, and the embedded server is used to send a web page file to a remote terminal in response to a web page request. The embedded server is also used to perform the following steps after detecting a remote control request initiated by the remote terminal via a web page file: if the remote terminal is a relay terminal configured by the embedded server, then send video images to the remote terminal; if the remote terminal is not a relay terminal, then facilitate a peer-to-peer connection between the non-relay terminal and the relay terminal, so that the non-relay terminal can directly obtain video images from the relay terminal.

[0030] In the above embodiments, the embedded server in the image forming apparatus automatically sets up a relay terminal in the remote terminal that initiates the remote control request, and transmits video images to the non-relay terminal through the relay terminal. This processing method enables the non-relay terminal to avoid obtaining video images from the image forming apparatus, allowing direct communication between the non-relay terminal and the relay terminal via a peer-to-peer connection. The relay terminal forwards the video images obtained from the image forming apparatus to the non-relay terminal. When multiple terminals initiate remote control requests to the image forming apparatus, this processing method reduces the amount of data sent out by the image forming apparatus, thereby alleviating the technical problem of excessive processing burden caused by multiple terminals accessing the image forming apparatus.

[0031] To facilitate understanding of this embodiment, a detailed description of an image forming apparatus disclosed in this application embodiment will be provided first. This image forming apparatus can be a printing device, for example, a device capable of supporting remote control technology and performing functions such as printing, scanning, copying, and faxing.

[0032] See Figure 1The diagram shown is a schematic representation of an image forming apparatus provided in an embodiment of this application. Figure 1 As shown, the image forming apparatus 10 includes a display device 101 and an embedded server 102.

[0033] Display device 101 is used to display video images related to the image forming function.

[0034] The display device is an operation panel pre-set in the image forming equipment, wherein the operation panel is a button-controlled operation panel or a touch screen-controlled operation panel. The image forming function refers to the content displayed on the operation panel when the image forming equipment performs functions such as printing, scanning, and copying. The displayed image forming function can be: color mode (e.g., color printing or black and white printing), paper type (A4 paper), image quality settings (e.g., automatic adjustment), number of output copies (e.g., print one copy), paper tray selection (e.g., paper tray 1), and other setting parameters related to the image forming function.

[0035] In this embodiment, an embedded web server (EWS) is pre-configured within the image forming apparatus. Users can access the embedded server by entering the IP address of the image forming apparatus into their web browser. The embedded server can generate or store web page files for user access.

[0036] Embedded server 102 is used to send web page files to remote terminals in response to web page requests from remote terminals.

[0037] A user uses a web browser on a remote terminal to send a webpage request to the embedded server, which generates a webpage file and returns the file to the remote terminal; alternatively, the embedded server locates the webpage file in its local storage and returns the found file to the remote terminal. Figure 2 As shown, the webpage file contains equipment information, equipment status, and consumable information for the corresponding image forming equipment. Furthermore, users can use this webpage file to configure functions such as printing, copying, scanning, faxing, and networking. For example, users can use this webpage file to configure the printing function of the image forming equipment, specifically setting parameters such as color mode, paper type, image quality settings, number of copies, and paper tray selection. In addition, users can also use this webpage file to initiate remote control requests to the embedded server for the image forming equipment. If user actions are detected... Figure 2 The triggering of the "Remote Control" button indicates that a remote control request for the image forming device has been detected and sent to the embedded server.

[0038] The embedded server 200 is also configured to perform the following steps after detecting a remote control request initiated by a remote terminal via a web page file: If the remote terminal is a relay terminal configured by the embedded server, then send video footage to the remote terminal: If the remote terminal is not a relay terminal, a peer-to-peer connection is established between the non-relay terminal and the relay terminal so that the non-relay terminal can directly obtain video footage from the relay terminal.

[0039] After detecting a remote control request initiated by a remote terminal via a web page file, the embedded server determines whether the remote terminal is a relay terminal configured by the embedded server. If yes, the embedded server sends a video feed to the remote terminal, which includes a real-time view of the image forming device's control panel. If no, the remote terminal is determined to be a non-relay terminal. In this case, the embedded server needs to actively or passively facilitate a peer-to-peer connection between the non-relay terminal and the relay terminal, allowing the non-relay terminal to directly obtain the video feed from the relay terminal.

[0040] When multiple terminals initiate remote control requests to the image forming device, this processing method can reduce the amount of data sent out by the image forming device, thereby alleviating the technical problem of excessive processing burden caused by multiple terminals accessing the image forming device.

[0041] In this embodiment of the application, the embedded server 200 is further configured to: When a peer-to-peer connection is not established between a non-relay terminal and a relay terminal, signaling messages are forwarded between the non-relay terminal and the relay terminal based on the remote control request initiated by the non-relay terminal, so as to facilitate a peer-to-peer connection between the relay terminal and the remote terminal.

[0042] After receiving a remote control request initiated by a remote terminal via a web page file, the embedded server determines whether the remote terminal is a relay terminal. If the remote terminal is determined to be a non-relay terminal, it further determines whether a peer-to-peer connection has been established between the remote terminal and the relay terminal. If no peer-to-peer connection has been established between the remote terminal and the relay terminal, the embedded server forwards signaling messages between the non-relay terminal and the relay terminal. These signaling messages facilitate a peer-to-peer connection between the relay terminal and the remote terminal.

[0043] In this embodiment, a list is pre-created to record the terminal identifiers of non-relay terminals that have established a peer-to-peer connection with the relay terminal at the current time. When a non-relay terminal establishes a peer-to-peer connection with the relay terminal, the non-relay terminal is added to the list; when the peer-to-peer connection between the non-relay terminal and the relay terminal is broken, the non-relay terminal is removed from the list.

[0044] Therefore, when the embedded server determines that the remote terminal initiating the remote control request is a non-relay terminal, it searches for the terminal identifier of the non-relay terminal in the list. If the terminal identifier of the non-relay terminal is not found, it is determined that a peer-to-peer connection has not been established between the non-relay terminal and the relay terminal. In this case, the embedded server facilitates a peer-to-peer connection between the relay terminal and the remote terminal by forwarding signaling messages between the non-relay terminal and the relay terminal. At this point, the terminal identifier of the non-relay terminal can be added to the list.

[0045] In one possible implementation, the process by which the embedded server forwards signaling messages between non-relay terminals and relay terminals is described as follows: After receiving a remote control request from a non-relay terminal, the embedded server sends a peer-joined signaling message (i.e., a peer-to-peer connection establishment request) to the relay terminal. Then, the relay terminal sends an offer signaling message to the embedded server, which carries configuration information for establishing the peer-to-peer connection. Upon receiving the offer signaling message, the embedded server forwards it to the non-relay terminal. Upon receiving the offer signaling message, the non-relay terminal sends an answer signaling message to the embedded server, which also carries configuration information for establishing the peer-to-peer connection. Upon receiving the answer signaling message, the embedded server forwards it to the relay terminal. Subsequently, the relay terminal sends a candidate signaling message to the embedded server. Upon receiving this message, the embedded server forwards it to the non-relay terminal. Then, the non-relay terminal sends a reply message to the embedded server. By exchanging candidate signaling messages, ICE Candidates can be collected and exchanged, thus determining the possible network path information between the non-relay terminal and the relay terminal. Finally, after finding a path based on the ICE mechanism, a secure peer-to-peer connection is established.

[0046] As described above, the technical solution of this application uses WebRTC technology. WebRTC technology can establish peer-to-peer connections between terminals without the aid of an intermediary, enabling the transmission of video and / or audio streams or other arbitrary data. However, if the terminals establishing the peer-to-peer connection are located on different networks, the browsers used to initiate remote control requests on these two terminals can only know their own internal IP addresses, not their respective public IP addresses. This prevents cross-network data transmission, requiring the use of a STUN / TURN server to achieve NAT traversal for data transmission. Therefore, in this situation, WebRTC technology first needs to solve the NAT traversal problem. However, the embedded server proposed in this application is an embedded web server built into the image forming device. Since the terminals connected to the image forming device (e.g., printer) are all in the same local area network as the image forming device, this application only needs to use local area network IPs and does not involve public network IPs. In other words, this application can achieve point-to-point connections between terminals without solving the NAT traversal problem. Through this point-to-point connection, video images can be automatically sent to non-relay terminals through relay terminals, thereby effectively reducing the amount of data sent out by the image forming device and achieving unexpected results.

[0047] In this embodiment of the application, the webpage file includes first code, which is used to send video footage to a non-relay terminal that requests the video footage; the first code is embedded in the webpage file, and specifically, the first code is used to implement the following functions: The first code is used to enable the relay terminal to establish a peer-to-peer connection with the non-relay terminal based on the signaling messages forwarded by the embedded server when there is no connection between the relay terminal and the non-relay terminal; it is also used to send video images to the non-relay terminal that requests video images after the peer-to-peer connection is established.

[0048] In this embodiment of the application, the embedded server responds to a webpage request from a remote terminal by sending a webpage file to the remote terminal, wherein the webpage file sent to the remote terminal contains at least a first code.

[0049] In this embodiment, after receiving a remote control request initiated by a non-relay terminal, the embedded server determines whether a peer-to-peer connection has been established between the non-relay terminal and the relay terminal. If it is determined that no peer-to-peer connection has been established, the server forwards signaling messages between the non-relay terminal and the relay terminal according to the method described in the above embodiment. At this time, the first code in the webpage file sent to the relay terminal enables the relay terminal to establish a peer-to-peer connection with the non-relay terminal based on the signaling messages forwarded by the embedded server.

[0050] After determining that a peer-to-peer connection has been established between the relay terminal and the non-relay terminal based on the signaling messages forwarded by the embedded server, the first code instructs the relay terminal to send the video image to the non-relay terminal that requested the video image. Specifically, the first code can add the non-relay terminal to the sending queue. When the relay terminal receives the video image, it sends the received video image to the non-relay terminal in the sending queue through the established peer-to-peer connection.

[0051] The first code is used to enable the relay terminal to add the non-relay terminal requesting the video feed to the sending queue when a connection is established between the relay terminal and the non-relay terminal; it is also used to enable the relay terminal to send the video feed to at least one non-relay terminal in the sending queue through a peer-to-peer connection when the relay terminal receives the video feed.

[0052] In this embodiment of the application, the embedded server responds to a webpage request from a remote terminal by sending a webpage file to the remote terminal, wherein the webpage file sent to the remote terminal contains at least a first code.

[0053] In this embodiment, after receiving a remote control request initiated by a non-relay terminal, the embedded server determines whether a peer-to-peer connection has been established between the non-relay terminal and the relay terminal. If a peer-to-peer connection is established, an optional implementation involves the embedded server sending a notification message to the relay terminal to notify it to transmit video footage to the non-relay terminal. After receiving the notification message from the embedded server, the relay terminal can use a first code in the webpage file to add the non-relay terminal requesting the video footage to a sending queue. Furthermore, when the relay terminal receives the video footage, this first code can also enable it to send the video footage to the non-relay terminal in the sending queue via the peer-to-peer connection. If it is determined that a peer-to-peer connection has been established, another optional implementation is that the embedded server sends a notification message to the non-relay terminal, and the non-relay terminal requests video footage from the relay terminal. At this time, the first code in the web page file sent to the relay terminal can enable the relay terminal to add the non-relay terminal requesting video footage to the sending queue. When the relay terminal receives the video footage, the first code can also enable the relay terminal to send the video footage to the non-relay terminal in the sending queue through the peer-to-peer connection.

[0054] Here, one optional implementation is that the webpage file sent by the embedded server to the relay terminal contains the first code, while the webpage file sent by the embedded server to the non-relay terminal does not contain the first code. For example, after receiving a remote control request from a remote terminal, the embedded server determines whether a relay terminal exists. If it determines that no relay terminal exists, it sets the remote terminal as a relay terminal and returns a webpage file containing the first code to the remote terminal. If it determines that a relay terminal exists, it returns a webpage file without the first code to the remote terminal. Another optional implementation is that the webpage file sent by the embedded server to the relay terminal contains the first code, and the webpage file sent by the embedded server to the non-relay terminal also contains the first code. However, the first code contained in the webpage file sent to the non-relay terminal is not used to perform the functions described above. When the non-relay terminal is designated as a relay terminal, the first code performs the functions described above.

[0055] In this embodiment, the webpage file includes second code, which is used to receive video footage from a relay terminal. This second code is embedded within the webpage file and specifically performs the following functions: The second code is used to enable the non-relay terminal to establish a peer-to-peer connection with the relay terminal based on the signaling messages forwarded by the embedded server when there is no connection between the non-relay terminal and the relay terminal.

[0056] In this embodiment of the application, the embedded server responds to a webpage request from a remote terminal by sending a webpage file to the remote terminal, wherein the webpage file sent to the remote terminal contains at least a second code.

[0057] In this embodiment, after receiving a remote control request initiated by a non-relay terminal, the embedded server determines whether a peer-to-peer connection has been established between the non-relay terminal and the relay terminal. If it is determined that no peer-to-peer connection has been established, the server forwards signaling messages between the non-relay terminal and the relay terminal according to the method described in the above embodiment. At this time, the second code in the webpage file sent to the non-relay terminal and the first code in the webpage file sent to the relay terminal enable the non-relay terminal and the relay terminal to establish a peer-to-peer connection based on the signaling messages forwarded by the embedded server.

[0058] The first code in the webpage file sent to the relay terminal determines, based on the signaling messages forwarded by the embedded server, that a peer-to-peer connection has been established between the relay terminal and the non-relay terminal. This causes the relay terminal to send the video feed to the non-relay terminal that requested the video feed. Specifically, the first code can add the non-relay terminal to a sending queue. When the relay terminal receives the video feed, it sends the received video feed to the non-relay terminal in the sending queue through the established peer-to-peer connection.

[0059] The second code is used to request video feeds from the relay terminal only through the peer-to-peer connection after a peer-to-peer connection is established between the non-relay terminal and the relay terminal.

[0060] In this embodiment of the application, the embedded server responds to a webpage request from a remote terminal by sending a webpage file to the remote terminal, wherein the webpage file sent to the remote terminal contains at least a second code.

[0061] In this embodiment, after receiving a remote control request initiated by a non-relay terminal, the embedded server determines whether a peer-to-peer connection has been established between the non-relay terminal and the relay terminal. If a peer-to-peer connection is established, an optional implementation involves the embedded server sending a notification message to the relay terminal to notify it to transmit video footage to the non-relay terminal. After receiving the notification message from the embedded server, the relay terminal can use a first code in the webpage file to add the non-relay terminal requesting the video footage to a sending queue. Furthermore, when the relay terminal receives the video footage, this first code can also enable it to send the video footage to the non-relay terminal in the sending queue via the peer-to-peer connection. If it is determined that a peer-to-peer connection has been established, another optional implementation is that the embedded server sends a notification message to the non-relay terminal. After receiving the notification message, the non-relay terminal requests a video feed from the relay terminal through the second code. At this time, the first code in the webpage file sent to the relay terminal can enable the relay terminal to add the non-relay terminal requesting the video feed to the sending queue. When the relay terminal receives the video feed, the first code can also enable the relay terminal to send the video feed to the non-relay terminal in the sending queue through the peer-to-peer connection.

[0062] Here, one optional implementation is that the webpage file sent by the embedded server to the non-relay terminal contains the second code, while the webpage file sent by the embedded server to the relay terminal does not contain the second code. For example, after receiving a remote control request from a remote terminal, the embedded server determines whether a relay terminal exists. If it determines that no relay terminal exists, it sets the remote terminal as a relay terminal and returns a webpage file containing at least the first code to the remote terminal. If it determines that a relay terminal exists, it returns a webpage file containing at least the second code to the remote terminal. Another optional implementation is that the webpage file sent by the embedded server to the non-relay terminal contains the second code, and the webpage file sent by the embedded server to the relay terminal also contains the second code; however, the second code contained in the webpage file sent to the relay terminal is not used to perform the functions described above. Another optional implementation is that both the first code and the second code are included in the webpage files sent to both the non-relay terminal and the relay terminal.

[0063] By setting the first and second codes in the web page file, a peer-to-peer connection between the relay terminal and the non-relay terminal can be established without the user's awareness. The relay terminal can then transmit video footage to the non-relay terminal. This approach reduces user involvement, allowing users to remain completely undisturbed. As a result, video footage can be forwarded remotely without generating any information that might intrude on the user.

[0064] In this embodiment of the application, the embedded server is also used for: A relay terminal is specified in at least one remote terminal that initiates a remote control request via a web page file, wherein the specified terminal is determined according to the order in which the remote control request is initiated and / or the processing performance of the remote terminal.

[0065] After receiving a remote control request from any remote terminal, the embedded server designates a relay terminal among at least one of the remote terminals that initiated the remote control request; or, after receiving remote control requests from a specified number of remote terminals, the embedded server designates a relay terminal among multiple remote terminals that initiated the remote control requests. The embedded server may designate one remote terminal as a relay terminal, or it may designate multiple remote terminals as relay terminals.

[0066] The embedded server can designate a relay terminal among at least one remote terminal based on the order initiating remote control requests and / or the processing performance of the remote terminal. Specifically, the embedded server determines the relay terminal from the first N remote terminals that initiated remote control requests; alternatively, the embedded server determines a remote terminal with satisfactory processing performance from the remote terminals that initiated remote control requests as the relay terminal; or, the embedded server determines a remote terminal with satisfactory processing performance from the first N remote terminals that initiated remote control requests as the relay terminal.

[0067] In this embodiment of the application, when processing performance includes network response speed, the embedded server is further used for: Configure the first remote terminal that initiates a remote control request via a webpage file as the relay terminal.

[0068] or, Remote terminals whose network response speed meets a preset speed threshold are configured as relay terminals.

[0069] In one possible implementation, the embedded server identifies the first remote terminal to initiate a remote control request via a webpage file and configures that remote terminal as a relay terminal. Here, the first remote terminal to initiate a remote control request via a webpage file can be understood as the first remote terminal to initiate a remote control request to the embedded server when no other terminal is currently remotely controlling the image forming device. In this case, the embedded server sets that remote terminal as a relay terminal. For example, at time T1, remote terminal 1 and remote terminal 2 initiate remote control requests via a webpage file; subsequently, remote terminal 1 and remote terminal 2 sequentially end their remote control of the image forming device. Suppose that at time T1, remote terminal 3 initiates a remote control request via a webpage file. In this case, remote terminal 3 is identified as the first remote terminal to initiate a remote control request via a webpage file, and remote terminal 3 is configured as a relay terminal.

[0070] In another possible implementation, after receiving remote control requests initiated by multiple remote terminals via web page files, the embedded server identifies the remote terminal with a network response speed greater than or equal to a preset speed threshold as a relay terminal. Specifically, after receiving a remote control request, the embedded server acquires the network response speed of each remote terminal and determines whether the network response speed is greater than or equal to the preset threshold. If the threshold is met, the remote terminal is configured as a relay terminal; otherwise, the server continues to acquire the network response speed of the next remote terminal initiating a remote control request.

[0071] In another possible implementation, after receiving the first remote control request initiated via a webpage file, the embedded server determines whether the network response speed of the remote terminal is greater than or equal to a preset speed threshold. If the network response speed of the remote terminal is determined to be greater than or equal to the preset speed threshold, the remote terminal is directly configured as a relay terminal. Otherwise, it continues to determine whether the network response speed of the next remote terminal initiating a remote control request via a webpage file is greater than or equal to the preset speed threshold. If the network response speeds of M consecutive remote terminals initiating remote control requests are all less than the preset speed threshold, the remote terminal that initiated the first remote control request via a webpage file can be configured as a relay terminal. Alternatively, among the M remote terminals initiating remote control requests, the remote terminal with the smallest difference between its network response speed and the preset speed threshold can be determined and configured as a relay terminal.

[0072] By specifying the relay terminal in the initiation sequence, video images can be transmitted to the requesting remote terminal via the relay terminal as soon as the remote terminal initiates remote control, effectively reducing the amount of data that the image forming equipment simultaneously provides to the remote terminals. By specifying the relay terminal based on processing performance, video images can be stably transmitted to non-relay terminals while ensuring the normal operation of the relay terminal. Even if multiple remote terminals initiate remote control requests simultaneously, high-quality and stable transmission of video images can still be achieved.

[0073] In this embodiment of the application, when at least one remote terminal initiating a remote control request through the webpage file includes a first remote terminal and a second remote terminal, and the first remote terminal has been designated as a relay terminal, the embedded server is further configured to: Based on the order initiation of the remote control request and / or the processing performance of the remote terminal, determine whether the second remote terminal meets the specified conditions of the relay terminal. If the specified conditions for a relay terminal are met, then the second remote terminal will replace the first remote terminal as a relay terminal, or the second remote terminal will be designated as another relay terminal.

[0074] The device comprises one or more first remote terminals and one or more second remote terminals, wherein the first remote terminals are configured as relay terminals by the embedded server. The first and second remote terminals include the terminal currently initiating a remote control request to the image forming device.

[0075] The embedded server determines whether the current time is the designated time for the relay terminal. If it determines that the current time is the designated time for the relay terminal, it identifies a second remote terminal that meets the designated conditions for the relay terminal and replaces the remote terminal with the second remote terminal as a relay terminal, or designates the second remote terminal as another relay terminal.

[0076] In specific implementation, when the embedded server detects a new remote terminal initiating a remote control request via a webpage file, it determines whether the second remote terminal meets the specified conditions for a relay terminal based on the order initiation of the remote control requests and / or the processing performance of the remote terminal; or, when the embedded server detects that a relay terminal has ended remote control of the image forming device, it determines whether the second remote terminal meets the specified conditions for a relay terminal based on the order initiation of the remote control requests and / or the processing performance of the remote terminal; or, when the embedded server detects that the processing performance of an existing relay terminal cannot meet the transmission requirements of video images from all non-relay terminals, it determines whether the second remote terminal meets the specified conditions for a relay terminal based on the order initiation of the remote control requests and / or the processing performance of the remote terminal.

[0077] In this embodiment of the application, the embedded server can determine a second remote terminal that meets the specified conditions of a relay terminal based on the order initiation of the remote control request and / or the processing performance of the remote terminal, and replace the first remote terminal with the second remote terminal as a relay terminal, or designate the second remote terminal as another relay terminal.

[0078] In practical implementation, the embedded server can identify the first remote terminal initiating the remote control request among the second remote terminals, and determine that this second remote terminal meets the specified conditions for a relay terminal. For example, if the embedded server detects that the first remote terminal has ended its remote control of the image forming device, it identifies the first remote terminal initiating the remote control request among the second remote terminals and uses this second remote terminal as a relay terminal instead of the first. The embedded server can also identify a second remote terminal whose network response speed meets a preset speed threshold among the second remote terminals, and determine that this second remote terminal meets the specified conditions for a relay terminal. For example, if the embedded server detects that the first remote terminal has ended its remote control of the image forming device, or detects that the existing first remote terminal cannot meet the video transmission requirements of all non-relay terminals, it identifies a second remote terminal whose network response speed meets a preset speed threshold among the second remote terminals and uses this second remote terminal as a relay terminal instead of the first, or designates this second remote terminal as another relay terminal. The embedded server can also identify the first second remote terminal whose network response speed meets a preset speed threshold according to the initiation order, and determine that this second remote terminal meets the specified conditions for a relay terminal. For example, if the embedded server detects that the first remote terminal has ended its remote control of the image forming device, or detects that the existing first remote terminal cannot meet the transmission requirements of all non-relay terminals' video images, then it determines the first second remote terminal whose network response speed meets the preset speed threshold among the second remote terminals according to the initiation order, and replaces the first remote terminal as a relay terminal, or designates the second remote terminal as another relay terminal.

[0079] By using the above processing method, when a first remote terminal withdraws from remote control of the image forming device, or when the first remote terminal cannot meet the video transmission requirements of all non-relay terminals, the normal transmission of video images can be guaranteed by selecting a second remote terminal that meets the specified conditions of a relay terminal. Moreover, this processing method does not require human intervention in the process of specifying relay terminals, and can realize automatic switching of relay terminals, thereby improving the stability of the remote control technology in the image forming device proposed in the embodiments of this application.

[0080] In this embodiment of the application, the embedded server is also used for: The relay terminal receives trigger information from the non-relay terminal. The trigger information is generated by the non-relay terminal's image forming function in the video frame received from the relay terminal. Based on the relay terminal's storage of video footage associated with the trigger information, a method is determined to instruct the relay terminal to send the video footage to the non-relay terminal that generated the trigger information.

[0081] After receiving a video frame from a relay terminal, the non-relay terminal displays the video frame, which includes at least an image forming function. The non-relay terminal detects the user's trigger operation on the image forming function within the video frame and generates trigger information. This trigger information includes the image forming function triggered by the user and the trigger content for that function; for example, if the image forming function is a color mode, the trigger content might be to change the color mode to color printing. The non-relay terminal then sends this trigger information to the relay terminal, which forwards it to the embedded server. The embedded server determines whether the relay terminal has a video frame associated with the trigger information, thus obtaining the relay terminal's storage information for that video frame. Based on this storage information, the server decides how to instruct the relay terminal to send the video frame to the non-relay terminal that generated the trigger information.

[0082] Here, the embedded server determines whether the video frame has been sent to the relay terminal. If it determines that the video frame has been sent, it confirms that the relay terminal has a video frame associated with the trigger information; otherwise, it determines that the relay terminal does not have a video frame associated with the trigger information. In specific implementation, the embedded server first determines whether a video frame associated with the trigger information has been generated. If it determines that it has not been generated, it determines that the relay terminal does not have a video frame associated with the trigger information; if it determines that it has been generated, it searches the sent video frames for a video frame associated with the trigger information.

[0083] Here, the embedded server pre-sets a tag for each sent video frame. This tag indicates a combination of one or more information, such as trigger information associated with the sent video frame, timestamp information of the sent video frame, and UI name of the sent video frame. The sent video frame refers to the video frame that has been sent to the relay terminal. At this time, the embedded server searches the sent video frames according to the tag to see if there is a video frame associated with trigger information sent by a non-relay terminal. If a frame is found, it is determined that the embedded server has sent that video frame to the relay terminal; otherwise, it is determined that the embedded server has not sent that video frame to the relay terminal.

[0084] In this embodiment of the application, the embedded server is also used for: When it is determined that the relay terminal does not have any video footage related to the trigger information stored locally, a reconstructed video is generated based on the trigger information. The reconstructed image is sent to the relay terminal as a video image associated with the trigger information; The relay terminal is instructed to send the received reconstructed screen to the non-relay terminal that generated the trigger information.

[0085] When the embedded server determines that it has not sent a video frame associated with the trigger information to the relay terminal, it determines that the relay terminal does not locally store the video frame associated with that trigger information. At this point, the embedded server needs to generate a reconstructed frame based on the trigger information to obtain the video frame associated with that trigger information. Then, the embedded server sends the video frame associated with the trigger information and an instruction message to the relay terminal. This instruction message instructs the relay terminal to send the received reconstructed frame to the non-relay terminal that generated the trigger information. Afterward, the embedded server marks the reconstructed frame as a sent video frame and sets a tag for it, i.e., the trigger information described above.

[0086] The reconstructed screen can be a video frame constructed by the processor of the image forming apparatus according to set parameters, or it can be a non-real-time display screen. For example, if the display device displays the main menu interface at time T, and the user clicks on the video frame in the non-relay terminal to select the "Task List" function in the main menu, the processor of the image forming apparatus can generate a "Task List" interface based on the internally stored job list information. Its appearance can be the same as or similar to the interface displayed when the user clicks on the display device. In one embodiment, the reconstructed screen can be a series of image frames that can be updated only when the set parameters change, such as when an image forming task is added to or removed from the "Task List," and not updated when the set parameters do not change, in order to reduce unnecessary data transmission.

[0087] In this embodiment of the application, the embedded server is also used for: When it is determined that the relay terminal has video frames associated with the trigger information stored locally, the relay terminal is instructed to search for historical video frames associated with the trigger information in the locally stored video frames, and return the historical video frames to the non-relay terminal that generated the trigger information.

[0088] When the embedded server determines that it has sent a video frame associated with the trigger information to the relay terminal, it determines that the relay terminal has the video frame associated with the trigger information stored locally. At this time, the embedded server sends an instruction message to the relay terminal, instructing the relay terminal to search for the historical video frame associated with the trigger information in the locally stored video frame, and return the historical video frame to the non-relay terminal that generated the trigger information.

[0089] After receiving the instruction information, the relay terminal searches for historical video frames associated with the trigger information in its locally stored video frames. The video frames stored locally in the relay terminal carry tags, which indicate a combination of one or more information associated with the video frame, such as trigger information, timestamp information, and UI name. The UI name indicates the image forming function corresponding to the trigger information.

[0090] The relay terminal can search for the tag associated with the trigger information in the tags of the video frame stored locally, and determine the video frame corresponding to the associated tag as the historical video frame associated with the trigger information. Then, it returns the historical video frame to the non-relay terminal that generated the trigger information.

[0091] When a tag is used to indicate the trigger information associated with the video frame, the embedded server sends an indication message to the relay terminal containing trigger information from the non-relay terminal (hereinafter referred to as trigger information A). At this time, the relay terminal searches for the video frame corresponding to the tag used to indicate trigger information A, and identifies the found video frame as the historical video frame associated with the trigger information.

[0092] When the tag is used to indicate the timestamp information of the video frame, the embedded server sends an indication message to the relay terminal carrying the timestamp information of the historical video frame associated with the trigger information (hereinafter referred to as timestamp information A). At this time, the relay terminal searches for the video frame corresponding to the tag used to indicate timestamp information A, and identifies the found video frame as the historical video frame associated with the trigger information.

[0093] When the label is used to indicate the UI name of the video frame, the embedded server sends an indication message to the relay terminal containing the UI name of the historical video frame associated with the trigger information (hereinafter referred to as UI name A). At this time, the relay terminal searches for the video frame corresponding to the label used to indicate UI name A, and identifies the found video frame as the historical video frame associated with the trigger information.

[0094] In this embodiment, when it is determined that the relay terminal has video footage associated with the trigger information stored locally, the system can be configured to play the video footage stored locally on the relay terminal via a player on the non-relay terminal side. Users can drag the player's progress bar to find the video footage they need.

[0095] In this application, the trigger information carries the terminal identifier of the corresponding non-relay terminal, and the video frame associated with the trigger information transmitted from the relay terminal to each non-relay terminal carries the terminal identifier of the non-relay terminal.

[0096] The terminal identifier can be information that indicates the identity of the non-relay terminal, such as the ID of the non-relay terminal, the timestamp of the non-relay terminal sending the trigger information to the relay terminal, or the user information of the logged-in web page file.

[0097] The triggering information sent from a non-relay terminal to a relay terminal may carry one or more of the aforementioned terminal identifiers. The video footage associated with the triggering information transmitted from the relay terminal to each non-relay terminal carries the same terminal identifier.

[0098] When there are multiple non-relay terminals sending trigger information, this processing method can quickly and accurately identify and respond to each trigger information, avoiding the situation where video footage associated with the trigger information is mistakenly sent to other non-relay terminals.

[0099] In this embodiment, the video feed includes a real-time view of the operation panel of the image forming apparatus, and / or a function setting interface of the image forming apparatus, the function setting interface including at least one function setting item associated with the image forming function. The rendering of the function setting interface of the image forming apparatus can be as follows: Figure 2 The settings interface shown.

[0100] In this case, the embedded server is also used for: Receive the first trigger information generated by the non-relay terminal on the real-time screen and the second trigger information generated on the function setting interface, sent by the relay terminal; If the function setting effect corresponding to the second trigger information does not conflict with the function setting effect corresponding to the first trigger information, the relay terminal is instructed to send the corresponding video image to the non-relay terminal that generated the first trigger information and the non-relay terminal that generated the second trigger information.

[0101] If the embedded server receives a first trigger message for the real-time video feed and a second trigger message for the function settings interface, the embedded server needs to determine whether the function settings effect corresponding to the second trigger message conflicts with the function settings effect corresponding to the first trigger message. If a conflict is determined, the server instructs the relay terminal to transmit the relevant video feed only to the non-relay terminal that generated the first trigger message, or to transmit the corresponding video feed only to the non-relay terminal that generated the second trigger message, or to not transmit the corresponding video feed to either the non-relay terminal that generated the first or second trigger message. If no conflict is determined, the server instructs the relay terminal to send the corresponding video feed to both the non-relay terminal that generated the first trigger message and the non-relay terminal that generated the second trigger message.

[0102] In specific implementation, the embedded server determines whether there is a corresponding image forming function in the image forming functions corresponding to the first trigger information and the second trigger information. If there is a corresponding image forming function, and for that image forming function, it is determined that the setting information corresponding to the first trigger information is different from the setting information corresponding to the second trigger information, then it is determined that the function setting effect corresponding to the second trigger information conflicts with the function setting effect corresponding to the first trigger information.

[0103] When different users trigger the image forming function through different remote terminals, different triggering information will be generated. If the function settings corresponding to these different triggering information conflict, the embedded server will be unable to accurately provide the user with the corresponding video image, resulting in the user's failure to remotely control the image forming device and thus affecting the user's remote control experience. The above implementation method can reduce the probability of user's remote control failure of the image forming device and further improve the user's remote control experience.

[0104] In this embodiment of the application, the embedded server is also used to perform the following steps: Send a first notification to the relay terminal; wherein the first notification is used to prompt the relay terminal to actively establish a peer-to-peer connection with the non-relay terminal.

[0105] After receiving a remote control request initiated by a non-relay terminal via a web page file, if the embedded server determines that no peer-to-peer connection has been established between the non-relay terminal and the relay terminal, it needs to forward signaling messages between the non-relay terminal and the relay terminal to facilitate a peer-to-peer connection between the relay terminal and the remote terminal.

[0106] The embedded server sends a first notification to the relay terminal, informing it that a peer-to-peer connection needs to be established with the non-relay terminal. After confirming the establishment of the peer-to-peer connection with the non-relay terminal, the relay terminal sends an establishment confirmation message to the embedded server. Subsequently, the embedded server forwards signaling messages between the non-relay terminal and the relay terminal to facilitate the peer-to-peer connection between the relay terminal and the remote terminal.

[0107] The confirmation message can be automatically sent via the first code contained in a webpage file sent to the relay terminal, or it can be manually sent by the user. For example, after receiving the first notification, the relay terminal displays the first notification on its display interface. After detecting the user's confirmation of the first notification, it determines that the confirmation message has been detected.

[0108] In this embodiment of the application, the embedded server is also used to perform the following steps: Send a second notification to the non-relay terminal; wherein the second notification is used to prompt the non-relay terminal to actively establish a peer-to-peer connection with the relay terminal.

[0109] After receiving a remote control request initiated by a non-relay terminal via a web page file, if the embedded server determines that no peer-to-peer connection has been established between the non-relay terminal and the relay terminal, it needs to forward signaling messages between the non-relay terminal and the relay terminal to facilitate a peer-to-peer connection between the relay terminal and the remote terminal.

[0110] The embedded server sends a second notification to the non-relay terminal, informing it that a peer-to-peer connection needs to be established with the relay terminal. After confirming the establishment of the peer-to-peer connection, the non-relay terminal sends an establishment confirmation message to the embedded server. Subsequently, the embedded server forwards signaling messages between the non-relay terminal and the relay terminal to facilitate the peer-to-peer connection between the relay terminal and the remote terminal.

[0111] The confirmation message can be automatically sent via a second code contained in a webpage file sent to the non-relay terminal, or it can be sent manually by the user. For example, after receiving the second notification, the non-relay terminal displays the second notification on its display interface. After detecting the user's confirmation of the second notification, it is determined that the confirmation message has been detected.

[0112] In this embodiment, the embedded server first sends a first notification to the relay terminal. If it does not receive confirmation of connection from the relay terminal, it sends a second notification to the non-relay terminal. If it does not receive confirmation of connection from the non-relay terminal, it periodically sends the first notification to the relay terminal until the number of communication attempts exceeds a predetermined limit, or until it receives confirmation of connection from the relay terminal.

[0113] In the above embodiments, by sending a first notification to a relay terminal or a second notification to a non-relay terminal, different connection methods for peer-to-peer connections can be provided to meet the user's needs.

[0114] In this embodiment of the application, at least one remote terminal initiating a remote control request via a webpage file includes a first remote terminal and a second remote terminal, wherein the first remote terminal has been designated as a relay terminal; the embedded server is further used for: If a disconnection of the remote control connection with the first remote terminal is detected, it is determined whether the second remote terminal meets the specified conditions of the relay terminal. If it is determined that the second remote terminal meets the specified conditions of the relay terminal, then the second remote terminal is designated as a relay terminal.

[0115] The device comprises one or more first remote terminals and one or more second remote terminals, wherein the first remote terminals are configured as relay terminals by the embedded server. The first and second remote terminals include the terminal currently initiating a remote control request to the image forming device.

[0116] If the embedded server detects that the remote control connection between any of the first remote terminals and the embedded server is broken, it identifies a second remote terminal among the second remote terminals that meets the specified conditions for a relay terminal and designates the second remote terminal that meets the specified conditions as a relay terminal. Alternatively, if the embedded server detects that the remote control connection between all the first remote terminals and the embedded server is broken, it identifies a second remote terminal among the second remote terminals that meets the specified conditions for a relay terminal and designates the second remote terminal that meets the specified conditions as a relay terminal. Or, if the embedded server detects that the number of first remote terminals with broken remote control connections to the embedded server is greater than a specified number, it identifies a second remote terminal among the second remote terminals that meets the specified conditions for a relay terminal and designates the second remote terminal that meets the specified conditions as a relay terminal.

[0117] In this embodiment of the application, the embedded server can determine a second remote terminal that meets the specified conditions of a relay terminal based on the order initiation of the remote control request and / or the processing performance of the remote terminal, and replace the first remote terminal with the second remote terminal as a relay terminal, or designate the second remote terminal as another relay terminal.

[0118] In specific implementation, the embedded server can identify the first remote terminal initiating the remote control request among the second remote terminals, and determine that this second remote terminal meets the specified conditions for a relay terminal. For example, if the embedded server detects that the first remote terminal has ended its remote control of the image forming device, it identifies the first remote terminal initiating the remote control request among the second remote terminals and uses this second remote terminal as a relay terminal instead of the first. The embedded server can also identify a second remote terminal whose network response speed meets a preset speed threshold among the second remote terminals, and determine that this second remote terminal meets the specified conditions for a relay terminal. For example, if the embedded server detects that the first remote terminal has ended its remote control of the image forming device, it identifies a second remote terminal whose network response speed meets a preset speed threshold among the second remote terminals and designates this second remote terminal as a relay terminal. The embedded server can also identify the first second remote terminal whose network response speed meets the preset speed threshold according to the initiation order, and determine that this second remote terminal meets the specified conditions for a relay terminal. For example, if the embedded server detects that the first remote terminal has ended its remote control of the image forming device, it determines the first second remote terminal whose network response speed meets the preset speed threshold in the order of initiation, and designates the second remote terminal as a relay terminal.

[0119] By using the above processing method, when a first remote terminal exits remote control of the image forming device, the normal transmission of video images can be guaranteed by filtering a second remote terminal that meets the specified conditions of the relay terminal. Moreover, this processing method does not require human intervention in the process of specifying the relay terminal, and can realize automatic switching of the relay terminal, thereby improving the stability of the remote control technology in the image forming device proposed in the embodiments of this application.

[0120] Reference Figure 3 The diagram shown is a flowchart of an image forming method provided in an embodiment of this application. Figure 3 As shown, the method includes the following steps: S301: In response to a webpage request from a remote terminal, send a webpage file to the remote terminal.

[0121] A user uses a web browser on a remote terminal to send a webpage request to the embedded server, which generates a webpage file and returns the file to the remote terminal; alternatively, the embedded server locates the webpage file in its local storage and returns the found file to the remote terminal. Figure 2As shown, the webpage file contains equipment information, equipment status, and consumable information for the corresponding image forming device. Furthermore, users can use this webpage file to configure functions such as printing, copying, scanning, faxing, and networking. For example, users can use this webpage file to configure the printing function of the image forming device, specifically setting parameters such as color mode, paper type, image quality settings, number of copies, and paper tray selection. In addition, users can also use this webpage file to initiate remote control requests for the image forming device to the embedded server.

[0122] S302: If a remote control request initiated by a remote terminal via a web page file is detected, perform the following steps: S303: If the remote terminal is a relay terminal configured by the embedded server, then send video footage to the remote terminal: If the remote terminal is not a relay terminal, S304 will facilitate a peer-to-peer connection between the non-relay terminal and the relay terminal so that the non-relay terminal can directly obtain video footage from the relay terminal.

[0123] After detecting a remote control request initiated by a remote terminal via a web page file, the embedded server determines whether the remote terminal is a relay terminal configured by the embedded server. If it is, the embedded server sends a video feed to the remote terminal, which includes a real-time view of the image forming device's control panel. If it is not, the remote terminal is determined to be a non-relay terminal. In this case, the embedded server facilitates a peer-to-peer connection between the non-relay terminal and the relay terminal, allowing the non-relay terminal to directly obtain the video feed from the relay terminal.

[0124] When multiple terminals initiate remote control requests to the image forming device, this processing method can reduce the amount of data sent out by the image forming device, thereby alleviating the technical problem of excessive processing burden caused by multiple terminals accessing the image forming device.

[0125] Reference Figure 4 The diagram shown is a flowchart of another image forming method provided in an embodiment of this application. Figure 4 As shown, the method includes the following steps: S401: Remote terminal 1 initiates a remote control request to the embedded server via a web page file; S402: The embedded server determines whether a relay terminal exists; S403: If the embedded server determines that there is no relay terminal, it will configure remote terminal 1 as a relay terminal; S404: The embedded server sends a configuration success message for the relay terminal to remote terminal 1; S405: The embedded server sends video footage to remote terminal 1.

[0126] Reference Figure 5 The diagram shown is a flowchart of another image forming method provided in an embodiment of this application. Figure 5 As shown, the method includes the following steps: S501: Remote terminal 2 initiates a remote control request to the embedded server via a web page file; S502: The embedded server determines whether a relay terminal exists; S503: When the embedded server detects the existence of a relay terminal, it determines whether a peer-to-peer connection has been established between remote terminal 1 and remote terminal 2. S504: When the embedded server determines that a peer-to-peer connection has been established, it instructs remote terminal 2 to request video feed from remote terminal 1. S505: Remote terminal 2 requests video feed from remote terminal 1; S506: Remote terminal 1 sends video footage to remote terminal 2.

[0127] Reference Figure 6 The diagram shown is an interactive flowchart of an image forming method provided in an embodiment of this application. Figure 6 As shown, the method includes the following steps: S601: Remote terminal 1 initiates a remote control request to the embedded server via a web page file; S602: The embedded server configures remote terminal 1 as a relay terminal and sends a successful configuration message to remote terminal 1. S603: Remote terminal 2 initiates a remote control request to the embedded server via a web page file; S604: If the embedded server determines that remote terminal 2 and remote terminal 1 have not established a peer-to-peer connection, it sends a first notification to remote terminal 1. S605: Remote terminal 1 sends an acknowledgment of establishment information to the embedded server; S606: The embedded server sends a peer-joined signaling message to remote terminal 1; S607: Remote terminal 1 sends an offer signaling message to the embedded server; S608: The embedded server sends an offer signaling message to remote terminal 2; S609: Remote terminal 2 sends an answer signaling message to the embedded server; S610: The embedded server sends an answer signaling message to remote terminal 1; S611: Remote terminal 1 sends a candidate signaling message to the embedded server; S612: The embedded server sends a candidate signaling message to remote terminal 2; S613: Remote terminal 2 sends a reply message to the embedded server for the candidate signaling message; S614: The embedded server sends a reply message to the candidate signaling message to remote terminal 1; At this point, the peer-to-peer connection between remote terminal 1 and remote terminal 2 has been established.

[0128] S615: The embedded server sends a video feed to remote terminal 1; S616: Remote terminal 1 stores video footage locally; S617: Remote terminal 1 forwards the video feed to remote terminal 2; S618: Remote terminal 2 updates video feed; S619: Remote terminal 2 sends trigger information to remote terminal 1; S620: Remote terminal 1 sends trigger information to the embedded server; S621: The embedded server returns a video frame associated with the trigger information to remote terminal 1; Another possible implementation of S621 is that when it is determined that the relay terminal has video frames associated with the trigger information stored locally, the embedded server instructs the relay terminal to search for historical video frames associated with the trigger information in the locally stored video frames.

[0129] S622: Remote terminal 1 forwards the video frame associated with the trigger information to remote terminal 2; S623: Remote terminal 2 updates video feed; S624: Remote terminal 1 sends a request to the embedded server to disconnect the remote control connection; S625: The embedded server sends a notification to remote terminal 2 that the relay terminal has been destroyed, and requests that remote terminal 2 be configured as a relay terminal. S626: Remote terminal 2 sends a message to the embedded server confirming the creation of a relay terminal.

[0130] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the image forming method described in the above-described method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0131] This application also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the image forming method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0132] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0133] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0134] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0135] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0136] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0137] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the technical scope disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. An image forming apparatus, characterized in that, include: A display device for displaying video images related to image forming functions; An embedded server is used to send web page files to a remote terminal in response to a web page request. The embedded server is further configured to perform the following steps after detecting a remote control request initiated by the remote terminal through the web page file: If the remote terminal is a relay terminal configured by the embedded server, then the video feed is sent to the remote terminal: If the remote terminal is a non-relay terminal, a peer-to-peer connection is established between the non-relay terminal and the relay terminal, so that the non-relay terminal can directly obtain the video image from the relay terminal.

2. The image forming apparatus according to claim 1, characterized in that, The embedded server is also used for: When no peer-to-peer connection is established between the non-relay terminal and the relay terminal, signaling messages are forwarded between the non-relay terminal and the relay terminal based on the remote control request initiated by the non-relay terminal, so as to facilitate a peer-to-peer connection between the relay terminal and the remote terminal.

3. The image forming apparatus according to claim 1, characterized in that, The webpage file includes first code, which is used to send the video feed to a non-relay terminal that requests the video feed; wherein... The first code is used to enable the relay terminal to establish a peer-to-peer connection with the non-relay terminal based on the signaling message forwarded by the embedded server when there is no connection between the relay terminal and the non-relay terminal; it is also used to send the video image to the non-relay terminal that requested the video image after the peer-to-peer connection is established; or The first code is used to cause the relay terminal to add the non-relay terminal requesting the video image to the sending queue when a connection is established between the relay terminal and the non-relay terminal; it is also used to cause the relay terminal to send the video image to at least one non-relay terminal in the sending queue through the peer-to-peer connection when the relay terminal receives the video image.

4. The image forming apparatus according to claim 1, characterized in that, The webpage file includes second code, which is used to receive the video feed from the relay terminal; wherein... The second code is used to enable the non-relay terminal to establish a peer-to-peer connection with the relay terminal based on the signaling messages forwarded by the embedded server when there is no connection between the non-relay terminal and the relay terminal; and The second code is used to request the video feed from the relay terminal only through the peer-to-peer connection after a peer-to-peer connection is established between the non-relay terminal and the relay terminal.

5. The image forming apparatus according to claim 1, characterized in that, The embedded server is also used for: The relay terminal is specified in at least one of the remote terminals that initiate the remote control request via the web page file, wherein the specification is based on the order in which the remote control request is initiated and / or the processing performance of the remote terminal.

6. The image forming apparatus according to claim 5, characterized in that, The processing performance includes network response speed; the embedded server is also used for: Configure the first remote terminal that initiates a remote control request via a webpage file as the relay terminal; or, Remote terminals whose network response speed meets a preset speed threshold are configured as relay terminals.

7. The image forming apparatus according to claim 5, characterized in that, The at least one remote terminal that initiates the remote control request through the web page file includes a first remote terminal and a second remote terminal, wherein the first remote terminal has been designated as a relay terminal. The embedded server is also used for: Based on the order initiation of the remote control requests and / or the processing performance of the remote terminal, determine whether the second remote terminal meets the specified conditions of the relay terminal; If it is determined that the specified conditions of the relay terminal are met, then the second remote terminal is used as a relay terminal instead of the first remote terminal, or the second remote terminal is designated as another relay terminal.

8. The image forming apparatus according to claim 1, characterized in that, The embedded server is also used for: The relay terminal receives trigger information from a non-relay terminal, the trigger information being generated by the non-relay terminal in response to the image forming function in the video frame received from the relay terminal; Based on the storage status of the video footage associated with the trigger information by the relay terminal, a method is determined to instruct the relay terminal to send the video footage to the non-relay terminal that generated the trigger information.

9. The image forming apparatus according to claim 8, characterized in that, The embedded server is also used for: When it is determined that the relay terminal does not have any video footage associated with the trigger information stored locally, a reconstructed video is generated based on the trigger information; The reconstructed image is sent to the relay terminal as a video image associated with the trigger information; The relay terminal is instructed to send the received reconstructed screen to the non-relay terminal that generated the trigger information.

10. The image forming apparatus according to claim 8, characterized in that, The embedded server is also used for: When it is determined that the relay terminal has video frames associated with the trigger information stored locally, the relay terminal is instructed to search for historical video frames associated with the trigger information in the locally stored video frames, and return the historical video frames to the non-relay terminal that generated the trigger information.

11. The image forming apparatus according to claim 8, characterized in that, The triggering information carries the terminal identifier of the corresponding non-relay terminal, and the video frame associated with the triggering information transmitted by the relay terminal to each of the non-relay terminals carries the terminal identifier of the non-relay terminal.

12. The image forming apparatus according to claim 1, characterized in that, The video feed includes a real-time view of the operation panel of the image forming device, and / or a function setting interface of the image forming device, the function setting interface including at least one function setting item associated with the image forming function.

13. The image forming apparatus according to claim 12, characterized in that, The embedded server is also used for: Receive the first trigger information generated by the non-relay terminal on the real-time screen and the second trigger information generated on the function setting interface, sent by the relay terminal; If the function setting effect corresponding to the second trigger information does not conflict with the function setting effect corresponding to the first trigger information, the relay terminal is instructed to send the corresponding video image to the non-relay terminal that generated the first trigger information and the non-relay terminal that generated the second trigger information.

14. The image forming apparatus according to claim 1, characterized in that, The embedded server is also used to perform the following steps: Send a first notification to the relay terminal; wherein the first notification is used to prompt the relay terminal to actively establish a peer-to-peer connection with the non-relay terminal; or, Send a second notification to the non-relay terminal; wherein the second notification is used to prompt the non-relay terminal to actively establish a peer-to-peer connection with the relay terminal.

15. The image forming apparatus according to claim 1, characterized in that, The at least one remote terminal that initiates the remote control request through the web page file includes a first remote terminal and a second remote terminal, wherein the first remote terminal has been designated as a relay terminal. The embedded server is also used for: If a disconnection of the remote control connection with the first remote terminal is detected, it is determined whether the second remote terminal meets the specified conditions of the relay terminal. If it is determined that the second remote terminal meets the specified conditions of the relay terminal, then the second remote terminal is designated as a relay terminal.

16. An image forming method, characterized in that, include: In response to a webpage request from a remote terminal, a webpage file is sent to the remote terminal; If a remote control request initiated by the remote terminal through the web page file is detected, the following steps are performed: If the remote terminal is a relay terminal configured by the embedded server, then send video footage to the remote terminal: If the remote terminal is not a relay terminal, a peer-to-peer connection is established between the non-relay terminal and the relay terminal, so that the non-relay terminal can directly obtain the video feed from the relay terminal.

Images