Processing method and electronic equipment

Abstract

The invention discloses a processing method and electronic equipment, and a method applying first electronic equipment comprises the following steps: in response to input data of second electronic equipment corresponding to an input device, determining an input action corresponding to the input data; generating an output picture corresponding to the input action according to a target processing rule corresponding to the input action; and transmitting the output picture to the second electronic equipment, so that the second electronic equipment outputs the output picture, and synchronously outputs an icon picture corresponding to the input device.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a processing method and an electronic device. Background Art

[0002] Currently, multiple computing terminals are deployed in the cloud, and users access the computing terminals through thin terminals to realize corresponding functions.

[0003] However, this method is limited by network resources, and the amount of data transmission between the thin terminal and the computing terminal is large, resulting in untimely response after the user operates the computing terminal on the thin terminal, resulting in lag. Summary of the invention

[0004] In view of this, the present application provides a processing method and an electronic device as follows:

[0005] A processing method, using a first electronic device, the method comprising:

[0006] In response to input data corresponding to the input device of the second electronic device, determining an input action corresponding to the input data;

[0007] generating an output screen corresponding to the input action according to a target processing rule corresponding to the input action;

[0008] The output screen is transmitted to the second electronic device, so that the second electronic device outputs the output screen and synchronously outputs an icon screen corresponding to the input device.

[0009] In the above method, preferably, generating an output screen corresponding to the input action according to a target processing rule corresponding to the input action includes:

[0010] According to the target processing rule corresponding to the input action, the action type and / or the position coordinates corresponding to the input action are processed to obtain the output picture;

[0011] The action type includes at least one of the following:

[0012] Cursor selection type;

[0013] Cursor movement type;

[0014] Scroll wheel scroll type;

[0015] Key input type.

[0016] A processing method is applied to a second electronic device, the method comprising:

[0017] In response to obtaining an output picture transmitted by the first electronic device, outputting the output picture;

[0018] Based on the input data of the input device of the second electronic device, an icon screen corresponding to the input device is generated in the output screen.

[0019] The above method, preferably, further comprises:

[0020] Based on a target transmission rule, the input data corresponding to the input device and the output screen is transmitted to the first electronic device.

[0021] The above method preferably transmits the input data corresponding to the input device and the output screen to the first electronic device according to the target transmission rule, including:

[0022] monitoring whether the input device performs an input action on the output picture;

[0023] When the input device performs an input action on the output screen, obtaining first input data according to the action type corresponding to the input action and the position coordinates of the input action corresponding to the output screen; and transmitting the first input data to the first electronic device;

[0024] When the input device has no input action on the output screen, second input data is obtained according to the position coordinates of the device icon corresponding to the input device on the output screen; in response to the transmission condition being met, the second input data is transmitted to the first electronic device.

[0025] In the above method, preferably, the transmission condition includes: the interval time between the historical moment of the previous transmission of the second input data reaches the target time;

[0026] The second input data includes: the position coordinates of the input device accumulated in the output screen within the interval time.

[0027] The above method, preferably, further comprises:

[0028] Obtaining an action frequency of the input device performing an input action on the output picture;

[0029] The target duration is updated according to the action frequency.

[0030] The above method, preferably, further comprises:

[0031] Obtaining a predicted duration of the next input action performed by the input device on the output screen;

[0032] The target duration is updated according to the predicted duration.

[0033] An electronic device, the electronic device being a first electronic device, comprises:

[0034] The processor is configured to determine an input action corresponding to the input data in response to the input data of the second electronic device corresponding to the input device; and generate an output screen corresponding to the input action according to a target processing rule corresponding to the input action;

[0035] The transmission module is used to transmit the output screen to the second electronic device, so that the second electronic device outputs the output screen and synchronously outputs the icon screen corresponding to the input device.

[0036] An electronic device, the electronic device serving as a second electronic device, comprising:

[0037] The display is configured to output the output picture transmitted by the first electronic device in response to obtaining the output picture transmitted by the first electronic device;

[0038] The processor is configured to generate an icon screen corresponding to the input device in the output screen based on input data of the input device of the second electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 A flowchart of a processing method provided in an embodiment of the present application;

[0041] Figure 2 This is an example diagram of the interaction between the first electronic device and the second electronic device in the embodiment of the present application;

[0042] Figure 3 This is an example diagram of input actions in an embodiment of the present application;

[0043] Figure 4 This is an example diagram of a first electronic device transmitting a device screen image to a second electronic device in an embodiment of the present application;

[0044] Figure 5 A flowchart of another processing method provided in an embodiment of the present application;

[0045] Figure 6 A partial flow chart of another processing method provided in an embodiment of the present application;

[0046] Figure 7 Another partial flow chart of another processing method provided in an embodiment of the present application;

[0047] Figure 8 A schematic diagram of the structure of a processing device provided in an embodiment of the present application;

[0048] Fig. 9 A schematic diagram of the structure of another processing device provided in an embodiment of the present application;

[0049] Fig.10 Another structural schematic diagram of another processing device provided in an embodiment of the present application;

[0050] Fig.11 Another structural schematic diagram of another processing device provided in an embodiment of the present application;

[0051] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0052] Fig.13 A schematic diagram of the structure of another electronic device provided in an embodiment of the present application;

[0053] Fig.14 A schematic diagram of the structure of a processing system provided in an embodiment of the present application;

[0054] Fig.15 Another structural schematic diagram of a processing system provided in an embodiment of the present application;

[0055] Fig.16 This is a schematic diagram of the architecture of thin terminals and fat clients in the cloud gaming service of e-sports hotels applicable to this application;

[0056] Fig.17 This is an example diagram of data transmission between a thin terminal and a host in a cloud gaming service applicable to an e-sports hotel in this application;

[0057] Fig.18 This is a flowchart of the adaptive scheduling algorithm proposed in the cloud gaming service of the e-sports hotel according to the embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] refer to Figure 1As shown, it is a flowchart of an implementation of a processing method provided in an embodiment of the present application. The method can be applied to electronic devices capable of performing data processing, such as Figure 2 The first electronic device shown, the first electronic device establishes a data connection with the second electronic device, for example, the first electronic device establishes a data connection with the second electronic device as a host, and for another example, the virtual machine on the first electronic device establishes a data connection with the second electronic device, and the first electronic device can synchronize its output screen to the second electronic device, for example, the output screen can be the device screen of the first electronic device, or the output screen can be the running screen of the virtual machine or application deployed on the first electronic device. For example, the first electronic device outputs the device screen or the running screen of the virtual machine to the second electronic device. The user can view the output screen synchronized from the first electronic device through the second electronic device and perform input operations on the output screen through input devices such as a mouse and a keyboard, and then transmit the input data to the first electronic device. The technical solution in this embodiment is mainly used to improve the user experience of the electronic device.

[0060] Specifically, the method in this embodiment may include the following steps:

[0061] Step 101: In response to input data corresponding to an input device of a second electronic device, determining an input action corresponding to the input data.

[0062] The input data refers to the data related to the input device transmitted by the second electronic device to the first electronic device. Specifically, the input data includes the command frame and / or position frame of the input device such as the mouse or keyboard. The command frame represents the type of action performed by the input device on the output screen, and the position frame represents the position coordinates of the input device on the output screen where the input operation is performed. For example, taking the mouse as an example, the input data may include the click action of the mouse and the position where the mouse is clicked on the output screen.

[0063] Specifically, in this embodiment, data analysis, such as data decoding and data packet analysis, may be performed on the input data received from the second electronic device to determine the input action corresponding to the input data.

[0064] The input action may correspond to an action type and / or a position coordinate, and the action type may include at least one of: a cursor selection type, a cursor movement type, a scroll wheel scrolling type, a key input type, and the like.

[0065] like Figure 3As shown in , taking the mouse as an example, the input action may correspond to the action type of the mouse cursor selecting a control and the position coordinates of the control; for another example, taking the mouse as an example, the input action may correspond to the position coordinates of the cursor in the process of moving from a first coordinate to a second coordinate; for another example, taking the mouse as an example, the input action may correspond to the action type of the mouse wheel rolling; for another example, taking the keyboard as an example, the input action may correspond to the action type of any key on the keyboard being pressed.

[0066] Step 102: Generate an output screen corresponding to the input action according to the target processing rule corresponding to the input action.

[0067] Different input actions correspond to different target processing rules. For example, the action type selected by the cursor and the action type moved by the cursor correspond to different processing rules; for another example, the position coordinates selected by the cursor are different, and the corresponding processing rules are different, and accordingly, the generated output screens are different.

[0068] Specifically, in step 102 , the action type and / or position coordinates corresponding to the input action may be processed according to the target processing rule corresponding to the input action to obtain an output screen.

[0069] For example, taking the action type selected by the cursor as an example, the corresponding target processing rule may be: according to the object type of the selected object at the position coordinate, perform corresponding processing. For example, if the position coordinate is a folder icon, then in step 102, the folder corresponding to the position coordinate selected by the cursor is opened; if the coordinate position is a file icon, then in step 102, the file corresponding to the position coordinate selected by the cursor, such as music, video, etc., is opened; if the position coordinate is a control, then in step 102, the interface or page connected to the control can be entered, and so on.

[0070] For another example, taking the action type of cursor movement as an example, the corresponding target processing rule may be: executing corresponding processing according to whether the object is selected before the cursor moves. For example, if the game character is selected (such as by mouse or keyboard) before the cursor moves, then the current perspective of the game character is switched according to the position coordinates of the cursor movement.

[0071] For another example, taking the action types of cursor selection and cursor movement as an example, the corresponding target processing rule may be: according to the object type contained in the screen area where the position coordinates selected by the cursor are located, the corresponding processing is performed. For example, if the screen area where the position coordinates selected by the cursor are located contains a game character, then according to the position coordinates moved by the cursor, the current perspective of the game character is switched; if the screen area where the position coordinates selected by the cursor are located contains a file icon, then according to the position coordinates moved by the cursor, the file icon can be moved to the corresponding screen position, and so on.

[0072] For example, taking the action type of scrolling with a scroll wheel as an example, the corresponding target processing rule may be: according to the current position of the cursor, switch the output area of ​​the output screen. For example, if the current position of the cursor is on the document progress bar, then switch the output area of ​​the document to achieve document browsing.

[0073] For another example, taking the action types of cursor selection and scroll wheel rolling as an example, the corresponding target processing rule may be: perform corresponding processing according to the position coordinates selected by the cursor and the scroll amount of the scroll wheel. For example, if the position coordinates selected by the cursor are a line of text in a document, then according to the scroll amount of the cursor scroll wheel, all text within the scroll amount range starting from the line of text is selected.

[0074] For another example, taking the key input type as an example, the corresponding target processing rule can be: according to the interface type of the current interface, based on one or more characters corresponding to the operated key, perform corresponding processing. For example, if the characters "j" and "k" are operated in the document input interface, then these two characters are displayed at the cursor position in the document input interface; if the character "F5" is operated in the web browsing interface, then the web page can be refreshed.

[0075] Step 103: Transmit the output screen to the second electronic device, so that the second electronic device outputs the output screen and simultaneously outputs the icon screen corresponding to the input device.

[0076] The first electronic device may transmit the output image to the second electronic device through a data connection between the first electronic device and the second electronic device.

[0077] For example, Figure 4 As shown in , the first electronic device transmits the device screen image to the second electronic device, and the second electronic device synchronously outputs the icon image of the mouse on the device screen image according to the position of the mouse. In this process, the icon image corresponding to the input device is processed on the second electronic device, and the first electronic device does not need to merge the icon image corresponding to the input device into the output image and then transmit it to the second electronic device.

[0078] For another example, the first electronic device transmits the operating interface of the virtual machine to the second electronic device, and the second electronic device synchronously outputs the icon screen of the mouse on the operating interface of the virtual machine according to the position of the mouse. In this process, the icon screen corresponding to the input device is processed on the second electronic device, and the first electronic device does not need to merge the icon screen corresponding to the input device into the output screen and then transmit it to the second electronic device.

[0079] It can be seen from the above technical scheme that the present application implements a processing method, which, after determining the input action on the second electronic device on the first electronic device, only needs to generate the corresponding output screen according to the target processing rule, and the icon screen corresponding to the input device is synchronously output by the first electronic device on the output screen, and there is no need for the second electronic device to merge the icon screen into the output screen and then output it to the first electronic device. This can not only reduce the screen processing flow, but also reduce the amount of data transmitted to the first electronic device, thereby speeding up the rate at which the output screen is transmitted to the first electronic device, thereby avoiding the output jam of the output screen on the first electronic device, thereby improving the user experience of the first electronic device and the second electronic device.

[0080] refer to Figure 5 , is a flowchart of an implementation of a processing method provided in an embodiment of the present application, and the method can be applied to electronic devices capable of data processing, such as Figure 2 The second electronic device shown, the first electronic device and the second electronic device establish a data connection, for example, the first electronic device as a host establishes a data connection with the second electronic device, for example, the virtual machine on the first electronic device establishes a data connection with the second electronic device, the first electronic device can synchronize its output screen to the second electronic device, for example, the output screen can be the device screen of the first electronic device, or the output screen can be the running screen of the virtual machine or application deployed on the first electronic device. For example, the first electronic device outputs the device screen or the running screen of the virtual machine to the second electronic device. The user can view the output screen synchronized from the first electronic device through the second electronic device and perform input operations on the output screen through input devices such as a mouse and keyboard, and then transmit the input data to the first electronic device. The technical solution in this embodiment is mainly used to improve the user experience of electronic devices.

[0081] Specifically, the method in this embodiment may include the following steps:

[0082] Step 501: In response to obtaining an output picture transmitted by a first electronic device, output the output picture.

[0083] The output image transmitted by the first electronic device may be a device screen image, a virtual machine operation interface, or an application program operation interface.

[0084] It should be noted that the output image transmitted by the first electronic device may be based on Figure 1 The output screen generated by the embodiment shown.

[0085] Step 502: Based on input data of an input device of a second electronic device, generate an icon screen corresponding to the input device in an output screen.

[0086] Specifically, the input data may include an action type and a position coordinate. Based on this, in step 502, an icon screen of the input device under a corresponding action type may be rendered at a corresponding position in the output screen according to the position coordinate of the input device in the output screen, such as an icon screen of a mouse arrow or an icon screen of a mouse hand shape. Figure 4 as shown in .

[0087] It can be seen from the above technical scheme that the present application implements a processing method, which outputs an icon screen corresponding to the input device in the output screen based on the input data of the input device after the output screen transmitted by the first electronic device is output on the second electronic device. The second electronic device does not need to merge the icon screen into the output screen and then output it to the first electronic device. This not only reduces the screen processing flow, but also reduces the amount of data transmitted to the first electronic device, thereby speeding up the rate at which the output screen is transmitted to the first electronic device, thereby avoiding output jams of the output screen on the first electronic device, thereby improving the user experience of the first electronic device and the second electronic device.

[0088] Based on the above implementation scheme, the method in this embodiment may further include the following steps: Figure 6 As shown in:

[0089] Step 503: Based on the target transmission rule, the input data corresponding to the input device and the output screen is transmitted to the first electronic device.

[0090] The input data is generated based on the input action performed by the input device on the output screen. The input action corresponds to an action type and / or a position coordinate. In the second electronic device, the input data can be transmitted to the first electronic device at a specific time interval according to the refresh frequency corresponding to the input device.

[0091] Based on this, the first electronic device can Figure 1 The illustrated embodiment determines the corresponding input action for the received input data and generates the corresponding output screen, which is then transmitted back to the second electronic device. Thus, as the input operation is performed on the second electronic device, the first electronic device can respond in a timely manner and feed back the corresponding output screen to the user of the second electronic device.

[0092] The target transmission rule is related to the input action on the output screen monitored by the input device on the second electronic device.

[0093] In one implementation, in step 503, when the input data corresponding to the input device and the output screen is transmitted to the first electronic device according to the target transmission rule, it can be implemented in the following manner: Figure 7 As shown in:

[0094] Step 701: monitor whether the input device performs input action on the output screen. If the input device performs input action on the output screen, execute step 702; if the input device does not perform input action on the output screen, execute step 703.

[0095] Whether the input device performs an input action on the output screen refers to whether the input device performs input actions of action types such as cursor selection, cursor movement, key input, and scroll wheel rolling on the output screen.

[0096] Step 702: Obtain first input data according to the action type corresponding to the input action and the position coordinates corresponding to the input action on the output screen, and transmit the first input data to the first electronic device.

[0097] Specifically, in this embodiment, the action type and position coordinates corresponding to the input action can be packaged, encrypted, etc. to obtain first input data, and the first input data is transmitted to the first electronic device. For example, the click operation of the mouse on the second electronic device and the position of the mouse click on the control are packaged, and the obtained input data is transmitted to the first electronic device.

[0098] Step 703: Obtain second output data according to the position coordinates of the device icon corresponding to the input device on the output screen; in response to the transmission condition being met, transmit the second input data to the first electronic device.

[0099] Specifically, in this embodiment, the position coordinates of the input device on the output screen can be packaged, encrypted, etc. to obtain second input data, and the second input data is transmitted to the first electronic device. For example, the mouse position of the mouse on the second electronic device on the device screen or the running interface of the virtual machine is packaged, and the obtained input data is transmitted to the first electronic device.

[0100] It should be noted that, in this embodiment, detecting whether the input device performs an input action on the output screen can be understood as whether the input device performs a valid action on the output screen.

[0101] Specifically, in this embodiment, the operation data of the input device can be received first to determine whether the operation data is associated with an element in the output screen. If the operation data is associated with an element in the output screen, then it can be determined that the input device performs an input action on the output screen; if the operation data is not associated with an element in the output screen, then it can be determined that the input device does not perform an input action on the output screen.

[0102] For example, if the mouse is associated with a game character in the output screen, such as when the mouse selects the game character, then when the mouse moves, it can be determined that the mouse has performed an action of moving the mouse on the output screen. At this time, the action type of the mouse movement and the coordinates of the mouse movement position can be packaged into first input data and transmitted to the first electronic device; if the mouse is not associated with a game character in the output screen, such as when the mouse does not select the game character, then when the mouse moves, it can be determined that the mouse has no effective action on the output screen. At this time, the position coordinates of the mouse on the output screen can be packaged into second input data and transmitted to the first electronic device when the transmission conditions are met.

[0103] It can be seen that in this embodiment, whether to transmit input data to the first electronic device in a timely manner or to wait until the transmission condition is met before transmitting the input data to the first electronic device is determined based on whether the input device performs an input action on the output screen. This does not affect the generation of the output screen by the first electronic device according to the input action, and the input data can be accumulated by setting the transmission condition. The input data is transmitted when the transmission condition is met. This can reduce the number of times the input data is transmitted to the first electronic device, thereby reducing the number of times the data is transmitted between the first electronic device and the second electronic device, avoiding excessive occupation of transmission resources, and further improving the rate at which the first electronic device transmits the output screen to the second electronic device and the rate at which the second electronic device transmits the input data to the first electronic device to a greater extent, thereby further avoiding output jams of the output screen on the first electronic device, thereby improving the user experience of the first electronic device and the second electronic device.

[0104] Based on the above implementation scheme, the transmission condition can be: the interval between the historical moment of the last transmission of the second input data reaches the target duration. In other words, if the input device does not perform an effective action on the input screen, the input data can be transmitted to the first electronic device again after the target duration, thereby reducing the number of times the input data is transmitted to the first electronic device.

[0105] At this time, the second transmission data transmitted to the first electronic device may include: the position coordinates of the input device accumulated in the output screen within the interval time.

[0106] For example, taking the target duration as the duration for refreshing 20 position frames (i.e., position coordinates) according to the current screen refresh frequency, if the mouse has no effective action on the device screen or the running interface of the virtual machine, such as simply shaking the mouse on the running interface, the mouse position frames will be accumulated for 20 frames and then transmitted to the first electronic device. This can reduce the number of times input data is transmitted to the first electronic device. After receiving these position frames on the first electronic device, even if the mouse icon is refreshed again after the target duration, it will not affect the normal response of the device screen or the running interface of the virtual machine.

[0107] In a specific implementation, in this embodiment, the target duration in the transmission condition may also be updated.

[0108] In one implementation, in this embodiment, the action frequency of the input device performing input actions on the output screen, such as click frequency, scroll wheel scroll frequency, etc., can be obtained, and then the target duration is updated according to the action frequency.

[0109] Among them, in this embodiment, the target duration can be reduced or increased according to the frequency of the action. For example, taking the mouse as an example, the higher the frequency of the mouse action, the lower the target duration can be; the lower the frequency of the mouse action, the longer the target duration can be. In other words, if the mouse action is frequent, the target duration can be appropriately reduced, and the accumulated mouse position coordinates are transmitted to the first electronic device once at a shorter interval, so that the mouse icon on the device screen or the running interface of the virtual machine can be refreshed for the user of the first electronic device in time; if the mouse action is not very frequent, the target duration can be appropriately increased, so that the accumulated mouse position coordinates can be transmitted to the first electronic device once at a longer interval, so that the number of times the input data is transmitted to the first electronic device can be reduced without affecting the refresh of the mouse icon on the first electronic device.

[0110] In another implementation, in this embodiment, the predicted duration of the next input action of the input device on the output screen can be obtained, such as predicting the future moment of the next mouse click, and then updating the target duration based on the predicted duration.

[0111] Among them, in this embodiment, the target duration can be increased or decreased according to the size of the predicted duration. For example, taking the mouse as an example, if the moment of the next mouse action is predicted to be far away, that is, the interval duration is long, the target duration can be increased; if the moment of the next mouse action is predicted to be close, that is, the interval duration is short, then the target duration can be reduced. In other words, if the next mouse action is predicted to be close, then the target duration can be appropriately reduced to timely refresh the device screen or the mouse icon on the running interface of the virtual machine for the user of the first electronic device; if the next mouse action is predicted to be far away, then the target duration can be appropriately increased, and the accumulated mouse position coordinates can be transmitted to the first electronic device once with a longer interval duration, so that the number of times input data is transmitted to the first electronic device can be reduced without affecting the refresh of the mouse icon on the first electronic device.

[0112] refer to Figure 8 , is a schematic diagram of a processing device provided in an embodiment of the present application, which can be applied to Figure 2 In the first electronic device shown, the device may include the following units:

[0113] An action determination unit 801 is used to determine an input action corresponding to the input data in response to the input data of the second electronic device corresponding to the input device;

[0114] A screen generating unit 802, configured to generate an output screen corresponding to the input action according to a target processing rule corresponding to the input action;

[0115] The screen transmission unit 803 is used to transmit the output screen to the second electronic device, so that the second electronic device outputs the output screen and synchronously outputs the icon screen corresponding to the input device.

[0116] It can be seen from the above technical scheme that the present application implements a processing device provided by which, after determining the input action on the second electronic device on the first electronic device, it is only necessary to generate a corresponding output screen according to the target processing rule, and the icon screen corresponding to the input device is synchronously output by the first electronic device on the output screen, and there is no need for the second electronic device to merge the icon screen into the output screen and then output it to the first electronic device. This not only reduces the screen processing flow, but also reduces the amount of data transmitted to the first electronic device, thereby speeding up the rate at which the output screen is transmitted to the first electronic device, thereby avoiding output jams of the output screen on the first electronic device, thereby improving the user experience of the first electronic device and the second electronic device.

[0117] In one implementation, the screen generation unit 802 is specifically configured to: process the action type and / or position coordinates corresponding to the input action according to the target processing rule corresponding to the input action to obtain the output screen; wherein the action type includes at least one of the following:

[0118] Cursor selection type; cursor movement type; scroll wheel scroll type; key input type.

[0119] refer to Fig. 9 , is a schematic diagram of a processing device provided in an embodiment of the present application, which can be applied to Figure 2 The second electronic device shown may include the following units:

[0120] A picture output unit 901, configured to output the output picture in response to obtaining the output picture transmitted by the first electronic device;

[0121] The icon generating unit 902 is configured to generate an icon screen corresponding to the input device in the output screen based on the input data of the input device of the second electronic device.

[0122] It can be seen from the above technical scheme that the present application implements a processing device, which outputs an icon screen corresponding to the input device in the output screen based on the input data of the input device after outputting the output screen transmitted by the first electronic device on the second electronic device. There is no need for the second electronic device to merge the icon screen into the output screen and then output it to the first electronic device. This not only reduces the screen processing flow, but also reduces the amount of data transmitted to the first electronic device, thereby speeding up the rate at which the output screen is transmitted to the first electronic device, thereby avoiding output jams of the output screen on the first electronic device, thereby improving the user experience of the first electronic device and the second electronic device.

[0123] In one implementation, the device in this embodiment may further include the following units: Fig.10 As shown in:

[0124] The data transmission unit 903 is used to transmit the input data corresponding to the input device and the output screen to the first electronic device based on a target transmission rule.

[0125] Among them, the data transmission unit 903 is specifically used to: monitor whether the input device performs an input action on the output screen; when the input device performs an input action on the output screen, obtain first input data according to the action type corresponding to the input action and the position coordinates corresponding to the input action on the output screen; transmit the first input data to the first electronic device; when the input device does not perform an input action on the output screen, obtain second input data according to the position coordinates of the device icon corresponding to the input device on the output screen; in response to the transmission condition being met, transmit the second input data to the first electronic device.

[0126] In one implementation, the transmission condition includes: the interval between the historical moment of the previous transmission of the second input data reaches the target duration; wherein the second input data includes: the position coordinates of the input device accumulated in the output screen within the interval duration.

[0127] Based on the above implementation, the device in this embodiment may further include the following units: Fig.11 As shown in:

[0128] The condition updating unit 904 is used to obtain the action frequency of the input device performing input actions on the output screen; and update the target duration according to the action frequency.

[0129] Alternatively, the condition updating unit 904 is configured to obtain a predicted duration of the next input action performed by the input device on the output screen; and update the target duration according to the predicted duration.

[0130] It should be noted that the specific implementation method of each unit in this embodiment can refer to the corresponding content in the previous text and will not be described in detail here.

[0131] refer to Fig.12 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 2 The first electronic device shown in the figure may include the following structure:

[0132] The processor 1201 is used to determine an input action corresponding to the input data of the second electronic device corresponding to the input device in response to the input data of the second electronic device; and generate an output screen corresponding to the input action according to the target processing rule corresponding to the input action;

[0133] The transmission module 1202, such as a network card, is used to transmit the output screen to the second electronic device, so that the second electronic device outputs the output screen and simultaneously outputs the icon screen corresponding to the input device.

[0134] It can be seen from the above technical solution that an electronic device provided by the implementation of the present application, after determining the input action on the second electronic device on the first electronic device, only needs to generate a corresponding output screen according to the target processing rule, and the icon screen corresponding to the input device is synchronously output by the first electronic device on the output screen, and there is no need for the second electronic device to merge the icon screen into the output screen and then output it to the first electronic device. This can not only reduce the screen processing flow, but also reduce the amount of data transmitted to the first electronic device, thereby speeding up the rate at which the output screen is transmitted to the first electronic device, thereby avoiding the output jam of the output screen on the first electronic device, thereby improving the user experience of the first electronic device and the second electronic device.

[0135] refer to Fig.13 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, and the electronic device as a second electronic device may include the following structure:

[0136] The display 1301 is configured to output the output picture transmitted by the first electronic device in response to obtaining the output picture transmitted by the first electronic device;

[0137] The processor 1302 is configured to generate an icon screen corresponding to the input device in the output screen based on input data of the input device of the second electronic device.

[0138] It can be seen from the above technical solution that an electronic device provided by the implementation of the present application outputs an output screen transmitted by a first electronic device on a second electronic device, and then outputs an icon screen corresponding to an input device in the output screen based on the input data of the input device. There is no need for the second electronic device to merge the icon screen into the output screen and then output it to the first electronic device. This not only reduces the screen processing flow, but also reduces the amount of data transmitted to the first electronic device, thereby speeding up the rate at which the output screen is transmitted to the first electronic device, thereby avoiding output jams of the output screen on the first electronic device, thereby improving the user experience of the first electronic device and the second electronic device.

[0139] refer to Fig.14 , is a schematic diagram of the structure of a processing system provided in an embodiment of the present application, and the processing system may include the following devices:

[0140] At least one first electronic device 1401 and at least one second electronic device 1402;

[0141] The first electronic device 1401 determines an input action corresponding to the input data in response to the input data of the second electronic device 1402 corresponding to the input device; then generates an output screen corresponding to the input action according to the target processing rule corresponding to the input action; and then transmits the output screen to the second electronic device 1402;

[0142] In response to obtaining the output screen transmitted by the first electronic device 1401, the second electronic device 1402 outputs the output screen; based on the input data of the input device of the second electronic device 1402, an icon screen corresponding to the input device is generated in the output screen. Further, based on the target transmission rule, the second electronic device 1402 transmits the input data corresponding to the input device and the output screen to the first electronic device 1401.

[0143] It can be seen from the above technical solution that in a processing system provided by the implementation of the present application, after determining the input action on the second electronic device on the first electronic device, it is only necessary to generate a corresponding output screen according to the target processing rule, and the icon screen corresponding to the input device is synchronously output by the first electronic device on the output screen, and there is no need for the second electronic device to merge the icon screen into the output screen and then output it to the first electronic device. This can not only reduce the screen processing flow, but also reduce the amount of data transmitted to the first electronic device, thereby speeding up the rate at which the output screen is transmitted to the first electronic device, thereby avoiding the output jam of the output screen on the first electronic device, thereby improving the user experience of the first electronic device and the second electronic device.

[0144] There may be a point-to-point data connection between the first electronic device and the second electronic device, and the data connection is used to transmit input data and output images between the first electronic device and the second electronic device.

[0145] For example, taking the system architecture of an e-sports hotel as an example, the first electronic device can be a thin terminal deployed in a hotel room, and the second electronic device can be a fat terminal deployed in the cloud of a communication service operator. The fat terminal and the thin terminal are connected point-to-point through switches, optical fiber connectors, etc. The fat terminal transmits the device screen image of the fat terminal to the thin terminal. The user of the thin terminal can input the device screen image of the fat terminal through input devices such as a mouse and a keyboard. The thin terminal transmits the corresponding input data to the fat terminal according to the target transmission rule. The fat terminal responds to the input action therein to generate a corresponding operation interface, and then returns it to the thin terminal. The thin terminal renders the icon image of the mouse on the device screen image, thereby reducing the amount of data transmission and the number of transmissions between the fat terminal and the thin terminal, thereby avoiding the output jam of the operation interface on the thin terminal, thereby improving the user experience.

[0146] Furthermore, the first electronic device is in a first network, the second electronic device is in a second network, and the first network is different from the second network. Moreover, the processing system may also include the following structure, such as Fig.15 As shown in:

[0147] The third electronic device 1403 can be deployed in the first network as a management server. The third electronic device 1403 can provide operation data to the first electronic device, and the first electronic device can operate according to the operation data and transmit the corresponding generated output screen to the second electronic device.

[0148] For example, the operating data may be a system image file of a fat terminal. The third electronic device may provide the image file to the first electronic device. The first electronic device may deploy and run the host system to generate a device screen image and transmit the device screen image to the second electronic device.

[0149] In addition, the processing system of this embodiment may further include: a control component connected between a target pin on the mainboard of the first electronic device and the third electronic device; the target pin is a pin that can trigger the restart of the first electronic device.

[0150] The control component is specifically used for: receiving a restart instruction sent by a third electronic device, and triggering a target pin according to the restart instruction to restart the first electronic device.

[0151] Taking the Cloud Gaming service of the e-sports hotel as an example, the technical solution of this application is described in detail below:

[0152] In the Cloud Gaming architecture, this application innovatively deploys high-performance hosts (i.e., the fat terminal mentioned above) to the cloud computer room, and achieves precise management through the intelligent scheduling server (i.e., the third electronic device mentioned above). AIO (All In One) is deployed locally as a thin terminal on the display side, and the host and the thin terminal are connected by a LAN or optical fiber, which not only efficiently transmits video streams, but also realizes instant response of keyboard and mouse reverse control.

[0153] like Fig.16 As shown in the figure, the high-performance host (i.e. the fat terminal mentioned above) is deployed in the cloud computer room, the cloud switch is connected to the Internet through a router, and is precisely managed through the intelligent scheduling server (i.e. the third electronic device mentioned above). The intelligent scheduling server obtains the image file of the host system from the image server and provides it to the high-performance host, while the cloud switch is connected to the local switch of the e-sports hotel through an optical fiber connector. AIO is deployed locally as a thin terminal on the display side and connected to the local switch.

[0154] The technical solution of this application is described in detail below:

[0155] First of all, the present application proposes a relative mouse solution, which is an efficient and low-latency local mouse control optimization algorithm. In the traditional thin client and cloud fat client architecture, each frame of position data of the mouse operation needs to be transmitted to the fat client through the network in real time for processing, which not only takes up a lot of network bandwidth, but also causes the user experience to decline due to network delays, especially in application scenarios with high real-time requirements. This technology is often limited by network conditions and computing resources, and it is difficult to achieve a high frame rate and low-latency mouse control experience. In view of this, the present application proposes an efficient and low-latency local mouse control optimization algorithm, which intelligently separates the mouse position frame (that is, the position coordinates in the previous text) and the command frame (that is, the action type in the previous text), and uses the local computing power of the thin terminal for pre-processing, such as Fig.17 As shown in , the mouse is monitored locally on the thin terminal to see if there is a mouse command frame. If so, the mouse command frame is transmitted, but the position frame is not continuously synchronized. The cloud host (i.e., the fat client, also called the fat terminal) responds to the command frame and returns the game video stream (i.e., the output screen in the previous article). This can significantly reduce the amount of data transmitted over the network, thereby reducing network bandwidth occupancy and latency, and improving user experience.

[0156] The specific plan is as follows:

[0157] 1. Mouse data separation and local processing:

[0158] a. On the thin terminal side, the raw data of the mouse input is divided into two categories: position frame and command frame. The position frame records the continuous position change of the mouse, while the command frame contains mouse commands such as click, double-click, scroll, etc.

[0159] b. The position frame can adopt a non-real-time transmission strategy, where every 10 to 20 frames (i.e., the target duration of 20 frames in the previous text) are packaged into a data packet for transmission, effectively reducing the frequency of data transmission.

[0160] c. The command frame is monitored in real time by the local thin terminal real-time monitoring system. Once the mouse command is detected, it is immediately packaged and sent to the fat client to ensure an immediate response to the command.

[0161] 2. Differentiated processing of fat clients (i.e. fat terminals):

[0162] a. After the fat client receives the data packet of the position frame, it is mainly used for synchronous update of the video screen (i.e., the output screen mentioned above, such as the device screen or the running interface of the virtual machine), rather than directly controlling the movement of the mouse pointer, thereby reducing the processing burden of the fat client.

[0163] b. For the command frame, the fat client immediately parses and executes the corresponding mouse command, such as simulating mouse clicks, etc., and feeds back the execution result to the thin terminal to achieve fast response.

[0164] 3. Local computing power optimization:

[0165] a. The thin terminal uses its own computing power to pre-process the mouse input, including but not limited to smoothing, acceleration prediction, etc., to further improve transmission efficiency and user experience.

[0166] b. Through algorithm optimization, the thin terminal can also locally predict the mouse position in the next few frames, reducing the mouse position lag caused by network delay.

[0167] 4. Dynamic adjustment mechanism:

[0168] According to network conditions and user needs, the present application also provides a dynamic adjustment mechanism that can automatically adjust the packing frequency of location frames and the transmission priority of command frames to adapt to different application scenarios and network environments.

[0169] It can be seen that this application can produce the following beneficial effects:

[0170] 1. Significantly reduce network bandwidth usage: By separating the position frame and the command frame and reducing the transmission frequency of the position frame, the network bandwidth usage is greatly reduced.

[0171] 2. Improve user experience: Due to the real-time transmission of command frames and intelligent processing of position frames, mouse operations are smoother and users can hardly perceive delays.

[0172] 3. Enhance system adaptability: The dynamic adjustment mechanism enables the system to automatically optimize the transmission strategy according to the network environment, improving the system's adaptability and stability.

[0173] 4. Reduce computing resource consumption: The fat client only needs to process command frames and perform video synchronization, which reduces the consumption of computing resources.

[0174] Secondly, this application proposes an adaptive scheduling algorithm, such as Fig.18 As shown in the figure, it is a flow chart of the adaptive scheduling algorithm. Based on the algorithm, in the process of building a high-performance cloud computing environment, this application implements an adaptive unattended scheduling system for high-performance cloud computing host deployment in the cloud. The system realizes the automation of the entire process from host placement to automatic deployment, greatly improving deployment efficiency and reducing labor costs.

[0175] Based on this system, this application has the following core advantages and technical highlights:

[0176] 1. Intelligent detection and adaptive matching: The system's built-in intelligent management server has real-time network detection capabilities, and can instantly capture the media access control address MAC (Media Access Control Address) of the newly added host when the fat client is powered on for the first time and boots up using a diskless network. By quickly comparing these MAC addresses with the pre-stored database, the system can accurately identify the newly added devices, ensuring that each new terminal can be effectively identified and included in the management scope. Fig.18 As shown in , after the fat terminal is started, it is determined whether it is a new MAC address. If so, it is added to the fat terminal list such as list of the management server, and then a restart command is issued to the fat terminal through a control component such as a remote control card RCC (remote control card), so that the fat terminal can run normally based on the installed game software after restart.

[0177] 2. Seamless integration and unified scheduling: For successfully identified new fat terminals, the system will not only automatically add their MAC addresses to the database of the local management host, but also immediately start a unified scheduling and allocation mechanism. This process does not require manual intervention and is completely completed by the system, achieving seamless integration from device identification to resource allocation.

[0178] 3. Remote shutdown and automatic deployment: After confirming that the new terminal is ready, the management server will remotely send a shutdown command to the newly deployed host. This marks the successful completion of the automatic deployment process, and also reserves space for subsequent formal activation or further configuration. Through this design, this application ensures that each host can be put into operation in the best condition, avoiding potential problems caused by improper manual operation.

[0179] 4. Unattended, efficient and convenient: The entire deployment process is truly unattended. Users only need to place the high-performance cloud computing host in the designated location of the computer room and connect it to the power supply, and all the remaining work will be completed automatically by the system. This not only greatly saves deployment time, but also significantly reduces dependence on human resources, allowing the operation and maintenance team to focus more on other core tasks.

[0180] In summary, the technical solution of this application can bring the following significant advantages:

[0181] 1. Efficient resource utilization and cost savings: Adaptive intelligent scheduling algorithms can be introduced to achieve a 1:N efficient ratio between high-performance hosts and local AIO thin terminals. This strategy greatly improves the utilization of hardware resources, effectively reduces overall equipment costs, and brings a more economical and efficient gaming experience to enterprises and users.

[0182] 2. Ultra-low latency "local mouse" experience: The cutting-edge "local mouse" solution makes the mouse control of the local AIO thin client respond quickly, with almost no latency. This improvement completely solves the pain point of mouse operation lag in traditional cloud games, allowing players to enjoy an immersive operating experience.

[0183] 3. Ultra-high-definition video transmission technology: Relying on mature video encoding and decoding algorithms, the Cloud Gaming architecture can achieve smooth video transmission at 60FPS at 2.5K QHD (Quarter High Definition) resolution, while maintaining an extremely low latency of about 2ms. This technological breakthrough brings users an unprecedented visual feast, making the game screen more delicate and realistic, and the action more coherent and smooth.

[0184] 4. Lossless transmission of host performance: Through the 1-to-1 direct connection technology of the optical fiber network, the local AIO thin terminal can losslessly receive and display the full performance of the high-performance host. This means that players can enjoy the graphics processing capabilities of desktop graphics cards, whether it is large-scale 3D games or high-load graphics applications, they can easily handle it with uncompromising performance.

[0185] 5. Modular design, convenient maintenance and upgrade: The AIO thin terminal in the Cloud Gaming architecture adopts a modular design, and the motherboard module is pluggable, which is convenient for daily maintenance and quick upgrade. This design not only reduces maintenance costs, but also improves the scalability and flexibility of the system, ensuring that users can continue to enjoy the latest gaming experience and technical support.

[0186] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0187] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0188] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0189] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A processing method, using a first electronic device, the method comprising: In response to input data corresponding to the input device of the second electronic device, determining an input action corresponding to the input data; generating an output screen corresponding to the input action according to a target processing rule corresponding to the input action; The output screen is transmitted to the second electronic device, so that the second electronic device outputs the output screen and synchronously outputs an icon screen corresponding to the input device.

2. The method according to claim 1, wherein generating an output screen corresponding to the input action according to a target processing rule corresponding to the input action comprises: According to the target processing rule corresponding to the input action, the action type and / or the position coordinates corresponding to the input action are processed to obtain the output picture; The action type includes at least one of the following: Cursor selection type; Cursor movement type; Scroll wheel scroll type; Key input type.

3. A processing method, applied to a second electronic device, the method comprising: In response to obtaining an output picture transmitted by the first electronic device, outputting the output picture; Based on the input data of the input device of the second electronic device, an icon screen corresponding to the input device is generated in the output screen.

4. The method according to claim 3, further comprising: Based on a target transmission rule, the input data corresponding to the input device and the output screen is transmitted to the first electronic device.

5. The method according to claim 4, transmitting the input data corresponding to the input device and the output screen to the first electronic device according to the target transmission rule, comprises: monitoring whether the input device performs an input action on the output picture; When the input device performs an input action on the output screen, obtaining first input data according to an action type corresponding to the input action and position coordinates corresponding to the input action on the output screen; transmitting the first input data to the first electronic device; When the input device has no input action on the output screen, obtaining second input data according to the position coordinates of the device icon corresponding to the input device on the output screen; In response to a transmission condition being satisfied, the second input data is transmitted to the first electronic device.

6. The method according to claim 5, wherein the transmission condition comprises: The interval duration between the historical moment of the last transmission of the second input data reaches the target duration; The second input data includes: the position coordinates of the input device accumulated in the output screen within the interval time.

7. The method according to claim 6, further comprising: Obtaining an action frequency of the input device performing an input action on the output picture; The target duration is updated according to the action frequency.

8. The method according to claim 6, further comprising: Obtaining a predicted duration of the next input action performed by the input device on the output screen; The target duration is updated according to the predicted duration.

9. An electronic device, the electronic device being a first electronic device, comprising: a processor, configured to determine an input action corresponding to the input data in response to the second electronic device corresponding to the input device; generating an output screen corresponding to the input action according to a target processing rule corresponding to the input action; The transmission module is used to transmit the output screen to the second electronic device, so that the second electronic device outputs the output screen and synchronously outputs the icon screen corresponding to the input device.

10. An electronic device, the electronic device serving as a second electronic device, comprising: The display is configured to output the output picture transmitted by the first electronic device in response to obtaining the output picture transmitted by the first electronic device; The processor is configured to generate an icon screen corresponding to the input device in the output screen based on input data of the input device of the second electronic device.

Images