Interaction instruction processing method and device, electronic equipment and storage medium

Abstract

The invention provides an interaction instruction processing method and apparatus, an electronic device and a storage medium. The method comprises the steps of displaying a game picture of a cloud game in a graphical user interface of a television; wherein the television provides the cloud game of a touch type through the connected cloud server, the television is in signal connection with the remote controller, and the touch operation mode of the cloud game corresponds to the key operation mode on the remote controller; in response to a first key operation for the remote controller, sending a first simulation touch event corresponding to the first key operation to the cloud server according to the corresponding relationship; and in response to a processing result returned by the cloud server according to the first simulation touch event, updating a game picture of the graphical user interface according to the processing result. Through the technology, cloud games can be conveniently and rapidly played on the television by using the remote controller.

Description

Technical Field

[0001] The present disclosure relates to the field of game technology, and in particular to a method, device, electronic device, and storage medium for processing interactive instructions. Background Art

[0002] With the development of cloud computing technology, the variety of cloud gaming is increasing. Cloud gaming refers to games running on cloud servers, which are computing services that provide simple, efficient, and elastic processing capabilities. Cloud servers are responsible for game logic operations, image rendering, and other functions. Users only need to use the game display components (such as touch screens) to play games.

[0003] Existing cloud games are typically developed for touch-screen devices like mobile phones, requiring users to tap the screen to control them. These applications are considered touch-based. Traditional TVs generally lack touch functionality and require peripherals like game controllers for control. If cloud games developed for touch-screen devices like mobile phones are directly ported to TVs, the TVs won't recognize the remote control's button commands, making convenient cloud gaming impossible. Summary of the Invention

[0004] The purpose of the present disclosure is to provide a method, device, electronic device and storage medium for processing interactive instructions, so as to conveniently play cloud games on a TV through a remote control.

[0005] An embodiment of the present disclosure provides a method for processing interactive instructions, the method comprising: displaying a game screen of the cloud game in a graphical user interface of a television; wherein the television provides the touch-type cloud game through a connected cloud server, the television is connected to a remote control signal, and there is a correspondence between the touch operation mode of the cloud game and the button operation mode on the remote control; in response to a first button operation on the remote control, sending a first simulated touch event corresponding to the first button operation to the cloud server according to the correspondence; in response to a processing result returned by the cloud server according to the first simulated touch event, updating the game screen of the graphical user interface according to the processing result.

[0006] In the second aspect, the embodiment of the present disclosure also provides a device for processing interactive instructions, the device comprising: a first display module, for displaying the game screen of the cloud game in the graphical user interface of the TV; wherein the TV provides the touch-type cloud game through a connected cloud server, the TV is connected to the remote control signal, and there is a correspondence between the touch operation mode of the cloud game and the button operation mode on the remote control; a touch event sending module, for responding to a first button operation on the remote control, and sending a first simulated touch event corresponding to the first button operation to the cloud server according to the correspondence; a second display module, for responding to the processing result returned by the cloud server according to the first simulated touch event, and updating the game screen of the graphical user interface according to the processing result.

[0007] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the above-mentioned method for processing interactive instructions.

[0008] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the processing method of the above-mentioned interactive instructions.

[0009] The disclosed embodiments provide a method, apparatus, electronic device, and storage medium for processing interactive commands. By setting a cloud gaming mode, the television's response mode can be switched. In key mode, the remote control's key operations are processed in the aforementioned manner. In non-key mode, the remote control's key operations can be processed in the existing manner or in other manners, effectively giving the remote control's key operations more functionality. For example, while existing remote controls can only control television programs, this embodiment can build on this to also enable cloud gaming control. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 A flowchart of a method for processing interactive instructions provided by an embodiment of the present disclosure; Figure 2A schematic diagram of key operations provided in an embodiment of the present disclosure; Figure 3 A schematic diagram of an interaction for simulating a single touch operation according to an embodiment of the present disclosure; Figure 4 A schematic diagram of an interaction for a continuous touch operation on a direction control area provided by an embodiment of the present disclosure; Figure 5 A schematic diagram of an interaction for a continuous touch operation on a viewing angle control area provided by an embodiment of the present disclosure; Figure 6 A schematic diagram of the structure of a device for processing interactive instructions provided in an embodiment of the present disclosure; Figure 7 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0012] The following will clearly and completely describe the technical solutions of the present disclosure in conjunction with the embodiments. Obviously, the embodiments described are only a part of the embodiments of the present disclosure, not all of them. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present disclosure.

[0013] Today's televisions can access broadband networks through set-top boxes or computers, providing services such as digital television, time-shifted television, and interactive television. The emergence of Internet television has brought people a new way to watch television. Users can choose programs of interest to play on TV, realizing functions such as on-demand viewing and stop-and-see.

[0014] To further expand the functionality of televisions, this embodiment provides a method, device, electronic device, and storage medium for processing interactive commands. This technology simulates touch controls via a remote control, enabling televisions to utilize the touch-driven approach of cloud gaming, effectively reducing the development cost of porting cloud gaming to televisions.

[0015] The remote control of this embodiment is also called a TV remote control. It has functions such as controlling TV programs, such as turning the TV on and off, switching channels, etc. The remote control includes basic buttons such as a power button, a menu button, a confirmation button, and directional keys. The directional keys specifically include up, down, left, and right directional buttons.

[0016] See also Figure 1 The flowchart of a method for processing interactive instructions is shown. The television in this embodiment is a terminal device with a display function. The method includes the following steps: Step S102, displaying the game screen of the cloud game in the graphical user interface of the TV; wherein, the TV provides touch-type cloud games through the connected cloud server, the TV is connected to the remote control signal, and there is a correspondence between the touch operation mode of the cloud game and the button operation mode on the remote control.

[0017] Touch-type cloud games refer to cloud games driven by touch operations. Touch operations usually include the touch object and operation method. The touch object can be a virtual object or a certain area in the game, etc. The operation method can include clicking, long pressing, sliding, double-clicking, triple-clicking, etc.

[0018] The aforementioned cloud game screen can be accessed by selecting the cloud game icon on the remote control to open the cloud game, which then displays the cloud game screen. Some cloud game login interfaces require user authentication, which can be performed by entering user information via the remote control or scanning a QR code with a mobile phone. Once authenticated, the cloud game screen will be displayed on the TV's graphical user interface.

[0019] The above-mentioned game screen refers to the screen corresponding to the cloud game application provided or displayed through a graphical user interface, which includes a UI interface and game scene for players to interact. In an optional embodiment, the UI interface may include game controls (such as skill controls, movement controls, function controls, etc.), indicators (such as direction indicators, character indicators, etc.), information display areas (such as the number of kills, game time, virtual prizes, etc.), or game setting controls (such as system settings, stores, gold coins, etc.), or game menu items, etc. In an optional embodiment, the game scene is a display screen corresponding to the virtual scene displayed on the TV, and the game scene may include virtual objects such as game characters, NPCs (Non-Player Characters), and AI (Artificial Intelligence) characters that execute game logic in the virtual scene.

[0020] The touch operation mode of cloud games may include: clicking or single-clicking the virtual object in the game screen, dragging the virtual object, long pressing the virtual object, multiple clicks (such as double-clicking or triple-clicking the virtual object, etc.). The virtual object may be a virtual character, a virtual control, or a menu item. The key operation mode on the remote control may include single-clicking, long pressing, or multiple clicks on keys such as direction keys, menu keys, or confirmation keys. The correspondence between the touch operation mode and the key operation mode is one-to-one. Different cloud games may have different correspondences. For example, each cloud game is configured with a set of correspondences between the touch operation mode of the cloud game and the key operation mode on the remote control. The correspondence can be pre-configured on the TV. Alternatively, the correspondence can also be pre-configured in the client of the cloud game, and the TV interacts with the cloud server through the client of the cloud game to play the cloud game.

[0021] Step S104 , in response to a first button operation on the remote controller, a first simulated touch event corresponding to the first button operation is sent to the cloud server according to the above correspondence.

[0022] After the user performs the first button operation on the remote control, the TV side will receive the event corresponding to the operation. The TV will identify the operation object (such as button identifier) ​​and operation type (such as single click, double click, long press, triple click, etc.) based on the event. After identifying this information, the TV sends a first simulated touch event to the cloud server. The first simulated touch event is used to represent the touch operation mode corresponding to the first button operation.

[0023] The object of the first key operation can be a direction key, a confirmation key, or a menu key, and the operation type can be a long press or a click. Among them, the long press operation corresponds to a key press duration greater than the set press threshold, and the click operation, also known as a single-click operation, corresponds to a key press duration shorter than the set press threshold.

[0024] The first simulated touch event may include a sliding event, a pressing event, a lifting event, etc. corresponding to a virtual object (such as a virtual character, a virtual control, or a menu item).

[0025] Step S106 , in response to the processing result returned by the cloud server according to the first simulated touch event, updating the game screen of the graphical user interface according to the processing result.

[0026] After receiving the above-mentioned first simulated touch event, the cloud server will process it according to the first simulated touch event. Different first simulated touch events may correspond to different processing methods. For example: if the first simulated touch event is a press event corresponding to a jump control, the cloud server will control the player's virtual character in the above-mentioned game screen to perform a jump action. If the direction control area corresponding to the first simulated touch event is slid to the left, the cloud server will control the player's virtual character in the above-mentioned game screen and the touch point of the direction control area to move to the left.

[0027] The above method is for touch-type cloud games. It configures the correspondence between the touch operation mode of the cloud game and the button operation mode on the remote control on the TV. Then, it can respond to the button operation of the remote control according to the correspondence, and send the simulated touch event corresponding to the button operation to the cloud server. The cloud server can then process it according to the simulated touch event. This process does not require increasing hardware costs or changing the touch driving method of the cloud game. Cloud games can be conveniently played on the TV through the remote control, effectively expanding the functions of the TV and the application scenarios of cloud games.

[0028] As a possible implementation, before responding to the key operation on the remote control, the method further includes: responding to the mode setting operation for the cloud game, switching the response mode of the TV; wherein the response mode of the TV includes key mode and non-key mode; accordingly, the response to the first key operation on the remote control is an operation in the key mode. In this way, the TV can switch between the key mode and the non-key mode. If the TV is in the key mode, the above-mentioned operation is executed. Figure 1 Steps S104 and S106 in the method shown.

[0029] The aforementioned non-key modes may include: TV remote control mode, simulated mouse interaction mode, game controller mode, etc. For example, if the TV is in TV remote control mode, it can respond to remote control button operations in the usual TV processing manner; if the TV is in simulated mouse interaction mode, it can also display a simulated mouse in the TV's graphical user interface, and use the remote control to operate the simulated mouse for cloud gaming; if the TV is in game controller mode, cloud gaming can be played on the TV using the game controller.

[0030] The above-mentioned mode setting operation can be an operation on the remote control, or an operation of selecting a virtual control on the graphical user interface by simulating a mouse, or, if the TV supports voice control, a voice control operation, etc.

[0031] The above method can switch the TV's response mode through the cloud game mode setting operation. In the key mode, the remote control key operation is processed in the above manner. In the non-key mode, the remote control key operation can be processed according to the existing method or other methods, effectively giving the remote control key operation more functions. For example, while the existing remote control can only realize the control function of TV programs, this embodiment can realize the control function of cloud games on this basis.

[0032] As a possible implementation method, the above-mentioned cloud game is configured with a direction control area (such as the area corresponding to the direction virtual joystick) and a perspective control area (such as the area corresponding to the perspective virtual joystick), and the key mode includes a first key mode corresponding to the direction control area and a second key mode corresponding to the perspective control area; wherein, the direction control area is used to control the movement direction of the player's virtual character in the game screen, for example, it can control the character's forward, backward, left and right movement. The perspective control area is used to control the direction of the player's virtual character, for example, it can help players better observe the game screen and find enemy virtual characters. Accordingly, the above-mentioned response mode switching process of the TV includes: setting the response mode after switching according to the response mode before the TV switches and the preset mode switching sequence.

[0033] Different response modes can correspond to the same mode setting operation. Taking the mode setting operation of double-clicking the menu button on the remote control as an example, if the default response mode after the TV enters the game screen is the first key mode, the simulated touch point is displayed on the direction control area; if the preset mode switching order is: first key mode → second key mode → simulated mouse interaction mode → first key mode, then the first time you double-click the menu button, the TV switches from the first key mode to the second key mode, and the simulated touch point is displayed on the perspective control area; if you double-click the menu button again, the TV switches from the second key mode to the simulated mouse interaction mode, cancels the display of the simulated touch point, and displays the simulated mouse in the middle of the game screen. In this mode, the simulated touch point is the position of the simulated mouse.

[0034] The above method further subdivides the key mode into a first key mode corresponding to the direction control area and a second key mode corresponding to the viewing angle control area, so that the response mode of the TV can be subdivided into the first key mode and the second key mode, thereby making it more convenient to control the direction control area and the viewing angle control area in the game screen separately.

[0035] As a possible implementation, the above-mentioned setting of the response mode after switching according to the response mode before the television is switched and the preset mode switching sequence includes the following steps: (1) If the response mode of the television before switching is the first key mode, and the next mode after the first key mode in the preset mode switching sequence is the second key mode, then the response mode after switching is determined to be the second key mode, and a simulated touch point is displayed on the viewing angle control area; (2) If the response mode of the television before switching is the second key mode, and the next mode after the second key mode in the preset mode switching sequence is the non-key mode, then the response mode after switching is determined to be the non-key mode, and a simulated mouse is displayed at a specified position on the game screen; (3) If the response mode before the TV is switched is the non-key mode, and the next mode after the non-key mode in the preset mode switching sequence is the first key mode, then the response mode after switching is determined to be the first key mode, and a simulated touch point is displayed on the direction control area.

[0036] The above-mentioned method of switching the response mode in turn according to the preset mode switching sequence does not require setting too many different buttons on the remote control to trigger the response mode switching, and is simple to implement; by switching between the first key position mode, the second key position mode and the non-key position mode, it can meet the user's application needs in different scenarios and facilitate user operation.

[0037] As a possible implementation, the non-key mode is a simulated mouse interaction mode. Accordingly, the method further includes: (1) when the TV is in the non-key mode, in response to a second button operation on the remote control, the current position of the simulated mouse in the game screen is used as a simulated touch point, and a second simulated touch event corresponding to the simulated touch point is sent to the cloud server according to the second button operation; (2) the processing result returned by the cloud server based on the second simulated touch event is displayed on the graphical user interface. When the TV is in this non-key mode, if a second button operation is performed on the remote control, the TV side will receive an event corresponding to the second button operation. Based on the event, the TV identifies the second button and operation mode, as well as the current position of the simulated mouse, which can be represented by the coordinate information of the simulated mouse in the above-mentioned game screen. After identifying this information, the TV sends a second simulated touch event to the cloud server. The second simulated touch event is used to indicate that a simulated touch operation corresponding to the second button operation has occurred at the current position of the simulated mouse. This method can interact with the cloud server through simulated mouse and remote control button operations in the game interface and play cloud games on the TV. This method further expands the way touch-type cloud games are implemented on the TV side and improves the flexibility of users in playing cloud games on TV.

[0038] As a possible implementation method, the simulated touch event sent by the TV to the cloud server is related to the current response mode of the TV. Sending the first simulated touch event corresponding to the first button operation to the cloud server according to the above correspondence may include: determining the current response mode of the TV; searching for the first touch operation corresponding to the first button operation in the current response mode in the correspondence; and sending the first simulated touch event corresponding to the first touch operation to the cloud server.

[0039] Taking the correspondence shown in Table 1 as an example, if the first button operation is a single-click confirmation button, the TV first determines the current response mode. It can be seen from Table 1 below that different response modes correspond to different touch operations for single-click confirmation buttons. If the current response mode is the first key position mode or the second key position mode, a simulated touch event corresponding to the click operation at the location of the first skill is sent to the cloud server; if the current response mode is the non-key position mode, a simulated touch event corresponding to the click operation at the location of the simulated mouse is sent to the server.

[0040] Table 1

[0041] The above correspondence includes the touch operations corresponding to the remote control button operations in different response modes, and then the button operations can be converted into touch operations, and simulated touch events are sent to the cloud server. The TV is equivalent to a terminal with touch function. The cloud server side still treats the simulated touch events sent by the TV in the same way as the touch terminal, and controls the progress of the cloud game without modifying the touch trigger method of the cloud game. This reduces the development cost of migrating cloud games from touch terminals to terminals such as TVs that do not use touch operations, and is suitable for promotion and application.

[0042] As a possible implementation, the mode setting operation is a double-click operation or a long press operation on the menu key of the remote controller. This mode setting operation implemented by double-clicking or long pressing the menu key of the remote controller is simple and easy.

[0043] As a possible implementation, the key operation mode in the above correspondence includes a key identifier and an operation type. The key identifier includes a direction key, a confirmation key, and a menu key. The operation types include single-click, long-press, double-click, and triple-click. Correspondingly, the touch operation mode in the above correspondence includes a single touch operation on a first virtual object and a continuous touch operation on a second virtual object. The first virtual object includes a direction control area, a view control area, a skill control, and a cloud game menu item; the second virtual object includes a direction control area and a view control area. Specifically, a single click on a direction key on the remote control corresponds to a touch drag operation on the virtual joystick (direction or view joystick) in the game screen in the direction corresponding to the direction key; a long press on a direction key on the remote control corresponds to a touch drag operation on the virtual joystick (direction or view joystick) in the game screen in the direction corresponding to the direction key; and a multiple click on a direction key on the remote control corresponds to a touch operation on the skill control in the game screen. This correspondence enables game control functions corresponding to various touch operations in cloud gaming, meeting the needs of users for cloud gaming on a TV.

[0044] In order to accurately identify whether the specific type of the above-mentioned key operation is a single-click operation, a long-press operation, a double-click operation, or a triple-click operation, the above-mentioned sending the first simulated touch event corresponding to the first key operation to the cloud server according to the correspondence may include the following steps: (1) In response to a click operation event including a first button identifier sent by a remote control, determining the operation type of the first button operation based on the number of consecutively received click operation events (i.e., event.getRepeatCount()); wherein the time interval between consecutively received click operation events is less than a first threshold. Specifically, the click operation event can be implemented using the following three instruction messages: (11) Whether it is pressed. If event.getAction() == KeyEvent.ACTION_DOWN, it is pressed; (12) Whether it is a long press: If event.getRepeatCount() ≥ 3, it is a long press; otherwise, it is a click; (13) Key type: event.getKeyCode() KeyEvent.KEYCODE_DPAD_DOWN, corresponding to the down key; KeyEvent.KEYCODE_DPAD_UP, corresponding to the up key; KeyEvent.KEYCODE_DPAD_LEFT, corresponding to the left button; KeyEvent.KEYCODE_DPAD_RIGHT, corresponding to the right button; KeyEvent.KEYCODE_DPAD_CENTER, corresponding to the confirmation key; If it is a click, it can be determined whether it is a single click, double click, triple click or N click event based on the interval between adjacent click events, where N is an integer greater than 3. Figure 2 As shown in the key operation diagram, the arrow pointing downwards indicates a key press event, and the arrow pointing upwards indicates a key release event. Figure 2 As shown, when a remote control key is pressed, a key-down event is sent to the TV. When a remote control key is released (released), a key-up event is sent to the TV. Each time the TV receives a key-up event, it waits 200ms (this waiting time is adjustable). If no key-up event is received within 200ms, the key operation is considered complete; otherwise, it continues to record and count received key-up events. In particular, if only the double-click operation type is configured in the above correspondence, and the triple-click or N-click operation types are not configured, then pressing and releasing the key three times in a row will also be considered a double-click operation, thereby speeding up the response to the remote control key. Figure 2 In the example, if a key-up event is received at 380ms, two consecutive key-up events have been received. Then, the device waits 200ms and checks again at 580ms. If no key-up event is received between 380 and 580ms, the operation is recorded as a double-click. The 200ms delay in this example is the fixed delay sent to the TV by the background configuration.

[0045] (2) searching for the first touch operation mode corresponding to the first button identifier and the operation type in the above correspondence relationship; (3) Sending a first simulated touch event corresponding to the first touch operation mode to the cloud server.

[0046] By determining the number of consecutive click operation events received, the operation type of the key operation can be determined more accurately, thereby ensuring the reliability of interaction with the cloud server.

[0047] The above-mentioned simulated touch events may include sliding events, pressing events, lifting events, etc. There is a correspondence between the simulated touch events and the buttons and button operation modes on the remote control, and the correspondence may be pre-configured in the TV system or the TV's cloud game client.

[0048] See also Figure 3 The interactive diagram of simulating a single touch operation is shown, where the correspondence between button operations and touch operations in cloud games includes: double-clicking the up button on the remote control corresponds to a single touch operation of the jump control in the cloud game. The process between the remote control, TV, and cloud server includes the following steps: Step S301: The remote control sends a press event and a lift event corresponding to the up button to the TV.

[0049] Step S302: The TV records one consecutive click and starts a timer (with a timing duration of 200ms).

[0050] Step S303: The remote controller sends a press event and a lift event corresponding to the up button to the television, wherein the interval between the lift event and the previous lift event is less than the timing duration of the timer.

[0051] Step S304: The TV records two consecutive clicks and starts a timer (200ms). If no up-key event is received during the timer, the timer times out and step S305 is executed.

[0052] Step S305: The TV sends a touch_down event to the cloud server.

[0053] Step S306: The TV sends a touch_up event to the cloud server.

[0054] Based on the two recorded clicks of the up button, the TV determines that a double-click operation of the up button has occurred. The TV checks the above correspondence to determine that the operation is a single touch operation for the jump control, and therefore sends the above touch_down event and touch_up event to the cloud server in sequence. The touch_down event and touch_up event carry the coordinate information of the jump control. The coordinate information is determined based on the coordinate system of the game screen. The origin of the coordinate system can be located at the upper left vertex of the interface, the x-axis extends to the right along the upper side of the interface, and the y-axis extends downward along the left side of the interface. Based on this, the above coordinate information includes the x-coordinate and y-coordinate of the jump control in the coordinate system.

[0055] Step S307: The cloud server controls the player's virtual character in the cloud game to perform a jumping action.

[0056] In specific applications, the resolution of the TV screen may be different from the resolution of the cloud server corresponding to the cloud game. After the cloud server receives the above coordinate information, it needs to convert the coordinate information according to the resolution of the TV screen and the resolution of the cloud server to determine that the coordinate information corresponds to the jump control. For example, if the TV screen resolution is 1920×1080, the game screen and the TV's graphical user interface overlap (i.e., the game screen is also 1920×1080), and the cloud server's resolution is 1280×720, assuming the jump control's x-coordinate in the game screen's coordinate system is 382 and its y-coordinate is 540. The jump control's aspect ratio on the TV screen is (382 ÷ 1920 = 20%, 540 ÷ 1080 = 50%). Multiplying this value by the cloud server's resolution (1280, 720) yields the coordinates (0.2, 0.5) * (1280, 720) = (256, 360) on the cloud server's game screen. Based on this, the cloud server can determine that the event corresponds to the jump control in the game screen and control the player's avatar to perform a jump action. If the TV screen resolution is the same as the cloud server's, the cloud server can directly use this coordinate information to determine the corresponding virtual control or virtual object for subsequent control operations.

[0057] For a single touch operation, the above-mentioned sending of the first simulated touch event corresponding to the first touch operation mode to the cloud server may include: if the first touch operation mode is a single touch operation on the first virtual object (such as a single drag of the virtual joystick or a single click of the virtual control, etc.), sending the first simulated touch event corresponding to the first virtual object to the cloud server; wherein the first simulated touch event includes a simulated touch press (touch_down) event and a simulated touch lift (touch_up) event sent sequentially. The touch press event and touch lift event sent sequentially are usually events corresponding to a finger clicking on a touch terminal. Here, the simulated touch press event and simulated touch lift event sent sequentially are used to simulate a single touch operation. The above-mentioned method can effectively notify the cloud server that a single touch operation on the first virtual object has occurred in the cloud game on the TV side by sequentially sending the simulated touch press event and the simulated touch lift event, thereby triggering the cloud server to perform relevant processing.

[0058] For a continuous touch operation, the sending of the first simulated touch event corresponding to the first touch operation mode to the cloud server includes: if the first touch operation mode is a continuous touch operation on a second virtual object, sending the first simulated touch event corresponding to the second virtual object to the cloud server; wherein the first simulated touch event includes a simulated touch press event, at least one simulated touch move event, and a simulated touch lift event sent sequentially, and the simulated touch lift event is sent after the end event of the continuous touch operation is triggered. The end event of the continuous touch operation can be an event corresponding to a single click of any directional key on the remote control, or an event corresponding to a long press of other keys, where other keys refer to keys on the remote control other than the keys corresponding to the triggering of the continuous touch operation.

[0059] The simulated touch press event, at least one simulated touch move event and simulated touch lift event sent sequentially are events that imitate the corresponding events of a finger long pressing the touch screen. By sending the simulated touch press event, at least one simulated touch move event and simulated touch lift event to the cloud server sequentially, the cloud server can be effectively notified that a continuous touch operation has occurred on the second virtual object in the cloud game on the TV side, thereby triggering the cloud server to perform relevant processing.

[0060] As a possible implementation, the second virtual object is a direction control area, that is, the object currently being operated is the direction control area, and the touch point of the direction control area can move within the area; accordingly, sending the first simulated touch event corresponding to the second virtual object to the cloud server includes: sequentially sending a simulated touch press event and a simulated touch move event containing coordinate information corresponding to the direction control area to the cloud server; when it is determined that an end event of a continuous touch operation has occurred, sending a simulated touch lift event containing coordinate information corresponding to the direction control area to the cloud server. Figure 4 The diagram below shows an interactive diagram of a continuous touch operation on the directional control area. The correspondence between button operations and touch operations in a cloud game includes: long pressing the up button on the remote control corresponds to a continuous upward drag operation on the directional control area in the cloud game. When the TV's response mode is the first key position mode corresponding to the directional control area, the process between the remote control, TV, and cloud server includes the following steps: In steps S401 to S403 , the remote control sends a press event corresponding to the up button to the TV.

[0061] In step S404, the television identifies a long press operation based on the number of received press events.

[0062] The TV receives the number of press events corresponding to the up key each time, event.getRepeatCount(). If event.getRepeatCount() ≥ 3, it is a long press operation.

[0063] Step S405: The TV sends a touch_down event to the cloud server.

[0064] Step S406: The TV sends a touch_move event to the cloud server.

[0065] Based on the three recorded up button presses, the TV determines that a long press of the up button has occurred. By checking the corresponding relationship, the TV determines that the operation is a continuous upward drag operation on the directional control area. Therefore, the TV sequentially sends the touch_down event and touch_move event to the cloud server. The touch_down event and touch_move event carry the coordinate information of the directional control area. The details of this coordinate information can be found in the description of the above embodiment and will not be repeated here.

[0066] Step S407: The cloud server controls the player's virtual character in the cloud game to move forward continuously.

[0067] Step S408: The remote control sends a lift event corresponding to the up button to the TV.

[0068] Step S409: The remote control sends a press event and a lift event corresponding to the down button to the TV.

[0069] Step S410: The TV sends a touch_up event to the cloud server.

[0070] The TV receives the lift event corresponding to the up key, the press event and the lift event corresponding to the down key, and determines that the end event of the continuous touch operation has occurred, so it sends a touch_up event to the cloud server.

[0071] In step S411, the cloud server cancels the control of the player's virtual character in the cloud game to move forward.

[0072] The simulated touch points in the direction control area during the above-mentioned interaction process usually have certain boundaries, and continuous touch operations will not cause the touch points to move to the boundaries of the graphical user interface. Therefore, the player's virtual character can be continuously controlled in movement according to the duration of continuous key presses on the remote control. Until the end event of the continuous touch operation occurs, a simulated touch lift event is sent to the cloud server to end the continuous drag operation of the direction control area, so that the cloud server stops the movement control of the player character, thereby ensuring the rationality of the operation control.

[0073] As a possible implementation, the second virtual object is a view control area; the touch point in the view control area typically has no fixed boundaries, and a sustained touch operation may cause the touch point to move to the boundary of the graphical user interface and become unable to respond to the sustained touch operation. Based on this, the sending of the first simulated touch event corresponding to the second virtual object to the cloud server includes: sequentially sending a simulated touch press event and multiple simulated touch move events containing coordinate information corresponding to the view control area to the cloud server; in response to the simulated touch point corresponding to the view control area reaching the boundary of the graphical user interface, sequentially sending a simulated touch lift event, a simulated touch press event, and multiple simulated touch move events containing coordinate information corresponding to the view control area to the cloud server; and when it is determined that an end event of the sustained touch operation has occurred, sending a simulated touch lift event containing information corresponding to the direction control area to the cloud server. The touch point in the view control area can move 2% of the screen distance each time. For example, upon receiving a simulated touch move event in which the view control area is dragged upward, the cloud server controls the touch point in the view control area to move 2% of the screen distance until the touch point moves to the top of the screen.

[0074] See also Figure 5 The diagram below illustrates an interactive diagram of a continuous touch operation on the view control area. The correspondence between key operations and touch operations in a cloud game includes: long pressing the up button on the remote control corresponds to a continuous upward drag operation on the view control area in the cloud game. When the TV's response mode is the second key position mode corresponding to the view control area, the process between the remote control, TV, and cloud server includes the following steps: In steps S501 to S503 , the remote control sends a press event corresponding to the up button to the TV.

[0075] Step S504: Identify a long press operation based on the number of press events received.

[0076] The TV receives event.getRepeatCount() for the number of times the up button is pressed. If event.getRepeatCount() ≥ 3, it is a long press operation.

[0077] Step S505: The TV sends a touch_down event to the cloud server.

[0078] Step S506: The TV sends a touch_move event to the cloud server.

[0079] Based on the three recorded up button presses, the TV determines that a long press of the up button has occurred. By checking the corresponding relationship, the TV determines that the operation is a continuous upward drag operation on the viewing angle control area. Therefore, the TV sequentially sends the touch_down event and touch_move event to the cloud server. The touch_down event and touch_move event carry the coordinate information of the viewing angle control area. The details of this coordinate information can be found in the description of the above embodiment and will not be repeated here.

[0080] Step S507: The remote control sends a lift event corresponding to the up button to the TV.

[0081] Because the TV has not yet processed the upward continuous drag operation of the viewing angle control area, it will not process the lift event and continue to execute step S508.

[0082] Step S508: The TV continues to send touch_move events to the cloud server.

[0083] In step S509, the cloud server controls the perspective of the player's virtual character in the cloud game to continue moving forward.

[0084] Step S509 can be performed after step S506 occurs, that is, after the cloud server receives the touch_move event, it controls the perspective of the player's virtual character to move forward, so that the player can see virtual objects in the scene farther away.

[0085] In step S510 , the television recognizes that the touch point in the viewing angle control area reaches the boundary of the graphical user interface, and then executes step S511 .

[0086] Step S511: The TV sends a touch_up event to the cloud server.

[0087] Step S512: The TV sends a touch_down event to the cloud server.

[0088] Step S513: The TV continues to send touch_move events to the cloud server.

[0089] Step S514: The remote control sends a press event and a lift event corresponding to the down button to the TV.

[0090] Step S515: The TV sends a touch_up event to the cloud server.

[0091] The TV has received the lift event corresponding to the up key in step S507, and has received the press event and lift event corresponding to the down key in step S514, and determines that the end event of the continuous touch operation has occurred, so it sends a touch_up event to the cloud server.

[0092] Step S516: The cloud server cancels the forward movement of the perspective of the player's virtual character in the cloud game.

[0093] The above-mentioned interaction process can move the simulated touch point to the initial position when the simulated touch point reaches the boundary of the graphical user interface by sending a simulated touch lift event, a simulated touch press event and multiple simulated touch move events to the cloud server, and re-respond to the continuous touch operation. When there is an end event, a simulated touch lift event is sent to the cloud server to end the control operation of the viewing angle control area, thereby ensuring the rationality of the operation control.

[0094] In this embodiment, when the direction control area or the viewing angle control area is dragged, the touch point will be displayed on the game screen, and the touch point will move along with the dragging operation.

[0095] The above-mentioned game interaction method pre-configures the correspondence between the touch operation mode of the cloud game and the key operation mode on the remote control. When the TV receives the key operation of the remote control, it identifies the target key according to the key position identifier in the key event of the key operation sent by the remote control. The target key is one of the up key, down key, left key, right key, confirmation key or menu key. Then, based on the reception time and number of the press event and the lift event in the key event, the target operation type corresponding to the key operation is identified. The target operation type is one of a single-click operation, a double-click operation, a triple-click operation and a long-press operation. According to the identified target button and target operation type, the above-mentioned correspondence is searched to obtain the corresponding touch operation, and then the event corresponding to the touch operation is sent to the cloud server, which can trigger the cloud server to respond to the key operation and control the progress of the cloud game. As a possible implementation method, refer to Table 2 for the correspondence between the touch operation method of cloud games and the single-click button operation method on the remote control. In this table, the direction control area is represented by the area corresponding to the direction virtual joystick, and the perspective control area is represented by the area corresponding to the perspective virtual joystick. In order to simplify the correspondence, the direction control area is called the direction virtual joystick, and the perspective control area is called the perspective virtual joystick.

[0096] Table 2

[0097] See Table 3 for the correspondence between the touch operation method of cloud games and the long-press button operation method on the remote control. In this table, the direction control area is represented by the area corresponding to the direction virtual joystick, and the view control area is represented by the area corresponding to the view virtual joystick. In order to simplify the correspondence, the direction control area is called the direction virtual joystick, and the view control area is called the view virtual joystick.

[0098] Table 3

[0099] See Table 4 for the correspondence between the touch operation method of cloud games and the double-click button operation method on the remote control. In this table, the direction control area is represented by the area corresponding to the direction virtual joystick, and the view control area is represented by the area corresponding to the view virtual joystick. In order to simplify the correspondence, the direction control area is called the direction virtual joystick, and the view control area is called the view virtual joystick.

[0100] Table 4

[0101] If there are many skill controls in the cloud game, you can also configure the three-click button operation mode on the remote control with the single-click touch operation of other skill controls. The specific correspondence can be flexibly configured according to the actual touch operation needs in the cloud game, and this embodiment does not limit this.

[0102] According to the corresponding relationships in Tables 2 to 4 above, after the TV recognizes the target touch operation corresponding to the current key operation, it will simulate the target touch operation and send the corresponding event to the cloud server. The simulation process is described as follows: (1) If the target touch operation is to click on the skill control, the touch screen pressing and lifting at the location of the skill control will be simulated, and the pressing event and lifting event corresponding to these two actions will be sent to the cloud server in sequence.

[0103] (2) If the target touch operation is to drag the viewing angle control area upward, the actions of pressing the finger at the center point of the viewing angle control area, sliding the finger upward by a preset configuration distance, and lifting the finger are simulated in sequence, and the corresponding press event, slide (or move) event, and lift event of these three actions are sent to the cloud server in sequence.

[0104] (3) The target touch operation is to drag the direction control area upward. The simulation process is similar to (2) above. The only difference is that the first action simulates the finger pressing the center point of the direction control area.

[0105] (4) The target touch operation is to drag the direction control area upwards continuously until clicking any direction, or long pressing other directions to stop; then the finger is simulated to press the center point of the direction control area, and the finger continues to slide upward for a preset configuration distance until it is interrupted (for example: other direction keys are pressed), then the finger is simulated to lift up, and the press event, slide (or move) event and lift event corresponding to these three actions are sent to the cloud server in sequence.

[0106] (5) If the target touch operation is to drag the view control area upwards continuously until the button is released, the following simulations are performed in sequence: (a) the finger presses at the center point of the view control area; (b) the finger continuously slides upwards for a preset configuration distance; (c) (b) (d) is repeated every 32ms. If the upward slide reaches the edge of the graphical user interface, the finger is simulated to be lifted, and 16ms is waited before simulating the finger pressing at the center point of the view control area, and the timed slide is continued; (e) until it is interrupted (for example, the button is released), the finger is simulated to be lifted, and the timer is terminated. The press event, slide (or move) event, and lift event corresponding to the actions in (a) to (e) are sent to the cloud server in sequence.

[0107] In actual applications, the TV can configure a status field for the buttons on the remote control to record the operation status of each button and perform subsequent interaction processing based on the operation status.

[0108] Corresponding to the above method, this embodiment also provides a processing device for interactive instructions, see Figure 6 , the device includes the following modules: A first display module 62 is configured to display the game screen of the cloud game in the graphical user interface of the television; wherein the television provides touch-type cloud games through a connected cloud server, the television is connected to the remote control signal, and the touch operation mode of the cloud game corresponds to the button operation mode on the remote control; A touch event sending module 64 is configured to respond to a first button operation on the remote controller and send a first simulated touch event corresponding to the first button operation to the cloud server according to a corresponding relationship; The second display module 66 is configured to respond to a processing result returned by the cloud server according to the first simulated touch event and update a game screen of the graphical user interface according to the processing result.

[0109] The above-mentioned device can switch the response mode of the television through the cloud gaming mode setting operation. In the key mode, the above-mentioned response processing is performed on the remote control key operation. In the non-key mode, the remote control key operation can be processed according to the existing method or other methods, effectively giving the remote control key operation more functions. For example, while the existing remote control can only realize the control function of TV programs, this embodiment can build on this to realize the control function of cloud gaming.

[0110] As a possible implementation, the touch event sending module 64 is also used for: before responding to the key operation performed on the remote control, in response to the mode setting operation for the cloud game, performing the response mode switching process of the TV; wherein, the response mode of the TV includes a key mode and a non-key mode; the response to the first key operation on the remote control is the operation under the key mode. Through the mode setting operation of the cloud game, the response mode switching process of the TV can be performed, and the above-mentioned response process can be performed on the key operation of the remote control in the key mode. In the non-key mode, the key operation of the remote control can be processed in the existing manner or in other ways, effectively giving the key operation of the remote control more functions. For example: the existing remote control can only realize the control function of TV programs. The present invention can realize the control function of cloud games on this basis.

[0111] As a possible implementation, the cloud game is configured with a direction control area and a perspective control area, and the key pattern includes a first key pattern corresponding to the direction control area and a second key pattern corresponding to the perspective control area; the direction control area is used to control the movement direction of the player's virtual character in the game screen, and the perspective control area is used to control the orientation of the player's virtual character; accordingly, the touch event sending module 64 is also used to set the response mode after switching according to the response mode before the TV is switched and the preset mode switching sequence. By further subdividing the key pattern into a first key pattern corresponding to the direction control area and a second key pattern corresponding to the perspective control area, the response mode of the TV can be subdivided into a first key pattern and a second key pattern, thereby making it more convenient to control the direction control area and the perspective control area in the game screen separately.

[0112] As a possible embodiment, the touch event sending module 64 is further configured to: if the response mode before the television is switched is the first key mode, and the next mode after the first key mode in the preset mode switching sequence is the second key mode, determine the response mode after switching to the second key mode, and display a simulated touch point on the viewing angle control area; if the response mode before the television is switched is the second key mode, and the next mode after the second key mode in the preset mode switching sequence is the non-key mode, determine the response mode after switching to the non-key mode, and display a simulated mouse at a specified position on the game screen; if the response mode before the television is switched is the non-key mode, and the next mode after the non-key mode in the preset mode switching sequence is the first key mode, determine the response mode after switching to the first key mode, and display a simulated touch point on the direction control area. By switching the response mode in turn according to the preset mode switching sequence, there is no need to set too many different buttons on the remote control to trigger the response mode switching, which is simple to implement; by switching between the first key mode, the second key mode, and the non-key mode, the user's application needs in different scenarios can be met, and user operation is convenient.

[0113] As a possible implementation, the first display module 62 is further configured to: when the TV is in non-key mode, in response to a second button operation on the remote control, use the current position of the simulated mouse in the game screen as a simulated touch point, and send a second simulated touch event corresponding to the simulated touch point to the cloud server according to the second button operation; and display on the graphical user interface the processing result returned by the cloud server based on the second simulated touch event. When the TV is in non-key mode, users can interact with the cloud server through the simulated mouse in the game screen and button operations on the remote control to play cloud games on the TV. This method further expands the implementation of touch-type cloud games on the TV side and enhances the flexibility of users playing cloud games on the TV.

[0114] As a possible implementation, the touch event sending module 64 is further configured to: determine the current response mode of the TV; search the corresponding relationship for the first touch operation corresponding to the first button operation in the current response mode; and send a first simulated touch event corresponding to the first touch operation to the cloud server. The corresponding relationship includes touch operations corresponding to remote control button operations in different response modes, thereby converting button operations into touch operations and sending simulated touch events to the cloud server. This allows the cloud server to continue controlling the progress of the cloud game in accordance with the response mode of the touch terminal without modifying the touch triggering mode of the cloud game. This reduces the development cost of migrating cloud games from touch terminals to terminals that do not use touch operation, such as TVs, and is suitable for promotion and application.

[0115] As a possible implementation, the mode setting operation is a double-click operation or a long press operation on the menu key of the remote controller. The mode setting operation achieved by double-clicking or long pressing the menu key of the remote controller is simple and easy.

[0116] As a possible implementation, the key operation mode in the correspondence includes a key identifier and an operation type, wherein the key identifier includes a direction key, a confirmation key, and a menu key, and the operation type includes a single-click operation, a long-press operation, a double-click operation, and a triple-click operation. The touch operation mode in the correspondence includes: a single touch operation of a first virtual object and a continuous touch operation of a second virtual object; wherein the first virtual object includes a direction control area, a perspective control area, a skill control, and a menu item of the cloud game; and the second virtual object includes: the direction control area and the perspective control area. Through this correspondence, the game control functions corresponding to various touch operations of cloud games can be implemented, meeting the user's needs for playing cloud games on a TV.

[0117] As a possible implementation, the touch event sending module 64 is further configured to: in response to a click operation event sent by the remote control and containing a first button identifier, determine an operation type of the first button operation based on the number of consecutively received click operation events; wherein the time interval between consecutively received click operation events is less than a first threshold; search the corresponding relationship for a first touch operation mode corresponding to the first button identifier and the operation type; and send a first simulated touch event corresponding to the first touch operation mode to the cloud server. By determining the operation type of the button operation based on the number of consecutively received click operation events, the reliability of interaction with the cloud server can be ensured.

[0118] As a possible implementation, the touch event sending module 64 is further configured to: if the first touch operation mode is a single touch operation on the first virtual object, send a first simulated touch event corresponding to the first virtual object to the cloud server; wherein the first simulated touch event includes a simulated touch press event and a simulated touch lift event sent sequentially. By sequentially sending the simulated touch press event and the simulated touch lift event to the cloud server, the cloud server can be effectively notified that a single touch operation on the first virtual object has occurred in the cloud game on the TV side, thereby triggering the cloud server to perform relevant processing.

[0119] As a possible implementation, the touch event sending module 64 is further configured to: if the first touch operation mode is a continuous touch operation on the second virtual object, send a first simulated touch event corresponding to the second virtual object to the cloud server; wherein the first simulated touch event includes a simulated touch press event, at least one simulated touch move event, and a simulated touch lift event sent sequentially, and the simulated touch lift event is sent after the end event of the continuous touch operation is triggered. By sequentially sending the simulated touch press event, at least one simulated touch move event, and the simulated touch lift event to the cloud server, the cloud server can be effectively notified that a continuous touch operation on the second virtual object has occurred in the cloud game on the TV side, thereby triggering the cloud server to perform relevant processing.

[0120] As a possible implementation, the second virtual object is the directional control area; the touch event sending module 64 is further configured to sequentially send a simulated touch down event and a simulated touch move event containing coordinate information corresponding to the directional control area to the cloud server; and upon determining the end of the sustained touch operation, send a simulated touch up event containing coordinate information corresponding to the directional control area to the cloud server. The touch points of the directional control area typically have certain boundaries, and sustained touch operations will not cause the touch points to move to the boundaries of the graphical user interface. Therefore, upon the end of the sustained touch operation, a simulated touch up event can be sent to the cloud server to end the control operation of the directional control area, thereby ensuring the rationality of the operational control.

[0121] As a possible implementation, the second virtual object is the perspective control area; the above-mentioned touch event sending module 64 is also used to: send simulated touch pressing events and multiple simulated touch moving events containing the coordinate information corresponding to the perspective control area to the cloud server in sequence; in response to the simulated touch point corresponding to the perspective control area reaching the boundary of the graphical user interface, send simulated touch lifting events, simulated touch pressing events and multiple simulated touch moving events containing the coordinate information corresponding to the perspective control area to the cloud server in sequence; when it is determined that the end event of the continuous touch operation has occurred, send a simulated touch lifting event containing the direction control area to the cloud server. The touch points in the viewing angle control area usually do not have fixed boundaries. Continuous touch operations will cause the touch points to move to the boundary of the graphical user interface and become unable to continue responding to the continued touch operations. Therefore, when the simulated touch point reaches the boundary of the graphical user interface, the simulated touch point can be moved to the initial position by sending a simulated touch lift event, a simulated touch press event, and multiple simulated touch move events to the cloud server, and then respond to the continued touch operation again. When there is an end event, a simulated touch lift event is sent to the cloud server to end the control operation of the viewing angle control area, thereby ensuring the rationality of the operation control.

[0122] Each of the above-mentioned cloud games is configured with a set of corresponding relationships between the touch operation mode of the cloud game and the button operation mode on the remote control.

[0123] Before the aforementioned TV has the aforementioned correspondence, the user will not respond to the aforementioned touch-type cloud games by operating the buttons on the remote control, because these cloud games require a touch driver and cannot recognize the button operations on the remote control. In other words, the TV does not have a touch driver function, and therefore cannot control touch-type cloud games, making it difficult to conveniently play cloud games on the TV. However, the aforementioned method and device can, after starting the cloud game, convert the button functions of the remote control based on the correspondence between the touch operation mode of the cloud game and the button operation mode on the remote control, converting the button operations on the remote control into simulated touch events in the correspondence, and then continuing to use the touch driver mode to control the cloud game, thereby achieving a simple porting of touch-type cloud games from touch terminals such as mobile phones to TVs.

[0124] An embodiment of the present disclosure provides an interactive instruction processing device, whose implementation principle and technical effects are the same as those of the aforementioned method embodiment. For the sake of brief description, for parts not mentioned in the embodiment of an interactive instruction processing device, reference may be made to the corresponding content in the aforementioned interactive instruction processing method embodiment.

[0125] The term "and / or" herein simply describes an association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent the existence of three situations: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0126] The present disclosure also provides an electronic device, such as Figure 7 As shown, it is a structural diagram of the electronic device, wherein the electronic device includes a processor 71 and a memory 70, the memory 70 stores computer executable instructions that can be executed by the processor 71, and the processor 71 executes the computer executable instructions to implement the above-mentioned method for processing an interactive instruction.

[0127] exist Figure 7 In the illustrated embodiment, the electronic device further includes a bus 72 and a communication interface 73 , wherein the processor 71 , the communication interface 73 and the memory 70 are connected via the bus 72 .

[0128] Among them, the memory 70 may include high-speed random access memory (RAM), and may also include non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 73 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 72 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 72 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0129] The processor 71 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 71 or by software instructions. The above processor 71 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present disclosure may be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor 71 reads the information in the memory and completes the steps of the method for processing an interactive instruction in the above embodiment in combination with its hardware.

[0130] An embodiment of the present disclosure also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned method for processing interactive instructions. The specific implementation can be found in the aforementioned method embodiment, which will not be repeated here.

[0131] The embodiments of the present disclosure provide a method for processing interactive instructions, an apparatus, and a computer program product for an electronic device, including a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. For specific implementation, please refer to the method embodiments and will not be repeated here.

[0132] Unless otherwise specifically stated, the relative steps, numerical expressions and numerical values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0133] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0134] In the description of this disclosure, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate the description of this disclosure and simplify the description. They do not indicate or imply that the devices or components referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on this disclosure. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0135] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

Claims

1. A method for processing interactive instructions, characterized in that: The method comprises: Displaying a game screen of a cloud game in a graphical user interface of a television; wherein the television provides the touch-type cloud game through a connected cloud server, the television is connected to a remote control signal, and a touch operation mode of the cloud game corresponds to a button operation mode on the remote control; In response to a first key operation on the remote controller, sending a first simulated touch event corresponding to the first key operation to the cloud server according to the corresponding relationship; In response to a processing result returned by the cloud server according to the first simulated touch event, a game screen of the graphical user interface is updated according to the processing result.

2. The method according to claim 1, characterized in that: Before responding to a key operation performed on the remote controller, the method further includes: in response to a mode setting operation for the cloud game, performing a response mode switching process of the television; wherein the response mode of the television includes a key mode and a non-key mode; The response to the first key operation on the remote controller is an operation in the key position mode.

3. The method according to claim 2, characterized in that The cloud game is configured with a direction control area and a viewing angle control area, and the key position mode includes a first key position mode corresponding to the direction control area and a second key position mode corresponding to the viewing angle control area; the direction control area is used to control the movement direction of the player's virtual character in the game screen, and the viewing angle control area is used to control the orientation of the player's virtual character; The response mode switching process of the television includes: setting the response mode after switching according to the response mode of the television before switching and a preset mode switching sequence.

4. The method according to claim 3, characterized in that Setting the response mode after switching according to the response mode before the television is switched and the preset mode switching sequence includes: If the response mode of the television before switching is the first key mode, and the next mode of the first key mode in the preset mode switching sequence is the second key mode, determining that the response mode after switching is the second key mode, and displaying a simulated touch point on the viewing angle control area; If the response mode of the television before switching is the second key mode, and the next mode of the second key mode in the preset mode switching sequence is the non-key mode, then determining that the response mode after switching is the non-key mode, and displaying a simulated mouse at a designated position on the game screen; If the response mode of the TV before switching is a non-key mode, and the next mode of the non-key mode in the preset mode switching sequence is the first key mode, then the response mode after switching is determined to be the first key mode, and a simulated touch point is displayed on the direction control area.

5. The method according to claim 4, characterized in that The method further comprises: When the television is in a non-key mode, in response to a second key operation on the remote controller, using the current position of the simulated mouse in the game screen as a simulated touch point, and sending a second simulated touch event corresponding to the simulated touch point to the cloud server according to the second key operation; The processing result returned by the cloud server based on the second simulated touch event is displayed on the graphical user interface.

6. The method according to claim 3, characterized in that Sending a first simulated touch event corresponding to the first key operation to the cloud server according to the corresponding relationship includes: determining a current response mode of the television; Searching the corresponding relationship for a first touch operation corresponding to the first button operation in the current response mode; A first simulated touch event corresponding to the first touch operation is sent to the cloud server.

7. The method according to claim 2 or 3, characterized in that: The mode setting operation is a double-click operation or a long-press operation on a menu key of the remote controller.

8. The method according to claim 1, characterized in that: The key operation mode in the corresponding relationship includes a key identifier and an operation type, the key identifier includes a direction key, a confirmation key and a menu key, and the operation type includes a single-click operation, a long-press operation, a double-click operation and a triple-click operation; The touch operation methods in the corresponding relationship include: a single touch operation of the first virtual object and a continuous touch operation of the second virtual object; wherein the first virtual object includes a direction control area, a perspective control area, a skill control and a menu item of the cloud game; the second virtual object includes: the direction control area and the perspective control area.

9. The method according to claim 8, characterized in that Sending a first simulated touch event corresponding to the first key operation to the cloud server according to the corresponding relationship includes: In response to a click operation event including a first button identifier sent by the remote controller, determining an operation type of the first button operation according to the number of consecutively received click operation events; wherein a time interval between consecutively received click operation events is less than a first threshold; Searching the corresponding relationship for a first touch operation mode corresponding to the first button identifier and the operation type; A first simulated touch event corresponding to the first touch operation mode is sent to the cloud server.

10. The method according to claim 9, characterized in that Sending a first simulated touch event corresponding to the first touch operation mode to the cloud server includes: If the first touch operation mode is a single touch operation on the first virtual object, a first simulated touch event corresponding to the first virtual object is sent to the cloud server; wherein the first simulated touch event includes a simulated touch press event and a simulated touch lift event sent sequentially.

11. The method according to claim 9, characterized in that Sending a first simulated touch event corresponding to the first touch operation mode to the cloud server includes: If the first touch operation mode is a continuous touch operation on the second virtual object, a first simulated touch event corresponding to the second virtual object is sent to the cloud server; wherein the first simulated touch event includes a simulated touch press event, at least one simulated touch move event and a simulated touch lift event sent sequentially, and the simulated touch lift event is sent after the end event of the continuous touch operation is triggered.

12. The method according to claim 11, characterized in that The second virtual object is the direction control area; and sending the first simulated touch event corresponding to the second virtual object to the cloud server includes: Sending a simulated touch press event and a simulated touch move event containing coordinate information corresponding to the direction control area to the cloud server in sequence; When it is determined that an end event of the continuous touch operation occurs, a simulated touch lift event corresponding to the direction control area is sent to the cloud server.

13. The method according to claim 11, characterized in that The second virtual object is the viewing angle control area; and sending the first simulated touch event corresponding to the second virtual object to the cloud server includes: Sending a simulated touch press event and multiple simulated touch move events containing coordinate information corresponding to the viewing angle control area to the cloud server in sequence; In response to the simulated touch point corresponding to the viewing angle control area reaching the boundary of the graphical user interface, sequentially sending to the cloud server a simulated touch lift event, a simulated touch press event, and multiple simulated touch move events containing coordinate information corresponding to the viewing angle control area; When it is determined that an end event of the continuous touch operation occurs, a simulated touch lift event corresponding to the direction control area is sent to the cloud server.

14. The method according to claim 1, characterized in that Each cloud game is configured with a set of corresponding relationships between the touch operation mode of the cloud game and the button operation mode on the remote control.

15. A device for processing interactive instructions, characterized in that: The device comprises: A first display module is used to display the game screen of the cloud game in the graphical user interface of the television; wherein the television provides the touch-type cloud game through a connected cloud server, the television is connected to a remote control signal, and the touch operation mode of the cloud game corresponds to the button operation mode on the remote control; a touch event sending module, configured to respond to a first key operation on the remote controller and send a first simulated touch event corresponding to the first key operation to the cloud server according to the corresponding relationship; The second display module is used to respond to the processing result returned by the cloud server according to the first simulated touch event, and update the game screen of the graphical user interface according to the processing result.

16. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 14.

Images