Mouse synchronization method and system for cloud electronic sports, storage medium and processing equipment

Abstract

The invention discloses a mouse synchronization method and system for cloud electronic sports, a storage medium and processing equipment, and the method comprises the steps: locally collecting first information of a physical mouse, and meanwhile, remotely collecting second information of a virtual mouse, the physical mouse being a main device of the virtual mouse; the local sends the first information to the remote end, and the remote end obtains third information based on integration of the second information and the picture information and returns the third information to the local; and locally performing optimization synchronization on the cursor position of the physical mouse based on the third information. In the application, the first information of the physical mouse and the second information of the virtual mouse are collected and are integrated at the far end by combining with the picture information to obtain the third information, and then the third information is returned to the local so as to adjust the cursor position of the physical mouse in real time. According to the method and the device, the physical mouse and the virtual mouse can be matched and synchronized while the remote picture information is considered, so that the movement of the mouse cursor is smoother and more stable, and the user experience can be improved.

Description

Technical Field

[0001] The present application relates to the field of cloud e-sports technology, and in particular to a mouse synchronization method, system, storage medium and processing device for cloud e-sports. Background Art

[0002] With the advancement of technology and the gradual maturity of the game market, cloud gaming and cloud e-sports have become an important part of the future gaming industry. Games cannot be separated from the use of keyboards and mice, especially some games where the use of mice is particularly important. The refresh rate of the mouse has a great impact on the response speed and sensitivity of the mouse. Generally speaking, the higher the refresh rate of the mouse, the smoother the movement of the mouse on the screen and the faster the response speed. When performing high-precision operations, such as games, choosing a mouse with a high refresh rate can improve the smoothness and response speed of the game.

[0003] However, current cloud gaming is limited by the client and remote instance transmission interaction technology, making it difficult to match the report rate and DPI of a high-refresh rate mouse, affecting the player's gaming experience. Summary of the invention

[0004] The purpose of this application is to provide a cloud e-sports mouse synchronization method, system, storage medium and processing device to solve the technical problem that the existing cloud e-sports mouse refresh rate is difficult to meet the local mouse refresh rate and affects the user experience. The many technical effects that can be produced by the preferred technical solution among the many technical solutions provided in this application are described in detail below.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] The first aspect of the present application provides a mouse synchronization method for cloud e-sports, including: locally collecting first information of a physical mouse and remotely collecting second information of a virtual mouse, wherein the physical mouse is the master device of the virtual mouse; locally sending the first information to the remote end, and the remote end integrating the second information and screen information to obtain third information and transmitting the third information back to the local end; locally optimizing and synchronizing the cursor position of the physical mouse based on the third information.

[0007] In some embodiments, the method further includes: remotely requesting a local USB device list; and locally sending the USB device list to the remote end, wherein the USB device list includes device information of the physical mouse.

[0008] In some embodiments, the method further includes: the remote end selects the physical mouse from the USB device list, and requests the local end to select and import the device information of the physical mouse; after the import is successful, the remote end requests the local end for the first information of the physical mouse.

[0009] In some embodiments, the first information is a refresh rate of the physical mouse, and the second information is a refresh rate of the virtual mouse.

[0010] In some embodiments, the method further includes: locally accessing the physical mouse through a request response process, and redirecting the physical mouse to a remote end; and remotely accessing the virtual mouse through a request response process, and redirecting the virtual mouse to the local end.

[0011] In some embodiments, the picture information is a remote rendering frame rate; and the third information is a remote video stream information.

[0012] In some embodiments, the local optimizing synchronization of the cursor position of the physical mouse based on the third information includes: synchronizing the cursor position of the physical mouse to the cursor position of the virtual mouse in the video stream.

[0013] The second aspect of the present application provides a mouse synchronization system, comprising: a local driver module and a local application module arranged locally, and a remote driver module and a remote application module arranged remotely; the local driver module is configured to collect first information of a physical mouse; the remote driver module is configured to collect second information of a virtual mouse; the local application module is configured to send the first information collected by the local driver module to the remote end; the remote application module is configured to receive the first information, and transmit the third information obtained by the remote end based on the integration of the second information and the screen information back to the local application module.

[0014] A third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed, the mouse synchronization method for cloud e-sports as described above is implemented.

[0015] The fourth aspect of the present application provides a processing device, including: one or more processors; a memory for storing one or more computer programs, and one or more of the processors are used to execute one or more computer programs stored in the memory, so that one or more of the processors execute the mouse synchronization method for cloud e-sports as described above.

[0016] Implementing one of the above technical solutions of the present application has the following advantages or beneficial effects: In the present application, by collecting the first information of the physical mouse and the second information of the virtual mouse, and integrating them remotely with the screen information to obtain the third information, the third information is then transmitted back to the local to adjust the cursor position of the physical mouse in real time. In this case, the physical mouse and the virtual mouse can be matched and synchronized while considering the remote screen information, so that the movement of the mouse cursor is smoother and more stable, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. It is obvious that the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 is a flowchart of a mouse synchronization method according to an embodiment of the present application;

[0019] Figure 2 It is a structural block diagram of the processing device of an embodiment of the present application.

[0020] In the figure: 1. processing device; 10. memory; 11. processor. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present application clearer, the various exemplary embodiments to be described below will refer to the corresponding drawings, which constitute a part of the exemplary embodiments, wherein various exemplary embodiments that may be used to implement the present application are described. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present disclosure. It should be understood that they are only examples of processes, methods, and devices that are consistent with some aspects disclosed in the present application as detailed in the attached claims, and other embodiments may also be used, or the embodiments listed herein may be modified in structure and function without departing from the scope and essence of the present application.

[0022] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", etc. indicate the orientation or positional relationship based on the drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the elements referred to must have a specific orientation, be constructed and operated in a specific orientation. The terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. The term "multiple" means two or more. The terms "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, an integral connection, a mechanical connection, an electrical connection, a communication connection, a direct connection, an indirect connection through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0023] In order to illustrate the technical solution described in the present application, a specific embodiment is provided below, and only the parts related to the embodiment of the present application are shown.

[0024] The embodiments of the present application can be applied to application scenarios such as cloud e-sports or cloud games. The core principle of cloud games is to complete the running and rendering process of the game remotely, and then transmit the game screen to the user's terminal device connected to the local in the form of a video stream. The user can input operations and send these operation instructions back to the remote server. For example, the user can operate through the physical mouse of the terminal device, collect the operation information of these physical mice locally, and send the operation information to the remote end as an operation instruction.

[0025] like Figure 1 As shown, the present application provides a mouse synchronization method for cloud e-sports, which may include the following steps (step S1 to step S3):

[0026] S1. Locally collect first information of a physical mouse and remotely collect second information of a virtual mouse, wherein the physical mouse is a master device of the virtual mouse.

[0027] In some embodiments, the local may refer to a local server. The local may be connected to a user's terminal device, such as a desktop computer, a notebook, etc. The local may collect information about a USB device of the terminal device, such as collecting information about a physical mouse of the terminal device.

[0028] In some embodiments, a virtual mouse may be set up at a remote end, for example, a SDK or API may be used to create a virtual mouse. The user's operation on a physical mouse may be mapped to the virtual mouse. The virtual mouse may refer to a mouse device inside a virtual machine.

[0029] In some embodiments, the local can establish a communication connection with the remote, for example, based on the TCP / IP protocol. The local and remote can communicate by sending data packets to each other. For example, the local can integrate the information of the physical mouse into a data packet and send it to the remote. The information of the physical mouse may include at least one of the mouse position information, the mouse movement information, the mouse button status, and the mouse wheel status.

[0030] In some embodiments, the mouse synchronization method of cloud e-sports may also include: the local accesses the physical mouse through a request response process, and redirects the physical mouse to the remote end; the remote end accesses the virtual mouse through a request response process, and redirects the virtual mouse to the local end.

[0031] In some embodiments, redirecting the physical mouse to the remote end may refer to transmitting the input operation of the physical mouse to the remote end, and performing the input operation on the remote end. Correspondingly, redirecting the virtual mouse to the local end may refer to transmitting the input operation of the virtual mouse to the local end, and performing the input operation on the local end.

[0032] In some embodiments, the user's terminal device may include a USB hub and a USB host controller. The USB hub may be used to manage USB devices connected thereto, such as a physical mouse. When a physical mouse is connected to the USB hub, the USB hub will notify the terminal device that a new device is connected. The USB host controller may be used to connect the USB device to the terminal device, and the USB host controller may be used to process the USB protocol and communicate with the terminal device.

[0033] In some embodiments, the request response process for the mouse can be performed by a USB driver. The USB driver can be used to send request information to the mouse and receive response information from the mouse. For example, the USB driver can be used to send an IO request to the mouse.

[0034] In some embodiments, accessing the mouse through the request response process can obtain real-time information of the corresponding mouse, such as a physical mouse or a virtual mouse. The real-time information may include input operations and states of the mouse.

[0035] In some embodiments, the first information may be the refresh rate of the physical mouse, and the second information may be the refresh rate of the virtual mouse. Since the physical mouse and the virtual mouse need to be redirected in the cloud gaming scenario, the refresh rate of the physical mouse may not match the refresh rate of the virtual mouse; at the same time, since the performance of the physical mouse, such as the refresh rate and DPI, is constantly improving, the refresh rate of the physical mouse is much greater than the refresh rate of the virtual mouse, making it difficult to use the physical mouse stably. Therefore, the embodiment of the present application collects the refresh rates of the physical mouse and the virtual mouse for matching processing, which can keep the physical mouse smooth during operation, thereby improving the user experience.

[0036] In some embodiments, the mouse synchronization method of cloud e-sports may further include: remotely requesting a local USB device list; locally sending the USB device list to the remote end, wherein the USB device list may include device information of the physical mouse. In some embodiments, the device information may include at least one of a device name, a device ID, a serial number, a device class, a device subclass, a protocol, an interface, an endpoint, driver information, a connection status, a device speed, and power management information.

[0037] In some embodiments, the remote end may send an OP_REQ_DEVLIST instruction to the local end to request a list of currently available USB devices. After receiving the OP_REQ_DEVLIST instruction, the local end may return an OP_REP_DEVLIST packet, which may include a list of USB devices. In some embodiments, the USB device list may also include other input devices, such as keyboards, game controllers, storage devices, audio devices, or video devices.

[0038] In some embodiments, the mouse synchronization method of cloud e-sports may also include: the remote end selects a physical mouse from the USB device list, and requests the local end to select and import the device information of the physical mouse; after the import is successful, the remote end requests the local end for the first information of the physical mouse.

[0039] In some embodiments, the remote end may send an OP_REQ_IMPORT packet to the local end to request that a specific USB device be imported into the local end. By sending the OP_REQ_IMPORT packet, the remote end may import the physical mouse in the USB device list. After receiving the OP_REQ_IMPORT packet, the local end may return an OP_REP_IMPORT packet to the remote end, at which point it may be considered that the physical mouse has been successfully imported.

[0040] In some embodiments, the remote end may also send a USBIP_CMD_SUBMIT packet to the local end for submitting a URB, and the local end may return a USBIP_RET_SUBMIT packet. In some embodiments, the remote end may also send a USBIP_CMD_SUBMIT packet to the local end for canceling a previously submitted URB, and the local end may return a USBIP_RET_UNLINK packet. Among them, URB (USB Request Block) is a data structure used to describe a USB request in USB communication.

[0041] S2. The local end sends the first information to the remote end, and the remote end integrates the second information and the picture information to obtain the third information and transmits the third information back to the local end.

[0042] In some embodiments, the picture information may be a rendering frame rate of the remote end. In some embodiments, the third information may be video stream information of the remote end.

[0043] In some embodiments, after receiving the first information, the remote end may compare the first information with the second information, and adjust the remote end screen according to the comparison result. The remote end screen may refer to a game screen displayed on the remote end server.

[0044] In some embodiments, the remote end may adjust the rendering frame rate based on the display refresh rate of the local terminal device, and adjust the remote image according to the adjusted rendering frame rate.

[0045] In some embodiments, the remote end can adjust the remote end image through a fusion video image analysis device.

[0046] S3. Locally optimize and synchronize the cursor position of the physical mouse based on the third information.

[0047] In some embodiments, optimizing and synchronizing the cursor position of the physical mouse locally based on the third information may include: synchronizing the cursor position of the physical mouse to the cursor position of the virtual mouse in the video stream.

[0048] In some embodiments, the cursor position of the physical mouse can be synchronized to the cursor position in the video stream based on the third information. The cursor position of the virtual mouse in the video stream can be obtained by parsing the third information. In some embodiments, the remote end can reflect the input operation of the physical mouse through the video stream.

[0049] The application also relates to a mouse synchronization system.

[0050] In some embodiments, the mouse synchronization system may include a local driver module and a local application module disposed locally, and a remote driver module and a remote application module disposed remotely. The local driver module may be configured to collect first information of a physical mouse. The remote driver module may be configured to collect second information of a virtual mouse. The local application module may be configured to send the first information collected by the local driver module to the remote end. The remote application module may be configured to receive the first information, and transmit the third information obtained by the remote end based on the integration of the second information and the screen information back to the local application module.

[0051] In some embodiments, the local driver module may include a pnptls driver, a stub driver, and a local USB driver. The remote driver module may include a remote USB driver and a vhci driver.

[0052] In some embodiments, the pnptls driver can be used to load a stub driver for the physical mouse, the stub driver can be used to redirect the physical mouse to a remote end, and the local USB driver can be used to collect first information of the physical mouse and access the physical mouse through a request response process.

[0053] In some embodiments, the remote USB driver can be used to collect the second information of the virtual mouse and access the virtual mouse through a request response process, and the vhci driver can be used to redirect the virtual mouse to the local.

[0054] In some embodiments, the local application module may include a local USB application program. The local USB application program may be communicatively connected with the pnptls driver, the stub driver, and the local USB driver.

[0055] In some embodiments, the local USB application can be used to forward the relevant information of the stub driver, the USB request information used to execute the request response process, and the USB response message through the network. The local USB application can communicate with the pnptls driver to enumerate all USB device information, that is, the device information of each USB device in the USB device list. The local USB application can notify the pnptls driver to load the stub driver to the specified USB device, such as a physical mouse.

[0056] In some embodiments, the remote application module may include a remote USB application program. The remote USB application program may be communicatively connected with both the remote USB driver and the vhci driver.

[0057] In some embodiments, the remote USB application can be used to forward the USB request message and USB response message of the vhci driver through the network, that is, the request and response process of accessing the virtual mouse, and can be used to pass the information of the physical mouse to the vhci driver.

[0058] Those skilled in the art can understand that all or part of the features / steps of the above-mentioned method embodiments can be implemented by methods, data processing systems or computer programs, and these features can be implemented without hardware, entirely by software, or by a combination of hardware and software. The aforementioned computer program can be stored in one or more computer-readable storage media, and the storage medium stores the computer program. When the computer program is executed (such as by a processor), the steps of the above-mentioned cloud e-sports mouse synchronization method embodiment are executed.

[0059] The aforementioned storage media that can store program codes include: static hard disks, solid-state hard disks, random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), optical storage devices, magnetic storage devices, flash memory, magnetic disks or optical disks and / or combinations of the above devices, that is, they can be implemented by any type of volatile or non-volatile storage device or a combination thereof.

[0060] like Figure 2As shown, the present application also provides an embodiment of a processing device 1, comprising one or more processors 11 and a memory 10; wherein the memory 10 is used to store one or more computer programs, and the one or more processors 11 are used to execute one or more computer programs stored in the memory 10, so that the processor 11 executes the features / steps of the above-mentioned cloud e-sports mouse synchronization method embodiment.

[0061] The above is only the preferred embodiment of the present application. Those skilled in the art will appreciate that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present application. In addition, under the guidance of the present application, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present application. Therefore, the present application is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the present application belong to the protection scope of the present application.

Claims

1. A mouse synchronization method for cloud e-sports, characterized in that: include: locally collecting first information of a physical mouse and remotely collecting second information of a virtual mouse, wherein the physical mouse is a master device of the virtual mouse; The local end sends the first information to the remote end, and the remote end integrates the second information and the picture information to obtain the third information and transmits the third information back to the local end; The cursor position of the physical mouse is optimized and synchronized locally based on the third information.

2. The mouse synchronization method for cloud e-sports according to claim 1, characterized in that: The method further comprises: the remote end requests a local USB device list; the local end sends the USB device list to the remote end, wherein the USB device list includes device information of the physical mouse.

3. The mouse synchronization method for cloud e-sports according to claim 2, characterized in that: The method further includes: the remote end selects the physical mouse from the USB device list, and requests the local end to select and import the device information of the physical mouse; after the import is successful, the remote end requests the local end for the first information of the physical mouse.

4. The mouse synchronization method for cloud e-sports according to claim 1, characterized in that: The first information is a refresh rate of the physical mouse, and the second information is a refresh rate of the virtual mouse.

5. The mouse synchronization method for cloud e-sports according to claim 1, characterized in that: The method further comprises: the local accesses the physical mouse through a request response process, and redirects the physical mouse to the remote end; the remote end accesses the virtual mouse through a request response process, and redirects the virtual mouse to the local end.

6. The mouse synchronization method for cloud e-sports according to claim 1, characterized in that: The picture information is the rendering frame rate of the remote end; the third information is the video stream information of the remote end.

7. The mouse synchronization method for cloud e-sports according to claim 6, characterized in that: The locally optimizing and synchronizing the cursor position of the physical mouse based on the third information includes: synchronizing the cursor position of the physical mouse to the cursor position of the virtual mouse in the video stream.

8. A mouse synchronization system, characterized in that: include: A local driver module and a local application module are arranged locally, and a remote driver module and a remote application module are arranged remotely; The local driving module is configured to collect first information of the physical mouse; The remote driving module is configured to collect second information of the virtual mouse; The local application module is configured to send the first information collected by the local driving module to a remote end; The remote application module is configured to receive the first information, and transmit the third information obtained by remote integration based on the second information and the picture information back to the local application module.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which, when executed, implements the mouse synchronization method for cloud e-sports as described in any one of claims 1-7.

10. A processing device, characterized in that: include: one or more processors; The memory is used to store one or more computer programs, and one or more processors are used to execute the one or more computer programs stored in the memory, so that one or more processors execute the mouse synchronization method for cloud e-sports as described in any one of claims 1-7.

Images