Method, device and system for remotely testing cloud rendering performance
By obtaining and controlling the real-time video stream of the tested terminal, the problems of performance loss and single-touch limitation in cloud rendering performance testing are solved, and accurate and efficient cloud rendering performance testing is achieved.
Patent Information
- Application Number
- CN202410962331.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-15
AI Technical Summary
In the prior art, cloud rendering performance testing is difficult to be carried out accurately, especially when testing in a network shielded box, the web cloud real machine solution causes performance loss in local equipment and only supports single touch, affecting cloud rendering performance.
The real-time video stream of the tested terminal is obtained through the relay terminal, and the operation instructions are received, and the terminal is transmitted to the tested terminal for control, so as to realize multi-touch operation, avoiding the encoding and transmission functions on the tested terminal, and reducing performance losses.
Accurate testing of cloud rendering performance is achieved, avoiding performance losses, and supporting multi-touch operations, improving the accuracy and efficiency of testing.
Smart Images

Figure CN118972548A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and more specifically, to a method, device and system for remote testing of cloud rendering performance. Background Art
[0002] With the development of science and technology, rendering technology has been widely used in many fields such as film and television special effects, advertising production and game production. Cloud rendering is an application solution proposed by cloud servers for the rendering industry. The cloud rendering is to submit the rendering tasks that originally need to be performed on the local device to the remote cloud server cluster for processing, and then send the rendering results to the local device for display to the user. The cloud rendering performance experience is very important to the user. Therefore, before using the cloud rendering, the cloud rendering performance needs to be tested. The flow rate, jamming rate and delay time in the cloud rendering performance are all affected by the network. Network fluctuations, signal interference, etc. will cause performance changes. In order to reduce the impact of network interference on local devices, the local device is placed in a network shielding box for testing. However, the local device cannot be directly operated when it is placed in a network shielding box.
[0003] In the prior art, in order to test the cloud rendering performance of local devices in a network shielding box, a web cloud real machine solution is proposed. When controlling the local device through the web cloud real machine, the encoding and transmission functions need to be run on the local device, which has additional loss on the performance of the local device and affects the cloud rendering performance. In addition, the web cloud real machine only supports single-point touch local devices, and the control performance is poor, which will also affect the cloud rendering performance.
[0004] To sum up, how to test cloud rendering performance more accurately is a problem that needs to be solved at present. Summary of the invention
[0005] In view of this, an embodiment of the present invention provides a method, device and system for remotely testing cloud rendering performance. By adopting the cloud rendering testing method, cloud rendering performance can be accurately tested.
[0006] In a first aspect, an embodiment of the present invention provides a method for remotely testing cloud rendering performance, the method comprising:
[0007] Acquire a real-time video stream sent by a transfer terminal, wherein the real-time video stream is a screen display image of the tested terminal captured by the transfer terminal;
[0008] receiving an operation instruction for performing a touch operation on any frame image in the real-time video stream, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal;
[0009] The operation instruction is sent so that the tested terminal is controlled according to the operation instruction.
[0010] Optionally, the method further includes:
[0011] Sending a start-shooting instruction to the transfer terminal, wherein the start-shooting instruction is used to control the transfer terminal to turn on a camera module to shoot a screen of the tested terminal.
[0012] Optionally, the method further includes:
[0013] The real-time video stream is decoded and rendered to obtain each frame image in the real-time video stream.
[0014] Optionally, because the coordinate position of the operation point included in the operation instruction is different from the actual coordinate position on the screen of the tested terminal, sending the operation instruction specifically includes:
[0015] Transforming the coordinate position of at least one operation point in the operation instruction to generate an updated operation instruction;
[0016] The updated operation instruction is sent.
[0017] Optionally, the sending the operation instruction specifically includes:
[0018] The operation instruction is sent to the transfer terminal, and the operation instruction is forwarded to the tested terminal through the transfer terminal.
[0019] Optionally, the sending the operation instruction further includes:
[0020] Sending the operation instruction to the tested terminal.
[0021] In a second aspect, an embodiment of the present invention provides a method for remotely testing cloud rendering performance, the method comprising:
[0022] Receiving a start-shooting instruction sent by the control terminal, wherein the start-shooting instruction is used to control the transfer terminal to start a camera module to shoot the screen of the tested terminal;
[0023] Acquire a real-time video stream, wherein the real-time video stream is a screen display image of the tested terminal captured by the transfer terminal;
[0024] The real-time video stream is sent to the control terminal.
[0025] Optionally, the method further includes:
[0026] receiving an operation instruction sent by the control terminal, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal;
[0027] Sending the operation instruction to the tested terminal.
[0028] Optionally, because the coordinate position of the operation point included in the operation instruction is different from the actual coordinate position on the screen of the tested terminal, sending the operation instruction to the tested terminal specifically includes:
[0029] Transforming the coordinate position of at least one operation point in the operation instruction to generate an updated operation instruction;
[0030] The updated operation instruction is sent to the tested terminal.
[0031] Optionally, the method further includes:
[0032] receiving an operation instruction sent by the control terminal, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal;
[0033] The clicking device is controlled to perform a multi-touch operation on the tested terminal according to the operation instruction.
[0034] In a third aspect, an embodiment of the present invention provides a device for remotely testing cloud rendering performance, the device comprising:
[0035] A first acquisition unit is used to acquire a real-time video stream sent by a transfer terminal, wherein the real-time video stream is a screen display image of the tested terminal taken by the transfer terminal;
[0036] A first receiving unit, configured to receive an operation instruction for performing a touch operation on any frame image in the real-time video stream, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal;
[0037] The first sending unit is used to send the operation instruction so that the tested terminal is controlled according to the operation instruction.
[0038] Optionally, the first sending unit is further used for:
[0039] Sending a start-shooting instruction to the transfer terminal, wherein the start-shooting instruction is used to control the transfer terminal to turn on a camera module to shoot a screen of the tested terminal.
[0040] Optionally, the first acquiring unit is further used for:
[0041] The real-time video stream is decoded and rendered to obtain each frame image in the real-time video stream.
[0042] Optionally, because the coordinate position of the operation point included in the operation instruction is different from the actual coordinate position on the screen of the tested terminal, the first sending unit is specifically configured to:
[0043] Transforming the coordinate position of at least one operation point in the operation instruction to generate an updated operation instruction;
[0044] The updated operation instruction is sent.
[0045] Optionally, the first sending unit is specifically configured to:
[0046] The operation instruction is sent to the transfer terminal, and the operation instruction is forwarded to the tested terminal through the transfer terminal.
[0047] Optionally, the first sending unit is further configured to:
[0048] Sending the operation instruction to the tested terminal.
[0049] In a fourth aspect, an embodiment of the present invention provides a device for remotely testing cloud rendering performance, the device comprising:
[0050] A second receiving unit is used to receive a start-shooting instruction sent by the control terminal, wherein the start-shooting instruction is used to control the transfer terminal to start the camera module to shoot the screen of the tested terminal;
[0051] A second acquisition unit is used to acquire a real-time video stream, wherein the real-time video stream is a screen display image of the tested terminal taken by the transfer terminal;
[0052] The second sending unit is used to send the real-time video stream to the control terminal.
[0053] Optionally, the second receiving unit is further used for:
[0054] receiving an operation instruction sent by the control terminal, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal;
[0055] The second sending unit is further used for sending the operation instruction to the tested terminal.
[0056] Optionally, because the coordinate position of the operation point included in the operation instruction is different from the actual coordinate position on the screen of the tested terminal, the second sending unit is specifically configured to:
[0057] Transforming the coordinate position of at least one operation point in the operation instruction to generate an updated operation instruction;
[0058] The updated operation instruction is sent to the tested terminal.
[0059] Optionally, the second receiving unit is further used for:
[0060] receiving an operation instruction sent by the control terminal, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal;
[0061] The apparatus further includes: a control unit, configured to control a pointing device to perform a multi-touch operation on the tested terminal according to the operation instruction.
[0062] In a fifth aspect, an embodiment of the present invention provides a system for remotely testing cloud rendering performance, the system comprising:
[0063] The transfer terminal is used to receive a start-shooting instruction sent by the control terminal, wherein the start-shooting instruction is used to control the transfer terminal to start the camera module to shoot the screen of the tested terminal; obtain a real-time video stream, wherein the real-time video stream is a screen display image of the tested terminal shot by the transfer terminal; and send the real-time video stream to the control terminal;
[0064] A control terminal is used to obtain a real-time video stream sent by a transfer terminal, wherein the real-time video stream is a screen display image of a tested terminal taken by the transfer terminal; receive an operation instruction for performing a touch operation on any frame image in the real-time video stream, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal; and send the operation instruction so that the tested terminal is controlled according to the operation instruction.
[0065] Optionally, the system further includes:
[0066] Shielding device, used to place the terminal under test and shield network interference.
[0067] Optionally, the system further includes:
[0068] The storage device is used to place the transfer terminal and the terminal device under test.
[0069] Optionally, the system further includes:
[0070] A network device is used to provide a network for the terminal under test, and the network device is placed inside the shielding device.
[0071] Optionally, the system further includes:
[0072] The clicking device is used to receive the operation instruction sent by the transfer terminal and perform multi-touch operation on the tested terminal.
[0073] In an embodiment of the present invention, a real-time video stream sent by a transfer terminal is obtained, wherein the real-time video stream is a screen display image of the tested terminal taken by the transfer terminal; an operation instruction for performing a touch operation on any frame image in the real-time video stream is received, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal; and the operation instruction is sent so that the tested terminal is controlled according to the operation instruction. Through the above method, since the transfer terminal obtains the real-time video stream of the tested terminal instead of the tested terminal, the cloud rendering performance of the tested terminal can be accurately tested without affecting the performance of the tested terminal. BRIEF DESCRIPTION OF THE DRAWINGS
[0074] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
[0075] Figure 1 is a system schematic diagram of a remote test of cloud rendering performance in an embodiment of the present invention;
[0076] Figure 2 is a schematic diagram of another system for remote testing of cloud rendering performance in an embodiment of the present invention;
[0077] Figure 3 is a system schematic diagram of another cloud rendering performance remote test in an embodiment of the present invention;
[0078] Figure 4 is a schematic diagram of a system for remote testing of cloud rendering performance in another embodiment of the present invention;
[0079] Figure 5 is a schematic diagram of a shielding device in an embodiment of the present invention;
[0080] Figure 6 is a schematic diagram of a storage device in an embodiment of the present invention;
[0081] Figure 7 This is a flow chart of a method for remote testing of cloud rendering performance in an embodiment of the present invention;
[0082] Figure 8 is a schematic diagram on a screen of a control terminal in an embodiment of the present invention;
[0083] Fig. 9 This is a flow chart of another method for remote testing of cloud rendering performance in an embodiment of the present invention;
[0084] Fig.10 This is a flow chart of another method for remote testing of cloud rendering performance in an embodiment of the present invention;
[0085] Fig.11 This is a flow chart of a method for remote testing of cloud rendering performance in another embodiment of the present invention;
[0086] Fig.12 This is a flow chart of a method for remote testing of cloud rendering performance in an embodiment of the present invention;
[0087] Fig.13 is an interactive schematic diagram in an embodiment of the present invention;
[0088] Fig.14 is a schematic diagram of a device for remote testing of cloud rendering performance in an embodiment of the present invention;
[0089] Fig.15 is a schematic diagram of another device for remote testing of cloud rendering performance in an embodiment of the present invention;
[0090] Fig.16 It is a schematic diagram of the hardware structure of a host computer in an embodiment of the present invention. DETAILED DESCRIPTION
[0091] The present application is described below based on embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, some specific details are described in detail. It is possible for those skilled in the art to fully understand the present application without the description of these details. In order to avoid confusing the essence of the present application, known methods, processes, flows, components and circuits are not described in detail.
[0092] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.
[0093] Unless the context clearly requires otherwise, the words "include", "comprising" and similar words throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is "including but not limited to".
[0094] In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.
[0095] In the embodiment of the present invention, the application scenarios of cloud rendering are very wide, including film and television and animation production, game development, architectural design and interior design, industrial design and product visualization, as well as virtual reality and augmented reality. As an efficient and convenient rendering method, cloud rendering is gradually changing the traditional design and production mode. With the continuous advancement and improvement of cloud rendering technology, the application prospects of cloud rendering will be broader. Taking the application of cloud rendering in game production as an example, when a user uses a cloud game, the user operates the local device to issue an operation instruction, and the cloud server responds to the operation instruction and displays the result on the screen of the local device. The cloud rendering performance experience is very important to the user. Therefore, before using the cloud rendering, the cloud rendering performance needs to be tested. The flow degree, jamming rate and delay time in the cloud rendering performance are all affected by the network. Network fluctuations, signal interference, etc. will cause performance changes. In order to reduce the impact of network interference on the local device, the local device is placed in a network shielding box for testing. However, when the local device is placed in the network shielding box, the local device cannot be directly operated.
[0096] In the prior art, in order to test the cloud rendering performance of the local device in the network shielding box, a web cloud real machine solution is proposed. When the local device is controlled by the web cloud real machine, the functions of obtaining the encoding and transmission of the local device screen image need to be run on the local device, which has additional loss on the performance of the local device and affects the cloud rendering performance; and the web cloud real machine only supports single-point touch local devices. The single touch can only perform a touch operation on the screen of the local device at a time, and multi-point touch cannot be achieved. The control performance is poor, which will also affect the cloud rendering performance. Since the hardware of the local device (for example, the processor model, memory size, etc. of the terminal device) and the network environment are related, different local devices can test different cloud rendering performance, and it is also necessary to test the cloud rendering performance for different local devices separately. Therefore, how to test the cloud rendering performance more accurately is a problem that needs to be solved at present.
[0097] In order to solve the above problems, in the embodiment of the present invention, a system for remote testing of cloud rendering performance is proposed, specifically: Figure 1 As shown, the system comprises:
[0098] Transfer terminal 101 and control terminal 102;
[0099] Among them, the transfer terminal 101 is used to receive a start-shooting instruction sent by the control terminal, wherein the start-shooting instruction is used to control the transfer terminal to turn on the camera module to shoot the screen of the tested terminal; obtain a real-time video stream, wherein the real-time video stream is a screen display image of the tested terminal photographed by the transfer terminal; send the real-time video stream to the control terminal; the control terminal 102 is used to obtain the real-time video stream sent by the transfer terminal, wherein the real-time video stream is a screen display image of the tested terminal photographed by the transfer terminal; receive an operation instruction for performing a touch operation on any frame image in the real-time video stream, wherein the operation instruction is used to be transmitted through to the tested terminal to control the tested terminal; send the operation instruction so that the tested terminal is controlled according to the operation instruction.
[0100] In a possible implementation, the transfer terminal is a device with shooting and data transmission functions, specifically a camera device, mobile phone, tablet, etc. with data transmission function, which can not only receive the start shooting instruction sent by the control terminal, but also shoot the real-time video stream and send the real-time video stream to the real-time control terminal. In the embodiment of the present invention, the transfer terminal can also be provided with a control function, and after receiving the operation instruction sent by the control terminal, the click device is controlled according to the operation instruction.
[0101] In a possible implementation, the control terminal is a single-chip microcomputer development board with an operating system, which can be a mobile phone, a tablet, etc., which can receive an operation instruction for performing a touch operation on any frame image in the real-time video stream, and send the operation instruction so that the tested terminal is controlled according to the operation instruction. For example, Raspberry Pi, specifically, the Raspberry Pi is a single-board microcomputer based on Linux. The Raspberry Pi has multiple advantages. First, the Raspberry Pi is small and portable, with a size of only a credit card, which is easy to carry and install on the top of the upper box body. The power consumption of the Raspberry Pi is low, and a 5V USB power supply can be used to power it. Thirdly, the operating system of the Raspberry Pi is based on Linux, and users can freely customize and modify it according to their own needs. It is suitable for the remote performance testing process of any terminal under test.
[0102] In a possible implementation, the terminal under test may be a device such as a mobile phone or a tablet that can load cloud games. The screen of the terminal under test may also be referred to as a display screen, a touch screen, etc. of the terminal under test, which is not limited in the embodiment of the present invention.
[0103] In the embodiment of the present invention, Figure 1 Based on this, a system for remote testing of cloud rendering performance is proposed. Figure 2 As shown, the system also includes:
[0104] Shielding device 103 and storage device 104;
[0105] Among them, the shielding device 103 is used to place the storage device, and an openable and closable component is provided on the shielding device. When the openable and closable component is in an open state, the storage object is placed in the box body, and the openable and closable component is closed to make the shielding device in a closed state; the storage device is used to place the transfer terminal and the terminal device under test, and the transfer terminal is placed above the storage device. A shooting hole is provided above the storage device, and the transfer terminal shoots the screen display image of the terminal under test inside the storage device through the shooting hole.
[0106] In an embodiment of the present invention, the shielding device is made of a material that can shield electromagnetic interference and is used to shield network interference. Since the shielding device shields the network, a communication hole can be set on the shielding device and the storage device. The transfer terminal and the tested terminal can be connected to an external control terminal through a data line through the communication hole for data transmission.
[0107] In a possible implementation, the storage device is made of light-shielding material.
[0108] In the embodiment of the present invention, Figure 2 Based on this, a system for remote testing of cloud rendering performance is proposed. Figure 3 As shown, the system also includes:
[0109] The network device 105 is used to provide a network for the tested terminal and the transfer terminal, and the network device is placed inside the shielding device.
[0110] In the embodiment of the present invention, Figure 2 Based on this, a system for remote testing of cloud rendering performance is proposed. Figure 4 As shown, the system also includes:
[0111] The clicking device 106 is used to receive the operation instruction sent by the transfer terminal and perform multi-touch operation on the terminal under test.
[0112] Specifically, the pointing device may be a multi-tentacle mechanical arm, or a multi-head pointing device or other device that can simultaneously point to the screen of the terminal under test.
[0113] In a possible implementation, the shielding device may further include other components, such as Figure 5As shown, the shielding device includes at least one fan component, and the fan component can be arranged on one side or multiple sides of the shielding device to dissipate heat for the transfer terminal and the tested terminal inside the shielding device.
[0114] In a possible implementation, the storage device may further include other components, such as Figure 6 As shown, the storage device includes a fixing component for fixing the tested terminal, the fixing component makes the screen of the tested terminal face the shooting hole, and the fixing component can also be provided with an adjustable telescopic structure, and the adjustable telescopic structure can fix the tested terminals of different sizes.
[0115] In the embodiment of the present invention, a height adjustment component is further provided inside the storage device for adjusting the distance between the tested terminal and the shooting hole.
[0116] In a possible implementation, one or more storage devices may be provided in the shielding device.
[0117] In the embodiment of the present invention, since the main components of the system for remote testing of cloud rendering performance are a transfer terminal and a control terminal, the processing flows of the transfer terminal and the control terminal are described in detail below.
[0118] In an embodiment of the present invention, a method for remotely testing cloud rendering performance is provided, specifically: Figure 7 As shown, the specific steps include:
[0119] Step S701: Acquire a real-time video stream sent by a transfer terminal, wherein the real-time video stream is a screen display image of the tested terminal captured by the transfer terminal.
[0120] Specifically, the control terminal obtains the real-time video stream sent by the transfer terminal. The real-time video stream is generated by the transfer terminal using a media encoder (Media Codec) to encode in accordance with the digital video compression format H264. The digital video compression format is determined based on actual conditions.
[0121] Step S702: receiving an operation instruction for performing a touch operation on any frame image in the real-time video stream, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal.
[0122] Specifically, the control terminal decodes and renders the real-time video stream, obtains each frame of the real-time video stream, and then displays each frame of the image on the screen of the control terminal. Specifically, the schematic diagram on the screen of the control terminal is as follows: Figure 8 As shown, Figure 8In the embodiment, the screen of the terminal under test is displayed on the screen of the control terminal, and the screen of the terminal under test is framed by a rectangular selection box. Zoom adjustment and focus adjustment are also set on the screen of the control terminal to adjust the size and resolution of the rectangular selection box.
[0123] In an embodiment of the present invention, a user performs touch interaction with the image displayed on the screen of the control terminal, and the touch interaction can be called on Touch Event. When the user touches or slides on the screen of the control terminal, that is, when the user touches or slides on the image displayed on the screen of the control terminal, a series of Motion Events, that is, the operation instructions, are generated.
[0124] Step S703: Send the operation instruction so that the tested terminal is controlled according to the operation instruction.
[0125] Specifically, when the control terminal sends the operation instruction, there are two situations, as follows:
[0126] Situation 1:
[0127] The control terminal sends the operation instruction to the transfer terminal, and the transfer terminal forwards the operation instruction to the tested terminal.
[0128] In a possible implementation manner, the operation instruction may be parsed at the control terminal or the transfer terminal, and the parsed operation instruction is sent to the terminal under test.
[0129] In an embodiment of the present invention, since the coordinate position of the operation point contained in the operation instruction is different from the actual coordinate position on the screen of the tested terminal, it is necessary to transform the coordinate position of at least one operation point in the operation instruction to generate an updated operation instruction, and then send the updated operation instruction.
[0130] In a possible implementation, the coordinate transformation is to geometrically transform the coordinate position of at least one operation point in the operation instruction in combination with the resolution, angle and frame selection position of the current frame image in the real-time video stream. The frame selection position is to convert the coordinate position of at least one operation point in the operation instruction into a geometrical value. Figure 8 What is displayed on the screen of the control terminal is the screen of the tested terminal framed by a rectangular selection frame, and the size of the rectangular selection frame can be adjusted, so the coordinate change is variable.
[0131] Situation 2:
[0132] The control terminal sends the operation instruction to the tested terminal.
[0133] In a possible implementation, the operation instruction is parsed by the control terminal, and the parsed operation instruction is directly sent to the terminal under test.
[0134] In an embodiment of the present invention, since the coordinate position of the operation point contained in the operation instruction is different from the actual coordinate position on the screen of the tested terminal, it is necessary to transform the coordinate position of at least one operation point in the operation instruction to generate an updated operation instruction, and then send the updated operation instruction.
[0135] In an embodiment of the present invention, another method for remote testing of cloud rendering performance is provided. Before step S701, other steps are also included. Specifically, Fig. 9 As shown, the specific steps include:
[0136] Step S704: Sending a start-shooting instruction to the transfer terminal, wherein the start-shooting instruction is used to control the transfer terminal to turn on a camera module to shoot the screen of the tested terminal.
[0137] Specifically, the control terminal sends a start shooting instruction to the transfer terminal.
[0138] In a possible implementation, before or after the control terminal sends the start-shooting instruction, it is also necessary to send a standby-test instruction to the tested terminal to put the tested terminal into a standby-test state.
[0139] In an embodiment of the present invention, a method for remotely testing cloud rendering performance is provided, specifically: Fig.10 As shown, the specific steps include:
[0140] Step S1001: receiving a start-shooting instruction sent by a control terminal, wherein the start-shooting instruction is used to control the transfer terminal to start a camera module to shoot a screen of the terminal under test.
[0141] Specifically, the transfer terminal receives a start shooting instruction sent by the control terminal.
[0142] Step S1002: Acquire a real-time video stream, wherein the real-time video stream is a screen display image of the tested terminal captured by the transfer terminal.
[0143] Specifically, the transfer terminal captures the screen display image of the tested terminal through the camera module and generates a real-time video stream.
[0144] In the embodiment of the present invention, a camera application is provided in the transfer terminal, and parameters such as preview frame rate, focal length and angle can be set in the camera application.
[0145] Step S1003: Send the real-time video stream to the control terminal.
[0146] Specifically, the transfer terminal sends the real-time video stream to the control terminal.
[0147] In the embodiment of the present invention, the transfer terminal and the control terminal may be connected via a wireless network or a wired data line.
[0148] In an embodiment of the present invention, another method for remote testing of cloud rendering performance is provided. After step S1003, other steps are further included. Specifically, Fig.11 As shown, the specific steps include:
[0149] Step S1004: receiving an operation instruction sent by the control terminal, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal.
[0150] In an embodiment of the present invention, the transfer terminal receives an operation instruction sent by the control terminal. Since the coordinate position of the operation point contained in the operation instruction is different from the actual coordinate position on the screen of the tested terminal, it is necessary to transform the coordinate position of at least one operation point in the operation instruction to generate an updated operation instruction.
[0151] Step S1005: Send the operation instruction to the tested terminal.
[0152] Specifically, the transfer terminal may also send the updated operation instruction to the tested terminal.
[0153] In a possible implementation, if the operation instruction has completed the coordinate conversion at the control terminal, the transfer terminal can directly send the received operation instruction to the terminal under test.
[0154] In a possible implementation, the transfer terminal sends the operation instruction to an input manager (Input Manager) of the terminal under test, and the operation instruction is injected through the input manager.
[0155] In an embodiment of the present invention, another method for remote testing of cloud rendering performance is provided. After step S1004, other steps are further included. Specifically, Fig.12 As shown, the specific steps include:
[0156] Step S1006: Control the clicking device to perform a multi-touch operation on the tested terminal according to the operation instruction.
[0157] Specifically, the transfer terminal can perform a multi-touch operation on the tested terminal by controlling a clicking device, and the clicking device can be a multi-tentacle mechanical arm or a multi-head clicking device.
[0158] In the embodiment of the present invention, the control terminal may be a mobile phone, which may also be called a server, the transfer terminal and the tested terminal may also be mobile phones, the transfer terminal and the tested terminal may be called a server, and the interaction diagram of the control terminal, the transfer terminal and the tested terminal is as shown in FIG. Fig.13 As shown, the transfer terminal sends a real-time video stream to the control terminal, and the screen of the control terminal renders the picture in the real-time video stream in real time. The control terminal sends a real-time control stream to the transfer terminal, wherein the real-time control stream includes at least one operation instruction, and the real-time control stream is transmitted to the tested terminal through control stream conversion.
[0159] In an embodiment of the present invention, a device for remotely testing cloud rendering performance is provided. Fig.14 As shown, specifically including:
[0160] A first acquiring unit 1401, a first receiving unit 1402 and a first sending unit 1403;
[0161] Among them, the first acquisition unit 1401 is used to acquire the real-time video stream sent by the transit terminal, wherein the real-time video stream is a screen display image of the tested terminal taken by the transit terminal; the first receiving unit 1402 is used to receive an operation instruction for performing a touch operation on any frame image in the real-time video stream, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal; the first sending unit 1403 is used to send the operation instruction so that the tested terminal is controlled according to the operation instruction.
[0162] Furthermore, the first sending unit is further used for:
[0163] Sending a start-shooting instruction to the transfer terminal, wherein the start-shooting instruction is used to control the transfer terminal to turn on a camera module to shoot a screen of the tested terminal.
[0164] Furthermore, the first acquiring unit is further used for:
[0165] The real-time video stream is decoded and rendered to obtain each frame image in the real-time video stream.
[0166] Furthermore, since the coordinate position of the operation point included in the operation instruction is different from the actual coordinate position on the screen of the tested terminal, the first sending unit is specifically configured to:
[0167] Transforming the coordinate position of at least one operation point in the operation instruction to generate an updated operation instruction;
[0168] The updated operation instruction is sent.
[0169] Furthermore, the first sending unit is specifically configured to:
[0170] The operation instruction is sent to the transfer terminal, and the operation instruction is forwarded to the tested terminal through the transfer terminal.
[0171] Furthermore, the first sending unit is further configured to:
[0172] Sending the operation instruction to the tested terminal.
[0173] In an embodiment of the present invention, another device for remotely testing cloud rendering performance is provided, such as Fig.15 As shown, specifically including:
[0174] A second receiving unit 1501, a second acquiring unit 1502 and a second sending unit 1503;
[0175] Among them, the second receiving unit 1501 is used to receive a start shooting instruction sent by the control terminal, wherein the start shooting instruction is used to control the transit terminal to turn on the camera module to shoot the screen of the tested terminal; the second acquiring unit 1502 is used to acquire a real-time video stream, wherein the real-time video stream is a screen display image of the tested terminal photographed by the transit terminal; the second sending unit 1503 is used to send the real-time video stream to the control terminal.
[0176] Furthermore, the second receiving unit is further used for:
[0177] receiving an operation instruction sent by the control terminal, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal;
[0178] The second sending unit is further used for sending the operation instruction to the tested terminal.
[0179] Further, since the coordinate position of the operation point included in the operation instruction is different from the actual coordinate position on the screen of the tested terminal, the second sending unit is specifically used to:
[0180] Transforming the coordinate position of at least one operation point in the operation instruction to generate an updated operation instruction;
[0181] The updated operation instruction is sent to the tested terminal.
[0182] Furthermore, the second receiving unit is further used to: receive an operation instruction sent by the control terminal, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal;
[0183] The apparatus further includes: a control unit, configured to control a pointing device to perform a multi-touch operation on the tested terminal according to the operation instruction.
[0184] Through the above-mentioned embodiments, it is possible to avoid the problem that when the tested terminal directly sends a real-time video stream to the control terminal, there is additional loss in the performance of the tested terminal because the functions of obtaining the encoding of the tested terminal screen image and transmitting the image need to be run on the tested terminal. In the embodiments of the present invention, the screen of the tested terminal is photographed by an additional transit terminal, and the functions of obtaining the encoding of the screen image and transmitting the image are all run on the transit terminal, which can ensure that the performance of the tested terminal is lossless; and it can solve the problem that multi-touch cannot be implemented on the tested terminal in the web cloud real machine solution, and multi-touch operation on the tested terminal can be implemented.
[0185] Fig.16 Schematic diagram of the hardware structure of the host computer in the embodiment of the present invention. Fig.16 As shown, it includes a general computer hardware structure, which at least includes a processor 1601 and a memory 1602. The processor 1601 and the memory 1602 are connected via a bus 1603. The memory 1602 is suitable for storing instructions or programs executable by the processor 1601. The processor 1601 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 1601 executes the instructions stored in the memory 1602, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 1603 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 1604 and the display device and the input / output (I / O) device 1605. The input / output (I / O) device 1605 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 1605 is connected to the system through an input / output (I / O) controller 1606.
[0186] Among them, the instructions stored in the memory 1602 are executed by at least one processor 1601 to achieve: obtaining a real-time video stream sent by the transit terminal, wherein the real-time video stream is a screen display image of the tested terminal taken by the transit terminal; receiving an operation instruction for performing a touch operation on any frame image in the real-time video stream, wherein the operation instruction is used to be transmitted through to the tested terminal to control the tested terminal; sending the operation instruction so that the tested terminal is controlled according to the operation instruction.
[0187] Specifically, the electronic device includes: one or more processors 1601 and a memory 1602, Fig.16 Take a processor 1601 as an example. The processor 1601 and the memory 1602 may be connected via a bus or other means. Fig.16 The example of the bus connection is taken as an example. The memory 1602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The processor 1601 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions and modules stored in the memory 1602, that is, the above-mentioned method for determining the remote test of cloud rendering performance is realized.
[0188] The memory 1602 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store a list of options, etc. In addition, the memory 1602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 1602 may optionally include a memory remotely arranged relative to the processor 1601, and these remote memories may be connected to an external device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0189] One or more modules are stored in the memory 1602, and when executed by one or more processors 1601, the method of remote testing of cloud rendering performance in any of the above method embodiments is executed.
[0190] As will be appreciated by those skilled in the art, various aspects of the embodiments of the present invention may be implemented as a system, method or computer program product. Therefore, various aspects of the embodiments of the present invention may take the form of a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.), or an implementation that combines software aspects with hardware aspects, which may generally be referred to herein as a "circuit," "module," or "system." In addition, various aspects of the embodiments of the present invention may take the form of a computer program product implemented in one or more computer-readable media having a computer-readable program code implemented thereon.
[0191] Any combination of one or more computer-readable media can be utilized. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, (but not limited to) electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or apparatuses, or any suitable combination of the foregoing. More specific examples (non-exhaustive enumeration) of computer-readable storage media will include the following: an electrical connection with one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of an embodiment of the present invention, a computer-readable storage medium can be any tangible medium that can contain or store a program used by an instruction execution system, device or apparatus or a program used in conjunction with an instruction execution system, device or apparatus.
[0192] A computer readable signal medium may include a propagated digital signal having a computer readable program code implemented therein, such as in baseband or as part of a carrier wave. Such propagated signals may take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any of the following computer readable media: not a computer readable storage medium, and may communicate, propagate, or transmit a program used by or in conjunction with an instruction execution system, device, or apparatus.
[0193] Program code embodied on a computer readable medium may be transmitted using any appropriate medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0194] The computer program code for performing operations for various aspects of the embodiments of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, etc., and conventional process programming languages such as "C" programming language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer as a stand-alone software package; partially on the user's computer and partially on a remote computer; or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., by using the Internet of an Internet service provider).
[0195] The flowchart legend and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present invention described above describe various aspects of the embodiment of the present invention.It will be understood that each block of the flowchart legend and / or block diagram and the combination of the blocks in the flowchart legend and / or block diagram can be implemented by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that the instructions (executed by the processor of the computer or other programmable data processing device) create a device for implementing the function / action specified in the flowchart and / or block diagram block or block.
[0196] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing device, or other apparatus to operate in a particular manner, so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / actions specified in the flowchart and / or block diagram blocks or blocks.
[0197] The computer program instructions may also be loaded onto a computer, other programmable data processing device or other apparatus so that a series of operable steps are performed on the computer, other programmable device or other apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the functions / actions specified in the flowchart and / or block diagram block or blocks.
[0198] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for remote testing of cloud rendering performance, characterized in that: The method comprises: Acquire a real-time video stream sent by a transfer terminal, wherein the real-time video stream is a screen display image of the tested terminal captured by the transfer terminal; receiving an operation instruction for performing a touch operation on any frame image in the real-time video stream, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal; The operation instruction is sent so that the tested terminal is controlled according to the operation instruction.
2. The method according to claim 1, characterized in that The method further comprises: Sending a start-shooting instruction to the transfer terminal, wherein the start-shooting instruction is used to control the transfer terminal to turn on a camera module to shoot a screen of the tested terminal.
3. The method according to claim 1, characterized in that The method further comprises: The real-time video stream is decoded and rendered to obtain each frame image in the real-time video stream.
4. The method according to claim 1, characterized in that Since the coordinate position of the operation point included in the operation instruction is different from the actual coordinate position on the screen of the tested terminal, sending the operation instruction specifically includes: Transforming the coordinate position of at least one operation point in the operation instruction to generate an updated operation instruction; The updated operation instruction is sent.
5. The method according to claim 1, characterized in that The sending of the operation instruction specifically includes: The operation instruction is sent to the transfer terminal, and the operation instruction is forwarded to the tested terminal through the transfer terminal.
6. The method according to claim 1, characterized in that The sending of the operation instruction specifically includes: Sending the operation instruction to the tested terminal.
7. A method for remote testing of cloud rendering performance, characterized in that: The method comprises: Receiving a start-shooting instruction sent by the control terminal, wherein the start-shooting instruction is used to control the transfer terminal to start a camera module to shoot the screen of the tested terminal; Acquire a real-time video stream, wherein the real-time video stream is a screen display image of the tested terminal captured by the transfer terminal; The real-time video stream is sent to the control terminal.
8. The method according to claim 7, characterized in that The method further comprises: receiving an operation instruction sent by the control terminal, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal; Sending the operation instruction to the tested terminal.
9. The method according to claim 8, characterized in that Since the coordinate position of the operation point included in the operation instruction is different from the actual coordinate position on the screen of the tested terminal, sending the operation instruction to the tested terminal specifically includes: Transforming the coordinate position of at least one operation point in the operation instruction to generate an updated operation instruction; The updated operation instruction is sent to the tested terminal.
10. The method according to claim 7, characterized in that The method further includes: receiving an operation instruction sent by the control terminal, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal; The clicking device is controlled to perform a multi-touch operation on the tested terminal according to the operation instruction.
11. A device for remote testing of cloud rendering performance, characterized in that: The device comprises: A first acquisition unit is used to acquire a real-time video stream sent by a transfer terminal, wherein the real-time video stream is a screen display image of the tested terminal taken by the transfer terminal; A first receiving unit, configured to receive an operation instruction for performing a touch operation on any frame image in the real-time video stream, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal; The first sending unit is used to send the operation instruction so that the tested terminal is controlled according to the operation instruction.
12. A device for remote testing of cloud rendering performance, characterized in that: The device comprises: A second receiving unit is used to receive a start-shooting instruction sent by the control terminal, wherein the start-shooting instruction is used to control the transfer terminal to start the camera module to shoot the screen of the tested terminal; A second acquisition unit is used to acquire a real-time video stream, wherein the real-time video stream is a screen display image of the tested terminal taken by the transfer terminal; The second sending unit is used to send the real-time video stream to the control terminal.
13. A system for remote testing of cloud rendering performance, characterized in that: The system comprises: The transfer terminal is used to receive a start-shooting instruction sent by the control terminal, wherein the start-shooting instruction is used to control the transfer terminal to start the camera module to shoot the screen of the tested terminal; obtain a real-time video stream, wherein the real-time video stream is a screen display image of the tested terminal shot by the transfer terminal; and send the real-time video stream to the control terminal; A control terminal is used to obtain a real-time video stream sent by a transfer terminal, wherein the real-time video stream is a screen display image of a tested terminal taken by the transfer terminal; receive an operation instruction for performing a touch operation on any frame image in the real-time video stream, wherein the operation instruction is used to be transparently transmitted to the tested terminal to control the tested terminal; and send the operation instruction so that the tested terminal is controlled according to the operation instruction.
