Data processing method and related device

Abstract

The invention provides a data processing method and a related device. The method can be implemented as follows: electronic equipment runs a game application; the electronic equipment obtains first state information of the game application in the first scene, wherein the first state information comprises the type of the first scene; rendering and drawing a first picture frame of the game application in the first scene; synthesizing, sending and displaying the first picture frame; processing information of the first picture frame is acquired, and the processing information of the first picture frame comprises a processing record of the first picture frame in a rendering and drawing stage and a processing record of the first picture frame in a synthesis and display stage; if it is determined that frame loss occurs in the first picture frame based on the processing information of the first picture frame, first maintenance and measurement information is sent to a server, and the first maintenance and measurement information comprises the processing information of the first picture frame and first state information of the game application in the first scene. In this way, the maintenance and measurement personnel can obtain the maintenance and measurement information during frame loss of the game application through the server, and then the frame loss problem of the game application is solved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a data processing method and related devices. Background Art

[0002] With the continuous upgrading of electronic devices, the applications in electronic devices are also developing at an alarming speed. However, when the application is running, there is a problem of frame loss in the application, which easily leads to a poor user experience. The maintenance personnel need to find the cause of the frame loss of the application in order to solve the problem of frame loss of the application. Therefore, how to collect the running data of the application when the frame is lost has become an urgent problem to be solved in this field. Summary of the invention

[0003] The present application provides a data processing method and related devices, which realize collecting first dimension measurement information of a first picture frame when a first picture frame of a game application in a first scene is lost.

[0004] In a first aspect, the present application provides a data processing method, which is applied to an electronic device, which runs a game application, the method comprising: obtaining first status information of the game application in a first scenario, the first status information including the type of the first scenario; rendering and drawing a first screen frame of the game application in the first scenario; synthesizing and displaying the first screen frame; obtaining processing information of the first screen frame, wherein the processing information of the first screen frame includes a processing record of the first screen frame in a rendering and drawing phase and a processing record in a synthesizing and displaying phase; if it is determined based on the processing information of the first screen frame that frame loss occurs in the first screen frame, sending first dimensional measurement information to a server, wherein the first dimensional measurement information includes the processing information of the first screen frame and the first status information of the game application in the first scenario.

[0005] Through a data processing method provided in the present application, first dimensional measurement information of a first screen frame of a game application in a first scenario can be collected, and the first dimensional measurement information includes processing information of the first screen frame and first status information of the game application in the first scenario, wherein the processing information of the first screen frame includes processing records of the first screen frame in a rendering and drawing phase and processing records in a synthesis and display phase. In this way, when frame loss occurs in the first screen frame, the measurement personnel can analyze the cause of frame loss of the first screen frame based on the first dimensional measurement information of the first screen frame, thereby solving the frame loss problem of the game application.

[0006] In a possible implementation, the electronic device also runs a graphics interface and a buffer queue management module; the rendering and drawing of the first screen frame of the game application in the first scene specifically includes: after receiving the application synchronization signal, the game application sends a buffer dequeue request to the buffer queue management module; in response to the buffer dequeue request, the buffer queue management module sends the address of the buffer to the game application; the game application renders and draws the first screen frame through the graphics interface, and adds the rendered first screen frame to the buffer queue managed by the buffer queue management module through the buffer. In this way, the electronic device can render and draw the first screen frame, so that the first screen frame can be better perceived by the human eye, giving the user a better visual experience.

[0007] In a possible implementation, the electronic device also runs a layer synthesizer SF; the synthesis and display of the first screen frame specifically includes: after receiving the layer synthesis synchronization signal, the SF sends a buffer acquisition request to the buffer queue management module; in response to the buffer acquisition request, the buffer queue management module sends the address corresponding to the buffer in the buffer queue storing the first screen frame after rendering to the SF; the SF takes out the first screen frame after rendering from the buffer storing the first screen frame after rendering, and synthesizes and displays the first screen frame after rendering; after synthesizing and displaying the first screen frame after rendering, the SF sends a buffer release request to the buffer queue management module; in response to the buffer release request, the buffer queue management module releases the buffer storing the first screen frame after rendering. In this way, the electronic device can synthesize and display the first screen frame, so that the user can see the first screen frame, and at the same time, releasing the buffer storing the first screen frame after rendering can save the memory space of the electronic device.

[0008] In a possible implementation, the period of the application synchronization signal is the same as the period of the layer synthesis synchronization signal.

[0009] In a possible implementation, obtaining the processing information of the first screen frame specifically includes: if the first scene is a designated game scene, obtaining the processing information of the first screen frame of the game application in the first scene.

[0010] In a possible implementation, the game application scene includes a game lobby scene and a game battle scene. The designated game scene is a game battle scene.

[0011] In a possible implementation, the electronic device further runs a maintenance and testing module; obtaining the first state information of the game application in the first scene specifically includes: the maintenance and testing module obtains the first state information sent by the game application; if the first scene is a designated game scene, obtaining the processing information of the first screen frame of the game application in the first scene specifically includes: if the maintenance and testing module determines that the first scene is a designated game scene based on the first state information, the maintenance and testing module obtains the processing information of the first screen frame of the game application in the first scene from the buffer queue management module, the graphic interface and the SF. In this way, the electronic device can obtain the processing information of the first screen frame when the first scene is a designated game scene.

[0012] In a possible implementation, the processing record of the first screen frame in the rendering and drawing stage includes one or more of the following: a record of buffer dequeuing, a record of rendering and drawing, and a record of buffer enqueuing; the processing record of the first screen frame in the synthesis and display stage includes one or more of the following: a record of obtaining the buffer, a record of synthesis and display, and a record of releasing the buffer; if the maintenance and measurement module determines that the first scene is a specified game scene based on the first status information, the maintenance and measurement module obtains the processing information of the first screen frame of the game application in the first scene from the buffer queue management module, the graphics interface and the SF, specifically including: if the maintenance and measurement module determines that the first scene is a specified game scene based on the first status information, the maintenance and measurement module obtains the processing information of the first screen frame of the game application in the first scene from the buffer queue management module, the graphics interface and the SF, specifically including: When the first status information determines that the first scene is a designated game scene, the maintenance and measurement module sends a first dot request to the buffer queue management module, sends a second dot request to the graphics interface, and sends a third dot request to the SF; in response to the first dot request, the buffer queue management module sends a buffer dequeue record, and / or a buffer enqueue record, and / or a buffer acquisition record, and / or a buffer release record to the maintenance and measurement module; in response to the second dot request, the graphics interface sends a rendering and drawing record to the maintenance and measurement module; in response to the third dot request, the SF sends a synthesis and display record to the maintenance and measurement module. In this way, the maintenance and testing module in the electronic device can obtain the records of buffer dequeue, rendering and drawing, buffer enqueue, buffer acquisition, synthesis and display, and buffer release of the first screen frame, thereby helping maintenance and testing personnel to solve the frame loss problem of game applications based on the records of buffer dequeue, rendering and drawing, buffer enqueue, buffer acquisition, synthesis and display, and buffer release of the first screen frame when frame loss occurs.

[0013] In a possible implementation, the record of the buffer dequeue includes one or more of the following: the time when the game application receives the application synchronization signal, the time when the game application sends the buffer dequeue request to the buffer queue management module, the time when the buffer queue management module sends the buffer address to the game application, and whether the buffer queue management module successfully sends the buffer address to the game application. In this way, the maintenance personnel can analyze whether the frame loss occurs when the first picture frame is dequeued based on the record of the buffer dequeue.

[0014] In a possible implementation, the rendering record includes one or more of the following: the start time of the rendering of the first screen frame, the end time of the rendering of the first screen frame, and whether the first screen frame is successfully rendered. In this way, the maintenance personnel can analyze whether the frame loss occurs when the first screen frame is rendered based on the rendering record.

[0015] In a possible implementation, the record of buffer entry includes one or more of the following: the start time of the buffer entry into the buffer queue, the end time of the buffer entry into the buffer queue, and the layer information of the first picture frame in the buffer. In this way, the maintenance personnel can analyze whether the first picture frame is lost when the buffer storing the first picture frame is entered into the queue based on the record of buffer entry.

[0016] In a possible implementation, the record of obtaining the buffer includes one or more of the following: the moment when the SF receives the layer synthesis synchronization signal, the moment when the SF sends the buffer acquisition request to the buffer queue management module, the moment when the buffer queue management module sends the address corresponding to the buffer in the buffer queue storing the first screen frame after rendering and drawing to the SF, and whether the buffer queue management module successfully sends the address corresponding to the buffer in the buffer queue storing the first screen frame after rendering and drawing to the SF. In this way, the maintenance and testing personnel can analyze whether the frame loss of the first screen frame occurs when the SF obtains the buffer storing the first screen frame after rendering and drawing based on the record of obtaining the buffer.

[0017] In a possible implementation, the record of synthesis and display includes one or more of the following: the start time of synthesis and display of the first picture frame, the end time of synthesis and display of the first picture frame, and whether the first picture frame is successfully synthesized and displayed. In this way, the maintenance personnel can analyze whether the frame loss occurs when the first picture frame is synthesized and displayed based on the record of synthesis and display.

[0018] In a possible implementation, the record of releasing the buffer includes one or more of the following: the time when the SF sends the buffer release request to the buffer queue management module, the time when the buffer queue management module releases the buffer storing the first screen frame after rendering and drawing, and whether the buffer queue management module successfully releases the buffer storing the first screen frame after rendering and drawing. In this way, the maintenance personnel can analyze whether the frame loss occurs when the buffer queue management module releases the buffer storing the first screen frame after rendering and drawing based on the record of releasing the buffer.

[0019] In a possible implementation, the first status information also includes one or more of the following: game stage, status of players on the same screen, number of players on the same screen, and display of game props.

[0020] In a possible implementation, the method also includes: obtaining processing information of a second picture frame, wherein the second picture frame is a picture frame that is before and adjacent to the first picture frame, and the processing information of the second picture frame includes processing records of the second picture frame in a rendering and drawing phase and processing records in a synthesis and display phase; and determining that frame loss has occurred in the first picture frame based on the processing information of the first picture frame specifically includes: determining that frame loss has occurred in the first picture frame based on the processing information of the first picture frame and the processing information of the second picture frame.

[0021] In a possible implementation, the processing information of the second screen frame includes the time when the buffer queue management module releases the buffer storing the second screen frame after rendering and drawing; the determination of the first screen frame having frame loss based on the processing information of the first screen frame and the processing information of the second screen frame specifically includes: if the difference between the time when the buffer queue management module releases the buffer storing the second screen frame after rendering and drawing and the time when the buffer queue management module releases the buffer storing the first screen frame after rendering and drawing is greater than the period of the synchronization signal, it is determined that the first screen frame has frame loss. In this way, the electronic device can determine whether the first screen frame has frame loss based on the processing information of the first screen frame and the processing information of the second screen frame.

[0022] In a possible implementation, the method also includes: when the game application sends the buffer dequeue request to the buffer queue management module, the dimension measurement module applies for a first dimension measurement buffer; after the dimension measurement module obtains the processing information of the first screen frame, the processing information of the first screen frame is stored in the first dimension measurement buffer.

[0023] In a possible implementation, the method further includes: after determining that frame loss occurs in the first picture frame, generating and saving first dimensional measurement information; after saving the first dimensional measurement information, releasing the first dimensional measurement buffer.

[0024] In a second aspect, the present application provides an electronic device comprising one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer instructions, so that when the one or more processors execute the computer instructions, the electronic device executes the data processing method in any possible implementation of the above-mentioned first aspect.

[0025] In a third aspect, the present application provides a chip, which is applied to an electronic device, and the chip includes a processing circuit and an interface circuit, the interface circuit is used to receive instructions and transmit them to the processing circuit, and the processing circuit is used to run instructions to execute the data processing method in any possible implementation of the first aspect above.

[0026] In a fourth aspect, the present application provides a computer-readable storage medium, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute a data processing method in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of the hardware structure of a server provided in an embodiment of the present application;

[0029] Figure 3A-3B A schematic diagram of an interface of a game application provided by the present embodiment;

[0030] Figure 4 A schematic diagram of the architecture of a software and hardware system provided in an embodiment of the present application;

[0031] Figure 5 An interactive flow chart of processing information of recording picture frames provided in an embodiment of the present application;

[0032] Fig. 6A A schematic diagram of performing different operations on a picture frame based on a synchronization signal provided in an embodiment of the present application;

[0033] Figure 6B A schematic diagram of storing processing information of a picture frame in a measurement buffer provided by an embodiment of the present application;

[0034] Figure 7 A schematic diagram of frame loss detection based on a measurement buffer provided in an embodiment of the present application;

[0035] Figure 8A flowchart of a data processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0037] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0038] The following introduces the hardware structure of an electronic device provided in an embodiment of the present application.

[0039] Figure 1 A schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application is shown.

[0040] It should be understood that Figure 1 The electronic device 100 shown is only an example, and the electronic device 100 may have more Figure 1 More or fewer components may be shown, two or more components may be combined, or there may be a different configuration of components. Figure 1 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0041] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an acceleration sensor 180E, and a touch sensor 180K, etc.

[0042] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0043] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0044] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0045] The processor 110 may also be provided with a memory for storing instructions and data.

[0046] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0047] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0048] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0049] The modulation and demodulation processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor.

[0050] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0051] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0052] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0053] Display screen 194 is used to display images, videos, etc.

[0054] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor. ISP is used to process data fed back by camera 193. Camera 193 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. NPU is a neural network (NN) computing processor, which can quickly process input information by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, and can also continuously self-learn.

[0055] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121.

[0056] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0057] The audio module 170 is used to convert digital audio information into analog audio signal output, and also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals.

[0058] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0059] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0060] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0061] The earphone jack 170D is used to connect a wired earphone.

[0062] The pressure sensor 180A is used to sense pressure signals and convert the pressure signals into electrical signals.

[0063] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. The gyro sensor 180B can be used for anti-shake shooting. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0064] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0065] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0066] The buttons 190 include a power button, a volume button, etc. The motor 191 can generate a vibration prompt. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power change, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.

[0067] In the embodiment of the present application, the device type of the electronic device 100 can be a mobile phone, a tablet computer, a handheld computer, a desktop computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), as well as smart home devices such as smart large screens and smart speakers, wearable devices such as smart bracelets, smart watches, and smart glasses, extended reality (extended reality, XR) devices such as augmented reality (AR), virtual reality (VR), and mixed reality (MR), vehicle-mounted devices or smart city devices, and the like.

[0068] The following introduces the hardware structure of a server provided in an embodiment of the present application.

[0069] Figure 2 A schematic diagram of the hardware structure of a server 200 provided in an embodiment of the present application is shown.

[0070] like Figure 2 As shown, the server 200 may include a communication interface 201 and a processor 202. Optionally, the server 200 may also include a memory 203. The memory 203 may be disposed inside the server 200 or outside the server 200. The functions implemented by the server 200 in the embodiment may all be implemented by the processor 202. The processor 202 receives a data stream through the communication interface 201 and is used to implement the method of any embodiment. During the implementation process, each step of the processing flow may complete the method of any embodiment through an integrated logic circuit of hardware in the processor 202 or instructions in software form. The memory 203 is coupled to the processor 202.

[0071] The processor 202 involved in the embodiment of the present application can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware processor to be executed, or the hardware and software modules in the processor can be combined and executed.

[0072] The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, modules or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, modules or modules.

[0073] The processor 202 may operate in conjunction with the memory 203. The memory 203 may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or a volatile memory (volatile memory), such as a random-access memory (RAM). The memory 203 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0074] In the embodiment of the present application, the specific connection medium between the communication interface 201, the processor 202 and the memory 203 is not limited. For example, the memory 203, the processor 202 and the communication interface 201 can be connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0075] The following is a schematic diagram of an interface of a game application provided in an embodiment of the present application.

[0076] Figure 3A and Figure 3B A schematic diagram of an interface of a game application provided in an embodiment of the present application is shown.

[0077] For example, Figure 3A As shown, before entering the game battle scene, the electronic device 100 can display an interface before entering the game battle scene, for example, a game lobby interface 310. Before entering the game battle scene, the amount of CPU calculation in the electronic device 100 is small, so the CPU load is low, and the CPU can have more processing power to cope with various computing tasks in the game application, thereby reducing the delay of the CPU processing tasks and reducing the risk of frame loss.

[0078] For example, Figure 3B As shown, when entering the game battle scene, the electronic device 100 can display the game battle interface 320. In the game battle scene, since the CPU needs to process a large amount of real-time calculations and network data exchange, the CPU load will increase. If the CPU load is too high, the CPU will not be able to process computing tasks (e.g., graphics rendering) in time, resulting in frame loss.

[0079] Therefore, the embodiment of the present application provides a data processing method applied to an electronic device, which runs a game application; the electronic device obtains first state information of the game application in a first scene, the first state information includes the type of the first scene; renders and draws the first screen frame of the game application in the first scene; synthesizes and displays the first screen frame; obtains processing information of the first screen frame, wherein the processing information of the first screen frame includes the processing record of the first screen frame in the rendering and drawing stage and the processing record in the synthesis and display stage; if it is determined based on the processing information of the first screen frame that the first screen frame has frame loss, send first dimension measurement information to the server, wherein the first dimension measurement information includes the processing information of the first screen frame and the first state information of the game application in the first scene. In this way, the maintenance and measurement personnel can obtain the maintenance and measurement information when the game application loses frames through the server, thereby solving the frame loss problem of the game application.

[0080] The following is a schematic diagram of the architecture of a software and hardware system provided in an embodiment of the present application.

[0081] like Figure 4 As shown, the electronic device 100 may include an application layer, a native service layer (native), a hardware abstraction layer (HAL), a kernel layer (kernel) and a hardware layer.

[0082] The application layer may include gaming applications.

[0083] The local service layer may include a graphics interface (including a graphics programming interface (open graphics library, OPENGL), a graphics programming interface for embedded systems (open graphics library for embedded systems, OPENGL ES), etc.), a buffer queue management module, a measurement module and a surface flinger (SF). Among them, the buffer queue management module may apply for a buffer queue (BufferQueue) in the memory and manage the applied buffer queue. The measurement module may apply for a measurement buffer queue (HiviewBufferQueue) in the memory and manage the applied measurement buffer queue.

[0084] The hardware abstraction layer may include a hardware composer (HWC). The kernel layer may include a GPU driver, a communication driver, and a display driver. The hardware layer may include a GPU, a communication module, and a display screen.

[0085] In combination with the above-mentioned hardware and software architecture, a data processing method provided by an embodiment of the present application is described in detail below:

[0086] 1. The hardware synthesizer can periodically send an application synchronization signal (app_vsyncsignal) to the game application.

[0087] The application synchronization signal can be used to trigger the game application to render a frame and provide the rendered frame to SF. The process of the game application rendering a frame and providing the rendered frame to SF can refer to the subsequent steps 3 to 13.

[0088] 2. The hardware synthesizer can periodically send a layer synthesis synchronization signal (sf_vsync signal) to the layer synthesizer.

[0089] The layer synthesis synchronization signal can be used to trigger the layer synthesizer to synthesize and display a picture frame. The process of the layer synthesizer synthesizing and displaying a picture frame can refer to the subsequent steps 14 to 19.

[0090] The period of the application synchronization signal (app_vsyncsignal) is the same as the period of the layer synthesis synchronization signal (sf_vsyncsignal).

[0091] 3. After receiving the application synchronization signal, the game application can send a buffer dequeue request to the buffer queue management module.

[0092] The buffer dequeue request is used to request to obtain an idle buffer from the buffer queue management module.

[0093] 4. After receiving the buffer dequeue request sent by the game application, the buffer queue management module can allocate free buffer to the game application.

[0094] In response to the buffer dequeue request, the buffer queue management module sends the address of the idle buffer to the game application.

[0095] 5. When the game application is running, it can send status information to the maintenance module. The status information may include game scene (e.g., the scene of the current screen), game stage (e.g., parachuting stage), number of players on the same screen (e.g., number of players in the current screen), status of players on the same screen (e.g., players are running), and display of game props (e.g., game props have reached their positions).

[0096] Among them, the maintenance and measurement module can determine whether the current game scene is in a game battle scene based on the status information. If the current game scene is in a game battle scene, the maintenance and measurement module can record the processing information of the picture frame. If the current game scene is not in a game battle scene, the maintenance and measurement module does not record the processing information of the picture frame. Among them, the processing information of the picture frame may include the processing record of the picture frame in the rendering and drawing stage and the processing record in the synthesis and display stage. Among them, the processing record of the picture frame in the rendering and drawing stage may include one or more of the following: the record of buffer dequeuing, the record of rendering and drawing, and the record of buffer enqueuing. The processing record of the picture frame in the synthesis and display stage may include one or more of the following: the record of obtaining the buffer, the record of synthesis and display, and the record of releasing the buffer.

[0097] 6. If the current game scene is in a game battle scene, the maintenance and testing module can send a second dotting request to the graphics interface.

[0098] The second dot request is used to request the graphics interface to obtain the rendering record, which may include one or more of the following: the start time of the picture frame rendering, the end time of the picture frame rendering, and the result of the picture frame rendering (for example, whether the picture frame is successfully drawn).

[0099] In one possible implementation, the rendering and drawing record may include one or more of the following: the start time of the screen frame rendering, the end time of the screen frame rendering, the start time of the screen frame drawing, the end time of the screen frame drawing, the result of the screen frame rendering (for example, whether the screen frame is rendered successfully), the result of the screen frame drawing (for example, whether the screen frame is drawn successfully), etc.

[0100] 7. If the current game scene is in a game battle scene, the maintenance and testing module can send a first dotting request to the buffer queue management module.

[0101] The first dot request is used to request the buffer queue management module to obtain buffer dequeue records, and / or buffer enqueue records, and / or obtain buffer records, and / or release buffer records.

[0102] Among them, the record of buffer dequeue may include one or more of the following: the moment when the game application receives the application synchronization signal, the moment when the game application sends the buffer dequeue request to the buffer queue management module, the moment when the buffer queue management module sends the address of the buffer to the game application, and the result of the buffer queue management module sending the address of the buffer to the game application (for example, whether the buffer queue management module sends the address of the buffer to the game application successfully, whether it times out, whether it is an exception, etc.).

[0103] Among them, the record of buffer enqueuing may include one or more of the following: the start time of the buffer joining the buffer queue, the end time of the buffer joining the buffer queue, and the layer information of the picture frame in the buffer (for example, the layer on which the picture frame is located).

[0104] Among them, the record of obtaining the buffer may include one or more of the following: the moment when SF receives the layer synthesis synchronization signal, the moment when SF sends a buffer acquisition request to the buffer queue management module, the moment when the buffer queue management module sends the address corresponding to the buffer in the buffer queue that stores the rendered and drawn screen frame to SF, and the result of the buffer queue management module sending the address corresponding to the buffer in the buffer queue that stores the rendered and drawn screen frame to SF (for example, whether the buffer queue management module sends the address corresponding to the buffer in the buffer queue that stores the rendered and drawn screen frame to SF successfully, whether it times out, whether it is abnormal, etc.).

[0105] Among them, the record of releasing the buffer may include one or more of the following: the moment when the SF sends a buffer release request to the buffer queue management module, the moment when the buffer queue management module releases the buffer storing the rendered and drawn picture frames, and the result of the buffer queue management module releasing the buffer storing the rendered and drawn picture frames (for example, whether the buffer queue management module releases the buffer storing the rendered and drawn picture frames successfully, whether it times out, whether it is abnormal, etc.).

[0106] 8. If the current game scene is in a game battle scene, the maintenance module can send a third dot request to the layer synthesizer.

[0107] The third dot request is used to request the layer synthesizer to obtain a synthesis display record. The synthesis display record may include one or more of the following: the start time of the picture frame synthesis display, the end time of the picture frame synthesis display, and the result of the picture frame synthesis display (for example, whether the picture frame is successfully synthesized and displayed).

[0108] In one possible implementation, the record of synthesis and display may include one or more of the following: the start time of picture frame synthesis, the end time of picture frame synthesis, the start time of picture frame display, the end time of picture frame display, the result of picture frame synthesis (for example, whether the picture frame is synthesized successfully), the result of picture frame display (for example, whether the picture frame is displayed successfully), etc.

[0109] 9. After receiving the application synchronization signal, the game application can send rendering instructions to the graphics interface.

[0110] 10. After receiving the rendering instruction sent by the game application, the graphics interface can send the rendering instruction to the GPU driver.

[0111] The GPU driver is used to convert the rendering instruction into an instruction recognizable by the GPU.

[0112] 11. After converting the rendering instruction into an instruction recognizable by the GPU, the GPU driver can send the rendering instruction to the GPU.

[0113] The rendering instruction is used to instruct the GPU to render a picture frame, and store the rendered picture frame in the above buffer.

[0114] In one possible implementation, the CPU and GPU may work together to render a drawing frame.

[0115] 12. After sending the rendering instruction to the graphics interface, the game application can send a frame sending instruction to the graphics interface. The frame sending instruction carries the address of the buffer.

[0116] 13. After receiving the frame sending instruction sent by the game application, the graphics interface can add the buffer containing the picture frame to the buffer queue.

[0117] 14. After receiving the layer synthesis synchronization signal, the layer synthesizer can send a buffer acquisition request to the buffer queue management module.

[0118] The buffer acquisition request is used to request to obtain the picture frame from the buffer queue management module.

[0119] 15. After receiving the buffer acquisition request sent by the layer compositor, the buffer queue management module can provide the buffer storing the picture frame to the layer compositor (SF).

[0120] In response to the buffer acquisition request, the buffer queue management module sends the address corresponding to the buffer in the buffer queue storing the rendered picture frame to the SF.

[0121] The layer synthesizer may obtain the picture frames from the buffer storing the picture frames, and perform layer synthesis on the picture frames.

[0122] In one possible implementation, the layer synthesizer can also obtain the remaining layer information (for example, various components displayed and interacted on the electronic device 100, such as buttons, text boxes, drop-down menus, check boxes, etc.), and synthesize the picture frame with the remaining layer information.

[0123] 16. After the layer synthesizer performs layer synthesis on the picture frame, the layer synthesizer can send the picture frame after layer synthesis to the hardware synthesizer.

[0124] 17. After receiving the picture frame sent by the layer synthesizer, the hardware synthesizer can synthesize the picture frame after layer synthesis into a frame of interface including the picture frame, and send the frame of interface including the picture frame to the display driver.

[0125] 18. After receiving the interface including the picture frame, the display driver can drive the display screen to display the interface including the picture frame.

[0126] For example, the above Figure 3A The game lobby interface 310 shown, or the above Figure 3B The game battle interface 320 is shown.

[0127] 19. After compositing a layer of a picture frame, the layer synthesizer may send a buffer release request to the buffer queue management module, which is used to instruct the buffer queue management module to release the buffer storing the picture frame.

[0128] After receiving the buffer release request, the buffer queue management module may clear the data in the buffer storing the picture frame and delete it from the buffer queue.

[0129] 20. After receiving the second dot request sent by the maintenance and measurement module, the graphics interface may send the rendering record to the maintenance and measurement module.

[0130] 21. After receiving the first dot request sent by the maintenance and measurement module, the buffer queue management module can send buffer dequeuing records, and / or buffer enqueuing records, and / or obtain buffer records, and / or release buffer records to the maintenance and measurement module.

[0131] 22. After receiving the request for obtaining the third dot sent by the maintenance and measurement module, the layer synthesizer can send the record of the synthesis and display to the maintenance and measurement module.

[0132] 23. When the maintenance and measurement module determines that a frame loss occurs based on the processing information of the picture frame in the maintenance and measurement buffer queue, the maintenance and measurement information of the picture frame can be sent to the communication driver.

[0133] The maintenance information of the picture frame may include processing information of the picture frame and status information of the game application in the game battle scene. The communication driver may include a Wi-Fi driver, a Bluetooth driver, etc. The communication driver is used to instruct the maintenance information to be converted into maintenance information recognizable by the communication module.

[0134] When the game application sends the buffer dequeue request to the buffer queue management module, the maintenance and testing module applies for a maintenance and testing buffer from the memory. After acquiring the processing information of the picture frame, the maintenance and testing module stores the processing information of the picture frame in the maintenance and testing buffer and puts the maintenance and testing buffer into the maintenance and testing buffer queue.

[0135] 24. After the communication driver converts the maintenance and measurement information sent by the maintenance and measurement module into maintenance and measurement information recognizable by the communication module, the maintenance and measurement information can be sent to the communication module.

[0136] Among them, the communication module may include a Wi-Fi module, a Bluetooth module, etc.

[0137] If the maintenance module determines that frame loss occurs based on the maintenance information in the maintenance buffer queue, the maintenance information is sent to the Wi-Fi driver, and the Wi-Fi driver can send the maintenance information to the Wi-Fi module. If the maintenance module determines that frame loss occurs based on the maintenance information in the maintenance buffer queue, the maintenance information is sent to the Bluetooth driver, and the Bluetooth driver can send the maintenance information to the Bluetooth module.

[0138] The communication module may send the maintenance information to the server 200 .

[0139] In a possible implementation, after determining that a frame loss occurs and acquiring the measurement information of the frame, the electronic device 100 may release the measurement buffer storing the measurement information of the frame.

[0140] In the embodiment of the present application, the maintenance and measurement module is not limited to detecting frame loss and collecting maintenance and measurement information on the screen frames of the above-mentioned game application, but can also detect screen frames in other frame loss scenarios and collect maintenance and measurement information. For example, the maintenance and measurement module can also detect frame loss on the dynamic effect screen when the application is started, collect maintenance and measurement information, and so on.

[0141] The following is a schematic diagram of processing information for recording picture frames provided in an embodiment of the present application.

[0142] like Figure 5 As shown, the software system of the electronic device 100 may include a game application, a graphics interface, a buffer queue management module and a layer synthesizer (SF). The following specifically introduces the content of the processing information of the picture frame, wherein the processing information of the picture frame may include one or more of the following: dot data 1, dot data 2, dot data 3, dot data 4, dot data 5, dot data 6:

[0143] S501. A game application may receive an application synchronization signal (app_vsync signal).

[0144] Among them, the game application can receive the application synchronization signal periodically sent by the hardware synthesizer.

[0145] S502: The game application may send a buffer dequeue (dequeuebuffer) request to the buffer queue management module.

[0146] The game application waits for the arrival of the application synchronization signal. When the game application receives the application synchronization signal, it can send a buffer dequeue request to the buffer queue management module and send the arrival time of the received application synchronization signal to the buffer queue management module.

[0147] S503: After receiving the buffer dequeue request sent by the game application, the buffer queue management module may allocate a free buffer (return free buffer) to the game application.

[0148] When allocating idle buffers to game applications, the buffer queue management module can record dot data 1. Dot data 1 can include a record of buffer dequeuing. The content of dot data 1 can refer to the content of the record of buffer dequeuing, which will not be described in detail here.

[0149] S504: After receiving the idle buffer allocated by the buffer queue management module, the game application may send a rendering instruction to the graphics interface.

[0150] S505 . After receiving the rendering instruction sent by the game application, the graphics interface may render a picture frame and store the rendered picture frame in the buffer.

[0151] When the graphics interface renders a drawing frame, it can record dotting data 2. The dotting data 2 can be a record of rendering and drawing, and the content of the dotting data 2 can refer to the content of the above-mentioned record of rendering and drawing, which will not be repeated here.

[0152] S506: After sending the rendering instruction, the game application may send a frame sending instruction to the graphics interface.

[0153] S507: After receiving the frame sending instruction sent by the game application, the graphics interface may send a buffer queue (queuebuffer) request to the buffer queue management module.

[0154] In response to the buffer queue entry request, the buffer queue management module may add the buffer storing the picture frame to the buffer queue. After adding the buffer storing the picture frame to the buffer queue, the buffer queue management module may record the dot data 3. The dot data 3 may include the record of the buffer queue entry, and the content of the dot data 3 may refer to the content of the record of the buffer queue entry, which will not be described in detail here.

[0155] S508: The layer synthesizer may receive a layer synthesis synchronization signal.

[0156] The layer synthesizer can receive a layer synthesis synchronization signal periodically sent by the hardware synthesizer.

[0157] S509: The layer compositor may send an acquirebuffer request to the buffer queue management module.

[0158] Among them, the layer synthesizer can wait for the arrival of the layer synthesis synchronization signal. When the layer synthesizer receives the layer synthesis synchronization signal, it can send a buffer acquisition request to the buffer queue management module and send the arrival time of the layer synthesis synchronization signal to the buffer queue management module.

[0159] S510. After receiving the buffer acquisition request sent by the layer compositor, the buffer queue management module may provide the buffer storing the picture frame (buffer that filled data) to the layer compositor.

[0160] When the buffer queue management module provides the buffer storing the picture frame to the layer synthesizer, it can record the dotting data 4. The dotting data 4 can include the record of the acquisition buffer, and the content of the dotting data 4 can refer to the content of the record of the acquisition buffer, which will not be repeated here.

[0161] S511. After obtaining the picture frame from the buffer storing the picture frame, the layer synthesizer may synthesize the picture frame for display.

[0162] When the layer synthesizer synthesizes and displays the picture frame, it can record the dot data 5. The dot data 5 can include the record of the synthesis and display, and the content of the dot data 5 can refer to the content of the record of the synthesis and display, which will not be repeated here.

[0163] S512: After synthesizing and displaying the picture frame, the layer synthesizer may send a buffer release (releasebuffer) request to the buffer queue management module.

[0164] In response to the buffer release request, the buffer queue management module can clear the data in the buffer storing the frame, delete the buffer storing the frame from the buffer queue, and record the dot data 6. The dot data 6 can include the record of releasing the buffer, and the content of the dot data 6 can refer to the content of the record of releasing the buffer, which will not be described in detail here.

[0165] In this way, when a frame loss event occurs, maintenance personnel can analyze the cause of the frame loss through the processing information of the picture frame, and then solve the frame loss problem and maintain the game application.

[0166] The following specifically describes a method for putting processing information of a picture frame into a HiviewBuffer provided in an embodiment of the present application.

[0167] When the maintenance module determines that the current game scene is a game battle scene based on the status information sent by the game application, the maintenance module can synchronously apply for a maintenance buffer (HiviewBuffer) when the game application applies for an idle buffer. The HiviewBuffer can be used to store the processing information of the current screen frame. Among them, HiviewBuffer can include one or more structures, which can be used to store data.

[0168] The electronic device 100 can detect the index of the buffer storing the picture frame and the identifier of the processing information, wherein the index of the buffer is used to indicate the frame number of the picture frame, and the identifier of the processing information is used to indicate the structure number of the processing information stored in HiviewBuffer.

[0169] When the game application in the electronic device 100 receives the application synchronization signal (app_vsyncsignal), the electronic device 100 can perform the operations of buffer dequeue, rendering and drawing picture frames, and buffer enqueue. Among them, the operation of buffer dequeue can refer to the above steps S501 to S503, and collect dotting data 1, and determine that the identifier of dotting data 1 is "1". The operation of rendering and drawing picture frames can refer to the above steps S504 to S505, and collect dotting data 2, and determine that the identifier of dotting data 2 is "2". The operation of buffer enqueue can refer to the above steps S506 to S507, and collect dotting data 3, and determine that the identifier of dotting data 3 is "3". Among them, the electronic device 100 can detect the index of the buffer storing the picture frame, so as to determine which frame the rendered and drawn picture frame is. Among them, the identifier of dotting data 1 is "1" to indicate that dotting data 1 is placed in the first structure in the corresponding HiviewBuffer. The identifier of dot data 2 is "2", which indicates that dot data 2 is put into the second structure in the corresponding HiviewBuffer. The identifier of dot data 3 is "3", which indicates that dot data 3 is put into the third structure in the corresponding HiviewBuffer.

[0170] When the layer synthesizer in the electronic device 100 receives the layer synthesis synchronization signal (sf_vsyncsignal), the electronic device 100 can perform the operations of obtaining the buffer, synthesizing the display screen frame, and releasing the buffer. Among them, the operation of obtaining the buffer can refer to the above steps S508 to S510, and collect the dot data 4, and determine that the identifier of the dot data 4 is "4". The operation of synthesizing the display screen frame can refer to the above step S511, and collect the dot data 5, and determine that the identifier of the dot data 5 is "5". The operation of releasing the buffer can refer to the above step S512, and collect the dot data 6, and determine that the identifier of the dot data 6 is "6". Among them, the electronic device 100 can detect the index of the buffer storing the screen frame, so as to determine which frame the synthesized display screen frame is. Among them, the identifier of the dot data 4 is "4" to indicate that the dot data 4 is placed in the fourth structure in the corresponding HiviewBuffer. The identifier of the dot data 5 is "5" to indicate that the dot data 5 is placed in the fifth structure in the corresponding HiviewBuffer. The identifier of dot data 6 is "6", which is used to indicate that dot data 6 is placed in the sixth structure in the corresponding HiviewBuffer.

[0171] Among them, the electronic device 100 can match the processing information of the same picture frame based on the index of the buffer storing the picture frame in the rendering and drawing stage and the index of the buffer storing the picture frame in the synthesis and display stage.

[0172] For example, Fig. 6A As shown, when the game application in the electronic device 100 receives an application synchronization signal, operations such as dequeuing the buffer of the Nth frame, rendering and drawing the frame, and enqueuing the buffer can be performed. When the layer synthesizer in the electronic device 100 receives a layer synthesis synchronization signal, operations such as obtaining the buffer of the N-1th frame, synthesizing and displaying the frame, and releasing the buffer can be performed. When the game application in the electronic device 100 receives the next application synchronization signal, operations such as dequeuing the buffer of the N+1th frame, rendering and drawing the frame, and enqueuing the buffer can be performed. When the layer synthesizer in the electronic device 100 receives the next layer synthesis synchronization signal, operations such as obtaining the buffer of the Nth frame, synthesizing and displaying the frame, and releasing the buffer can be performed.

[0173] For example, the electronic device 100 may place the dot data 1 into the Nth frame of the image processed in the application synchronization cycle, the identifier of the dot data 1 is "1", the identifier of the dot data 2 is "2", and the identifier of the dot data 3 is "3". Figure 6B In the first structure of the corresponding dimension measurement buffer _N shown in the figure, the dot data 2 is placed in Figure 6B In the second structure of the corresponding dimension measurement buffer _N shown in the figure, the dot data 3 is placed in Figure 6B The electronic device 100 can put the dot data 4 into the third structure in the corresponding dimension measurement buffer _N shown in the figure. The picture frame processed in the layer synthesis synchronization period is the Nth frame, the identifier of the dot data 4 is "4", the identifier of the dot data 5 is "5", and the identifier of the dot data 6 is "6". Figure 6B In the fourth structure of the corresponding dimension measurement buffer _N shown in the figure, the dot data 5 is placed in Figure 6B In the fifth structure of the corresponding dimension measurement buffer _N shown in FIG. 1 , the dot data 6 is placed in Figure 6B In the sixth structure in the corresponding dimension measurement buffer_N shown.

[0174] It should be understood that the application synchronization signal received by the game application in the electronic device 100 and the layer synthesis synchronization signal received by the layer synthesizer may be received at the same time or at different times.

[0175] It should be understood that within the same synchronization cycle, the picture frames processed by the electronic device 100 in the application synchronization cycle and the picture frames processed by the electronic device 100 in the layer synthesis synchronization cycle may not be the same picture frames. Therefore, it is necessary to match the processing information of the picture frames processed in the application synchronization cycle and the processing information of the picture frames processed in the layer synthesis synchronization cycle, and put them into the same HiviewBuffer to ensure that one HiviewBuffer collects the processing information of the same picture frame.

[0176] It should be understood that the state information of a plurality of consecutive picture frames may be the same, and therefore, the same state information may include a plurality of picture frames and HiviewBuffers of the plurality of picture frames for storing processing information.

[0177] In a possible implementation, the state information of the picture frame may also be stored in the dimensioning buffer corresponding to the picture frame. For example, the state information of the Nth picture frame is stored in the seventh structure in the dimensioning buffer_N.

[0178] A frame loss detection method provided in an embodiment of the present application is described in detail below.

[0179] The electronic device 100 can obtain processing information of multiple continuous picture frames. HiviewBuffer can be used to store processing information of picture frames. The electronic device 100 can determine whether a frame is lost and the number of lost frames based on processing information of the current picture frame and processing information of the previous picture frame, where the previous picture frame is the previous frame of the current picture frame.

[0180] When the previous screen frame is continuously displayed on the display screen for multiple synchronization cycles, the current screen frame has frame loss. For example, when the difference between the time when the buffer storing the previous screen frame after rendering and drawing is released and the time when the buffer storing the current screen frame after rendering and drawing is released is greater than the cycle of the synchronization signal, it can be determined that the current first screen frame has frame loss.

[0181] Exemplarily, the calculation formula may be as follows:

[0182]

[0183] Where, skip is the number of dropped frames. releaseBufferTime is the time when the current frame is released. lastReleaseBufferTime is the time when the previous frame is released. app_vsync_cycle is used to indicate the cycle of the application synchronization signal.

[0184] When skip is less than 0, it indicates that no frame loss event occurs. When skip is greater than 0, it indicates that a frame loss event occurs and the number of lost frames is [skip]. Wherein, [skip] is the largest integer less than skip, for example, if skip = 3.8, then [skip] = 3; if skip = -2.4, then [skip] = -3. In this way, the electronic device 100 can determine whether a frame is lost and the number of lost frames based on the measurement information of the current picture frame and the measurement information of the previous picture frame.

[0185] For example, Figure 7 As shown, the measurement buffer_N is used to store the processing information of the current Nth frame acquired by the electronic device 100, and the measurement buffer_N-1 is used to store the processing information of the N-1th frame acquired by the electronic device 100. The electronic device 100 can determine whether a frame loss occurs and the number of frame losses based on the skip value. When skip is greater than 0, it indicates that a frame loss event occurs and the number of frame losses is [skip].

[0186] When the electronic device 100 determines that the current picture frame has a frame loss based on the processing information of the current picture frame and the processing information of the previous picture frame, it can send the first measurement information to the server 200, wherein the measurement information is stored in the measurement buffer. After sending the measurement information to the server 200, the electronic device 100 can release the measurement buffer.

[0187] It should be understood that in the embodiment of the present application, it is only necessary to record the processing information of the current picture frame and the processing information of the previous picture frame to determine whether a frame is lost and the number of frames lost. Therefore, HiviewBufferQueue only needs to record the processing information of the current picture frame and the processing information of the previous picture frame. For example, HiviewBufferQueue records the processing information of the N-1th frame and the processing information of the Nth frame. When the processing information of the N+1th frame begins to be recorded, the processing information of the N-1th frame can be overwritten. In this way, there is no need to apply for a new Hiviewbuffer in HiviewBufferQueue for the processing information of each frame, thereby saving resources.

[0188] It should be understood that in the embodiment of the present application, the picture frame rendering and drawing stage and the picture frame synthesis and display stage are carried out simultaneously. Therefore, there may be a buffer accumulation. For example, when the picture frame synthesis and display stage records the measurement information of the Nth frame, the picture frame rendering and drawing stage has begun to record the processing information of the N+Xth frame (X=1, 2, 3, etc.). The electronic device 100 can set the number of Hiviewbuffers in the HiviewBufferQueue based on the storage situation of the memory and when the buffer is satisfied. In this way, it can be ensured that the processing information of the current picture frame and the processing information of the previous picture frame are recorded.

[0189] It should be understood that the method described in the embodiments of the present application can also be applied to other scenarios where the frame loss experience is sensitive, wherein the frame loss experience is sensitive to the scene can be used to indicate the situation where the loss of picture frames may affect the user experience during video playback or real-time video applications. For example: in real-time video scenarios such as video conferencing, video calls or live broadcasts, frame loss can cause the picture to freeze, delay or be incoherent, or when the application is started, the frame rate of the application animation drops, which will make the user experience bad.

[0190] The following is a flow chart of a data processing method provided in an embodiment of the present application.

[0191] Figure 8 A flow chart of a data processing method provided by an embodiment of the present application is shown. The data processing method is applied to an electronic device 100, in which a game application, a graphics interface, a buffer queue management module, a layer synthesizer SF and a maintenance module are running. The specific steps are as follows:

[0192] S801. Obtain first state information of a game application in a first scenario, where the first state information includes a type of the first scenario.

[0193] When the electronic device 100 runs the game application, it can obtain the first state information of the game application in the first scene. The first state information can also include the game stage (for example, the parachuting stage), the number of players on the same screen (for example, the number of players in the current screen), the status of the players on the same screen (for example, the players are running), and the display of game props (for example, the game props have arrived at the location), etc.

[0194] S802: Rendering and drawing a first screen frame of a game application in a first scene.

[0195] The process of rendering the frame of the game application in the first scene by the electronic device 100 can refer to the above Figure 4 Steps 3 to 13 in the above are not described in detail here.

[0196] S803, synthesize and display the first picture frame.

[0197] The process of synthesizing and displaying the first picture frame by the electronic device 100 can refer to the above Figure 4 In the process, steps 14 to 19 are not described in detail here.

[0198] S804. Obtain processing information of the first picture frame, wherein the processing information of the first picture frame includes processing records of the first picture frame in a rendering and drawing phase and processing records of the first picture frame in a synthesis and displaying phase.

[0199] The process of the electronic device 100 obtaining the processing information of the first picture frame can refer to the above Figure 4 Steps 5 to 22 are not described in detail here.

[0200] S805. If it is determined based on the processing information of the first frame that the first frame has frame loss, first dimensional measurement information is sent to the server, wherein the first dimensional measurement information includes the processing information of the first frame and first state information of the game application in the first scenario.

[0201] The process in which the electronic device 100 determines that the first picture frame has a frame loss based on the processing information of the first picture frame and sends the first dimension measurement information to the server can refer to the above Figure 4 Steps 23 to 24 are not described in detail here.

[0202] In a possible implementation, the electronic device 100 may obtain processing information of a second picture frame, which is a picture frame before and adjacent to the first picture frame, and the processing information of the second picture frame may include the time when the buffer storing the second picture frame after rendering and drawing is released. The electronic device 100 may determine that the first picture frame has a frame loss based on the difference between the time when the buffer storing the second picture frame after rendering and drawing is released and the time when the buffer storing the first picture frame after rendering and drawing is released being greater than the period of the synchronization signal.

[0203] After acquiring the processing information of the first picture frame, the electronic device 100 may store the processing information of the first picture frame in a first dimension buffer.

[0204] After determining that the first picture frame has frame loss, the electronic device 100 may generate and save the first dimensional measurement information. After saving the first dimensional measurement information, the electronic device 100 may release the first dimensional measurement buffer.

[0205] For the parts not described in detail in the embodiments of the present application, reference can be made to the aforementioned embodiments and will not be repeated here.

[0206] In a possible implementation, the electronic device 100 can send the maintenance information of the picture frame to other devices when detecting that the picture frame is lost. The maintenance personnel can obtain the maintenance information of the lost picture frame from the device storing the maintenance information, and analyze the cause of the lost picture frame based on the maintenance information of the lost picture frame, thereby solving the frame loss problem of the game application.

[0207] In a possible implementation, the electronic device 100 can save the maintenance information of the picture frame in a local file when detecting that a picture frame is lost. The maintenance personnel can obtain the maintenance information of the lost picture frame from the local file, and analyze the cause of the lost picture frame based on the maintenance information of the lost picture frame, thereby solving the frame loss problem of the game application.

[0208] In a possible implementation, when the electronic device 100 detects that the number of frame losses in the game application within a preset time (e.g., 10 minutes) is greater than a certain threshold (e.g., 100 frames), the electronic device 100 can send all the maintenance information of the lost frames within the preset time to the server or other devices. In this way, when abnormal frame loss occurs in the electronic device 100, the maintenance information of the abnormal frame loss can be collected, thereby eliminating the situation of normal frame loss.

[0209] Through a data processing method provided in the present application, first dimensional measurement information of a first screen frame of a game application in a first scenario can be collected, and the first dimensional measurement information includes processing information of the first screen frame and first status information of the game application in the first scenario, wherein the processing information of the first screen frame includes processing records of the first screen frame in a rendering and drawing phase and processing records in a synthesis and display phase. In this way, when frame loss occurs in the first screen frame, the measurement personnel can analyze the cause of frame loss of the first screen frame based on the first dimensional measurement information of the first screen frame, thereby solving the frame loss problem of the game application.

[0210] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0211] The embodiments of the present application also provide a computer program product. When the computer program product is executed on an electronic device, the electronic device can implement the steps in the above-mentioned various method embodiments.

[0212] The embodiment of the present application also provides a chip system, the chip system includes a processor, the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement the steps of any method embodiment of the present application. The chip system can be a single chip or a chip module composed of multiple chips.

[0213] The term "user interface (UI)" in the specification and drawings of this application refers to the medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface of an application is source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize, such as pictures, text, buttons and other controls. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, pictures and text. The properties and contents of controls in the interface are defined by tags or nodes, such as XML through <textview> 、 <imgview> 、 <videoview>The controls contained in the interface are specified by nodes such as <head> and <body>. A node corresponds to a control or attribute in the interface. After being parsed and rendered, the node is presented as user-visible content. In addition, many applications, such as hybrid applications, usually also contain web pages in their interfaces. A web page, also known as a page, can be understood as a special control embedded in the application interface. A web page is source code written in a specific computer language, such as hypertext markup language (HTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed as user-recognizable content by a browser or a web page display component with similar functions to a browser. The specific content contained in a web page is also defined by tags or nodes in the web page source code. For example, HTML is defined by <body> tags or nodes in the web page source code. 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0214] The most common form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operation that is displayed in a graphical manner. It can be an icon, window, control or other interface element displayed on the display screen of an electronic device, where a control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets and other visual interface elements.

[0215] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0216] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0217] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A data processing method, characterized in that: Applied to an electronic device, the electronic device runs a game application, and the method includes: Acquire first state information of the game application in a first scenario, where the first state information includes a type of the first scenario; Rendering and drawing a first screen frame of the game application in a first scene; synthesizing and displaying the first picture frame; Acquire processing information of the first picture frame, wherein the processing information of the first picture frame includes processing records of the first picture frame in a rendering and drawing phase and processing records in a synthesis and display transmission phase; If it is determined based on the processing information of the first frame that the first frame has frame loss, first dimensional measurement information is sent to the server, wherein the first dimensional measurement information includes the processing information of the first frame and first status information of the game application in the first scenario.

2. The method according to claim 1, characterized in that The electronic device also runs a graphics interface and a buffer queue management module; The rendering and drawing of the first screen frame of the game application in the first scene specifically includes: After receiving the application synchronization signal, the game application sends a buffer dequeue request to the buffer queue management module; In response to the buffer dequeue request, the buffer queue management module sends the address of the buffer to the game application; The game application renders and draws the first screen frame through the graphics interface, and adds the rendered first screen frame to the buffer queue managed by the buffer queue management module through the buffer.

3. The method according to claim 2, characterized in that The electronic device also runs a layer synthesizer SF; The synthesizing and displaying the first picture frame specifically includes: After receiving the layer synthesis synchronization signal, the SF sends a buffer acquisition request to the buffer queue management module; In response to the buffer acquisition request, the buffer queue management module sends the address corresponding to the buffer in the buffer queue storing the rendered first screen frame to the SF; The SF takes out the first rendered picture frame from the buffer storing the first rendered picture frame, and synthesizes the first rendered picture frame for display; After synthesizing and displaying the first picture frame after rendering and drawing, the SF sends a buffer release request to the buffer queue management module; In response to the buffer release request, the buffer queue management module releases the buffer storing the first picture frame after rendering.

4. The method according to claim 3, characterized in that The period of the application synchronization signal is the same as the period of the layer synthesis synchronization signal.

5. The method according to claim 3 or 4, characterized in that: The obtaining of the processing information of the first picture frame specifically includes: If the first scene is a designated game scene, processing information of a first screen frame of the game application in the first scene is obtained.

6. The method according to claim 5, characterized in that The electronic device also runs a maintenance and testing module; The obtaining the first state information of the game application in the first scenario specifically includes: The maintenance module obtains the first status information sent by the game application; If the first scene is a specified game scene, obtaining processing information of a first screen frame of the game application in the first scene specifically includes: If the maintenance module determines that the first scene is a designated game scene based on the first state information, the maintenance module obtains processing information of the first screen frame of the game application in the first scene from the buffer queue management module, the graphics interface and the SF.

7. The method according to claim 6, characterized in that The processing record of the first picture frame in the rendering and drawing stage includes one or more of the following: a buffer dequeue record, a rendering and drawing record, and a buffer enqueue record; The processing record of the first picture frame in the synthesis and display stage includes one or more of the following: a record of obtaining a buffer, a record of synthesizing and displaying, and a record of releasing a buffer; If the maintenance and detection module determines that the first scene is a designated game scene based on the first state information, the maintenance and detection module obtains processing information of a first screen frame of the game application in the first scene from the buffer queue management module, the graphics interface and the SF, specifically including: If the maintenance and measurement module determines that the first scene is a designated game scene based on the first state information, the maintenance and measurement module sends a first RBI request to the buffer queue management module, sends a second RBI request to the graphics interface, and sends a third RBI request to the SF; In response to the first dot request, the buffer queue management module sends the record of the buffer being dequeued, and / or the record of the buffer being enqueued, and / or the record of obtaining the buffer, and / or the record of releasing the buffer to the maintenance and measurement module; In response to the second dotting request, the graphics interface sends the rendering record to the maintenance and measurement module; In response to the third dotting request, the SF sends the record of the synthetic display to the maintenance module.

8. The method according to claim 7, characterized in that The record of the buffer dequeue includes one or more of the following: the moment when the game application receives the application synchronization signal, the moment when the game application sends the buffer dequeue request to the buffer queue management module, the moment when the buffer queue management module sends the buffer address to the game application, and whether the buffer queue management module successfully sends the buffer address to the game application.

9. The method according to claim 7 or 8, characterized in that: The rendering record includes one or more of the following: the start time of rendering the first frame, the end time of rendering the first frame, and whether the first frame is successfully rendered.

10. The method according to any one of claims 7 to 9, characterized in that: The record of the buffer zone being enqueued includes one or more of the following: the start time when the buffer zone is added to the buffer zone queue, the end time when the buffer zone is added to the buffer zone queue, and the layer information of the first picture frame in the buffer zone.

11. The method according to any one of claims 7 to 10, characterized in that: The record of obtaining the buffer includes one or more of the following: the moment when the SF receives the layer synthesis synchronization signal, the moment when the SF sends the buffer acquisition request to the buffer queue management module, the moment when the buffer queue management module sends the address corresponding to the buffer in the buffer queue storing the first screen frame after rendering and drawing to the SF, and whether the buffer queue management module successfully sends the address corresponding to the buffer in the buffer queue storing the first screen frame after rendering and drawing to the SF.

12. The method according to any one of claims 7 to 11, characterized in that: The record of synthesis and display includes one or more of the following: the start time of synthesis of the first picture frame, the end time of synthesis of the first picture frame, the start time of display of the first picture frame, the end time of display of the first picture frame, whether the first picture frame is successfully synthesized, and whether the first picture frame is successfully displayed.

13. The method according to any one of claims 7 to 12, characterized in that: The record of releasing the buffer includes one or more of the following: the time when the SF sends the buffer release request to the buffer queue management module, the time when the buffer queue management module releases the buffer storing the first screen frame after rendering and drawing, and whether the buffer queue management module successfully releases the buffer storing the first screen frame after rendering and drawing.

14. The method according to any one of claims 1 to 13, characterized in that The first status information also includes one or more of the following: game stage, status of players on the same screen, number of players on the same screen, and display of game props.

15. The method according to any one of claims 1 to 14, characterized in that The method further comprises: Acquire processing information of a second picture frame, wherein the second picture frame is a picture frame before and adjacent to the first picture frame, and the processing information of the second picture frame includes a processing record of the second picture frame in a rendering and drawing phase and a processing record of the second picture frame in a synthesis and displaying phase; If it is determined based on the processing information of the first picture frame that the first picture frame is lost, the method specifically includes: Based on the processing information of the first picture frame and the processing information of the second picture frame, it is determined that the first picture frame has frame loss.

16. The method according to claim 15, characterized in that The processing information of the second picture frame includes the time when the buffer queue management module releases the buffer storing the rendered and drawn second picture frame; The determining, based on the processing information of the first picture frame and the processing information of the second picture frame, that the first picture frame has frame loss specifically includes: If the difference between the time when the buffer queue management module releases the buffer storing the second screen frame after rendering and the time when the buffer queue management module releases the buffer storing the first screen frame after rendering and is greater than the period of the synchronization signal, it is determined that the first screen frame has been lost.

17. The method according to any one of claims 6 to 13, characterized in that: The method further comprises: The measurement module applies for a first measurement buffer when the game application sends the buffer dequeue request to the buffer queue management module; After acquiring the processing information of the first picture frame, the dimension measurement module stores the processing information of the first picture frame into the first dimension measurement buffer.

18. The method according to claim 17, characterized in that The method further comprises: After determining that the first picture frame has frame loss, generating and saving the first dimension measurement information; After saving the first dimensional measurement information, the first dimensional measurement buffer is released.

19. An electronic device, characterized in that: It includes one or more processors and one or more memories; wherein, the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer instructions, so that when the one or more processors execute the computer instructions, the electronic device executes the data processing method as described in any one of claims 1-18.

20. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a processor of an electronic device, enable the electronic device to execute the data processing method according to any one of claims 1 to 18.

Images