A data processing method and related device

Abstract

The present application provides a data processing method and related apparatus, which can achieve the following: an electronic device running a game application; the electronic device obtaining first status information of the game application in a first scenario, the first status information including the type of the first scenario; rendering and drawing a first screen frame of the game application in the first scenario; synthesizing and displaying the first screen frame; obtaining processing information of the first screen frame, wherein the processing information of the first screen frame includes a processing record of the first screen frame during the rendering and drawing phase and a processing record during the synthesizing and displaying phase; if it is determined based on the processing information of the first screen frame that the first screen frame has been lost, sending first quality assurance information to a server, wherein the first quality assurance information includes the processing information of the first screen frame and the first status information of the game application in the first scenario. In this way, quality assurance personnel can obtain quality assurance information when the game application loses frames through the server, thereby resolving the problem of frame loss in the game application.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a data processing method and related devices. Background Art

[0002] With the continuous upgrading of electronic devices, the applications they use are also developing at an astonishing rate. However, during application execution, frame drops can occur, which can easily lead to a poor user experience. Maintenance personnel need to identify the cause of application frame drops in order to resolve them. Therefore, how to collect application runtime data when frame drops occur has become a pressing issue in this field. Summary of the Invention

[0003] The present application provides a data processing method and related devices, which realize the collection of first-dimensional measurement information of the first picture frame when the first picture frame of the game application in the first scene is lost.

[0004] In a first aspect, the present application provides a data processing method, which is applied to an electronic device running a game application, the method comprising: obtaining first status information of the game application in a first scenario, the first status information including the type of the first scenario; rendering and drawing a first screen frame of the game application in the first scenario; synthesizing and displaying the first screen frame; obtaining processing information of the first screen frame, wherein the processing information of the first screen frame includes a processing record of the first screen frame in the rendering and drawing stage and a processing record in the synthesizing and displaying stage; if it is determined based on the processing information of the first screen frame that the first screen frame has frame loss, sending first dimensional measurement information to a server, wherein the first dimensional measurement information includes the processing information of the first screen frame and the first status information of the game application in the first scenario.

[0005] Through a data processing method provided in the present application, first-dimensional measurement information of the first screen frame of a game application in a first scenario can be collected. The first-dimensional measurement information includes processing information of the first screen frame and first status information of the game application in the first scenario, wherein the processing information of the first screen frame includes processing records of the first screen frame in the rendering and drawing stage and processing records in the synthesis and display stage. In this way, when frame loss occurs in the first screen frame, the measurement personnel can analyze the cause of the frame loss of the first screen frame based on the first-dimensional measurement information of the first screen frame, thereby solving the frame loss problem of the game application.

[0006] In one possible implementation, the electronic device also runs a graphics interface and a buffer queue management module; the rendering and drawing of the first screen frame of the game application in the first scenario specifically includes: the game application sends a buffer dequeue request to the buffer queue management module after receiving an application synchronization signal; in response to the buffer dequeue request, the buffer queue management module sends the address of the buffer to the game application; the game application renders and draws the first screen frame through the graphics interface, and adds the rendered first screen frame to the buffer queue managed by the buffer queue management module through the buffer. In this way, the electronic device can render and draw the first screen frame so that the first screen frame can be better perceived by the human eye, giving the user a better visual experience.

[0007] In one possible implementation, the electronic device further operates a layer synthesizer SF; the synthesizing and displaying the first screen frame specifically includes: after receiving a layer synthesis synchronization signal, the SF sends a buffer acquisition request to the buffer queue management module; in response to the buffer acquisition request, the buffer queue management module sends the address corresponding to the buffer in the buffer queue storing the rendered first screen frame to the SF; the SF retrieves the rendered first screen frame from the buffer storing the rendered first screen frame, and synthesizes and displays the rendered first screen frame; after synthesizing and displaying the rendered first screen frame, the SF sends a buffer release request to the buffer queue management module; in response to the buffer release request, the buffer queue management module releases the buffer storing the rendered first screen frame. In this way, the electronic device can synthesize and display the first screen frame, allowing the user to see the first screen frame. At the same time, releasing the buffer storing the rendered first screen frame can save memory space of the electronic device.

[0008] In a possible implementation, the period of the application synchronization signal is the same as the period of the layer synthesis synchronization signal.

[0009] In a possible implementation, obtaining the processing information of the first picture frame specifically includes: if the first scene is a designated game scene, obtaining the processing information of the first picture frame of the game application in the first scene.

[0010] In a possible implementation, the game application scenes include a game lobby scene and a game battle scene. The designated game scene is a game battle scene.

[0011] In one possible implementation, the electronic device further runs a maintenance and testing module; obtaining first status information of the game application in a first scenario specifically includes: the maintenance and testing module obtaining the first status information sent by the game application; and if the first scenario is a designated game scenario, obtaining processing information of a first screen frame of the game application in the first scenario specifically includes: if the maintenance and testing module determines that the first scenario is the designated game scenario based on the first status information, the maintenance and testing module obtains processing information of the first screen frame of the game application in the first scenario from the buffer queue management module, the graphics interface, and the SF. In this way, the electronic device can obtain processing information of the first screen frame when the first scenario is the designated game scenario.

[0012] In a possible implementation, the processing record of the first screen frame in the rendering and drawing stage includes one or more of the following: a record of buffer dequeuing, a record of rendering and drawing, and a record of buffer enqueuing; the processing record of the first screen frame in the synthesis and display stage includes one or more of the following: a record of obtaining the buffer, a record of synthesis and display, and a record of releasing the buffer; if the maintenance and detection module determines that the first scene is a specified game scene based on the first status information, the maintenance and detection module obtains the processing information of the first screen frame of the game application in the first scene from the buffer queue management module, the graphics interface and the SF, specifically including: ... When the first status information determines that the first scene is a designated game scene, the maintenance and measurement module sends a first dotting request to the buffer queue management module, sends a second dotting request to the graphics interface, and sends a third dotting request to the SF; in response to the first dotting request, the buffer queue management module sends a buffer dequeue record, and / or a buffer enqueue record, and / or a buffer acquisition record, and / or a buffer release record to the maintenance and measurement module; in response to the second dotting request, the graphics interface sends a rendering and drawing record to the maintenance and measurement module; in response to the third dotting request, the SF sends a synthesis and display record to the maintenance and measurement module. In this way, the maintenance and testing module in the electronic device can obtain the records of the buffer dequeueing, rendering and drawing, buffer enqueuing, buffer acquisition, synthesis and display, and buffer release of the first picture frame, thereby helping maintenance and testing personnel to solve the frame loss problem of the game application based on the records of the buffer dequeueing, rendering and drawing, buffer enqueuing, buffer acquisition, synthesis and display, and buffer release of the first picture frame when frame loss occurs.

[0013] In one possible implementation, the buffer dequeue record includes one or more of the following: the time when the game application received the application synchronization signal; the time when the game application sent the buffer dequeue request to the buffer queue management module; the time when the buffer queue management module sent the buffer address to the game application; and whether the buffer queue management module successfully sent the buffer address to the game application. This allows maintenance personnel to analyze whether the first frame loss occurred during the buffer dequeue based on the buffer dequeue record.

[0014] In one possible implementation, the rendering record includes one or more of the following: the start time of rendering the first frame, the end time of rendering the first frame, and whether the first frame was successfully rendered. This allows maintenance personnel to analyze whether the first frame loss occurred during rendering based on the rendering record.

[0015] In one possible implementation, the buffer enqueue record includes one or more of the following: the start time the buffer was added to the buffer queue, the end time the buffer was added to the buffer queue, and the layer information of the first frame in the buffer. This allows maintenance personnel to analyze, based on the buffer enqueue record, whether the first frame loss occurred when the buffer storing the first frame was enqueued.

[0016] In one possible implementation, the buffer acquisition record includes one or more of the following: the moment the SF receives the layer composition synchronization signal, the moment the SF sends the buffer acquisition request to the buffer queue management module, the moment the buffer queue management module sends the address corresponding to the buffer in the buffer queue storing the rendered first frame to the SF, and whether the buffer queue management module successfully sends the address corresponding to the buffer in the buffer queue storing the rendered first frame to the SF. In this way, based on the buffer acquisition record, maintenance personnel can analyze whether the frame loss in the first frame occurred when the SF acquired the buffer storing the rendered first frame.

[0017] In one possible implementation, the synthesis display record includes one or more of the following: the start time of synthesis display of the first frame, the end time of synthesis display of the first frame, and whether the synthesis display of the first frame was successfully synthesized and displayed. This allows maintenance personnel to analyze whether frame loss during synthesis display of the first frame occurs based on the synthesis display record.

[0018] In one possible implementation, the buffer release record includes one or more of the following: the time when the SF sends the buffer release request to the buffer queue management module, the time when the buffer queue management module releases the buffer storing the rendered first frame, and whether the buffer queue management module successfully releases the buffer storing the rendered first frame. This allows maintenance personnel to analyze, based on the buffer release record, whether the frame loss in the first frame occurred when the buffer queue management module released the buffer storing the rendered first frame.

[0019] In a possible implementation, the first status information further includes one or more of the following: game stage, status of players on the same screen, number of players on the same screen, and display of game props.

[0020] In one possible implementation, the method further includes: obtaining processing information of a second frame, wherein the second frame is a frame that is before and adjacent to the first frame, and the processing information of the second frame includes processing records of the second frame in the rendering and drawing stage and processing records in the synthesis and display stage; determining that frame loss has occurred in the first frame based on the processing information of the first frame specifically includes: determining that frame loss has occurred in the first frame based on the processing information of the first frame and the processing information of the second frame.

[0021] In one possible implementation, the processing information of the second frame includes the time when the buffer queue management module releases the buffer storing the rendered second frame; and determining whether the first frame has been lost based on the processing information of the first frame and the processing information of the second frame specifically includes: if the difference between the time when the buffer queue management module releases the buffer storing the rendered second frame and the time when the buffer queue management module releases the buffer storing the rendered first frame is greater than the period of the synchronization signal, determining whether the first frame has been lost. In this way, the electronic device can determine whether the first frame has been lost based on the processing information of the first frame and the processing information of the second frame.

[0022] In one possible implementation, the method further includes: when the game application sends the buffer dequeue request to the buffer queue management module, the dimension measurement module applies for a first dimension measurement buffer; after the dimension measurement module obtains the processing information of the first picture frame, the processing information of the first picture frame is stored in the first dimension measurement buffer.

[0023] In a possible implementation, the method further includes: after determining that frame loss occurs in the first picture frame, generating and saving first dimensional measurement information; and after saving the first dimensional measurement information, releasing the first dimensional measurement buffer.

[0024] In a second aspect, the present application provides an electronic device comprising one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer instructions, so that when the one or more processors execute the computer instructions, the electronic device executes the data processing method in any possible implementation of the above-mentioned first aspect.

[0025] In a third aspect, the present application provides a chip, which is applied to an electronic device, and the chip includes a processing circuit and an interface circuit, the interface circuit is used to receive instructions and transmit them to the processing circuit, and the processing circuit is used to run instructions to execute the data processing method in any possible implementation method of the above-mentioned first aspect.

[0026] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the data processing method in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of the hardware structure of a server provided in an embodiment of the present application;

[0029] Figure 3A-Figure 3B A schematic diagram of an interface of a game application provided in this embodiment;

[0030] Figure 4 A schematic diagram of the architecture of a software and hardware system provided in an embodiment of the present application;

[0031] Figure 5 An interactive flow chart of processing information for recording picture frames provided in an embodiment of the present application;

[0032] Figure 6A A schematic diagram of performing different operations on picture frames based on synchronization signals provided in an embodiment of the present application;

[0033] Figure 6B A schematic diagram of storing processing information of a picture frame in a measurement buffer provided by an embodiment of the present application;

[0034] Figure 7 A schematic diagram of frame loss detection based on a measurement buffer provided in an embodiment of the present application;

[0035] Figure 8A flowchart of a data processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The following is a clear and detailed description of the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship between related objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0037] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0038] The following describes the hardware structure of an electronic device provided in an embodiment of the present application.

[0039] Figure 1 A hardware structure diagram of an electronic device 100 provided in an embodiment of the present application is shown.

[0040] It should be understood that Figure 1 The electronic device 100 shown is only one example, and the electronic device 100 may have more Figure 1 More or fewer components may be shown, two or more components may be combined, or the components may be arranged differently. Figure 1 The various components shown in the drawings may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0041] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an acceleration sensor 180E, and a touch sensor 180K.

[0042] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0043] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0044] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0045] The processor 110 may also include a memory for storing instructions and data.

[0046] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0047] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0048] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves for radiation via the antenna 1.

[0049] The modem processor includes a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a medium- or high-frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is passed to the application processor.

[0050] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0051] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 , and antenna 2 is coupled to wireless communication module 160 , so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0052] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0053] The display screen 194 is used to display images, videos, etc.

[0054] The electronic device 100 can realize the shooting function through the ISP, camera 193, video codec, GPU, display screen 194 and application processor. The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. The NPU is a neural network (NN) computing processor that quickly processes input information by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, and can also continuously self-learn.

[0055] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage. The internal memory 121 can be used to store computer-executable program code, which includes instructions. The processor 110 executes the instructions stored in the internal memory 121 to execute various functional applications and data processing of the electronic device 100.

[0056] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0057] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals.

[0058] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0059] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0060] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0061] The headphone jack 170D is used to connect a wired headphone.

[0062] The pressure sensor 180A is used to sense pressure signals and convert the pressure signals into electrical signals.

[0063] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100, to prevent shaky images, and to navigate and experience physical gaming scenes.

[0064] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0065] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, in a location different from that of the display screen 194.

[0066] Buttons 190 include a power button and volume buttons. Motor 191 can generate vibration prompts. Indicator 192 can be an indicator light that can be used to indicate charging status, power level changes, messages, missed calls, notifications, etc. SIM card interface 195 is used to connect a SIM card.

[0067] In the embodiment of the present application, the device type of the electronic device 100 can be any of a mobile phone, a tablet computer, a handheld computer, a desktop computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), as well as smart home devices such as smart large screens and smart speakers, wearable devices such as smart bracelets, smart watches, and smart glasses, extended reality (XR) devices such as augmented reality (AR), virtual reality (VR), and mixed reality (MR), in-vehicle devices or smart city devices, etc.

[0068] The following describes the hardware structure of a server provided in an embodiment of the present application.

[0069] Figure 2 A schematic diagram of the hardware structure of a server 200 provided in an embodiment of the present application is shown.

[0070] like Figure 2 As shown, the server 200 may include a communication interface 201 and a processor 202. Optionally, the server 200 may further include a memory 203. The memory 203 may be provided inside the server 200 or outside the server 200. The functions implemented by the server 200 in the embodiment may all be implemented by the processor 202. The processor 202 receives a data stream through the communication interface 201 and is used to implement the method of any embodiment. During the implementation process, each step of the processing flow may complete the method of any embodiment by means of a hardware integrated logic circuit in the processor 202 or instructions in the form of software. The memory 203 is coupled to the processor 202.

[0071] The processor 202 involved in the embodiments of the present application can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0072] The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, modules or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, modules or modules.

[0073] Processor 202 may operate in conjunction with memory 203. Memory 203 may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or a volatile memory, such as random-access memory (RAM). Memory 203 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0074] The embodiments of the present application do not limit the specific connection medium between the communication interface 201, processor 202, and memory 203. For example, the memory 203, processor 202, and communication interface 201 may be connected via a bus. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0075] The following is a schematic diagram of an interface of a game application provided in an embodiment of the present application.

[0076] Figure 3A and Figure 3B A schematic diagram of an interface of a game application provided in an embodiment of the present application is shown.

[0077] For example, Figure 3A As shown, before entering the game battle scene, the electronic device 100 can display an interface before entering the game battle scene, for example, the game lobby interface 310. Before entering the game battle scene, the CPU in the electronic device 100 has a small amount of computing power. Therefore, the CPU load is low, and the CPU can have more processing power to handle various computing tasks in the game application, thereby reducing the delay of the CPU processing tasks and reducing the risk of frame loss.

[0078] For example, Figure 3B As shown, when entering the game battle scene, the electronic device 100 can display the game battle interface 320. In the game battle scene, since the CPU needs to handle a large amount of real-time calculations and network data exchange, the CPU load will become high. If the CPU load is too high, it will cause the CPU to be unable to process computing tasks (e.g., graphics rendering) in a timely manner, resulting in frame drops.

[0079] Therefore, an embodiment of the present application provides a data processing method applied to an electronic device that runs a game application; the electronic device obtains first state information of the game application in a first scenario, the first state information including the type of the first scenario; renders and draws a first screen frame of the game application in the first scenario; synthesizes and displays the first screen frame; obtains processing information of the first screen frame, wherein the processing information of the first screen frame includes a processing record of the first screen frame during the rendering and drawing phase and a processing record during the synthesis and display phase; if it is determined based on the processing information of the first screen frame that the first screen frame has lost frame, first quality assurance information is sent to a server, wherein the first quality assurance information includes the processing information of the first screen frame and the first state information of the game application in the first scenario. In this way, quality assurance personnel can obtain quality assurance information when the game application loses frame through the server, thereby resolving the problem of frame loss in the game application.

[0080] The following is a schematic diagram of the architecture of a software and hardware system provided by an embodiment of the present application.

[0081] like Figure 4 As shown, the electronic device 100 may include an application layer, a native service layer (native), a hardware abstraction layer (HAL), a kernel layer (kernel) and a hardware layer.

[0082] The application layer may include gaming applications.

[0083] The local service layer can include a graphics interface (including the Open Graphics Library (OPENGL) and the Open Graphics Library for Embedded Systems (OPENGL ES)), a buffer queue management module, a measurement module, and a surface flinger (SF). The buffer queue management module can apply for and manage buffer queues (BufferQueues) in memory. The measurement module can apply for and manage measurement buffer queues (HiviewBufferQueues) in memory.

[0084] The hardware abstraction layer may include the hardware composer (HWC). The kernel layer may include the GPU driver, communication driver, and display driver. The hardware layer may include the GPU, communication module, and display screen.

[0085] The following describes a data processing method provided by an embodiment of the present application in detail in combination with the above-mentioned software and hardware architecture:

[0086] 1. The hardware synthesizer can periodically send an application synchronization signal (app_vsyncsignal) to the game application.

[0087] The application synchronization signal can be used to trigger the game application to render a frame and provide the rendered frame to the SF. The process of the game application rendering a frame and providing the rendered frame to the SF can be referred to in subsequent steps 3 to 13.

[0088] 2. The hardware compositor can periodically send a layer synthesis synchronization signal (sf_vsync signal) to the layer compositor.

[0089] The layer synthesis synchronization signal can be used to trigger the layer synthesizer to synthesize and display a picture frame. The process of the layer synthesizer synthesizing and displaying a picture frame can be referred to the subsequent steps 14 to 19.

[0090] The period of the application synchronization signal (app_vsyncsignal) is the same as the period of the layer synthesis synchronization signal (sf_vsyncsignal).

[0091] 3. After receiving the application synchronization signal, the game application can send a buffer dequeue request to the buffer queue management module.

[0092] The buffer dequeue request is used to request an idle buffer from the buffer queue management module.

[0093] 4. After receiving the buffer dequeue request sent by the game application, the buffer queue management module can allocate idle buffers to the game application.

[0094] In response to the buffer dequeue request, the buffer queue management module sends the address of the idle buffer to the game application.

[0095] 5. When the game application is running, it can send status information to the maintenance module. This status information may include game scene (e.g., the scene of the current screen), game stage (e.g., the parachuting stage), number of players on the screen (e.g., the number of players on the current screen), player status on the screen (e.g., players running), and game item display status (e.g., game item arrival).

[0096] Among them, the maintenance and measurement module can determine whether the current game scene is in a game battle scene based on the status information. If the current game scene is in a game battle scene, the maintenance and measurement module can record the processing information of the picture frame. If the current game scene is not in a game battle scene, the maintenance and measurement module does not record the processing information of the picture frame. Among them, the processing information of the picture frame may include the processing record of the picture frame in the rendering and drawing stage and the processing record in the synthesis and display stage. Among them, the processing record of the picture frame in the rendering and drawing stage may include one or more of the following: the record of buffer dequeueing, the record of rendering and drawing, and the record of buffer enqueuing. The processing record of the picture frame in the synthesis and display stage may include one or more of the following: the record of obtaining the buffer, the record of synthesis and display, and the record of releasing the buffer.

[0097] 6. If the current game scene is in a game battle scene, the maintenance and testing module can send a second dotting request to the graphics interface.

[0098] The second dot request is used to request the graphics interface to obtain the rendering record, which may include one or more of the following: the start time of the frame rendering, the end time of the frame rendering, and the result of the frame rendering (for example, whether the frame is successfully rendered).

[0099] In one possible implementation, the rendering and drawing record may include one or more of the following: the start time of the picture frame rendering, the end time of the picture frame rendering, the start time of the picture frame drawing, the end time of the picture frame drawing, the result of the picture frame rendering (for example, whether the picture frame is rendered successfully), the result of the picture frame drawing (for example, whether the picture frame is drawn successfully), etc.

[0100] 7. If the current game scene is in a game battle scene, the maintenance and testing module can send a first dotting request to the buffer queue management module.

[0101] The first dot request is used to request the buffer queue management module to obtain buffer dequeue records, and / or buffer enqueue records, and / or buffer acquisition records, and / or buffer release records.

[0102] Among them, the record of buffer dequeueing may include one or more of the following: the moment when the game application receives the application synchronization signal, the moment when the game application sends the buffer dequeue request to the buffer queue management module, the moment when the buffer queue management module sends the address of the buffer to the game application, and the result of the buffer queue management module sending the address of the buffer to the game application (for example, whether the buffer queue management module sends the address of the buffer to the game application successfully, whether it times out, whether it is an exception, etc.).

[0103] Among them, the record of buffer enqueuing can include one or more of the following: the start time of the buffer joining the buffer queue, the end time of the buffer joining the buffer queue, and the layer information of the picture frame in the buffer (for example, the layer on which the picture frame is located).

[0104] Among them, the record of obtaining the buffer may include one or more of the following: the moment when SF receives the layer synthesis synchronization signal, the moment when SF sends a buffer acquisition request to the buffer queue management module, the moment when the buffer queue management module sends the address corresponding to the buffer in the buffer queue that stores the rendered and drawn picture frame to SF, and the result of the buffer queue management module sending the address corresponding to the buffer in the buffer queue that stores the rendered and drawn picture frame to SF (for example, whether the buffer queue management module sends the address corresponding to the buffer in the buffer queue that stores the rendered and drawn picture frame to SF successfully, whether it times out, whether it is abnormal, etc.).

[0105] Among them, the record of releasing the buffer may include one or more of the following: the moment when SF sends a buffer release request to the buffer queue management module, the moment when the buffer queue management module releases the buffer storing the rendered and drawn picture frame, and the result of the buffer queue management module releasing the buffer storing the rendered and drawn picture frame (for example, whether the buffer queue management module releases the buffer storing the rendered and drawn picture frame successfully, whether it times out, whether it is abnormal, etc.).

[0106] 8. If the current game scene is in a game battle scene, the maintenance module can send a third dotting request to the layer compositor.

[0107] The third dot request is used to request the layer compositor to obtain a composite display record. The composite display record may include one or more of the following: the start time of the composite display of the frame, the end time of the composite display of the frame, and the result of the composite display of the frame (for example, whether the composite display of the frame was successful).

[0108] In one possible implementation, the record of synthesis and display may include one or more of the following: the start time of picture frame synthesis, the end time of picture frame synthesis, the start time of picture frame display, the end time of picture frame display, the result of picture frame synthesis (for example, whether the picture frame is synthesized successfully), the result of picture frame display (for example, whether the picture frame is displayed successfully), etc.

[0109] 9. After receiving the application synchronization signal, the game application can send rendering instructions to the graphics interface.

[0110] 10. After receiving the rendering instruction sent by the game application, the graphics interface can send the rendering instruction to the GPU driver.

[0111] The GPU driver is used to convert the rendering instruction into an instruction recognizable by the GPU.

[0112] 11. After converting the rendering instruction into an instruction recognizable by the GPU, the GPU driver can send the rendering instruction to the GPU.

[0113] The rendering instruction is used to instruct the GPU to render and draw a picture frame, and store the rendered picture frame in the above buffer.

[0114] In one possible implementation, the CPU and GPU may work together to render a drawing frame.

[0115] 12. After sending the rendering command to the graphics interface, the game application can send a frame sending command to the graphics interface. The frame sending command carries the address of the buffer.

[0116] 13. After receiving the frame sending instruction sent by the game application, the graphics interface can add the buffer containing the picture frame to the buffer queue.

[0117] 14. After receiving the layer synthesis synchronization signal, the layer synthesizer can send a buffer acquisition request to the buffer queue management module.

[0118] The buffer zone acquisition request is used to request the buffer zone queue management module to acquire the picture frame.

[0119] 15. After receiving the buffer acquisition request sent by the layer compositor, the buffer queue management module can provide the buffer containing the picture frame to the layer compositor (SF).

[0120] In response to the buffer acquisition request, the buffer queue management module sends the address corresponding to the buffer in the buffer queue storing the rendered frame to the SF.

[0121] The layer synthesizer can obtain the picture frames from the buffer storing the picture frames and perform layer synthesis on the picture frames.

[0122] In one possible implementation, the layer compositor can also obtain the remaining layer information (for example, various components displayed and interacted on the electronic device 100, such as buttons, text boxes, drop-down menus, check boxes, etc.) and synthesize the picture frame and the remaining layer information.

[0123] 16. After the layer synthesizer performs layer synthesis on the picture frame, it can send the layer synthesized picture frame to the hardware synthesizer.

[0124] 17. After receiving the picture frame sent by the layer synthesizer, the hardware synthesizer can synthesize the picture frame after layer synthesis into a frame of interface including the picture frame, and send this frame of interface including the picture frame to the display driver.

[0125] 18. After receiving the interface including the picture frame, the display driver can drive the display screen to display the interface including the picture frame.

[0126] For example, the above Figure 3A The game lobby interface 310 shown, or the above Figure 3B The game battle interface 320 is shown.

[0127] 19. After compositing a frame, the layer compositor may send a buffer release request to the buffer queue management module. The buffer release request is used to instruct the buffer queue management module to release the buffer storing the frame.

[0128] After receiving the buffer release request, the buffer queue management module may clear the data in the buffer storing the picture frame and delete it from the buffer queue.

[0129] 20. After receiving the second dot request sent by the maintenance and testing module, the graphics interface may send the rendered record to the maintenance and testing module.

[0130] 21. After receiving the first dot request sent by the maintenance and measurement module, the buffer queue management module can send buffer dequeuing records, and / or buffer enqueuing records, and / or obtain buffer records, and / or release buffer records to the maintenance and measurement module.

[0131] 22. After receiving the request to obtain the third dot sent by the maintenance and testing module, the layer synthesizer can send the record of the synthesis and display to the maintenance and testing module.

[0132] 23. When the maintenance test module determines that a frame loss situation occurs based on the processing information of the picture frame in the maintenance test buffer queue, the maintenance test information of the picture frame can be sent to the communication driver.

[0133] The frame's maintenance information may include frame processing information and game application status information in a game scenario. Communication drivers may include Wi-Fi drivers, Bluetooth drivers, and the like. The communication driver instructs the communication module to convert the maintenance information into information recognizable by the communication module.

[0134] When the game application sends a buffer dequeue request to the buffer queue management module, the maintenance and testing module requests a maintenance and testing buffer from the memory. After obtaining the processing information of the picture frame, the maintenance and testing module stores the processing information of the picture frame in the maintenance and testing buffer and places the maintenance and testing buffer in the maintenance and testing buffer queue.

[0135] 24. After the communication driver converts the maintenance and measurement information sent by the maintenance and measurement module into maintenance and measurement information recognizable by the communication module, the maintenance and measurement information can be sent to the communication module.

[0136] Among them, the communication module may include a Wi-Fi module, a Bluetooth module, etc.

[0137] If the maintenance monitoring module determines that frame loss has occurred based on the maintenance monitoring information in the maintenance monitoring buffer queue, it sends the maintenance monitoring information to the Wi-Fi driver, which can then send the maintenance monitoring information to the Wi-Fi module. If the maintenance monitoring module determines that frame loss has occurred based on the maintenance monitoring information in the maintenance monitoring buffer queue, it sends the maintenance monitoring information to the Bluetooth driver, which can then send the maintenance monitoring information to the Bluetooth module.

[0138] The communication module may send the maintenance information to the server 200 .

[0139] In a possible implementation, after determining that a frame loss occurs and obtaining the measurement information of the frame, the electronic device 100 may release the measurement buffer storing the measurement information of the frame.

[0140] In an embodiment of the present application, the maintenance and testing module is not limited to detecting frame loss and collecting maintenance and testing information on the screen frames of the above-mentioned game application, but can also detect screen frames in other frame loss scenarios and collect maintenance and testing information. For example, the maintenance and testing module can also detect frame loss on the dynamic effect screen when the application is started, and collect maintenance and testing information, and so on.

[0141] The following is a schematic diagram of processing information for recording picture frames provided in an embodiment of the present application.

[0142] like Figure 5 As shown, the software system of the electronic device 100 may include a game application, a graphics interface, a buffer queue management module, and a layer synthesizer (SF). The following describes in detail the content of the processing information of the picture frame, wherein the processing information of the picture frame may include one or more of the following: dot data 1, dot data 2, dot data 3, dot data 4, dot data 5, dot data 6:

[0143] S501. A game application may receive an application synchronization signal (app_vsync signal).

[0144] The game application may receive an application synchronization signal periodically sent by the hardware synthesizer.

[0145] S502: The game application may send a buffer dequeue (dequeuebuffer) request to the buffer queue management module.

[0146] The game application waits for the arrival of the application synchronization signal. When the game application receives the application synchronization signal, it can send a buffer dequeue request to the buffer queue management module and send the arrival time of the application synchronization signal to the buffer queue management module.

[0147] S503: After receiving the buffer dequeue request sent by the game application, the buffer queue management module may allocate a free buffer (return free buffer) to the game application.

[0148] When allocating an idle buffer to a game application, the buffer queue management module may record RBI data 1. The RBI data 1 may include a record of buffer dequeueing. The content of the RBI data 1 may refer to the content of the buffer dequeueing record, which will not be described in detail here.

[0149] S504: After receiving the idle buffer allocated by the buffer queue management module, the game application may send a rendering instruction to the graphics interface.

[0150] S505 , after receiving the rendering instruction sent by the game application, the graphics interface may render and draw a picture frame, and store the rendered picture frame in the buffer.

[0151] When the graphics interface renders a frame, it can record dotting data 2. The dotting data 2 can be a record of rendering and drawing. The content of the dotting data 2 can refer to the content of the above-mentioned rendering and drawing record, which will not be described in detail here.

[0152] S506: After sending the rendering instruction, the game application may send a frame sending instruction to the graphics interface.

[0153] S507 : After receiving the frame sending instruction sent by the game application, the graphics interface may send a buffer queue (queuebuffer) request to the buffer queue management module.

[0154] In response to the buffer enqueue request, the buffer queue management module may add the buffer storing the image frame to the buffer queue. After adding the buffer storing the image frame to the buffer queue, the buffer queue management module may record dot data 3. The dot data 3 may include a record of the buffer enqueue. The content of dot data 3 can refer to the content of the buffer enqueue record described above and is not further described here.

[0155] S508: The layer synthesizer may receive a layer synthesis synchronization signal.

[0156] The layer synthesizer can receive the layer synthesis synchronization signal periodically sent by the hardware synthesizer.

[0157] S509: The layer compositor may send an acquirebuffer request to the buffer queue management module.

[0158] Among them, the layer synthesizer can wait for the arrival of the layer synthesis synchronization signal. When the layer synthesizer receives the layer synthesis synchronization signal, it can send a buffer acquisition request to the buffer queue management module and send the arrival time of the layer synthesis synchronization signal to the buffer queue management module.

[0159] S510 , after receiving the buffer acquisition request sent by the layer compositor, the buffer queue management module may provide the buffer storing the picture frame (buffer that filled data) to the layer compositor.

[0160] When the buffer queue management module provides the buffer containing the picture frame to the layer compositor, it can record the dotting data 4. The dotting data 4 can include the record of the acquisition buffer. The content of the dotting data 4 can refer to the content of the record of the acquisition buffer, which is not repeated here.

[0161] S511. After obtaining the picture frame from the buffer storing the picture frame, the layer compositor may synthesize the picture frame for display.

[0162] When the layer synthesizer synthesizes and displays the frame, it can record the dotting data 5. The dotting data 5 can include the record of the synthesis and display. The content of the dotting data 5 can refer to the content of the record of the synthesis and display, and will not be repeated here.

[0163] S512: After synthesizing and displaying the picture frame, the layer compositor may send a buffer release (releasebuffer) request to the buffer queue management module.

[0164] In response to the buffer release request, the buffer queue management module may clear the data in the buffer storing the frame, delete the buffer storing the frame from the buffer queue, and record dot data 6. The dot data 6 may include a record of the buffer release. The content of dot data 6 can refer to the content of the buffer release record, and will not be described in detail here.

[0165] In this way, when a frame loss event occurs, maintenance personnel can analyze the cause of the frame loss through the processing information of the picture frame, and then solve the frame loss problem and maintain the game application.

[0166] The following specifically describes the method for putting the processing information of the picture frame into the HiviewBuffer provided in an embodiment of the present application.

[0167] Based on the status information sent by the game application, the maintenance and monitoring module can determine that the current game scene is a game battle scene. When the game application requests an idle buffer, the maintenance and monitoring module can simultaneously request a maintenance and monitoring buffer (HiviewBuffer). This HiviewBuffer can be used to store processing information for the current frame. The HiviewBuffer can include one or more structures, which can be used to store data.

[0168] The electronic device 100 can detect the index of the buffer storing the picture frame and the identifier of the processing information, wherein the index of the buffer is used to indicate the frame number of the picture frame and the identifier of the processing information is used to indicate the structure number of the processing information stored in HiviewBuffer.

[0169] When the game application in the electronic device 100 receives the application synchronization signal (app_vsyncsignal), the electronic device 100 can perform the operations of dequeuing the buffer, rendering the drawing picture frame, and enqueuing the buffer. Among them, the operation of dequeuing the buffer can refer to the above steps S501 to S503, and collect the dotting data 1, and determine that the identifier of the dotting data 1 is "1". The operation of rendering the drawing picture frame can refer to the above steps S504 to S505, and collect the dotting data 2, and determine that the identifier of the dotting data 2 is "2". The operation of enqueuing the buffer can refer to the above steps S506 to S507, and collect the dotting data 3, and determine that the identifier of the dotting data 3 is "3". Among them, the electronic device 100 can detect the index of the buffer storing the picture frame, so as to determine which frame the rendered and drawn picture frame is. Among them, the identifier of the dotting data 1 is "1" and is used to indicate that the dotting data 1 is placed in the first structure in the corresponding HiviewBuffer. The identifier of dot data 2 is "2", which indicates that dot data 2 is put into the second structure in the corresponding HiviewBuffer. The identifier of dot data 3 is "3", which indicates that dot data 3 is put into the third structure in the corresponding HiviewBuffer.

[0170] When the layer synthesizer in the electronic device 100 receives the layer synthesis synchronization signal (sf_vsyncsignal), the electronic device 100 can perform the operations of obtaining the buffer, synthesizing the display frame, and releasing the buffer. Among them, the operation of obtaining the buffer can refer to the above steps S508 to S510, and collect the dotting data 4, and determine that the identifier of the dotting data 4 is "4". The operation of synthesizing the display frame can refer to the above step S511, and collect the dotting data 5, and determine that the identifier of the dotting data 5 is "5". The operation of releasing the buffer can refer to the above step S512, and collect the dotting data 6, and determine that the identifier of the dotting data 6 is "6". Among them, the electronic device 100 can detect the index of the buffer storing the picture frame to determine which frame the synthesized display frame is. Among them, the identifier of the dotting data 4 is "4" to indicate that the dotting data 4 is placed in the fourth structure in the corresponding HiviewBuffer. The identifier of the dotting data 5 is "5" to indicate that the dotting data 5 is placed in the fifth structure in the corresponding HiviewBuffer. The identifier of dot data 6 is “6”, which is used to indicate that dot data 6 is placed into the sixth structure in the corresponding HiviewBuffer.

[0171] The electronic device 100 may match the processing information of the same picture frame based on the index of the buffer storing the picture frame in the rendering and drawing stage and the index of the buffer storing the picture frame in the synthesis and display stage.

[0172] For example, Figure 6A As shown, when the game application in the electronic device 100 receives an application synchronization signal, it can dequeue the buffer of the Nth frame, render and draw the frame, and enqueue the buffer. When the layer synthesizer in the electronic device 100 receives a layer synthesis synchronization signal, it can obtain the buffer of the N-1th frame, synthesize and display the frame, and release the buffer. When the game application in the electronic device 100 receives the next application synchronization signal, it can dequeue the buffer of the N+1th frame, render and draw the frame, and enqueue the buffer. When the layer synthesizer in the electronic device 100 receives the next layer synthesis synchronization signal, it can obtain the buffer of the Nth frame, synthesize and display the frame, and release the buffer.

[0173] For example, the electronic device 100 may place the dot data 1 into the Nth frame processed in the application synchronization period, the identifier of the dot data 1 is "1", the identifier of the dot data 2 is "2", and the identifier of the dot data 3 is "3". Figure 6B In the first structure of the corresponding dimension buffer _N shown, put the dot data 2 into Figure 6B In the second structure of the corresponding dimension buffer _N shown in the figure, the dot data 3 is placed in Figure 6B The electronic device 100 can put the dot data 4 into the third structure in the corresponding dimension buffer _N based on the fact that the picture frame processed in the layer synthesis synchronization period is the Nth frame, the identifier of the dot data 4 is "4", the identifier of the dot data 5 is "5", and the identifier of the dot data 6 is "6". Figure 6B In the fourth structure of the corresponding dimension buffer _N shown in the figure, the dot data 5 is placed in Figure 6B In the fifth structure of the corresponding dimension buffer _N shown in FIG, the dot data 6 is placed Figure 6B In the sixth structure in the corresponding dimension buffer _N shown.

[0174] It should be understood that the application synchronization signal received by the game application in the electronic device 100 and the layer synthesis synchronization signal received by the layer synthesizer may be received at the same time or at different times.

[0175] It should be understood that within the same synchronization cycle, the picture frames processed by the electronic device 100 in the application synchronization cycle and the picture frames processed by the electronic device 100 in the layer synthesis synchronization cycle may not be the same picture frames. Therefore, it is necessary to match the processing information of the picture frames processed in the application synchronization cycle and the processing information of the picture frames processed in the layer synthesis synchronization cycle, and put them into the same HiviewBuffer to ensure that one HiviewBuffer collects the processing information of the same picture frame.

[0176] It should be understood that the state information of a plurality of consecutive picture frames may be the same. Therefore, the same state information may include a plurality of picture frames and HiviewBuffers used by the plurality of picture frames to store processing information.

[0177] In a possible implementation, the state information of the picture frame may also be stored in the dimensioning buffer corresponding to the picture frame. For example, the state information of the Nth picture frame is stored in the seventh structure in the dimensioning buffer_N.

[0178] The following describes in detail a frame loss detection method provided by an embodiment of the present application.

[0179] The electronic device 100 may obtain processing information of multiple consecutive frames. The HiviewBuffer may be used to store the processing information of the frames. The electronic device 100 may determine whether a frame has been dropped and the number of dropped frames based on the processing information of the current frame and the processing information of the previous frame, where the previous frame is the frame before the current frame.

[0180] When the previous frame is continuously displayed on the display screen for multiple synchronization periods, the current frame is dropped. For example, if the difference between the time when the buffer storing the rendered previous frame is released and the time when the buffer storing the rendered current frame is released is greater than the period of the synchronization signal, it can be determined that the current first frame has been dropped.

[0181] For example, the calculation formula may be as follows:

[0182]

[0183] Where skip is the number of dropped frames. releaseBufferTime is the time when the current frame is released. lastReleaseBufferTime is the time when the previous frame is released. app_vsync_cycle is used to indicate the cycle of the application synchronization signal.

[0184] When skip is less than 0, it indicates that no frame loss event has occurred. When skip is greater than 0, it indicates that a frame loss event has occurred and the number of lost frames is [skip]. Where [skip] is the largest integer less than skip. For example, if skip = 3.8, then [skip] = 3; if skip = -2.4, then [skip] = -3. In this way, the electronic device 100 can determine whether a frame has been lost and the number of lost frames based on the measurement information of the current frame and the measurement information of the previous frame.

[0185] For example, Figure 7 As shown, the measurement buffer _N is used to store the processing information of the current Nth frame acquired by the electronic device 100, and the measurement buffer _N-1 is used to store the processing information of the N-1th frame acquired by the electronic device 100. The electronic device 100 can determine whether frame loss has occurred and the number of lost frames based on the skip value. When skip is greater than 0, it indicates that a frame loss event has occurred and the number of lost frames is [skip].

[0186] When electronic device 100 determines that a frame loss has occurred in the current frame based on the processing information of the current frame and the processing information of the previous frame, it can send first quality measurement information to server 200. The quality measurement information is stored in a quality measurement buffer. After sending the quality measurement information to server 200, electronic device 100 can release the quality measurement buffer.

[0187] It should be understood that in the embodiment of the present application, it is only necessary to record the processing information of the current picture frame and the processing information of the previous picture frame to determine whether a frame is lost and the number of frames lost. Therefore, the HiviewBufferQueue only needs to record the processing information of the current picture frame and the processing information of the previous picture frame. For example, the HiviewBufferQueue records the processing information of the N-1th frame and the processing information of the Nth frame. When the processing information of the N+1th frame begins to be recorded, the processing information of the N-1th frame can be overwritten. In this way, there is no need to apply for a new Hiviewbuffer in the HiviewBufferQueue for the processing information of each frame, thereby saving resources.

[0188] It should be understood that in the embodiment of the present application, the picture frame rendering and drawing stage and the picture frame synthesis and display stage are carried out simultaneously. Therefore, there may be a buffer accumulation situation. For example, when the picture frame synthesis and display stage records the measurement information of the Nth frame, the picture frame rendering and drawing stage has begun to record the processing information of the N+Xth (X=1, 2, 3, etc.) frame. The electronic device 100 can set the number of Hiviewbuffers in the HiviewBufferQueue based on the storage situation of the memory and when the buffer is met. In this way, it can be ensured that the processing information of the current picture frame and the processing information of the previous frame are recorded.

[0189] It should be understood that the methods described in the embodiments of the present application can also be applied to other scenarios where the frame loss experience is sensitive. These scenarios can be used to indicate situations where frame loss may impact the user experience during video playback or real-time video applications. For example, in real-time video scenarios such as video conferencing, video calls, or live broadcasts, frame loss can cause image freezes, delays, or discontinuities, or the frame rate of application animations can drop when the application starts, resulting in a poor user experience.

[0190] The following is a flow chart of a data processing method provided in an embodiment of the present application.

[0191] Figure 8 A flow chart of a data processing method provided in an embodiment of the present application is shown. The data processing method is applied to an electronic device 100, which runs a game application, a graphics interface, a buffer queue management module, a layer synthesizer SF, and a maintenance and testing module. The specific steps are as follows:

[0192] S801: Obtain first state information of a game application in a first scenario, where the first state information includes a type of the first scenario.

[0193] When the electronic device 100 runs a game application, it can obtain first state information of the game application in a first scenario. The first state information may also include the game stage (e.g., parachuting stage), the number of players on the same screen (e.g., the number of players in the current screen), the status of players on the same screen (e.g., players running), and the display status of game props (e.g., game props in position).

[0194] S802: Rendering and drawing a first screen frame of the game application in a first scene.

[0195] The process of the electronic device 100 rendering the frame of the game application in the first scene can refer to the above Figure 4 Steps 3 to 13 are not described here in detail.

[0196] S803: synthesize and display the first picture frame.

[0197] The process of the electronic device 100 synthesizing and displaying the first picture frame can refer to the above Figure 4 Steps 14 to 19 are not described here in detail.

[0198] S804: Acquire processing information of the first frame, wherein the processing information of the first frame includes processing records of the first frame in a rendering and drawing phase and processing records in a synthesis and display phase.

[0199] The process of the electronic device 100 obtaining the processing information of the first picture frame can refer to the above Figure 4 Steps 5 to 22 are not described here in detail.

[0200] S805. If it is determined based on the processing information of the first frame that the first frame has frame loss, send first dimensional measurement information to the server, where the first dimensional measurement information includes the processing information of the first frame and first status information of the game application in the first scenario.

[0201] The electronic device 100 determines that the first frame has lost a frame based on the processing information of the first frame and sends the first dimension information to the server. Figure 4 Steps 23 to 24 are not described here in detail.

[0202] In one possible implementation, the electronic device 100 may obtain processing information for a second frame, which is a frame that is before and adjacent to the first frame. The processing information for the second frame may include the time when a buffer storing the rendered and drawn second frame is released. The electronic device 100 may determine that the first frame has been dropped based on the difference between the time when the buffer storing the rendered and drawn second frame is released and the time when the buffer storing the rendered and drawn first frame is released being greater than the period of the synchronization signal.

[0203] After acquiring the processing information of the first picture frame, the electronic device 100 may store the processing information of the first picture frame in a first dimension buffer.

[0204] After determining that the first picture frame is lost, the electronic device 100 may generate and save the first dimensional measurement information and release the first dimensional measurement buffer after saving the first dimensional measurement information.

[0205] For the parts not described in detail in the embodiments of the present application, reference can be made to the aforementioned embodiments and will not be repeated here.

[0206] In one possible implementation, upon detecting frame loss, the electronic device 100 can send frame loss information to other devices. Maintenance personnel can retrieve the frame loss information from the device storing the information and, based on the information, analyze the cause of the frame loss, thereby resolving the frame loss issue in gaming applications.

[0207] In one possible implementation, upon detecting frame loss, the electronic device 100 can save the frame loss information to a local file. A maintenance technician can retrieve the frame loss information from the local file and, based on the information, analyze the cause of the frame loss, thereby resolving the frame loss issue in the gaming application.

[0208] In one possible implementation, when the electronic device 100 detects that the number of frame drops in a game application exceeds a certain threshold (e.g., 100 frames) within a preset time (e.g., 10 minutes), the electronic device 100 can send all the maintenance information of the lost frames within the preset time to a server or other device. In this way, when abnormal frame drop occurs on the electronic device 100, the maintenance information of the abnormal frame drop can be collected, thereby eliminating the possibility of normal frame drop.

[0209] Through a data processing method provided in the present application, first-dimensional measurement information of the first screen frame of a game application in a first scenario can be collected. The first-dimensional measurement information includes processing information of the first screen frame and first status information of the game application in the first scenario, wherein the processing information of the first screen frame includes processing records of the first screen frame in the rendering and drawing stage and processing records in the synthesis and display stage. In this way, when frame loss occurs in the first screen frame, the measurement personnel can analyze the cause of the frame loss of the first screen frame based on the first-dimensional measurement information of the first screen frame, thereby solving the frame loss problem of the game application.

[0210] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0211] An embodiment of the present application further provides a computer program product. When the computer program product is run on an electronic device, the electronic device can implement the steps in the above-mentioned various method embodiments.

[0212] The present application also provides a chip system, comprising a processor coupled to a memory, the processor executing a computer program stored in the memory to implement the steps of any method embodiment of the present application. The chip system can be a single chip or a chip module composed of multiple chips.

[0213] The term "user interface (UI)" in the specification and drawings of this application refers to the media interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface of an application is a source code written in a specific computer language such as Java and Extensible Markup Language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize, such as pictures, text, buttons and other controls. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, pictures and text. The properties and contents of controls in the interface are defined by tags or nodes, such as XML through <textview> 、 <imgview> 、 <videoview>Nodes such as <head> and <body> are used to specify the controls contained in the interface. A node corresponds to a control or attribute in the interface, and the node is presented as user-visible content after parsing and rendering. In addition, many applications, such as hybrid applications, usually also contain web pages in their interfaces. A web page, also known as a page, can be understood as a special control embedded in the application interface. A web page is a source code written in a specific computer language, such as hypertext markup language (HTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed as user-recognizable content by a browser or a web page display component with similar functions to a browser. The specific content contained in a web page is also defined by tags or nodes in the web page source code, such as HTML through <body>. 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0214] A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operations that uses graphics. It can be an icon, window, control, or other interface element displayed on the display of an electronic device. Controls can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, and other visual interface elements.

[0215] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).

[0216] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0217] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A data processing method, characterized in that: Applied to an electronic device running a game application, the method includes: Acquire first state information of the game application in a first scenario, where the first state information includes a type of the first scenario; Rendering and drawing a first screen frame of the game application in a first scene; synthesizing and displaying the first picture frame; Acquire processing information of the first picture frame, wherein the processing information of the first picture frame includes processing records of the first picture frame in a rendering and drawing phase and processing records in a synthesis and displaying phase; If it is determined based on the processing information of the first frame that the first frame has been lost, first dimension measurement information is sent to the server, wherein the first dimension measurement information includes the processing information of the first frame and the first status information of the game application in the first scenario.

2. The method according to claim 1, characterized in that The electronic device also runs a graphics interface and a buffer queue management module; The rendering and drawing of the first screen frame of the game application in the first scene specifically includes: After receiving the application synchronization signal, the game application sends a buffer dequeue request to the buffer queue management module; In response to the buffer dequeue request, the buffer queue management module sends the address of the buffer to the game application; The game application renders and draws the first screen frame through the graphics interface, and adds the rendered first screen frame to the buffer queue managed by the buffer queue management module through the buffer.

3. The method according to claim 2, characterized in that The electronic device also runs a layer synthesizer SF; The synthesizing and displaying the first picture frame specifically includes: After receiving the layer synthesis synchronization signal, the SF sends a buffer acquisition request to the buffer queue management module; In response to the buffer acquisition request, the buffer queue management module sends the address corresponding to the buffer in the buffer queue storing the rendered first frame to the SF; The SF takes out the first rendered frame from the buffer storing the first rendered frame, and synthesizes the first rendered frame for display; After synthesizing and displaying the rendered first picture frame, the SF sends a buffer release request to the buffer queue management module; In response to the buffer release request, the buffer queue management module releases the buffer storing the rendered first screen frame.

4. The method according to claim 3, characterized in that The period of the application synchronization signal is the same as the period of the layer synthesis synchronization signal.

5. The method according to claim 3, characterized in that The acquiring of the processing information of the first picture frame specifically includes: If the first scene is a designated game scene, processing information of a first screen frame of the game application in the first scene is obtained.

6. The method according to claim 5, characterized in that The electronic device also runs a maintenance and testing module; The obtaining of the first state information of the game application in the first scenario specifically includes: The maintenance and testing module obtains the first status information sent by the game application; If the first scene is a designated game scene, obtaining processing information of a first screen frame of the game application in the first scene specifically includes: If the maintenance module determines that the first scene is a designated game scene based on the first state information, the maintenance module obtains processing information of the first screen frame of the game application in the first scene from the buffer queue management module, the graphics interface and the SF.

7. The method according to claim 6, characterized in that The processing record of the first picture frame in the rendering and drawing stage includes one or more of the following: a record of buffer dequeuing, a record of rendering and drawing, and a record of buffer enqueuing; The processing record of the first picture frame in the synthesis and display stage includes one or more of the following: a record of obtaining a buffer, a record of synthesizing and displaying, and a record of releasing a buffer; If the maintenance and detection module determines that the first scene is a designated game scene based on the first state information, the maintenance and detection module obtains processing information of the first screen frame of the game application in the first scene from the buffer queue management module, the graphics interface, and the SF, specifically including: If the maintenance and detection module determines that the first scene is a designated game scene based on the first state information, the maintenance and detection module sends a first RBI request to the buffer queue management module, sends a second RBI request to the graphics interface, and sends a third RBI request to the SF; In response to the first dotting request, the buffer queue management module sends the record of the buffer being dequeued, and / or the record of the buffer being enqueued, and / or the record of obtaining the buffer, and / or the record of releasing the buffer to the maintenance and testing module; In response to the second dotting request, the graphics interface sends the rendering record to the maintenance and testing module; In response to the third dotting request, the SF sends the record of the synthesized display to the maintenance module.

8. The method according to claim 7, characterized in that The record of the buffer dequeue includes one or more of the following: the moment when the game application receives the application synchronization signal, the moment when the game application sends the buffer dequeue request to the buffer queue management module, the moment when the buffer queue management module sends the buffer address to the game application, and whether the buffer queue management module successfully sends the buffer address to the game application.

9. The method according to claim 7 or 8, characterized in that The rendering record includes one or more of the following: a start time of rendering the first frame, an end time of rendering the first frame, and whether the first frame is successfully rendered.

10. The method according to claim 7 or 8, characterized in that The record of the buffer zone being enqueued includes one or more of the following: the start time of the buffer zone being added to the buffer zone queue, the end time of the buffer zone being added to the buffer zone queue, and the layer information of the first picture frame in the buffer zone.

11. The method according to claim 7 or 8, characterized in that The record of obtaining the buffer includes one or more of the following: the moment when the SF receives the layer synthesis synchronization signal, the moment when the SF sends the buffer acquisition request to the buffer queue management module, the moment when the buffer queue management module sends the address corresponding to the buffer in the buffer queue storing the first screen frame after rendering and drawing to the SF, and whether the buffer queue management module successfully sends the address corresponding to the buffer in the buffer queue storing the first screen frame after rendering and drawing to the SF.

12. The method according to claim 7 or 8, characterized in that The record of synthesis and display includes one or more of the following: the start time of synthesis of the first picture frame, the end time of synthesis of the first picture frame, the start time of display of the first picture frame, the end time of display of the first picture frame, whether the first picture frame is successfully synthesized, and whether the first picture frame is successfully displayed.

13. The method according to claim 7 or 8, characterized in that The record of releasing the buffer includes one or more of the following: the moment when the SF sends the buffer release request to the buffer queue management module, the moment when the buffer queue management module releases the buffer storing the first screen frame after rendering and drawing, and whether the buffer queue management module successfully releases the buffer storing the first screen frame after rendering and drawing.

14. The method according to any one of claims 1 to 8, characterized in that The first status information also includes one or more of the following: game stage, status of players on the same screen, number of players on the same screen, and display of game props.

15. The method according to any one of claims 1 to 8, characterized in that The method further comprises: Obtaining processing information of a second frame, wherein the second frame is a frame that is before and adjacent to the first frame, and the processing information of the second frame includes a processing record of the second frame in a rendering and drawing phase and a processing record of the second frame in a synthesis and display phase; If it is determined based on the processing information of the first picture frame that the first picture frame is lost, the method further includes: Based on the processing information of the first picture frame and the processing information of the second picture frame, it is determined that the first picture frame has been lost.

16. The method according to claim 15, characterized in that The processing information of the second picture frame includes the time when the buffer queue management module releases the buffer storing the rendered second picture frame; The determining, based on the processing information of the first picture frame and the processing information of the second picture frame, that the first picture frame has been lost specifically includes: If the difference between the time when the buffer queue management module releases the buffer storing the second screen frame after rendering and drawing and the time when the buffer queue management module releases the buffer storing the first screen frame after rendering and drawing is greater than the period of the synchronization signal, it is determined that the first screen frame has been lost.

17. The method according to any one of claims 6 to 8, characterized in that The method further comprises: The maintenance and testing module applies for a first maintenance and testing buffer when the game application sends the buffer dequeue request to the buffer queue management module; After acquiring the processing information of the first picture frame, the dimension measurement module stores the processing information of the first picture frame into the first dimension measurement buffer.

18. The method according to claim 17, characterized in that The method further comprises: After determining that the first picture frame has been lost, generating and saving the first dimension measurement information; After saving the first dimension measurement information, the first dimension measurement buffer is released.

19. An electronic device, characterized in that: The electronic device comprises one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer instructions, so that when the one or more processors execute the computer instructions, the electronic device executes the data processing method as described in any one of claims 1 to 18.

20. A computer-readable storage medium, characterized in that The method comprises instructions, which, when executed on a processor of an electronic device, enable the electronic device to execute the data processing method according to any one of claims 1 to 18.

Images