Detection method and device of terminal equipment, terminal equipment and storage medium

Abstract

The application discloses a terminal device detection method and device, a terminal device and a storage medium, and belongs to the technical field of terminal control. The method is applied to a terminal device and comprises the following steps: in the running process of a target application program, an interface rendering interface called by the target application program is listened to; rendering data information transmitted by the listened-to interface rendering interface is acquired; and the reason for the display abnormal state of the terminal device is determined based on the rendering data information, wherein the captured rendering data information comprises not only rendering image data output by a graphics processor but also texture data input to the graphics processor. When the reason for the display abnormal state of the terminal device is detected, more data outside the GPU end can be captured, the detection of the reason for the display abnormal state of the terminal device can be applied to more rendering scenes, the captured data is more comprehensive, and the accuracy of the detection of the terminal device is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal control, and particularly relates to a terminal device detection method and device, a terminal device and a storage medium. BACKGROUND

[0002] With the development of information technology and Internet technology, terminal devices are increasingly widely used in daily life. For example, people can install application programs (Apps) in terminal devices and start the application programs in the terminal devices to watch videos, view news, etc.

[0003] During the use of a terminal device, the terminal device may appear a screen full of random images due to various reasons. When the terminal device appears the screen full of random images, the terminal device needs to be detected. At present, the detection method for the screen full of random images of the terminal device is mostly to capture frame data inside the terminal device by using a frame capture tool, and then analyze problems in a rendering process.

[0004] Based on the above detection method, the captured frame data is usually data on a graphics processing unit (GPU) side, the captured data is not comprehensive enough, and problems of the screen full of random images are usually located on an App side, which reduces the accuracy of detection of the terminal device. SUMMARY

[0005] To solve the problems in the prior art, increase the comprehensiveness of data acquisition, and improve the accuracy of detection of the terminal device, an embodiment of the present application provides a terminal device detection method and device, a terminal device and a storage medium. The technical solution is as follows.

[0006] In one aspect, the present application provides a terminal device detection method applied to a terminal device, and the method comprises the following steps.

[0007] In the running process of a target application program, an interface rendering interface called by the target application program is listened to.

[0008] Rendering data information transmitted by the interface rendering interface is acquired, the rendering data information comprises texture data input to a graphics processing unit and rendering image data output by the graphics processing unit.

[0009] Based on the rendering data information, a reason for a display abnormal state of the terminal device is determined.

[0010] In one aspect, the present application provides a terminal device detection device applied to a terminal device, and the device comprises the following steps.

[0011] The interface monitoring module is configured to monitor an interface rendering interface called by the target application during running of the target application.

[0012] The information obtaining module is configured to obtain rendering data information transmitted by the interface rendering interface, the rendering data information including texture data input to a graphic processor and rendering image data output by the graphic processor.

[0013] The reason determining module is configured to determine a reason for the terminal device to be in the display abnormal state based on the rendering data information.

[0014] In another aspect, the present application provides a terminal device, which comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the detection method of the terminal device according to one aspect.

[0015] In another aspect, the present application provides a computer readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement the detection method of the terminal device according to one aspect.

[0016] In another aspect, the present application provides a computer program product, which, when running on a computer, causes the computer to execute the detection method of the terminal device according to one aspect.

[0017] In another aspect, the present application provides an application publishing platform, which is configured to publish a computer program product, and when the computer program product runs on a computer, causes the computer to execute the detection method of the terminal device according to one aspect.

[0018] The technical scheme provided by the present application embodiment has at least the following beneficial effects:

[0019] This application monitors the UI rendering interface called by the target application during its operation; acquires the rendering data transmitted by the monitored UI rendering interface; and determines the cause of the abnormal display state of the terminal device based on the rendering data. The captured rendering data includes not only the rendered image data output by the graphics processor but also the texture data input to the graphics processor. When detecting the cause of the abnormal display state of the terminal device, more data beyond the GPU can be captured, making the detection of the cause of the abnormal display state of the terminal device applicable to more rendering scenarios. The captured data is more comprehensive, improving the accuracy of the detection of the terminal device. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the interface of a terminal device according to an exemplary embodiment of this application;

[0022] Figure 2 This is a flowchart of a method for detecting a terminal device provided in an exemplary embodiment of this application;

[0023] Figure 3 This is a flowchart of a method for detecting a terminal device provided in an exemplary embodiment of this application;

[0024] Figure 4 This is a flowchart of a method for detecting a terminal device provided in an exemplary embodiment of this application;

[0025] Figure 5 This is a schematic diagram of the structure of an internal graphics library of a terminal device according to an exemplary embodiment of this application;

[0026] Figure 6 This is a flowchart of a method for detecting a terminal device according to an exemplary embodiment of this application;

[0027] Figure 7 This is a structural block diagram of a detection device for a terminal device provided in an exemplary embodiment of this application;

[0028] Figure 8 This is a schematic diagram of the structure of a terminal device provided in one embodiment of this application. Detailed Implementation

[0029] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is made with reference to the accompanying drawings in which like reference numerals refer to like elements, unless the context of use indicates otherwise. The following description of exemplary embodiments is not representative of all embodiments consistent with the present application. Rather, it is merely an example of apparatus and methods consistent with some aspects of the present application as detailed in the appended claims.

[0030] The scheme provided in the present application can be used in the scene where the terminal device appears a screen full of Chinese characters and needs to be detected in daily life. In order to facilitate understanding, the following first introduces some terms and application architecture related to the embodiments of the present application.

[0031] adb is the full name of Android Debug Bridge, which means Android Debug Bridge. It plays the role of a debugging bridge in the Android system. Android programs can be debugged through adb commands, which can be regarded as a kind of debug tool. adb is equivalent to a program installed on a client-server end. adb can communicate in the form of a listening port.

[0032] Common adb commands:

[0033] View device adb devices; this command is to view the currently connected devices, and the Android devices or emulators connected to the computer will be listed and displayed.

[0034] Install software adb install; this command installs the specified apk file to the device.

[0035] Uninstall software adb uninstall adb uninstall-k; if the-k parameter is added, the software is uninstalled but the configuration and cache files are retained.

[0036] Log in to the device shell adb shell; this command logs in to the device shell.

[0037] Send files from the computer to the device adb push; the push command can copy files or folders on the local computer to the device (mobile phone).

[0038] Download files from the device to the computer adb pull; the pull command can copy files or folders on the device (mobile phone) to the local computer.

[0039] Display help information adb help; this command will display help information.

[0040] Please refer to Figure 1This illustrates a schematic diagram of the interface of a terminal device according to an exemplary embodiment of this application. Figure 1 As shown, it includes a terminal device 110 and an application interface 120.

[0041] Optionally, the terminal device 110 can be a terminal device equipped with a display screen, such as a mobile phone, tablet computer, MP4 (Moving Picture Experts Group Audio Layer IV) player, e-book reader, smart TV, desktop computer, laptop computer, or other mobile terminal. Alternatively, the terminal device can also be a smart wearable device such as a smartwatch, smart glasses, or smart display.

[0042] Users can trigger corresponding applications on their terminal devices, causing the devices to display the application's interface. During the display process, if there are problems with the rendering process or the terminal's driver layer, screen flickering or glitches can occur, similar to... Figure 1 The application interface shown is 120. At this point, it is necessary to locate the cause of the screen flickering phenomenon.

[0043] Currently, terminal devices can be equipped with frame capture tools such as Renderdoc and Gapid to capture frame data and analyze problems that occur during the rendering process. However, these tools typically capture GPU-side data and cannot capture data uploaded from the CPU. Therefore, when detecting and locating the cause of display anomalies on terminal devices, the data uploaded from the CPU cannot be verified. Screen flickering issues are usually located on the app side. If the problem is in the driver layer inside the terminal device, it cannot be located in a timely manner and requires the chip manufacturer to intervene in the analysis, reducing the efficiency of detection. Furthermore, the accuracy of these frame capture tools in detecting terminal devices is also relatively low.

[0044] To address the problems existing in the aforementioned related technologies, increase the comprehensiveness of data acquisition, and improve the accuracy of terminal device detection, this application provides a terminal device detection method. The method captures rendering data information transmitted through the interface rendering interface, including not only the texture data input to the graphics processor but also the rendering image data output by the graphics processor. This makes the captured data more comprehensive and the accuracy of terminal device detection higher.

[0045] Please refer to Figure 2 The diagram illustrates a flowchart of a method for detecting a terminal device according to an exemplary embodiment of this application. This method for detecting a terminal device can be used as follows: Figure 1The terminal device shown in the figure. As Figure 2 The detection method of the terminal device shown in the figure can include the following steps:

[0046] Step 201, during the running of the target application, the interface rendering interface called by the target application is listened to.

[0047] The target application can be any application installed in the terminal device. Optionally, when the terminal device needs to listen to a certain application, the application is the target application, and the interface rendering interface called by the target application is listened to during the running of the target application. The target application can run in the foreground or in the background.

[0048] Optionally, the interface rendering interface is an interface called by the terminal device when displaying the interface of the target application. For example, an interface for transmitting rendering data of the interface and transmitting texture data.

[0049] Step 202, obtaining the rendering data information transmitted by the interface rendering interface, the rendering data information including the texture data input to the graphics processor and the rendering image data output by the graphics processor, wherein the rendering data information is used to detect the cause of the display abnormal state of the terminal device.

[0050] Optionally, the terminal device can capture the data transmitted by the interface to obtain the rendering data information transmitted by the interface rendering interface. In this application, the rendering data information captured by the terminal device can include the texture data input from the central processing unit (CPU) of the terminal device to the graphics processor (GPU) and the rendering image data output by the graphics processor.

[0051] Step 203, determining the cause of the display abnormal state of the terminal device based on the rendering data information.

[0052] That is, the obtained rendering data is used to detect the cause of the display abnormal state of the terminal device. The display abnormal state can be the phenomenon of the display screen of the terminal device appearing as a flower screen (such as the interface shown in the above Figure 1 The terminal device can detect itself based on the rendering data information, or send the rendering data information to a server to which the terminal device establishes a network connection, and the server detects the cause of the display abnormal state of the terminal device.

[0053] In conclusion, the application monitors the interface rendering interface called by the target application during the running of the target application, acquires rendering data information transmitted by the monitored interface rendering interface, determines the cause of the display abnormal state of the terminal device based on the rendering data information, wherein the captured rendering data information includes not only the rendering image data output by the graphic processor but also the texture data input to the graphic processor, more data outside the GPU can be captured when detecting the cause of the display abnormal state of the terminal device, the detection of the cause of the display abnormal state of the terminal device can be applied to more rendering scenes, the captured data is more comprehensive, and the accuracy of the detection of the terminal device is improved.

[0054] In a possible implementation manner, the terminal device sends the acquired rendering data information to a server to which the terminal device establishes a communication connection, and the server detects the cause of the display abnormal state of the terminal device.

[0055] Please refer to Figure 3 which shows a method flowchart of a terminal device detection method provided by an example embodiment of the application, and the terminal device detection method can be used in a server that establishes a communication connection with the terminal device. As shown in Figure 3 , the terminal device detection method can include the following steps:

[0056] Step 301: A data monitoring command is sent to the terminal device, and the data monitoring command is used to instruct the terminal device to monitor the interface rendering interface called by the target application during the running of the target application, and acquire rendering data information transmitted by the monitored interface rendering interface.

[0057] In the example embodiment of the application, the server can be a server, a server cluster composed of several servers, a virtualization platform, or a cloud computing service center.

[0058] Optionally, the server and the terminal device can be connected through a communication network, and the information transmission and reception between the two can be realized through a network protocol, such as polling, long polling, and long connection.

[0059] Optionally, the above communication network connection uses standard communication techniques and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a mobile, wired or wireless network, a private network or any combination of virtual private networks). In some embodiments, techniques and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged over the network. In addition, all or some links can be encrypted using conventional encryption techniques such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. In other embodiments, custom and / or dedicated data communication techniques can be used instead of or in addition to the above data communication techniques.

[0060] In the present application, a data monitoring command is sent to the terminal device by the server, which instructs the terminal device to monitor the interface rendering interface called by the target application program during the running process of the target application program, and to capture the rendering data information transmitted by the monitored interface rendering interface. Optionally, the data monitoring command can be an adb command. That is, the terminal device is notified to monitor the target application program and capture the rendering data information transmitted by the monitored interface rendering interface by using the adb command between the server and the terminal device. The target application program and the interface rendering interface can refer to the description in step 201 above, which will not be repeated here.

[0061] Step 302, receiving the rendering data information sent by the terminal device, the rendering data information including the texture data input to the graphic processor and the rendering image data output by the graphic processor.

[0062] Optionally, the terminal device sends the captured rendering data information to the server through the above communication network connection, and correspondingly, the server can receive the rendering data information returned by the terminal device through the communication network connection. The rendering data information captured by the terminal device can include the texture data input from the CPU end of the terminal device to the GPU end and the rendering image data output by the GPU end.

[0063] In step 303, the reason for the display abnormal state of the terminal device is detected according to the rendering data information.

[0064] Optionally, the server detects the reason for the display abnormal state of the terminal device according to the obtained rendering data information, so as to obtain the reason for the display abnormal state of the terminal device.

[0065] In summary, the server sends a data monitoring command to the terminal device, so that the terminal device monitors the interface rendering interface called by the target application program in the running process of the target application program, and obtains the rendering data information transmitted by the interface rendering interface. The server receives the rendering data information sent by the terminal device, and detects the reason for the display abnormal state of the terminal device according to the rendering data information. The rendering data information captured by the terminal device includes not only the rendering image data output by the graphics processor, but also the texture data input to the graphics processor. When detecting the reason for the display abnormal state of the terminal device, the server can use more data outside the GPU end captured by the terminal device, so that the data used in the detection is more comprehensive, and the accuracy of the detection of the terminal device is improved.

[0066] The above Figure 2 and Figure 3 The scheme shown in the above Figure 2 and Figure 3 The scheme is described below with respect to the interaction mode between the terminal device and the server.

[0067] Please refer to Figure 4 , which shows a method flowchart of a terminal device detection method according to an example embodiment of the present application. The terminal device detection method can be used in a system composed of a terminal device and a server, and the terminal device and the server perform respective steps. As shown in Figure 4 , the terminal device detection method can include the following steps:

[0068] In step 401, the server sends a data monitoring command to the terminal device.

[0069] The data monitoring command is used to instruct the terminal device to monitor the interface rendering interface called by the target application program in the running process of the target application program, and to capture the rendering data information transmitted by the interface rendering interface.

[0070] The server and the terminal device can establish a communication network connection, and the server and the terminal device adopt a data monitoring command sending mode, so that the terminal device monitors the target application program. For example, the data monitoring command can be in the form of an adb command, that is, the server sends an adb command to the terminal device, so that the terminal device monitors the target application program according to the adb command.

[0071] In step 402, the terminal device receives the data monitoring command sent by the server.

[0072] Correspondingly, the terminal device can receive the data monitoring command sent by the server through the communication network connection. The data monitoring command is an adb command, and the terminal device receives the adb command sent by the server.

[0073] In one possible implementation manner, before this step, the terminal device can obtain an interface image of the terminal device, and send the interface image to the server, so that the server sends the data monitoring command when the interface image of the terminal device is a target scene type. Correspondingly, the server can perform the step of receiving the interface image sent by the terminal device, and determining the scene type of the interface image according to the interface image. The target scene type is the type of display abnormality of the terminal device.

[0074] For example, the terminal device can automatically obtain the interface image of the terminal device, and send the obtained interface image to the server through the above communication network connection. The server identifies the interface image to determine the scene type of the interface image, and sends the data monitoring command to the terminal device when the interface image belongs to the target scene type. The target scene type is the type of the terminal device with a screen display. For example, the interface image sent by the terminal device to the server contains a screen display in a certain region. The server can determine that the interface image belongs to the target scene type by identification, and sends the adb command to the terminal device.

[0075] Alternatively, the terminal device can also have an image recognition function. Before this step, the terminal device can also obtain an interface image of the terminal device, determine the scene type of the interface image according to the interface image, and send the scene type to the server, so that the server sends the data monitoring command when the interface image of the terminal device is a target scene type. Correspondingly, the server can receive the interface scene type sent by the terminal device. That is, the terminal device sends the scene type of the current interface image to the server. After the server receives the scene type, when the interface image belongs to the target scene type, the server sends the adb command to the terminal device. That is, in the two modes, the above step 401 can be: when the interface scene type is the target scene type, send the data monitoring command to the terminal device.

[0076] Optionally, the target scene type includes any one of a black screen scene, a region garbled scene, or a full garbled scene.

[0077] In step 403, the terminal device determines the target application program according to the data monitoring command.

[0078] Optionally, the data monitoring command includes package name information of the target application program to be monitored. The terminal device can obtain the package name information set in the data monitoring command according to the received data monitoring command, so as to determine the target application program to be monitored.

[0079] In step 404, the terminal device monitors the interface rendering interface called by the target application program in the running process of the target application program.

[0080] Optionally, the terminal device monitors the interface rendering interface called by the target application program in the running process of the target application program. The target application program can be any application program installed in the terminal device, and the target application program can run in the foreground or run in the background.

[0081] Optionally, in the present application, the terminal device can monitor the target application program based on the graphics library in the terminal device. For example, the terminal device can monitor the interface rendering interface called by the target application program through the interface monitoring module in the graphics library. The graphics library can be an intermediate layer embedded between the App and the underlying GL library, which can intercept the gl instruction information transmitted from the App to the underlying GL library, so as to monitor the App.

[0082] Optionally, the interface rendering interface includes at least two of a rendering instruction interface, a texture data uploading interface, and a frame buffer interface; and the rendering data information can include texture data input to the graphics processor, rendering image data output by the graphics processor, viewport parameter information, and frame buffer parameter information. That is, the terminal device can monitor the rendering instruction interface, the texture data uploading interface, and the frame buffer interface through the interface monitoring module in the graphics library. Optionally, the interfaces that can be monitored by the interface monitoring module can be set by the developer or the operation and maintenance personnel.

[0083] Please refer to Figure 5 which shows a structure diagram of a graphics library in a terminal device according to an example embodiment of the present application. As shown in Figure 5As shown, the application layer 501, the intermediate layer 502, and the bottom layer 503 are included. The graphic library is embedded in the intermediate layer 502, and the gl instruction input by the application layer 501 can be intercepted. The graphic library listens to the App through the interface listening module, listens to the graphic instruction interface, listens to the texture data upload interface, listens to the frame buffer interface, and performs the data saving operation, and the rendering data information transmitted by the interface rendering interface is captured through the hook mode.

[0084] In step 405, the terminal device acquires the rendering data information transmitted by the interface rendering interface, and the rendering data information includes the texture data input to the graphic processor and the rendering image data output by the graphic processor.

[0085] The rendering data information is used to detect the reason for the abnormal display state of the terminal device.

[0086] Optionally, the terminal device captures the rendering data information transmitted by the interface rendering interface through the hook mode. That is, the terminal device captures the data transmitted by the interface through the graphic library in the active hook mode, and the obtained rendering data information includes the texture data input to the graphic processor and the rendering image data output by the graphic processor. For example, the terminal device captures the input information transmitted by the rendering instruction interface and the drawing instruction transmitted by the graphic API (Application Program Interface, application program interface) of the terminal device through the active hook mode, captures the texture data transmitted by the texture data upload interface through the hook mode, and can capture the viewport parameter, the frame buffer, and the frame buffer empty operation instruction through the hook mode. Optionally, the currently supported graphic API of the terminal device includes but is not limited to the following graphic API: OpenGL, OpenGLES, Vulkan, DirectX, and Metal.

[0087] In a possible implementation manner, the terminal device can also receive the target parameter sent by the server, the target parameter is used to indicate that the terminal device acquires the storage instruction corresponding to the target parameter; according to the target parameter, the storage instruction corresponding to the target parameter is acquired; and according to the storage instruction, the rendering data information is stored in the local storage of the terminal device. Correspondingly, the server needs to send the target parameter to the terminal device.

[0088] Optionally, the server can transmit the target parameter to the terminal device by sending an adb command, so as to instruct the terminal device to acquire the storage instruction corresponding to the target parameter. For example, the server transmits a target key value (which is the target parameter) to the terminal device. After receiving the target key value, the terminal device can acquire the storage instruction corresponding to the target key value according to the target key value, so as to store the rendering data information in the local storage of the terminal device. The target key value can be pre-set by a developer or an operation and maintenance personnel. In a possible implementation manner, the above target parameter can be carried in the above data monitoring command, the server can send the terminal device at one time, and the terminal device can acquire the target parameter from the data monitoring command, and then execute the subsequent steps.

[0089] Optionally, the target parameter also corresponds to a set value; the terminal device acquires the set value corresponding to the target parameter according to the target parameter, and the set value is used to represent the target rendering interface in the interface rendering interface; the data information belonging to the target rendering interface in the rendering data information is determined according to the set value; when the terminal device executes the step of storing the rendering data information in the local storage of the terminal device according to the storage instruction, the data information belonging to the target rendering interface in the rendering data information can be stored in the local storage of the terminal device according to the storage instruction.

[0090] For example, the target key value transmitted between the server and the terminal device also corresponds to a set value (set value), the terminal device acquires the set value corresponding to the target key value according to the target key value, and determines the data information belonging to the target rendering interface in the rendering data information according to the set value. For example, the target key value corresponds to the instruction of storing the rendering data information, the set value corresponds to the instruction of storing the management data interface, and the terminal device can determine the data information belonging to the management data interface in the rendering data information according to the set value, and store the information in the local storage of the terminal device.

[0091] Step 406, the terminal device sends the rendering data information to the server.

[0092] The terminal device can send the captured rendering data information to the server, so that the server detects the reason for the display abnormal state of the terminal device according to the rendering data information, generates a detection result, and sends the detection result to the terminal device. The detection result is used to indicate the reason for the display abnormal state of the terminal device.

[0093] Step 407, the server receives the rendering data information sent by the terminal device.

[0094] Correspondingly, the server can receive the rendering data information sent by the terminal device. The rendering data information includes the texture data input to the graphic processor and the rendering image data output by the graphic processor.

[0095] In step 408, the server detects the cause of the display abnormal state of the terminal device according to the rendering data information.

[0096] In a possible implementation, the server can compare the rendering data information with standard data information, the standard data information being the rendering data information corresponding to the display state of the terminal device in the normal display state when the target application program is normally running; when the rendering data information is the same as the standard data information, it is determined that the cause of the display abnormal state of the terminal device exists in the driver layer of the terminal device; and when the rendering data information is different from the standard data information, it is determined that the cause of the display abnormal state of the terminal device exists in the target application program.

[0097] The standard data information is the rendering data information corresponding to the display state of the terminal device in the normal display state when the target application program is normally running. Optionally, the terminal device can send the rendering data information corresponding to the display state of the terminal device in the normal display state when the target application program is normally running to the server in advance, and the server can store the standard data information. When the rendering data information is captured and compared with the standard data information, if the rendering data information is the same as the standard data information, it indicates that the rendering process on the App side is normal, and the cause of the display abnormal state of the terminal device may exist in the driver layer of the terminal device. The server can determine that the cause of the display abnormal state of the terminal device exists in the driver layer of the terminal device. If the rendering data information is different from the standard data information, it indicates that the rendering process on the App side is abnormal, and the cause of the display abnormal state of the terminal device exists in the rendering process of the target application program, so that the cause of the display abnormal state of the terminal device is determined to exist in the target application program.

[0098] In a possible implementation, the terminal device can also perform problem troubleshooting by itself. That is, the terminal device can detect the cause of the display abnormal state of the terminal device according to the rendering data information. The detection manner is similar to the detection manner of the server, that is, the terminal device does not need to send the captured rendering data information to the server, and can perform the step of detecting the cause of the display abnormal state of the terminal device according to the rendering data information by itself. The manner of detecting the cause of the display abnormal state of the terminal device according to the rendering data information by itself is similar to the manner of the server, which will not be described herein again.

[0099] In a possible implementation manner, the rendering data information further includes rendering instructions output by the graphics processor, and when the server performs detection, the rendering instructions included in the rendering data information can be compared with preset instructions; when the rendering instructions are different from the preset instructions, it is determined that the reason for the display abnormal state of the terminal device exists in the target application program. For example, in the terminal device, an instruction (glInvalidateFramebuffer) of explicitly calling the color buffer clearing is required in the rendering process, and when the instruction is not explicitly specified in the application program, the display screen of the terminal device will appear a screen tearing phenomenon. In the present scheme, the rendering data information captured can further include such rendering instructions and perform analysis, when the rendering instructions are different from the preset instructions, that is, when the instruction is not explicitly called in the application program, it is indicated that the reason for the display abnormal state of the terminal device exists in the target application program, and the background personnel of the target application program needs to be contacted for guidance and correction.

[0100] In step 409, a detection result is generated, and the detection result is used to indicate the reason for the display abnormal state of the terminal device.

[0101] Optionally, the server can generate a corresponding detection result based on the above detection manner, so as to indicate the reason for the display abnormal state of the terminal device.

[0102] In step 410, the detection result is sent to the terminal device.

[0103] Optionally, the server sends the generated detection result to the terminal device, and can carry a repair instruction, so that the terminal device performs repair and timely handles the screen tearing phenomenon of the display screen.

[0104] In step 411, the detection result returned by the server according to the rendering data information is received.

[0105] In step 412, the reason for the display abnormal state of the terminal device is determined according to the detection result.

[0106] In conclusion, the present application listens to the interface rendering interface called by the target application program in the running process of the target application program; obtains rendering data information transmitted by the listened interface rendering interface, and determines the reason for the display abnormal state of the terminal device based on the rendering data information, wherein the captured rendering data information includes not only rendering image data output by the graphics processor, but also texture data input to the graphics processor. When the reason for the display abnormal state of the terminal device is detected, more data outside the GPU can be captured, so that the detection of the reason for the display abnormal state of the terminal device can be applied to more rendering scenes, the captured data is more comprehensive, and the accuracy of the detection of the terminal device is improved.

[0107] In addition, the target parameter and the set value can be used to locally store the rendering data information, so that the server side can be flexibly used subsequently. The hook mechanism is adopted, and other tools are not needed for positioning. By monitoring a plurality of Apps, positioning of most App display abnormal problems can be supported, and the problem can be determined without relying on the intervention of a chip manufacturer.

[0108] For example, the terminal device and the server interact through the adb command. Please refer to Figure 6 , which shows a method flowchart of a terminal device detection method according to an example embodiment of the present application. The terminal device detection method can be used in a system composed of a terminal device and a server, and the terminal device and the server perform respective steps. As shown in Figure 6 , the terminal device detection method can include the following steps:

[0109] Step 601: The server sends an adb command to the terminal device.

[0110] Step 602: The terminal device monitors the target App according to the adb command.

[0111] Step 603: The terminal device captures rendering data information.

[0112] Optionally, the App package name information can be set in the adb command. After the App is started, the graphics library in the terminal device monitors the rendering instruction of the App through the interface monitoring module, actively hooks the input information of the GPU, intercepts the graphics API drawing instruction of the terminal device, captures the texture data on the App side, captures the frame buffer data, captures the viewport parameter information, captures the frame buffer area empty operation parameter, captures the binding texture information, and captures the thread number and other information. These information is the rendering data information captured by the terminal device.

[0113] Step 604: The terminal device performs a dump operation on the rendering data information locally.

[0114] Optionally, the terminal device can determine whether to perform the dump operation on the rendering data information by acquiring the target parameter passed by the adb. The description of the target parameter can refer to the description in the above Figure 4 embodiment, and details are not repeated here. For example, when the terminal device detects that the target parameter passed by the adb specifies that the dump operation needs to be performed on the captured texture data, the graphics library saves the texture data to the local memory through the Hook texture upload interface. The dump operation on the rendering data information is an additional operation performed by the terminal device, and does not modify the rendering process and the rendering data information.

[0115] In step 605, the terminal device sends the rendering data information to the server.

[0116] In step 606, the server detects the cause of the display abnormal state of the terminal device according to the rendering data information.

[0117] Optionally, the terminal device can send the rendering data information to the server. For example, in the detection scheme provided in the present application, when the terminal device appears abnormal conditions such as black screen scene or regional dithering scene, the terminal device can determine by the rendering data information captured by Hooking, and other display abnormal conditions can be determined by comparing the data captured after the terminal device appears abnormal conditions with the data before the terminal device appears abnormal conditions. The rendering data information before rendering input and the rendering data information after rendering can be viewed. That is, the rendering data information saved to the local can be viewed to determine.

[0118] Taking the non-game App rendering process as an example, the non-game App display rendering is relatively simple, and often a texture data is transmitted to the GPU side for display. When the non-game App display is abnormal, if the server or the terminal device judges that the input texture data is abnormal, it can be determined that the cause of this abnormal condition is in the App side, otherwise the cause of this abnormal condition is in the driver layer of the terminal device, and the chip manufacturer needs to be involved in analysis.

[0119] For foldable screen mobile phones, when the display area changes, the viewport parameter transmitted from the App side is abnormal, causing the black screen scene to appear. For example, if the screen size is 2150x962, the normal transmitted viewport parameter is (0, 0, 2159, 962), and the information transmitted when the screen is black is (0, 0, 0, 0). Optionally, the chip used by some terminal devices needs the app to explicitly call the clear color, depth buffer and other instructions to clear the data that does not need to be cleared. When there is no explicit specification, the behavior of the chip on such data is undefined, which leads to the fact that some areas will be dithered and the regional dithering scene appears. In the scheme provided in the present application, the rendering data information that can be captured by the terminal device through the graphics library can include these viewport parameter information, call clear color, depth buffer and other instructions, and be compared with the data when the display is normal. When the rendering data information is abnormal, it can be known that the driver layer of the terminal device has a problem.

[0120] In summary, the present application monitors the interface rendering interface called by the target application during the running of the target application, acquires rendering data information transmitted by the monitored interface rendering interface, determines the cause of the display abnormal state of the terminal device based on the rendering data information, wherein the captured rendering data information includes not only the rendering image data output by the graphics processor but also the texture data input to the graphics processor, more data outside the GPU can be captured when detecting the cause of the display abnormal state of the terminal device, so that the detection of the cause of the display abnormal state of the terminal device can be applied to more rendering scenes, the captured data is more comprehensive, and the accuracy of the detection of the terminal device is improved.

[0121] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the present application.

[0122] Please refer to Figure 7 , which shows a structure block diagram of a terminal device detection apparatus provided by an exemplary embodiment of the present application. The terminal device detection apparatus 700 can be used in a terminal device to execute all or part of the steps performed by the terminal device in the methods provided by the embodiments shown in Figure 2 、 Figure 4 or Figure 6 . The terminal device detection apparatus 700 includes:

[0123] An interface monitoring module 701, configured to monitor an interface rendering interface called by a target application during the running of the target application;

[0124] An information capturing module 702, configured to acquire rendering data information transmitted by the monitored interface rendering interface, wherein the rendering data information includes texture data input to a graphics processor and rendering image data output by the graphics processor;

[0125] A cause determining module 703, configured to determine the cause of the display abnormal state of the terminal device based on the rendering data information.

[0126] In conclusion, the application can monitor the interface rendering interface called by the target application during the running of the target application, acquire the rendering data information transmitted by the monitored interface rendering interface, determine the reason for the display abnormal state of the terminal device based on the rendering data information, wherein the acquired rendering data information includes not only the rendering image data output by the graphic processor but also the texture data input to the graphic processor, more data outside the GPU can be acquired when detecting the reason for the display abnormal state of the terminal device, the detection of the reason for the display abnormal state of the terminal device can be applied to more rendering scenes, the acquired data is more comprehensive, and the accuracy of the detection of the terminal device is improved.

[0127] In an optional embodiment, the apparatus further includes:

[0128] The first receiving module is configured to receive a data monitoring command sent by a server before the interface monitoring module 701 monitors the interface rendering interface called by the target application during the running of the target application.

[0129] The first determining module is configured to determine the target application according to the data monitoring command.

[0130] In an optional embodiment, the reason determining module 703 includes a first sending unit, a first receiving unit and a first determining unit.

[0131] The first sending unit is configured to send the rendering data information to the server after the information acquisition module 702 acquires the rendering data information transmitted by the monitored interface rendering interface.

[0132] The first receiving unit is configured to receive a detection result returned by the server according to the rendering data information, wherein the detection result is used to indicate the reason for the display abnormal state of the terminal device.

[0133] The first determining unit is configured to determine the reason for the display abnormal state of the terminal device according to the detection result.

[0134] In an optional embodiment, the apparatus further includes:

[0135] The third receiving module is configured to receive a target parameter sent by the server before the first sending module sends the rendering data information to the server, wherein the target parameter is used to indicate that the terminal device acquires a storage instruction corresponding to the target parameter.

[0136] The first acquiring module is configured to acquire the storage instruction corresponding to the target parameter according to the target parameter.

[0137] The first storage module is configured to store the rendering data information in a local storage of the terminal device according to the storage instruction.

[0138] In an optional embodiment, the target parameter further corresponds to a set value; and the device further includes:

[0139] The second acquisition module is configured to acquire the set value corresponding to the target parameter according to the target parameter, the set value being used to represent a target rendering interface in the interface rendering interface.

[0140] The second determination module is configured to determine the data information belonging to the target rendering interface in the rendering data information according to the set value.

[0141] The first storage module is specifically configured to:

[0142] According to the storage instruction, store the data information belonging to the target rendering interface in the rendering data information in a local storage of the terminal device.

[0143] In an optional embodiment, the device further includes a first sending module or a second sending module.

[0144] The first acquisition module is configured to acquire the interface image of the terminal device before the first receiving module receives the data monitoring command sent by the server; and send the interface image to the server, so that the server sends the data monitoring command when the interface image of the terminal device is a target scene type.

[0145] The second sending module is configured to acquire the interface image of the terminal device before the first receiving module receives the data monitoring command sent by the server; and according to the interface image, determine the interface scene type of the terminal device; and send the interface scene type to the server, so that the server sends the data monitoring command when the interface image of the terminal device is a target scene type.

[0146] The target scene type is a type in which the terminal device appears display abnormality.

[0147] In an optional embodiment, the target scene type includes any one of a black screen scene, a regional screen flower scene or a full screen flower scene.

[0148] In an optional embodiment, the cause determination module 703 includes a first comparison unit, a second determination unit and a third determination unit.

[0149] The first comparison unit is configured to compare the rendering data information with standard data information after the information grabbing module 702 grabs the rendering data information transmitted by the interface rendering interface, the standard data information being corresponding rendering data information when the display state of the terminal device is normal during normal operation of the target application program.

[0150] The second determination unit is configured to determine that the reason for the display abnormal state of the terminal device is in the driver layer of the terminal device when the rendering data information is the same as the standard data information.

[0151] The third determination unit is configured to determine that the reason for the display abnormal state of the terminal device is in the target application program when the rendering data information is different from the standard data information.

[0152] In an optional embodiment, the rendering data information further includes rendering instructions output by the graphic processor, and the reason determination module 703 further includes a second comparison unit and a fourth determination unit.

[0153] The second comparison unit is configured to compare the rendering instructions included in the rendering data information with preset instructions.

[0154] The fourth determination unit is configured to determine that the reason for the display abnormal state of the terminal device is in the target application program when the rendering instructions are different from the preset instructions.

[0155] In an optional embodiment, the interface rendering interface includes at least two of a rendering instruction interface, a texture data uploading interface, and a frame buffer interface.

[0156] The rendering data information further includes viewport parameter information and frame buffer parameter information.

[0157] In an optional embodiment, the information grabbing module 702 is specifically configured to,

[0158] grab the rendering data information transmitted by the interface rendering interface in a hook manner.

[0159] Please refer to Figure 8 which shows a structure schematic diagram of a terminal device provided by an example embodiment of the present application, as Figure 8 shown, the terminal device includes a processor 810, a memory 820, and a display component 830. The memory 820 stores at least one instruction, which is loaded and executed by the processor 810 to implement part or all of the steps performed by the terminal device in the detection method of the terminal device according to various embodiments described above.

[0160] The embodiment of the present application further provides a computer readable medium, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement all or part of steps performed by the terminal device or the server in the detection method of the terminal device according to various embodiments.

[0161] The embodiment of the present application further provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement all or part of steps performed by the terminal device or the server in the detection method of the terminal device according to various embodiments.

[0162] It should be noted that: the apparatus provided in the above embodiments is only used for example when performing information pushing or information display, and the above-mentioned division of the functional modules is used for example, and in actual application, the above-mentioned functions can be distributed to different functional modules to be completed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0163] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0164] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by program to instruct related hardware to complete, and the program can be stored in a computer readable storage medium, and the above-mentioned storage medium can be read-only memory, magnetic disk or optical disk, etc.

[0165] The above-mentioned is only the optional embodiment of the present application, and does not limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A detection method of a terminal device, characterized by, The method applied to a terminal device comprises: monitoring an interface rendering interface called by a target application during running of the target application; obtaining rendering data information transmitted by the interface rendering interface, the rendering data information comprising texture data input to a graphic processor and rendering image data output by the graphic processor; determining a cause of a display abnormal state of the terminal device based on the rendering data information; the determining of the cause of the display abnormal state of the terminal device based on the rendering data information comprises: comparing the rendering data information with standard data information, the standard data information being rendering data information corresponding to a normal display state of the terminal device when the target application runs normally; when the rendering data information is the same as the standard data information, determining that the cause of the display abnormal state of the terminal device exists in a driver layer of the terminal device; when the rendering data information is different from the standard data information, determining that the cause of the display abnormal state of the terminal device exists in the target application.

2. The method of claim 1, wherein, before the monitoring of the interface rendering interface called by the target application during running of the target application, the method further comprises: receiving a data monitoring command sent by a server; determining the target application according to the data monitoring command.

3. The method of claim 2, wherein, the determining of the cause of the display abnormal state of the terminal device based on the rendering data information comprises: sending the rendering data information to the server; receiving a detection result returned by the server according to the rendering data information, the detection result being used for indicating the cause of the display abnormal state of the terminal device; determining the cause of the display abnormal state of the terminal device according to the detection result.

4. The method of claim 3, wherein, before the sending of the rendering data information to the server, the method further comprises: receiving a target parameter sent by the server; obtaining a storage instruction corresponding to the target parameter according to the target parameter; storing the rendering data information in a local storage of the terminal device according to the storage instruction.

5. The method of claim 4, wherein, the method further comprises: obtaining a set value corresponding to the target parameter according to the target parameter, the set value being used for representing a target rendering interface in the interface rendering interface; determining data information belonging to the target rendering interface in the rendering data information according to the set value; the storing of the rendering data information in the local storage of the terminal device according to the storage instruction comprises: storing the data information belonging to the target rendering interface in the rendering data information in the local storage of the terminal device according to the storage instruction.

6. The method of claim 2, wherein, before the receiving of the data monitoring command sent by the server, the method further comprises: obtaining an interface image of the terminal device; and sending the interface image to the server, so that the server sends the data monitoring command when the interface image of the terminal device is a target scene type; or obtaining an interface image of the terminal device; determining an interface scene type of the terminal device according to the interface image; and sending the interface scene type to the server, so that the server sends the data monitoring command when the interface image of the terminal device is a target scene type. The target scene type is a type in which the terminal device displays abnormally.

7. The method of claim 6, wherein, The target scene type includes any one of a black screen scene, a regional screen flower scene, or a full screen flower scene.

8. The method of claim 1, wherein, The rendering data information further includes rendering instructions output by the graphic processor, and based on the rendering data information, a cause of the terminal device displaying abnormally is determined, including: comparing the rendering instructions included in the rendering data information with preset instructions; when the rendering instructions are different from the preset instructions, determining that the cause of the terminal device displaying abnormally is in the target application program.

9. The method according to any one of claims 1 to 8, characterized in that, The interface rendering interface includes at least two of a rendering instruction interface, a texture data upload interface, and a frame buffer interface. The rendering data information further includes viewport parameter information and frame buffer parameter information.

10. The method according to any one of claims 1 to 8, characterized in that, The rendering data information transmitted by the interface rendering interface and monitored is obtained, including: The rendering data information transmitted by the interface rendering interface and monitored is obtained by hooking.

11. A detection apparatus of a terminal device, characterized by comprising: The device is applied to a terminal device, and includes: An interface monitoring module is configured to monitor an interface rendering interface called by a target application program during running of the target application program. An information grabbing module is configured to obtain rendering data information transmitted by the interface rendering interface and monitored, the rendering data information including texture data input to a graphic processor and rendering image data output by the graphic processor. A cause determining module is configured to determine a cause of the terminal device displaying abnormally based on the rendering data information. The cause determining module includes a first comparison unit, a second determination unit, and a third determination unit. The first comparison unit is configured to compare the rendering data information with standard data information, the standard data information being rendering data information corresponding to a normal display state of the terminal device when the target application program runs normally. The second determination unit is configured to determine that the cause of the terminal device displaying abnormally is in a driver layer of the terminal device when the rendering data information is the same as the standard data information. The third determination unit is configured to determine that the cause of the terminal device displaying abnormally is in the target application program when the rendering data information is different from the standard data information.

12. A terminal device, comprising: The terminal device includes a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the detection method of the terminal device according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the detection method of the terminal device according to any one of claims 1 to 10.

Images