Play Component Compatibility Detection Method, Device, Computer Equipment and Storage Medium

Abstract

This application relates to the field of artificial intelligence technology, and provides a method, device, computer device, and storage medium for detecting the compatibility of a playback component. The method includes: loading the playback component into the detection process and creating an off-screen rendering environment, where the detection process is a process started when the application switches to the background. Decode the target video corresponding to the application to the off-screen rendering environment through the playback component, extract the video frame to be detected from the off-screen rendering environment, input the video frame to be detected into a preset image detection model to obtain the video frame detection result, where the image detection model is trained based on an image set including abnormal images, and determine the playback component compatibility detection result according to the video frame detection result. The detection process does not rely on the actual operations and feedback of users, improves the detection efficiency, and enables the compatibility detection to be completed without the user's awareness.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment and storage medium for detecting the compatibility of playback components. Background Art

[0002] As an important component of smart devices, the playback component is a playback software used to play multimedia. The system usually configures the system playback component when it leaves the factory, such as the MediaPlayer in the Android system, which is the player that comes with the Android system. As the basic component of the system, the system playback component needs to meet the function of playing videos in various applications. Therefore, the compatibility of the playback component needs to be tested.

[0003] However, in traditional methods, the detection of playback components basically relies on the feedback information of users after the actual operation process, and lacks dynamic detection technology, resulting in low efficiency of playback component compatibility testing. Summary of the invention

[0004] Based on this, it is necessary to provide a playback component compatibility detection method, device, computer equipment and storage medium that can improve test efficiency in response to the above technical problems.

[0005] A method for detecting compatibility of playback components, the method comprising:

[0006] Load the playback component into the detection process and create an off-screen rendering environment. The detection process is the process started when the application is switched to the background.

[0007] Decode the target video corresponding to the application to the off-screen rendering environment through the playback component;

[0008] Extracting a video frame to be detected from an off-screen rendering environment, inputting the video frame to be detected into a preset image detection model to obtain a video frame detection result, wherein the image detection model is trained based on an image set including abnormal images;

[0009] According to the video frame detection result, the playback component compatibility detection result is determined.

[0010] A playback component compatibility detection device, the device comprising:

[0011] The loading module is used to load the playback component into the detection process and create an off-screen rendering environment. The detection process is the process started when the application is switched to the background;

[0012] A decoding module, used for decoding a target video corresponding to the application to an off-screen rendering environment through a playback component;

[0013] A detection module, configured to extract a video frame to be detected from an off-screen rendering environment, input the video frame to be detected into a preset image detection model, and obtain a video frame detection result, where the image detection model is trained based on an image set including abnormal images;

[0014] A compatibility determination module, configured to determine a playback component compatibility detection result according to the video frame detection result.

[0015] A computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0016] Load a playback component into a detection process, and create an off-screen rendering environment, where the detection process is a process started when the application switches to the background;

[0017] Decode the target video corresponding to the application to the off-screen rendering environment through the playback component;

[0018] Extract a video frame to be detected from the off-screen rendering environment, input the video frame to be detected into a preset image detection model, and obtain a video frame detection result, where the image detection model is trained based on an image set including abnormal images;

[0019] Determine a playback component compatibility detection result according to the video frame detection result.

[0020] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0021] Load a playback component into a detection process, and create an off-screen rendering environment, where the detection process is a process started when the application switches to the background;

[0022] Decode the target video corresponding to the application to the off-screen rendering environment through the playback component;

[0023] Extract a video frame to be detected from the off-screen rendering environment, input the video frame to be detected into a preset image detection model, and obtain a video frame detection result, where the image detection model is trained based on an image set including abnormal images;

[0024] Determine a playback component compatibility detection result according to the video frame detection result.

[0025] The above playback component compatibility detection method, device, computer device, and storage medium load the playback component into the detection process based on the detection process started when the application switches to the background, load and start the playback component in the form of a background process, create an off-screen rendering environment, and decode the target video corresponding to the application to the off-screen rendering environment through the playback component, facilitating the video decoding of the playback component without the user's awareness. Since the image detection model is trained based on an image set including abnormal images, extract the video frames to be detected from the decoded data of the off-screen rendering environment, input the video frames to be detected into the preset image detection model, and determine whether the video frames are abnormal, thereby obtaining the playback component compatibility detection result based on the video frame detection result, realizing the playback component compatibility detection during the background operation of the application, and the detection process does not depend on the user's actual operations and feedback, improving the detection efficiency. Description of the Drawings

[0026] Figure 1 It is an application environment diagram of the playback component compatibility detection method in an embodiment;

[0027] Figure 2 It is a flowchart of the playback component compatibility detection method in an embodiment;

[0028] Figure 3 It is a flowchart of the playback component compatibility detection method in another embodiment;

[0029] Figure 4 It is a flowchart of the playback component compatibility detection method in still another embodiment;

[0030] Figure 5 It is a flowchart of the playback component compatibility detection method in yet another embodiment;

[0031] Figure 6 It is a flowchart of the playback component compatibility detection method in another embodiment;

[0032] Figure 7 It is a flowchart of the playback component compatibility detection method in another embodiment;

[0033] Figure 8 It is a schematic diagram of the relationship between the main process and the subprocess in the playback component compatibility detection method in an embodiment;

[0034] Figure 9 It is a flowchart of the playback component compatibility detection method in another embodiment;

[0035] Figure 10 It is a flowchart block diagram of the playback component compatibility detection method in an embodiment;

[0036] Figure 11 The structural block diagram of a playback component compatibility detection device in an embodiment;

[0037] Figure 12 The internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0039] The playback component compatibility detection method provided by the present application can be applied to an application environment as Figure 1 shown. The terminal 102 communicates with the server 104 through a network. Among them, the terminal 102 loads a playback component into a detection process and creates an off-screen rendering environment. The detection process is a process started when the application program switches to the background. The target video corresponding to the application program is decoded into the off-screen rendering environment through the playback component, and a video frame to be detected is extracted from the off-screen rendering environment. The video frame to be detected is input into a preset image detection model to obtain a video frame detection result. The image detection model is trained by the server 104 based on an image set including abnormal images and sent to the terminal 102. The terminal 102 determines the playback component compatibility detection result according to the video frame detection result.

[0040] In other embodiments, the terminal 102 can send the video frame to be detected to the server 104. The server 104 inputs the video frame to be detected into a preset image detection model to obtain a video frame detection result, and determines the playback component compatibility detection result according to the video frame detection result. The obtained playback component compatibility detection result can also be fed back from the server 104 to the terminal 102.

[0041] Among them, the terminal 102 can be but is not limited to various personal computers, laptop computers, smart phones, tablet computers and portable wearable devices. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0042] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in multiple fields. The solution provided by the embodiment relates to the field of computer vision technology of artificial intelligence. Computer vision is a science that studies how to make machines "see". Further, it refers to using cameras and computers to replace human eyes to perform machine vision such as target recognition, tracking, and measurement on targets, and further performing graphics processing to make the computer process into images that are more suitable for human eyes to observe or be transmitted to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, and video semantic understanding. It will be specifically described through the following embodiments:

[0043] In one embodiment, as Figure 2 shown, a method for detecting the compatibility of a playback component is provided. Taking the terminal in Figure 1 as an example, the method includes the following steps 202 to step 208.

[0044] Step 202, load the playback component into the detection process and create an off-screen rendering environment. The detection process is a process started when the application switches to the background.

[0045] The playback component refers to the object to be tested used to decode multimedia data to play multimedia data. In the embodiment, the playback component can be a system playback component or a playback component downloaded through a third party. For example, the MediaPlayer that comes with the Android system, or, for another example, QQPlayer, Adobe Flash Player, etc. downloaded from the app store or the official website.

[0046] The detection process refers to a subroutine independent of the main process of the application. When the main process of the application enters the background running state, this sub-process is started to perform compatibility detection.

[0047] When the detection process is started, the playback component can be loaded into the detection process by pulling up the playback component. During the process of loading the playback component, the output environment of the video frame decoded by the playback component is determined by creating an off-screen rendering environment.

[0048] Off-screen rendering is a rendering method outside the screen, that is, the rendering result will not be directly presented on the current screen. During normal video playback, generally, the video frames are sequentially rendered to the current screen through on-screen rendering. At this time, the rendering operation of the GPU (Graphics Processing Unit) is performed in the screen buffer currently used for display. However, during off-screen rendering, the GPU will create a buffer outside the current screen buffer for rendering operations and will not render to the current screen.

[0049] An application refers to an application that needs to use a playback component to play multimedia data. The application can be an object configured by the user according to needs for compatibility detection of the playback component, or an object determined based on the default policy of the terminal. For example, by default, each user needs to be used as a detection object to detect the compatibility of the playback component when a new application is installed.

[0050] Background running refers to a process that the operating system does not display and run on the front-end display interface. Background running is relative to front-end running such as on the desktop. When the desktop is executed, the user can directly perceive it, while the background is invisible to the user. Switching an application to the background is a process of switching the running program that can be directly seen on the desktop to a state where it cannot be directly seen. For example, minimizing or hiding the program being executed on the desktop so that the program page no longer appears on the desktop. Another example is to open other applications to cover the display content on the current desktop so that the program page no longer appears on the desktop.

[0051] In one embodiment, the terminal detects the running states of each application. When it detects that the state of one of the applications switches to the background running state and receives a detection command for that application, it starts a detection process. Among them, the detection command for that application can be generated based on user operations in the background or can be generated by triggering the default policy of the terminal (such as when the application is newly installed). By generating detection commands under different conditions, comprehensive compatibility detection of the playback component can be achieved for different applications.

[0052] Step 204, decode the target video corresponding to the application to the off-screen rendering environment through the playback component.

[0053] The target video corresponding to the application refers to the video used for compatibility detection. Among them, the target video can be obtained according to the video file address or the specified video file.

[0054] Decoding is a process of restoring digital codes to their original represented content using specific methods, or converting electrical pulse signals, optical signals, radio waves, etc. into the information, data, etc. they represent. Decoding is the process by which the recipient restores the received symbols or codes into information, corresponding to the encoding process. The playback component can convert video data into a continuous video frame in image format by decoding the target video, and the video can be played by sequentially rendering the continuous video frames on the interface.

[0055] In the detection process, the playback component is started to decode the target video to obtain video frames, and the obtained video frames are rendered into an off-screen rendering environment, avoiding direct rendering and playback on the current screen, realizing video decoding and off-screen playback without the user's awareness.

[0056] Step 206: Extract the video frames to be detected from the off-screen rendering environment, and input the video frames to be detected into a preset image detection model to obtain the video frame detection results.

[0057] The playback component decodes the target video into the off-screen rendering environment. The video frames to be detected can be all the video frames in the off-screen rendering environment or part of the video frames in the off-screen rendering environment. By using all the video frames as the video frames to be detected, all the video frames of the target video can be comprehensively detected to avoid missing abnormal pictures. However, since the number of video frames corresponding to the target video is huge, the detection speed can be improved by screening the video frames and using part of the video frames as the video frames to be detected. In the embodiment, the screening method can be random screening or screening according to set rules, such as screening according to a set ratio, screening according to a set frequency, etc. Among them, screening according to a set ratio means determining the number of video frames to be screened based on the total number of video frames, and then obtaining the number of video frames to be tested from the total video frames. Screening according to a set frequency means screening out one video frame as the video frame to be tested every N video frames in chronological order.

[0058] The image detection model refers to a neural network model used to detect the video frames to be tested. In the embodiment, the image detection model can be obtained by training with an image set of abnormal images.

[0059] Specifically, a training service for machine learning is built in the background, and a large number of materials such as flower screens, green screens, and black screens are provided for training to obtain the image detection model. Among them, flower screens, green screens, and black screens belong to display abnormal images. A flower screen means that there are stripes, spots, or color blocks different from the normal color on the display screen, or there are situations such as position inversion, disorder, screen jitter, and distortion. A green screen indicates a situation where there are local or all green stripes, spots, or color blocks on the display screen, and a black screen means that the display result on the display screen is all black.

[0060] The training process of the image detection model includes the following steps: using normal images as positive samples and abnormal images such as screen freeze, green screen, and black screen as negative samples to train the initial image detection model; until the model evaluation parameters meet the set conditions, an image detection model that can be used to perform image detection on the video frames to be detected is obtained.

[0061] During the detection process, by inputting the video frames to be detected into a preset image detection model, the video frame detection results output by the image detection model are obtained.

[0062] Step 208: Determine the playback component compatibility detection result according to the video frame detection result.

[0063] The video frame detection results include two cases: abnormal video frames with abnormal situations such as screen freeze, green screen, and black screen, and normal video frames without abnormal situations such as screen freeze, green screen, and black screen.

[0064] When the video frame detection result is that there are abnormal video frames, a detection result that the playback component is incompatible with the application is obtained. When the video frame detection result is that there are no abnormal video frames, a detection result that the playback component is compatible with the application is obtained.

[0065] The above playback component compatibility detection method is based on the detection process started when the application switches to the background. The playback component is loaded into the detection process and loaded and started in the form of a background process. By creating an off-screen rendering environment, the target video corresponding to the application is decoded by the playback component into the off-screen rendering environment, which is convenient for realizing video decoding by the playback component without the user's perception. Since the image detection model is trained based on an image set including abnormal images, the video frames to be detected are extracted from the decoded data of the off-screen rendering environment, and the video frames to be detected are input into a preset image detection model, so as to judge whether the video frames are abnormal. Thus, based on the video frame detection result, the playback component compatibility detection result is obtained, realizing the playback component compatibility detection during the background operation of the application. The detection process does not depend on the user's actual operation and feedback, improves the detection efficiency, and enables the compatibility detection to be completed without the user's perception.

[0066] In one embodiment, as Figure 3 shown, decoding the target video corresponding to the application by the playback component into the off-screen rendering environment, i.e., step 204, includes steps 302 to 304:

[0067] Step 302: Start the playback component and decode the target video through the playback component to obtain the decoded video frames.

[0068] Step 304: Render the video frames into the off-screen rendering environment according to the image rendering medium configured in the playback component.

[0069] An image rendering medium refers to a medium used to represent the video frame rendering environment. In an embodiment, taking the Android system as an example, the image rendering medium can be a Surface. A Surface is a handle of an image buffer in the Android system and serves as a medium for interaction between the playback component and the display view (View). The player can draw the decoded image of the player onto the video (View) through the Surface.

[0070] The image rendering medium is associated with an off-screen rendering environment. After the playback component decodes the target video to obtain the decoded video frame, based on the image rendering medium associated with the off-screen rendering environment and configured in the playback component, the video frame is rendered to the off-screen rendering environment. By configuring the image rendering medium, targeted output of the video frame according to the specified rendering environment is achieved, avoiding direct rendering to the current screen, so as to realize seamless decoding and video frame rendering for the user.

[0071] In one embodiment, as Figure 4 shown, before rendering the video frame to the off-screen rendering environment according to the image rendering medium configured in the playback component, it further includes steps 402 to 406.

[0072] Step 402: Construct a texture identifier corresponding to the off-screen rendering environment according to the off-screen rendering environment.

[0073] Step 404: Determine the image rendering medium corresponding to the texture object according to the texture object represented by the texture identifier.

[0074] Step 406: Configure the image rendering medium to the playback component.

[0075] The texture identifier, i.e., the texture ID (Identity document), is texture mapping information randomly generated based on the off-screen rendering environment and used to represent the association relationship with the off-screen rendering environment. In an embodiment, the texture identifier can be constructed through EGL, where EGL refers to OpenGL extended for Android and is equivalent to a rendering API (Application Programming Interface).

[0076] Based on the texture identifier, a texture object SurfaceTexture is generated. Through the SurfaceTexture, an image rendering medium Surface can be generated. After setting the Surface to the playback component, the data output environment of the playback component is determined, enabling the playback component to start normally.

[0077] In an embodiment, the texture identifier can be randomly generated based on an off-screen rendering environment. Different texture identifiers respectively represent different texture objects, and the image rendering media corresponding to different texture objects can also be different, but ultimately they all correspond to the same off-screen rendering environment. The image rendering media is configured to the playback component to realize the association between the playback component and the off-screen rendering environment.

[0078] In this embodiment, by constructing a texture identifier and determining the image rendering media based on the texture object represented by the texture identifier, different image rendering media can be constructed based on different texture identifiers, realizing the expansion of the association path between the playback component and the off-screen rendering environment, which facilitates the synchronous execution of multiple detection processes.

[0079] In one embodiment, extracting the video frame to be detected from the off-screen rendering environment includes: performing frame extraction on the video frames in the off-screen rendering environment at a preset frequency to obtain the video frame to be detected.

[0080] In an embodiment, EGL can continuously read the data in the Surface of the image rendering media to obtain the video frame decoded by the playback component. The video frame is frame-extracted at a certain frequency and input into the image detection model, for example, sending 1 frame every 5 frames.

[0081] By performing frame extraction at a set frequency, not only the number of video frames to be tested is reduced, but also the video frames to be tested can be screened at the same interval, ensuring the uniform distribution of the screening results in the total video frames, and accurate and reliable detection results can be obtained.

[0082] In one embodiment, as Figure 5 shown, decoding the target video corresponding to the application program to the off-screen rendering environment by the playback component, i.e., step 204, includes steps 502 and 504.

[0083] Step 502, when the playback mode of the off-screen rendering environment is silent playback, start the playback component to decode the target video, and render the decoded video frames to the off-screen rendering environment.

[0084] Step 504, when the playback mode of the off-screen rendering environment is non-silent playback, adjust the playback mode to silent mode, start the playback component to decode the target video, and render the decoded video frames to the off-screen rendering environment.

[0085] The playback modes of the off-screen rendering environment include silent playback and non-silent playback. The playback mode is the configuration data of the off-screen rendering environment and does not affect the sound effects of other processes on the terminal. Specifically, when the playback mode of the off-screen rendering environment is silent playback, when rendering the video frames decoded in the off-screen rendering environment, a silent rendering effect is achieved. When the playback mode of the off-screen rendering environment is non-silent playback, when rendering the video frames decoded in the off-screen rendering environment, the audio data will be played synchronously based on the sound effect setting parameters of the terminal. For example, the video frames will be rendered according to the current multimedia sound size set on the terminal, and video playback will be performed in the off-screen rendering environment.

[0086] By obtaining the playback mode of the off-screen rendering environment, when the playback mode of the off-screen rendering environment is silent playback, start the playback component to decode the target video, and render the decoded video frames to the off-screen rendering environment. When the playback mode of the off-screen rendering environment is non-silent playback, adjust the playback mode to silent mode, start the playback component to decode the target video, and render the decoded video frames to the off-screen rendering environment to achieve silent playback in the off-screen rendering environment. The entire process adopts the method of off-screen rendering and silent playback, which will not interfere with the normal use of the terminal user.

[0087] In one embodiment, as Figure 6 shown, before loading the playback component into the detection process and creating the off-screen rendering environment, that is, before step 202, steps 602 to 604 are further included.

[0088] Step 602, when it is detected that the application switches to the background, record the background running duration.

[0089] Step 604, when the background running duration reaches the preset duration, start the detection process.

[0090] After the user switches the application to the background, start the detection process after a certain waiting time, such as 2 seconds. When the background running duration reaches the preset duration, start the detection process to provide a stable startup environment for the detection process and prevent the detection process from being frequently started due to the user quickly switching between the foreground and background.

[0091] In one embodiment, the playback component compatibility detection method further includes: Step 702, when it is detected that the application switches from the background to the foreground, end the detection process.

[0092] In the embodiment, as Figure 8As shown, the running process of the application is the main process, and the detection program is the compatibility detection subprocess corresponding to the main process. The subprocess and the main process execute independently of each other. The start and shutdown of the subprocess depend on the running state of the main process and the execution result of the subprocess. Specifically, when the application switches from running in the foreground to running in the background, the compatibility detection subprocess starts. There are multiple ways for the compatibility detection subprocess to end. The first is to end the compatibility detection subprocess in advance, such as when the user switches the application from the background to the foreground to shut down the subprocess. The second is to end the compatibility detection subprocess after the detection is completed. For example, the compatibility detection subprocess executes for a specific time. For example, the set detection time is 5 seconds. After execution, the execution result is recorded and the detection is completed, then the subprocess is shut down. Another example is that if an abnormal image appears in the detection result, the detection result can be recorded and the detection is completed, and then the subprocess is shut down.

[0093] By monitoring the running state of the application, when the application switches from the background to the foreground, the detection process is ended to avoid affecting the normal use of the user.

[0094] In the embodiment, when the execution ends or the image is abnormal, the compatibility detection result of the application by the playback component can be obtained, and the application can be marked as a detected application. When it is monitored that the application switches from the background to the foreground, after ending the detection process, the application can also be marked as a to-be-detected application. When the running state of the application meets the detection conditions again next time, the detection process can be restarted for detection again until the compatibility detection result of the application by the playback component is obtained. By marking the application that has not completed the detection as a to-be-detected application and detecting it again, comprehensive compatibility detection can be achieved.

[0095] In one embodiment, as Figure 9 shown, a method for detecting the compatibility of a playback component is provided, including the following steps 902 to step 926.

[0096] Step 902, when it is monitored that the application switches to the background, record the background running duration.

[0097] Step 904, when the background running duration reaches the preset duration, start the detection process.

[0098] Step 906, load the playback component into the detection process and create an off-screen rendering environment.

[0099] Step 908, when the playback mode of the off-screen rendering environment is non-mute playback, adjust the playback mode to mute mode.

[0100] Step 910, start the playback component, decode the target video through the playback component to obtain the decoded video frames.

[0101] Step 912: construct a texture identifier corresponding to the off-screen rendering environment according to the off-screen rendering environment.

[0102] Step 914: Determine the image rendering medium corresponding to the texture object according to the texture object represented by the texture identifier.

[0103] Step 916, configuring the image rendering medium to the playback component.

[0104] Step 918, render the video frame to an off-screen rendering environment according to the image rendering medium.

[0105] Step 920 , performing frame extraction processing on the video frames in the off-screen rendering environment at a preset frequency to obtain the video frames to be detected.

[0106] Step 922: input the video frame to be detected into a preset image detection model to obtain a video frame detection result.

[0107] Step 924, determining the playback component compatibility detection result based on the video frame detection result.

[0108] Step 926, when it is detected that the application is switched from the background to the front end, the detection process is terminated.

[0109] The present application also provides an application scenario, which applies the above-mentioned playback component compatibility detection method. Specifically, the playback component compatibility detection method is applied to the compatibility detection of the player MediaPlayer that comes with the Android system, and the application in this application scenario is as follows:

[0110] First, you need to build a machine learning training service in the background, provide a large number of image materials such as flower screen, green screen, black screen, etc. to train the initial image detection model, and obtain the image detection model for detecting video frames. Then, based on the image detection model, the detection process includes two parts:

[0111] The first part is to build an independent process environment, and the second part is to run the detection program in the independent process and obtain the results. In the first part, when the terminal obtains the detection command from the background, it records the detection task for the APP and monitors the running status of the APP. The detection command can be issued from the background or the default policy that the terminal needs to detect every time a new user is installed. The detection command contains the video playback address or video file that needs to be detected. When it is detected that the user switches the App to the background, after a certain waiting time, such as 2 seconds, the detection process is immediately started to prevent the user from quickly switching between the front and back ends and causing the process to start frequently. There are two ways to end the detection process. The first is after the user switches the App back to the front end, and the other is after the process executes for a specific time, such as 5 seconds. After the execution is completed, the execution result is recorded and it can be destroyed.

[0112] The second part is to run the detection program in an independent process and obtain the results. This part is divided into two stages: loading the player and actually starting the player. Figure 9 As shown, you need to use EGL to create an OpenGL off-screen rendering environment, generate a texture ID through glBindTexture, and generate a SurfaceTexture with this texture ID, and then use this SurfaceTexture to generate a Surface, and then configure this Surface to the player. The player can use this Surface to render the decoded video frames to the off-screen rendering environment. The implementation of this method does not require any modification to the system player MediaPlayer, and will not affect the compatibility test results.

[0113] When actually starting the player, you need to mute it first, and then start the player. After the player starts decoding, the decoded data will be output to the Surface configured for the player. EGL can continuously read the data in the Surface to get the decoded data of the player. Then the data is sent to the inference module at a certain frequency, which can be 1 frame every 5 frames. After receiving the data, the inference module performs inference according to the trained image detection model and obtains the inference result, which mainly determines whether the image has abnormal conditions such as distorted screen, green screen, black screen, etc. If abnormal conditions such as distorted screen, green screen, black screen, etc. occur, the detection result can be recorded and the entire program can be stopped; otherwise, it will exit normally after running for a period of time according to a specific time. If there is an abnormality, the system player has a compatibility problem, otherwise it means no. The entire process adopts off-screen rendering and silent playback, so it will not disturb the user. It can well solve the scenario of using a large number of system players, such as TV platform playback.

[0114] It should be understood that although the steps in the various flowcharts involved in the above embodiments are sequentially shown according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the various flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0115] In one embodiment, as Figure 11 shown, a playback component compatibility detection device 1100 is provided. This device can be a software module, a hardware module, or a combination of both to become a part of a computer device. Specifically, this device includes: a loading module 1102, a decoding module 1104, a detection module 1106, and a result determination module 1108, where:

[0116] The loading module 1102 is used to load the playback component into the detection process and create an off-screen rendering environment. The detection process is a process started when the application switches to the background.

[0117] The decoding module 1104 is used to decode the target video corresponding to the application to the off-screen rendering environment through the playback component.

[0118] The detection module 1106 is used to extract the video frames to be detected from the off-screen rendering environment, input the video frames to be detected into a preset image detection model, and obtain the video frame detection result, where the image detection model is trained based on an image set including abnormal images.

[0119] The compatibility determination module 1108 is used to determine the playback component compatibility detection result according to the video frame detection result.

[0120] In one of the embodiments, the decoding module is further used to start the playback component, decode the target video through the playback component to obtain the decoded video frames; and render the video frames to the off-screen rendering environment according to the image rendering medium configured in the playback component, where the image rendering medium is associated with the off-screen rendering environment.

[0121] In one of the embodiments, the playback component compatibility detection device further includes an image rendering medium configuration module, which is used to construct a texture identifier corresponding to the off-screen rendering environment according to the off-screen rendering environment; determine the image rendering medium corresponding to the texture object according to the texture object represented by the texture identifier; and configure the image rendering medium to the playback component.

[0122] In one embodiment, the detection module is further configured to extract frames from video frames in the off-screen rendering environment at a preset frequency to obtain video frames to be detected.

[0123] In one embodiment, the decoding module is further configured to, when the playback mode of the off-screen rendering environment is silent playback, start the playback component to decode the target video, and render the decoded video frames to the off-screen rendering environment; when the playback mode of the off-screen rendering environment is non-silent playback, adjust the playback mode to silent mode, start the playback component to decode the target video, and render the decoded video frames to the off-screen rendering environment.

[0124] In one embodiment, the decoding module is further configured to, when it is detected that the application switches to the background, record the background running duration; when the background running duration reaches a preset duration, start a detection process.

[0125] In one embodiment, the monitoring module is further configured to end the detection process when it is detected that the application switches from the background to the foreground.

[0126] The above playback component compatibility detection device, based on the detection process started when the application switches to the background, loads the playback component into the detection process, loads and starts the playback component in the form of a background process, creates an off-screen rendering environment, decodes the target video corresponding to the application to the off-screen rendering environment through the playback component, which facilitates video decoding by the playback component without the user's awareness. Since the image detection model is trained based on an image set including abnormal images, extract the video frames to be detected from the decoded data of the off-screen rendering environment, input the video frames to be detected into a preset image detection model, and determine whether there are abnormalities in the video frames. Thus, based on the video frame detection results, obtain the playback component compatibility detection results, and realize the playback component compatibility detection during the background operation of the application. The detection process does not depend on the user's actual operations and feedback, and improves the efficiency of playback component compatibility detection.

[0127] For the specific limitations of the playback component compatibility detection device, reference can be made to the limitations of the playback component compatibility detection method in the above text, which will not be elaborated here. Each module in the above playback component compatibility detection device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so as to facilitate the processor to call and execute the operations corresponding to the above modules.

[0128] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 12As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, carrier network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it realizes a method for detecting the compatibility of playback components. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads provided on the casing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0129] Those skilled in the art can understand that Figure 12 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0130] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are realized.

[0131] In one embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are realized.

[0132] In one embodiment, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the above method embodiments.

[0133] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0134] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0135] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for detecting the compatibility of a playback component, characterized in that, the method includes: loading the playback component into a detection process and creating an off-screen rendering environment, where the detection process is a process started when the application switches to the background; the detection process is used to detect the compatibility between the application and the playback component; decoding the target video corresponding to the application to the off-screen rendering environment through the playback component; extracting a video frame to be detected from the off-screen rendering environment, and inputting the video frame to be detected into a preset image detection model to obtain a video frame detection result, where the image detection model is trained based on an image set including abnormal images; determining the playback component compatibility detection result according to the video frame detection result, where the playback component compatibility detection result indicates whether the playback component is compatible with the application.

2. The method according to claim 1, characterized in that, the decoding the target video corresponding to the application to the off-screen rendering environment through the playback component includes: starting the playback component, decoding the target video through the playback component to obtain a decoded video frame; rendering the video frame to the off-screen rendering environment according to the image rendering medium configured in the playback component, where the image rendering medium is associated with the off-screen rendering environment.

3. The method according to claim 2, characterized in that, before rendering the video frame to the off-screen rendering environment according to the image rendering medium configured in the playback component, it further includes: constructing a texture identifier corresponding to the off-screen rendering environment according to the off-screen rendering environment; determining an image rendering medium corresponding to the texture object according to the texture object represented by the texture identifier; configuring the image rendering medium to the playback component.

4. The method according to claim 1, characterized in that, the extracting a video frame to be detected from the off-screen rendering environment includes: performing frame extraction on the video frames in the off-screen rendering environment at a preset frequency to obtain a video frame to be detected.

5. The method according to claim 1, characterized in that, the decoding the target video corresponding to the application to the off-screen rendering environment through the playback component includes: when the playback mode of the off-screen rendering environment is silent playback, starting the playback component to decode the target video and rendering the decoded video frame to the off-screen rendering environment; when the playback mode of the off-screen rendering environment is non-silent playback, adjusting the playback mode to silent mode, starting the playback component to decode the target video and rendering the decoded video frame to the off-screen rendering environment.

6. The method according to claim 1, characterized in that, before loading the playback component into the detection process and creating the off-screen rendering environment, it further includes: when it is monitored that the application switches to the background, recording the background running duration; when the background running duration reaches a preset duration, starting the detection process.

7. The method according to claim 1, characterized in that, the method further includes: When it is detected that the application switches from the background to the foreground, end the detection process.

8. A playback component compatibility detection device, characterized in that, the device includes: A loading module, configured to load a playback component into a detection process and create an off-screen rendering environment, where the detection process is a process started when the application switches to the background; the detection process is used to detect the compatibility between the application and the playback component; A decoding module, configured to decode a target video corresponding to the application to the off-screen rendering environment through the playback component; A detection module, configured to extract a video frame to be detected from the off-screen rendering environment, input the video frame to be detected into a preset image detection model, and obtain a video frame detection result, where the image detection model is trained based on an image set including abnormal images; A compatibility determination module, configured to determine a playback component compatibility detection result according to the video frame detection result, where the playback component compatibility detection result indicates whether the playback component is compatible with the application.

9. The device according to claim 8, characterized in that, The decoding module is further configured to start the playback component, decode the target video through the playback component to obtain a decoded video frame; and render the video frame to the off-screen rendering environment according to an image rendering medium configured in the playback component, where the image rendering medium is associated with the off-screen rendering environment.

10. The device according to claim 8, characterized in that, The device further includes: An image rendering medium configuration module, configured to construct a texture identifier corresponding to the off-screen rendering environment according to the off-screen rendering environment; determine an image rendering medium corresponding to the texture object according to the texture object represented by the texture identifier; and configure the image rendering medium to the playback component.

11. The device according to claim 8, characterized in that, The detection module is further configured to perform frame extraction processing on video frames in the off-screen rendering environment at a preset frequency to obtain video frames to be detected.

12. The device according to claim 8, characterized in that, The decoding module is further configured to, when the playback mode of the off-screen rendering environment is silent playback, start the playback component to decode the target video and render the decoded video frame to the off-screen rendering environment; when the playback mode of the off-screen rendering environment is non-silent playback, adjust the playback mode to silent mode, start the playback component to decode the target video, and render the decoded video frame to the off-screen rendering environment.

13. The device according to claim 8, characterized in that, The decoding module is further configured to, when it is detected that the application switches to the background, record the background running duration; when the background running duration reaches a preset duration, start a detection process.

14. The device according to claim 8, characterized in that, The detection module is further configured to, when it is detected that the application switches from the background to the foreground, end the detection process.

15. A computer device, comprising a memory and a processor, where the memory stores a computer program, characterized in that, when the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

16. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images