Video playing method and related equipment

Abstract

The invention discloses a video playing method and related equipment. According to the video playing method, in the process that the electronic equipment decodes the video through the encoding and decoding assembly, the electronic equipment can correspondingly process the decoded image data, and then the texture corresponding to the processed image data is bound with the identifier, so that the video playing efficiency is improved. Subsequent electronic equipment can obtain the texture bound with the identifier through the application, drawing is carried out based on the texture, and finally the image to be displayed obtained through drawing is displayed on a display screen. According to the method, texture processing of the system side can be realized, that is, under the condition that the application side needs to decode and play the video through the coding and decoding capability of the system side, the system side can firstly process the video, so that the electronic equipment can display a better playing effect in each playing scene, and the user experience is improved. The situation that the final video playing effect is poor due to the fact that the application adopts different playing modes but the same processing flow is avoided.

Description

Technical Field

[0001] This application relates to the field of video playback technology, and in particular to a video playback method and related equipment. Background Technology

[0002] With the rapid development of video playback technology, video playback functionality is widely used in various applications, and the demand for video processing is also increasing. For some applications, although different playback methods are used to play videos, the same process is still applied to process the videos. This may result in poor video playback quality when using certain methods, such as darkened video images or incorrect video colors. Summary of the Invention

[0003] This application provides a video playback method and related equipment. According to the video playback method, before an electronic device plays a video, the electronic device can process the decoded image data on the system side, and then use the processed image data as a texture for manipulation. In this way, the application side of the electronic device can draw based on the texture, and finally display the drawn image on the display screen. This allows the electronic device to display a good playback effect in various playback scenarios, avoiding poor video playback effect caused by applications using different playback methods but the same processing flow.

[0004] In a first aspect, this application provides a video playback method. This method can be applied to an electronic device equipped with a display screen. The method may include: in response to a first operation, the electronic device can decode a first video, the first operation being used to trigger a first application to play the first video; if preset conditions are met, after decoding the first video to obtain Nth frame image data, the electronic device can process the Nth frame image data through an operating system to obtain a first texture, where N is a positive integer; the electronic device can use the first application to draw the Nth frame image data of the first video based on the first texture, and display the drawn Nth frame image data of the first video on the display screen.

[0005] It is understood that the image data / images involved in this application can refer to video frame data. The Nth frame image data involved in this application can refer to updated video frame data (or updated video frame data).

[0006] The process of an application playing video using a preset playback method can include: the application can notify the system to decode the video; the system can then decode the video and upload the decoded image data to the GPU, i.e., bind a texture, or manipulate the decoded image data as a texture; the application can then obtain the texture and draw and render based on it. This method is highly flexible, supports direct manipulation of textures, and real-time updates of texture content, thus presenting more complex and smoother rendering effects and providing more powerful rendering capabilities. The solution provided in this application further improves upon the above method. In the solution provided in this application, when an electronic device is preparing to play a video, the decoded image data can be processed on the system side before being uploaded to the GPU. Then, the processed image data is uploaded to the GPU. In this way, the decoded and processed image data can be manipulated as a texture. In this case, the application obtains the processed image data, or the texture obtained after processing the decoded image data. After obtaining the texture, the application can draw based on it and then display the drawn image, thereby achieving a better playback effect. The solution provided in this application enables electronic devices to not only present complex and smooth rendering effects, but also to further adjust and process the playback effects. Moreover, this processing is implemented on the system side of the electronic device, which means that the processing is applicable to all applications in the electronic device without requiring these applications to design and modify themselves. This allows the electronic device to adjust and correct the playback effects of each application when playing videos, resulting in high applicability.

[0007] It should be noted that, in some embodiments of this application, the electronic device can process each frame of image data (or video frame data) it decodes using the above method before displaying it, instead of waiting until the first video is fully decoded (i.e., all image data of the first video is decoded) before processing all the decoded image data using the above method. In other words, the above method can be continuously used during the process of the electronic device decoding the first video.

[0008] It is understood that the "system side" mentioned in this application refers to the operating system of the electronic device itself, mainly related to the capabilities of the operating system, and primarily involves the lower layers of the application layer in the electronic device's software architecture (e.g., framework layer, system libraries, HAL layer, and kernel layer). The "application side" mentioned in this application refers to the application itself, mainly related to its capabilities, and primarily involves the application layer in the electronic device's software architecture. It should be noted that the application side is relatively independent, especially for third-party applications, whose capabilities are primarily designed and developed by the third-party application developers. The application side can call upon the capabilities of the system side (i.e., the capabilities of the electronic device's operating system itself) to specifically implement the designed functions of the application side, such as video playback.

[0009] In some embodiments of this application, the first operation may be the first operation on the first control mentioned below.

[0010] In some embodiments of this application, the electronic device can decode the first video through an operating system. In one possible implementation, the electronic device can decode the first video through the codec component of the operating system. A description of the codec component is provided below and will not be elaborated upon here.

[0011] In some embodiments of this application, the first texture may be the corresponding texture obtained after processing the Nth frame image data as mentioned in step S104 below.

[0012] In some embodiments of this application, the electronic device can use the `setOnFrameAvailableListener()` method to listen for updates to the image data obtained by decoding the first video, that is, to listen for the Nth frame of image data obtained by decoding the first video, or to listen for updates to the Nth frame of image data. In simpler terms, the electronic device can use the `setOnFrameAvailableListener()` method to listen for the first frame of image data, the second frame of image data, etc., obtained by decoding the first video, so that the electronic device can promptly obtain the image data when a new frame of image data is decoded, thereby facilitating subsequent processing.

[0013] In some embodiments of this application, the electronic device draws the Nth frame image data based on the first texture through the first application. Specifically, the electronic device processes the first texture through the first application to obtain a second texture, obtains the processed second texture from the first application through the rendering thread, and then draws the Nth frame image data based on the second texture.

[0014] In some other embodiments of this application, the electronic device draws the Nth frame image data based on the first texture through the first application. Specifically, the electronic device may obtain the first texture through the first application, obtain the first texture from the first application through the rendering thread, and then draw the Nth frame image data based on the first texture.

[0015] It is understood that the playback effects involved in this application may include one or more of the following effects: brightness of the displayed image, filter, watermark, clarity, color, etc., and this application does not limit these effects. Of course, playback effects may also include other effects (e.g., transition effects, slow motion, split screen, fast motion, and other special visual effects), and this application does not limit these effects.

[0016] In conjunction with the first aspect, in one possible implementation, the preset conditions are met, which may specifically include: playing the first video using a preset playback method.

[0017] In the solution provided in this application, for applications that use a preset playback mode and require video playback, the electronic device can process the decoded image data on the system side. In this way, the electronic device can identify the playback scenario and process the decoded image data (e.g., the Nth frame image data) on the system side only when the preset playback mode is used. This allows for better playback performance in specific scenarios while minimizing power consumption.

[0018] In some embodiments of this application, the preset playback method can be a playback method based on TextureView. It should be noted that the preset playback method can also be other playback methods involving texture uploading (or texture binding), and this application does not limit this.

[0019] In some embodiments of this application, the electronic device may specifically process the decoded new image data through the operating system's graphics library (or graphics renderer). In one possible implementation, the electronic device may process the decoded new image data through the rendering module in the graphics library. It is understood that the processing of the decoded new image data through the operating system's graphics library is equivalent to the system-side processing of the decoded new image data.

[0020] As described below, since the system-side processing of the newly decoded image data occurs before the application processes the data and draws the image, the system-side processing of the newly decoded image data can also be referred to as preprocessing.

[0021] In some embodiments of this application, not only can the system-side processing of decoded image data be implemented for applications in electronic devices that employ a preset playback mode and require video playback, but system-side processing can also be implemented in specific playback scenarios. In one possible implementation, the electronic device processes the newly decoded image data on the system side only when a preset playback mode is employed and a first preset condition is met. In this way, the electronic device can identify the playback scenario and process the newly decoded image data on the system side only in specific scenarios, thus achieving better playback performance in specific scenarios while minimizing power consumption.

[0022] It is understood that the preset conditions involved in this application may include a first preset condition. The electronic device can identify the playback scene based on the first preset condition. When the first preset condition is met, the electronic device can determine that the current playback scene is a specific scene. When the first preset condition is not met, the electronic device can determine that the current playback scene is not a specific scene. In this case, after decoding the Nth frame image, the electronic device can directly convert the Nth frame image into a texture without processing the Nth frame image. Correspondingly, the first application can directly draw and display based on the converted texture.

[0023] In conjunction with the first aspect, in one possible implementation, satisfying the preset conditions may also include any one or more of the following: the first application is a preset application, the first operation is a preset operation, or the interface identifier to be drawn corresponding to the first operation is a preset interface identifier.

[0024] Optionally, satisfying preset conditions may specifically include: playing the first video using a preset playback method, and satisfying the first preset condition. The first preset condition may include any one or more of the following: the first application is a preset application, the first operation is a preset operation, or the interface identifier to be drawn corresponding to the first operation is a preset interface identifier. A description of the first preset condition can be found below, and will not be elaborated upon here.

[0025] In the solution provided in this application, the electronic device can process the new image data decoded on the system side only in a specific scenario. The specific scenario can be determined based on any one or more factors such as application, operation and interface identifier to be drawn. In this way, specific scenarios can be set from different aspects to meet the user's needs for playback effects in various scenarios.

[0026] In some embodiments of this application, the identifier of the interface to be drawn corresponding to the first operation can be the identifier corresponding to the interface to be drawn by the electronic device triggered by the first operation. In one possible implementation, the first operation can be the first operation for the first control mentioned below.

[0027] In some embodiments of this application, when the first operation is the first operation on the first control mentioned below, if satisfying the first preset condition includes the first operation being a preset operation, then satisfying the first preset condition may also include the first control being a preset control.

[0028] In conjunction with the first aspect, in one possible implementation, after decoding the first video to obtain the Nth frame image data, the electronic device processes the decoded Nth frame image data through the operating system. Specifically, the electronic device may determine the processing method based on the first application through the operating system, and process the Nth frame image data based on the processing method.

[0029] In the solution provided in this application, the specific method by which the electronic device processes the Nth frame image data on the system side can be determined based on the first application, that is, based on the application that currently needs to play the video. In this way, when the user plays the video using different applications, the electronic device can adopt corresponding processing methods on the system side. These corresponding processing methods can be the same or different; they are not necessarily the same. The playback effect can be specifically modified or adjusted when playing the video using different applications, thereby meeting the user's needs for playback effects when playing the video using different applications. Here, the corresponding processing method is the processing method corresponding to the application playing the video.

[0030] In some embodiments of this application, if a first preset condition is met, and the image data obtained by decoding the first video is updated (e.g., the Nth frame image data is obtained by decoding), the electronic device can process the updated image data (e.g., the Nth frame image data). Meeting the first preset condition includes the first application being a preset application. In this case, the electronic device can determine the processing method based on the first application. Of course, the electronic device can also determine the processing method based on other factors (e.g., the first control, the first operation, the interface identifier to be drawn corresponding to the first operation, etc.), and this application does not impose any limitations on this.

[0031] In some embodiments of this application, there is a correspondence between the identifier of the first application and the processing method. The electronic device determines the processing method based on the first application and processes the decoded Nth frame image data based on the processing method. Specifically, the electronic device may determine the processing method corresponding to the identifier of the first application based on the correspondence and use the processing method to process the Nth frame image data.

[0032] It is understood that the identifier of the first application may include the application name, package name, etc., and this application does not limit its specific form.

[0033] In some embodiments of this application, there is a correspondence between the category of the first application and the processing method. The electronic device determines the processing method based on the first application and processes the decoded Nth frame image data based on the processing method. Specifically, the electronic device may determine the processing method corresponding to the category of the first application based on the correspondence and use the processing method to process the Nth frame image data.

[0034] In some embodiments of this application, there is a correspondence between the list of first applications and the processing methods. The electronic device determines the processing method based on the first application and processes the decoded Nth frame image data based on the processing method. Specifically, the electronic device may determine the processing method corresponding to the list of first applications based on the correspondence and use the processing method to process the Nth frame image data.

[0035] Of course, the processing method determined by the electronic device based on the first application may include other specific implementation methods, and this application does not limit this.

[0036] In conjunction with the first aspect, in one possible implementation, in response to the first operation, the method may further include: displaying a first interface, the first interface being a user interface of a first application, the first interface including a first control. The first operation is an operation on the first control. After decoding the first video to obtain the Nth frame image data, the electronic device processes the decoded Nth frame image data through an operating system, specifically including: the electronic device can determine a processing method based on the first control and the first operation through the operating system, and process the Nth frame image data based on the processing method.

[0037] In the solution provided in this application, the specific way in which the electronic device processes the Nth frame image data on the system side can be determined based on the first control and the first operation. That is, the corresponding processing method is determined based on the control and operation that currently triggers playback. In this way, when the user triggers video playback through different controls and different operations, the electronic device can adopt the corresponding processing method on the system side, instead of using the same processing method. The playback effect can be specifically modified or adjusted when the video is triggered by different controls and different operations, thereby meeting the user's needs for playback effect in different playback scenarios.

[0038] In some embodiments of this application, if a first preset condition is met, and the image data obtained by decoding the first video is updated (e.g., the Nth frame image data is obtained by decoding), the electronic device can process the updated image data (e.g., the Nth frame image data). Meeting the first preset condition includes the first operation being a preset operation. In this case, the electronic device can determine the processing method based on the first operation. Of course, the electronic device can also determine the processing method based on other factors (e.g., the first application, the interface identifier to be drawn corresponding to the first operation, etc.), and this application does not impose any limitations on this.

[0039] In some embodiments of this application, if a first preset condition is met, and the image data obtained by decoding the first video is updated, the electronic device can process the updated image data. Furthermore, the first operation is an operation on a first control, and meeting the first preset condition includes the first operation being a preset operation and the first control being a preset control. In this case, the electronic device can determine the processing method based on the first control and the first operation. Of course, the electronic device can also determine the processing method based on other factors (e.g., the first application, the interface identifier to be drawn corresponding to the first operation, etc.), and this application does not impose any limitations on this.

[0040] In some embodiments of this application, there is a correspondence between the combination of controls and operations that trigger video playback and the processing method. The electronic device determines the processing method based on the first control and the first operation, and processes the decoded Nth frame image data based on the processing method. Specifically, the electronic device can determine the processing method corresponding to the combination of the first control and the first operation based on the correspondence, and process the Nth frame image data based on the processing method.

[0041] In some embodiments of this application, there is a correspondence between the combination of the control list and the operation list and the processing method. The electronic device determines the processing method based on the first control and the first operation, and processes the decoded Nth frame image data based on the processing method. Specifically, the electronic device can determine the processing method corresponding to the combination of the first control list and the first operation list based on the correspondence, and process the Nth frame image data based on the processing method.

[0042] In some embodiments of this application, a specific list corresponds to a specific processing method, and the specific list may include controls and / or operations. The electronic device determines the processing method based on a first control and a first operation, and processes the decoded Nth frame image data based on that processing method. Specifically, the electronic device may determine whether both the first control and the first operation are within the specific list. If both are within the specific list, the electronic device may process the Nth frame image data based on the specific processing method corresponding to the specific list. If neither is within the specific list, the electronic device may process the Nth frame image data based on other processing methods. These other processing methods may be different from the specific processing method, and this application does not limit this.

[0043] Of course, the electronic device may also include other specific implementation methods based on the first control and the first operation to determine the processing method, and this application does not limit this.

[0044] In conjunction with the first aspect, in one possible implementation, after decoding the first video to obtain the Nth frame image data, the electronic device processes the decoded Nth frame image data through the operating system. Specifically, this may include: the electronic device determining the interface identifier to be drawn corresponding to the first operation; the electronic device may also determine the processing method based on the interface identifier to be drawn corresponding to the first operation through the operating system, and process the Nth frame image data based on the processing method.

[0045] In the solution provided in this application, the specific way in which the electronic device processes the Nth frame image data on the system side can be determined based on the interface identifier to be drawn corresponding to the first operation. That is, the corresponding processing method is determined based on the identifier corresponding to the interface triggered by the current first operation. In this way, when the user triggers video playback and triggers the electronic device to draw the interface through different operations, the electronic device can adopt the corresponding processing method on the system side, without having to adopt the same processing method. The corresponding playback effect when triggering the drawing of different interfaces can be specifically corrected or adjusted, thereby meeting the user's needs for playback effect in different playback scenarios.

[0046] In some embodiments of this application, the first operation is an operation on a first control. In this case, the interface to be drawn corresponding to the first operation is the interface to be drawn by the electronic device triggered by the first operation on the first control.

[0047] In some embodiments of this application, if a first preset condition is met, and the image data obtained by decoding the first video is updated (e.g., the Nth frame image data is obtained by decoding), the electronic device can process the updated image data (e.g., the Nth frame image data). Meeting the first preset condition includes the interface identifier to be drawn corresponding to the first operation being a preset interface identifier. In this case, the electronic device can determine the processing method based on the interface identifier to be drawn corresponding to the first operation. Of course, the electronic device can also determine the processing method based on other factors (e.g., the first application, the first control, and the first operation, etc.), and this application does not limit this.

[0048] In some embodiments of this application, there is a correspondence between the interface identifier to be drawn and the processing method. The electronic device determines the processing method based on the interface identifier to be drawn corresponding to the first operation, and processes the decoded Nth frame image data based on the processing method. Specifically, the electronic device can determine the processing method corresponding to the interface identifier to be drawn corresponding to the first operation based on the correspondence, and process the Nth frame image data based on the processing method.

[0049] In conjunction with the first aspect, in one possible implementation, the processing method may include any one or more of the following: video compression, adding effects, or color gamut conversion. The added effects may include any one or more of the following: increasing / decreasing brightness, adding a watermark, or setting a high dynamic range (HDR) imaging effect.

[0050] In the solution provided in this application, the electronic device can process the decoded Nth frame image data in any one or more of a variety of processing methods, so that the playback effect can be improved based on different processing methods in different playback scenarios, thereby meeting the user's needs for playback effect in different playback scenarios.

[0051] In conjunction with the first aspect, in one possible implementation, the method may further include: in response to the first operation, under a preset playback mode, the electronic device can create a first surface texture object through the operating system and a second surface texture object through the first application. The electronic device processes the Nth frame image data through the operating system to obtain the first texture, specifically including: the electronic device can process the Nth frame image data through the operating system's graphics renderer, generate the first texture based on the processed Nth frame image data, and update the texture corresponding to the first surface texture object to the first texture. The electronic device draws the Nth frame image data based on the first texture through the first application, specifically including: the electronic device can obtain the first texture corresponding to the first surface texture object through the first application, update the texture corresponding to the second surface texture object to the first texture, or process the first texture to obtain the second texture, and update the texture corresponding to the second surface texture object to the second texture; the electronic device may also draw the Nth frame image data based on the texture corresponding to the second surface texture object.

[0052] In the solution provided in this application, the application side of the electronic device can process the decoded image data during video playback, or it can choose not to process it. If the application side processes the decoded image data during video playback, before performing this processing, the electronic device can correspondingly set the system-side processing method based on the application side's processing method. This allows the system-side processing method and the application-side processing method to cooperate, thereby achieving more complex and better playback effects. Conversely, if the application side does not process the decoded image data during video playback, the electronic device can process the decoded image data on the system side before rendering it on the application side. This achieves better playback effects and avoids poor playback quality due to the application side not performing any processing.

[0053] In some embodiments of this application, the method may further include: in response to a first operation, when a preset playback mode is adopted, the electronic device may determine a first identifier and a second identifier, wherein the first identifier is used to identify the texture corresponding to a first surface texture object, and the second identifier is used to identify the texture corresponding to a second surface texture object. In this case, the electronic device obtains the first texture corresponding to the first surface texture object through a first application, which may specifically include: the electronic device may obtain the first texture based on the first identifier through the first application. Updating the texture corresponding to the first surface texture object to the first texture may specifically include: binding the first texture to the first identifier. Updating the texture corresponding to the second surface texture object to the first texture may specifically include: binding the first texture to the second identifier. Updating the texture corresponding to the second surface texture object to the second texture may specifically include: binding the second texture to the second identifier.

[0054] It is understandable that when an electronic device binds a first texture to a second identifier using a second surface texture object, the texture corresponding to the second identifier obtained by the electronic device through the first application is the first texture. Conversely, when an electronic device processes the first texture using a second surface texture object to obtain a second texture, and then binds the second texture to the second identifier, the texture corresponding to the second identifier obtained by the electronic device through the first application is the second texture.

[0055] It is understood that the texture corresponding to the second identifier obtained by the electronic device through the first application can be the texture corresponding to the second surface texture object, or the texture in the second surface texture object.

[0056] It is understood that electronic devices can also acquire the texture corresponding to the image data in other ways, and this application does not limit this.

[0057] In a second aspect, this application provides an electronic device including a display screen, one or more memories, and one or more processors; the display screen is coupled to the one or more memories and the one or more processors, the display screen is used to display images, the memories are used to store computer program code including computer instructions, and the one or more processors call the computer instructions to cause the electronic device to perform the method described in the first aspect or any implementation thereof.

[0058] Thirdly, this application provides a computer storage medium. The computer storage medium includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect or any implementation thereof.

[0059] Fourthly, embodiments of this application provide a chip. This chip can be applied to an electronic device, and the chip includes one or more processors. The processors are used to invoke computer instructions to cause the electronic device to perform the methods described in the first aspect or any implementation thereof.

[0060] Fifthly, embodiments of this application provide a chip system. This chip system can be applied to an electronic device. The chip system includes at least one processor and an interface for receiving instructions and transmitting them to the at least one processor; the at least one processor executes the received instructions, causing the electronic device to perform the method described in the first aspect or any implementation thereof.

[0061] In some embodiments of this application, the chip system may be an application processor (AP) or a system on chip (SoC) including an AP. The method described in the first aspect or any implementation thereof may be implemented by an AP, and the method described in the second aspect or any implementation thereof may be implemented by an AP.

[0062] In some other embodiments of this application, the chip system may include an access point (AP) and other modules. These other modules may be a modem (also known as a baseband processor).

[0063] Sixthly, embodiments of this application provide a computer program product including instructions. When the computer program product is run on an electronic device, it causes the electronic device to perform the method described in the first aspect or any implementation thereof.

[0064] It is understood that the electronic device provided in the second aspect, the computer storage medium provided in the third aspect, the chip provided in the fourth aspect, the chip system provided in the fifth aspect, and the computer program product provided in the sixth aspect are all used to execute the method described in the first aspect or any implementation thereof. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects of any possible implementation of the first aspect, which will not be repeated here. Attached Figure Description

[0065] Figure 1 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0066] Figure 2 A schematic diagram of the software structure of an electronic device provided in an embodiment of this application;

[0067] Figure 3A flowchart illustrating a video playback method provided in this application embodiment;

[0068] Figure 4 A schematic diagram of the user interface provided for an embodiment of this application;

[0069] Figure 5 A flowchart illustrating yet another video playback method provided in this application embodiment. Detailed Implementation

[0070] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0071] It should be understood that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0072] It should be understood that the term "user interface" in the specification, claims, and drawings of this application refers to the medium interface through which an application or operating system interacts and exchanges information with the user. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0073] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0074] With the rapid development of video playback technology, video playback functionality is widely used in various applications. For example, applications under the Android operating system often use TextureView to host video sources for video playback. Furthermore, the demand for video processing is increasing during video playback. For some applications, despite using different playback methods, the same process is used to process the video, which may lead to poor playback quality with certain methods, such as a darkened video or incorrect colors. For instance, after a user sends a video through a communication application, in response to the user's click on the video, the application can use TextureView to play it. Specifically, the application can instruct the codec component to decode the video and capture the decoded image data using a SurfaceTexture. This decoded image data is then uploaded as a texture to the Graphics Processing Unit (GPU). The application can then obtain the texture uploaded to the GPU and process and display it accordingly. However, compared to viewing the video locally, the video played in the communication application appears washed out.

[0075] This application provides a video playback method and related equipment. According to this video playback method, during the decoding process of video by an electronic device through an encoding / decoding component, the electronic device can process the decoded image data accordingly, and then bind the processed image data as a texture to an identifier. Subsequently, the electronic device can obtain the texture bound to the identifier through an application and draw the texture onto the display screen. This method enables system-side texture processing. That is, when the application side needs to decode and play video using the encoding / decoding capabilities of the electronic device system, the system side can process the video first. This allows the electronic device to display good playback effects in various playback scenarios, avoiding poor final video playback effects caused by applications using different playback methods but with corresponding processing flows.

[0076] To better understand the video playback method provided in the embodiments of this application, some technical terms involved in this application will be explained below.

[0077] 1. Texture

[0078] A texture (or texture data) is a two-dimensional image, typically composed of pixels, whose pixel values ​​can be accessed via texture coordinates. Textures can be mapped onto the surface of a 3D model to give the model effects such as color, texture, and lighting.

[0079] Textures are typically used in conjunction with shape representations (usually 3D models) to render an entire object or scene. Together with the 3D model, they provide users with a more realistic and immersive visual experience. Texture data usually does not include a file header; instead, it is a contiguous area storing color values, allowing it to be directly used by shader programs. It is typically stored in video memory (VRAM). VRAM, also called the frame buffer, stores rendering data processed or about to be retrieved by the graphics card chip. In contrast, images are independent representations of objects or scenes. They generally refer to formatted files such as bitmaps (BMP), Joint Photographic Experts Group (JPEG / JPG), and Portable Network Graphics (PNG). Images consist of a file header and a data body, which may include compression algorithms to reduce file size. Images are usually located on the system hard drive and are loaded into memory for use by user-developed applications.

[0080] Furthermore, textures can be considered a special form of image that contains a host of additional information beyond color and transparency arrangements, such as sampler parameters and wrapping rules. This information enables textures to achieve more complex rendering effects in computer graphics.

[0081] 2. Surface

[0082] A surface is a canvas or buffer used for drawing images, on which drawing operations can be performed. A surface can be a screen, a texture, a framebuffer, etc.

[0083] 3. TextureView and SurfaceTexture

[0084] TextureView is a type of view control used to display updatable images or videos. It's a subclass of View, allowing images to be updated in a thread other than the UI thread. TextureView supports hardware acceleration, meaning it can utilize the GPU for faster processing and smoother image display. TextureView can also update in a background thread, avoiding UI thread blocking. Furthermore, TextureView can be overlaid with other views to achieve complex image composition effects.

[0085] SurfaceTexture is a core component for rendering in the Android operating system (or simply Android system). It can be simply understood as an "intermediate tool for producing textures," also known as a texture container. It typically works in conjunction with other views or functional modules. SurfaceTexture is used to interact with images within graphics libraries (such as OpenGL ES). It provides a mechanism to directly pass image data streams to the graphics library for rendering without copying the image data into the central processing unit (CPU) memory.

[0086] The graphics library can be referred to as a graphics renderer. OpenGL ES stands for Open Graphics Library for Embedded Systems. It is a 3D graphics application programming interface (API) specifically designed for embedded systems and mobile computing devices, and is the embedded version of OpenGL.

[0087] SurfaceTexture can receive image data from a camera, video decoder, or other image sources and provide it as texture data (e.g., OpenGL ES textures) to the Render Thread for processing. It can also be used to implement functions such as video playback, camera preview, and real-time image processing. The Render Thread's role and function in the Android system differ from application threads. Starting with Android 5.0, RenderThread was introduced as an independent rendering thread, primarily responsible for UI rendering, distinct from the Main Thread. The Main Thread is mainly responsible for handling all activities on the user interface, including receiving user input and processing lifecycle events, while RenderThread focuses on UI drawing and display. This separation helps improve rendering performance, avoids main thread blocking, and thus improves application responsiveness and user experience. In short, RenderThread is a thread managed by the operating system, not an application thread.

[0088] SurfaceTexture works by creating a Surface object that shares a context with OpenGL ES and passing it as an argument to the SurfaceTexture constructor. The application can then obtain the texture coordinate transformation matrix and apply it to the texture in the OpenGL ES rendering pipeline.

[0089] The apparatus involved in the embodiments of this application is described below.

[0090] The electronic devices involved in this application can be terminal devices, specifically mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), or dedicated cameras (e.g., SLR cameras, point-and-shoot cameras), etc. The embodiments of this application do not impose any restrictions on the specific type of electronic devices.

[0091] The hardware structure of the electronic device involved in the embodiments of this application is described below.

[0092] Please see Figure 1 , Figure 1 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0093] like Figure 1As shown, the electronic device may include: a processor, an external memory interface, internal memory, a Universal Serial Bus (USB) interface, a charging management module, a power management module, a battery, antenna 1, antenna 2, a mobile communication module, a wireless communication module, a sensor module, buttons, a motor, an indicator, a camera, a display screen, an audio module, and a Subscriber Identity Module (SIM) card slot, etc. The audio module may include a speaker, receiver, microphone, headphone jack, etc., and the sensor module may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, proximity sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0094] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. It is understood that the illustrated components can be implemented in hardware, software, or a combination of both. In some embodiments of this application, the electronic device may include more components than illustrated. For example, the electronic device may include other types of sensors. In still other embodiments of this application, the electronic device may include fewer components than illustrated, or combine some components, or split some components, or arrange different components. The interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device.

[0095] A processor may include one or more processing units, such as an application processor (AP), a modem (also known as a baseband processor), a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), and / or a neural network processing unit (NPU). The AP is responsible for running the operating system and applications. The modem is responsible for handling various communication protocols.

[0096] The wireless communication function of an electronic device can be implemented through antenna 1, antenna 2, a mobile communication module, a wireless communication module, and a modem. The modem can interact with the base station through antennas (e.g., antenna 1, antenna 2, etc.). In some embodiments, antenna 1 of the electronic device is coupled to the mobile communication module, and antenna 2 is coupled to the wireless communication module, enabling the electronic device to communicate with networks and other devices through wireless communication technology.

[0097] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0098] A GPU is a microprocessor for image processing, connected to a display screen and an application processor. A GPU performs mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information. A display screen is used to display images, videos, etc. In some embodiments, an electronic device may include one or more displays screens.

[0099] A camera is used to capture still images or videos. An ISP (Image Signal Processor) processes the data fed back from the camera. Light is transmitted through the lens to the camera's photosensitive element, where the light signal is converted into an electrical signal. The camera's photosensitive element then transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. Electronic devices may include one or more cameras.

[0100] Internal memory may include one or more RAMs and one or more non-volatile memory (NVMs). RAMs can be directly read and written by the processor and can be used to store executable programs (e.g., machine instructions) of the operating system or other running programs, as well as user and application data. NVMs can also store executable programs and user and application data, and can be pre-loaded into RAMs for direct processor access.

[0101] In this embodiment of the application, the code implementing the method described in this embodiment can be stored in non-volatile memory. When running an application that needs to play video (e.g., the first application), the electronic device can load the executable code stored in the non-volatile memory into random access memory.

[0102] External memory interfaces can be used to connect to external non-volatile memory, thereby expanding the storage capacity of electronic devices.

[0103] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0104] The operating system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses the layered architecture Android system as an example to exemplify the software structure of an electronic device. It should be noted that although this application uses the Android system as an example, its basic principles are equally applicable to electronic devices based on operating systems such as iOS or Windows.

[0105] Figure 2 This is a schematic diagram of the software structure of an electronic device provided in an embodiment of this application.

[0106] The software architecture of electronic devices adopts a layered architecture, dividing the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. Taking the Android system running on an application platform as an example, in some embodiments of this application, the Android system's software architecture is divided into five layers, from top to bottom: the application layer, the framework layer, the Android runtime and system libraries, the hardware abstraction layer (HAL), and the kernel layer.

[0107] The application layer can include a series of application packages. These application packages may include applications such as camera, gallery, calendar, call, map, WLAN, Bluetooth, music, video, and SMS. The application layer may also include note-taking applications. Furthermore, the application layer may include the system user interface (systemUI). The systemUI is used to display the electronic device's interface, such as displaying the note-taking interface, the signal icon corresponding to the SIM card, and the call interface.

[0108] It is understood that the application layer may also include other applications, such as the communication applications mentioned above, and this application does not limit this.

[0109] The framework layer provides APIs and a programming framework for the application layer. The framework layer can include predefined functions. For example, it may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc. The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build the application. The display interface can consist of one or more views. For example, a display interface including a text notification icon can include views for displaying text and views for displaying images. The view system can also include TextureView and SurfaceTexture. The meaning and function of TextureView and SurfaceTexture can be found above, and will not be repeated here.

[0110] The framework layer may also include a multimedia framework. A multimedia framework can be a set of APIs providing audio, video, and image support, including media playback, recording, editing, streaming media, and other related functions. A multimedia framework may include multimedia components from multiple framework layers, such as codec components within the framework layer. In some embodiments of this application, the codec component in the framework layer may be MediaCodec, an audio / video codec class provided by Android that allows applications to encode (compress) and decode (decompress) audio and video data. MediaCodec enables audio and video playback, recording, transcoding, and other functions. MediaCodec can access the underlying codec components; it does not possess codec capabilities itself but obtains them by invoking the underlying codec components.

[0111] It should be noted that the codec components mentioned above can include multiple layers of codec components in the software architecture of electronic devices, such as codec components at the framework layer, system library, and HAL layer. It can be understood that the codec components at the framework layer can also be called upper-layer codec components, and the codec components at the HAL layer (Media HAL mentioned below) can also be called lower-layer codec components.

[0112] The framework layer may also include an input module. The input module can collect, process, and distribute various input events (e.g., key events, touch events, etc.). In some embodiments of this application, the input module can be an Input Manager Service (IMS). IMS can monitor device nodes (e.g., all device nodes under / dev / input), and when a device node has data, it will process the data and find a suitable window to dispatch the input event to it. In some embodiments of this application, the input event management module can be an inputflinger. Inputflinger is a service responsible for handling input events; it can receive raw input events from hardware devices and convert them into events that the Android system can understand, such as touch events.

[0113] The runtime is responsible for system scheduling and management. The runtime includes the core libraries and the virtual machine. The core libraries consist of two parts: one part contains the functionalities that the programming language (e.g., Java) needs to call, and the other part is the system's core libraries. The application layer and framework layer run in the virtual machine. The virtual machine executes the programming files (e.g., Java files) of the application layer and framework layer into binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0114] The system library may include multiple functional modules, such as media libraries and graphics libraries. The graphics library may include 3D graphics processing libraries (e.g., OpenGL ES) and 2D graphics engines (e.g., SGL). The media library supports playback and recording of various common audio and video formats, as well as still image files. The media library can support various audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing library can be used to implement 3D graphics drawing, image rendering, compositing, and layer processing. The 2D graphics engine is a drawing engine for 2D graphics. In some embodiments of this application, the 2D graphics engine may include a hardware-accelerated rendering engine for UI (HWUI). HWUI is a GPU-accelerated 2D graphics engine. HWUI aims to provide efficient, stable, and high-quality 2D graphics rendering capabilities, providing technical support for the Android system's UI experience, such as implementing various Android View components through HWUI. The drawing instructions for the application interface can ultimately be executed in the rendering thread through the HWUI library (or libhwui).

[0115] The graphics library may also include a drawing and rendering module. This module is part of the hardware-accelerated rendering pipeline and is used to pass image data from producers to consumers, such as uploading textures to the GPU. In some embodiments of this application, the drawing and rendering module may include EGLConsumer. The EGLConsumer class is an important class in the Android system used as a consumer to process EGL (OpenGL ES) buffers in graphics rendering. It is part of the Android hardware-accelerated rendering pipeline and is used to pass image data from producers to consumers. The main function of the EGLConsumer class is to receive EGL buffers from producers and pass them to consumers for display or other processing. It provides a mechanism that allows multiple threads to access and process EGL buffers simultaneously, thereby achieving efficient graphics rendering.

[0116] The system library may also include a codec module. The codec module in the system library can also be understood as the codec component mentioned above located in the system library, which is responsible for communication involving codecs between the framework layer and the HAL layer. In some embodiments of this application, the media library may include a codec module.

[0117] The Hardware Abstraction Layer (HAL) is an interface layer located between the operating system kernel and upper-level software, its purpose being to abstract hardware. The HAL is an abstract interface for device kernel drivers, used to provide application programming interfaces (APIs) for accessing the underlying devices to higher-level Java API frameworks. The HAL can provide a standard interface to display device hardware functionality to higher-level Java API frameworks. The HAL contains several library modules (e.g., Camera HAL, Audio HAL, and Multimedia HAL). When the system framework layer API requests access to the portable device's hardware, the operating system loads the library module for that hardware component. The Multimedia HAL, also known as the Media HAL, provides a series of interfaces for accessing upper-level multimedia modules and lower-level multimedia devices, responsible for interacting with multimedia hardware (e.g., encoders, cameras, microphones, speakers, etc.).

[0118] The kernel layer is the foundation of the Android system. It is responsible for hardware drivers, networking, power, system security, and memory management. The kernel layer acts as an intermediary between hardware and software, relaying application requests to the hardware. It can include audio drivers, display drivers, camera drivers, and sensor drivers.

[0119] It should be noted that the application provided Figure 2 The illustrated software architecture diagram of the electronic device is merely an example and does not limit the specific module divisions within different layers of the Android system. For details, please refer to the descriptions of the Android system software architecture in conventional technologies. Furthermore, the method provided in this application can also be implemented on other operating systems, which will not be listed here.

[0120] The following describes a video playback method provided by an embodiment of this application.

[0121] Please see Figure 3 , Figure 3 A flowchart illustrating a video playback method provided in this application embodiment. This video playback method can be applied to electronic devices. The video playback method may include, but is not limited to, the following steps:

[0122] S101: The electronic device displays a first interface. The first interface is the user interface of a first application, and includes a first control used to trigger the playback of a first video.

[0123] When an electronic device runs a first application, it can display a first interface. The first interface may include a first control, which is used to trigger the first application to play a first video. It is understood that the first application can play videos, or that the first application is an application with video playback functionality. This application does not limit the specific category of the first application. For example, the first application can be a video application (which can be simply referred to as a video application), a communication application (which can be simply referred to as a communication application), or a news application (which can be simply referred to as a news application). It is understood that the video corresponding to the first control (or the video that the first control is used to trigger playback), i.e., the first video, can be referred to as the video to be played. In some embodiments of this application, the first control can be a video thumbnail.

[0124] For example, the first application can be as follows: Figure 4 The communication application shown can have a first control such as... Figure 4 The video thumbnail 11 or video thumbnail 12 shown below, correspondingly, the first video can be as follows: Figure 4 The video corresponding to video thumbnail 11 or video thumbnail 12 shown. When the electronic device runs the first application, it can display videos such as... Figure 4 The chat interface 1a shown is for the user and the chat partner (e.g., ...). Figure 4 The example shown is Lao Zhang's (Old Zhang's) chat interface. Chat interface 1a may include video thumbnails 11 and 12. Video thumbnail 11 is the thumbnail corresponding to the video shared by the chat partner to the user, while video thumbnail 12 is the thumbnail corresponding to the video shared by the user to their chat partner. The user can play the video shared by the chat partner by clicking video thumbnail 11, and can also play their own shared video by clicking video thumbnail 12.

[0125] Of course, the video playback function of the first application can be implemented based on the electronic device's operating system and display screen. For example, video decoding can be implemented based on the codec component of the electronic device's operating system, the graphics renderer can be implemented based on the electronic device's operating system to process the decoded image data, and the image data can be displayed based on the electronic device's display screen.

[0126] S102: In response to a first operation on the first control, when the first application adopts a preset playback mode, the electronic device creates a first surface texture object through the codec component and creates a second surface texture object through the first application.

[0127] When the electronic device is running a first application, the electronic device can display a first interface. In response to a first operation on a first control in the first interface, if the first application plays the video corresponding to the first control using a preset playback method, the electronic device can create a first surface texture object (i.e., a first SurfaceTexture object) through a codec component, and create a second surface texture object (i.e., a second SurfaceTexture object) through the first application.

[0128] It is understood that the preset playback method can be set according to actual needs, and this application does not impose any restrictions on it. In some embodiments of this application, the preset playback method can be a method of playing video based on TextureView.

[0129] In essence, a surface texture object is a SurfaceTexture object. The first surface texture object can convert image data (or video frame data) decoded by the codec component into a texture. The first surface texture object can monitor the image data decoded by the codec component. The second surface texture object can process the texture converted by the first surface texture object to obtain a processed texture for subsequent use in the rendering process, such as drawing based on this processed texture by the rendering thread. This means that the electronic device can update the texture corresponding to the first surface texture object (or the texture within the first surface texture object) based on the new image data decoded by the codec component. The second surface texture object can monitor the image data within the first surface texture object; that is, the second surface texture object can monitor the texture converted by the first surface texture object. This also means that the electronic device can update the texture corresponding to the second surface texture object (or the texture within the second surface texture object) based on the updated texture of the first surface texture object.

[0130] Understandably, the first operation can also be other operations, such as other touch, voice control, or gesture operations, to trigger the first application to play the first video.

[0131] S103: The electronic device decodes the first video through the codec component to obtain multiple frames of image data.

[0132] In response to the first operation on the first control, the electronic device can decode the first video frame by frame through the codec component to obtain multiple frames of image data. It can be understood that during the process of decoding the first video through the codec component, after each frame of image data (or one frame of video frame data) is decoded, the electronic device can execute subsequent steps S104-S106.

[0133] The following example illustrates steps S104-S106 by taking the decoding of new image data as the Nth frame image data (or the updated video frame data as the Nth frame image data). Here, N is a positive integer, and N is not greater than the total number of frames in the first video.

[0134] S104: After decoding the Nth frame image data, the electronic device updates the texture corresponding to the first surface texture object based on the Nth frame image data. If the first preset condition is met, during the process of updating the texture corresponding to the first surface texture object, the electronic device obtains the context through the graphics library, processes the Nth frame image data to obtain the corresponding texture, and binds the texture to the first identifier.

[0135] After decoding a new frame of image data, such as the Nth frame, the electronic device can acquire the Nth frame and update the texture corresponding to the first surface texture object based on it. In other words, during the decoding process of the first video by the electronic device's encoding / decoding component, after each frame of image data (or video frame) is decoded, the first surface texture object can update its own texture based on the decoded image data. Similarly, after each update of the texture by the first surface texture object, the second surface texture object can update its own texture based on the updated texture of the first surface texture object.

[0136] It should be noted that during the process of updating the texture corresponding to the first surface texture object, if the first preset condition is met, the electronic device can obtain the context based on the graphics library (for example, by obtaining the context through the drawing and rendering module), process the Nth frame image data, obtain the processed image data, and upload the processed image data to the GPU, such as by sending it to OpenGL ES. In this way, the processed image data can be manipulated as a texture, such as being called or processed.

[0137] Optionally, during the process of updating the texture corresponding to the first surface texture object, if the first preset condition is met, the electronic device can directly process the Nth frame image data to obtain the corresponding texture and upload the texture to the GPU.

[0138] In some embodiments of this application, during the process of updating the texture corresponding to the first surface texture object, the electronic device uploads the processed image data to the GPU. Specifically, this may include binding the processed image data with a first identifier. It is understood that after the electronic device binds the processed image data with the first identifier, the processed image data can be manipulated as a texture. This means that the meaning of binding the processed image data with the first identifier in this application is the same as binding the texture corresponding to the processed image data with the first identifier (i.e., binding the texture obtained after processing the Nth frame of image data with the first identifier).

[0139] It is understandable that the first identifier is the identifier corresponding to the first surface texture object, which is used to convert the image data obtained by the first surface texture object into a texture, and also to identify the texture corresponding to the first surface texture object.

[0140] It should be noted that the texture mentioned in this application is essentially image data. The difference between the texture and the decoded image data and the image data after processing the decoded image data mentioned in this application is that after the image data is uploaded to the GPU (or image data binding identifier, or image data sent to a graphics library such as OpenGL ES), the image data can be manipulated as a texture.

[0141] In some embodiments of this application, satisfying the first preset condition may specifically include: the application to which the first control belongs is a preset application, i.e., the first application is a preset application. In one possible implementation, the first application being a preset application may specifically include: the first application being an application of a preset category. In yet another possible implementation, the first application being a preset application may specifically include: the first application being an application in a preset list. It is understood that the preset category and preset list can be set according to actual needs, and this application does not impose any restrictions on them.

[0142] In some embodiments of this application, the electronic device can determine whether the first application is a preset application by using an identifier (e.g., package name, name, etc.) corresponding to the first application. It is understood that the identifier corresponding to the application can be set according to actual needs, and this application does not impose any restrictions on this.

[0143] Of course, the category of an application and the list to which it belongs can also be represented by an identifier. For details on how to implement this, please refer to the above text. This application will not elaborate on this further.

[0144] In some embodiments of this application, satisfying the first preset condition may specifically include: the first operation on the first control is a preset operation on the preset control, that is, the first control is a preset control and the first operation is a preset operation.

[0145] It is understood that preset controls and preset operations can be set according to actual needs, and this application does not impose any restrictions on them. In some embodiments of this application, the preset control can be a video thumbnail. In other embodiments of this application, the preset control can be a video link. In some embodiments of this application, the preset operation can be a click operation. In other embodiments of this application, the preset operation can be a long press operation.

[0146] In some embodiments of this application, satisfying the first preset condition may specifically include: the identifier corresponding to the interface to be drawn by the electronic device triggered by the first operation on the first control (hereinafter referred to as the interface to be drawn) is a preset interface identifier. The interface to be drawn is the user interface that the electronic device prepares to draw in response to the first operation on the first control. In some embodiments of this application, the electronic device can determine the identifier of the interface to be drawn through a graphics library. For example, the electronic device can obtain the identifier of the interface to be drawn through libhwui in the graphics library.

[0147] It is understood that preset interface identifiers can be set according to actual needs, and this application does not limit their specific content and form of expression. For example, preset interface identifiers may include 101, 102, and 103. The interface to be drawn, identified as 101, is the user interface that the electronic device prepares to draw in response to an operation on a video thumbnail in the chat interface of a communication application. The interface to be drawn, identified as 102, is the user interface that the electronic device prepares to draw in response to an operation on the application icon corresponding to a short video application. The interface to be drawn, identified as 103, is the user interface that the electronic device prepares to draw in response to an operation on a video thumbnail in the homepage interface of a short video application.

[0148] In some embodiments of this application, the identifier of the interface to be drawn can be the name of the interface to be drawn. Of course, the electronic device can also represent the identifier of the interface to be drawn in other forms (e.g., numbers, strings, etc.).

[0149] In some embodiments of this application, the electronic device can set a global variable. When the global variable has a first content, it indicates that the electronic device needs to process the Nth frame image data and bind the texture corresponding to the processed image data to a first identifier. When the global variable has a second content, it indicates that the electronic device does not need to process the Nth frame image data and can directly bind the texture corresponding to the Nth frame image data to the first identifier. Optionally, when the global variable has a first content, it indicates that the electronic device needs to perform a preprocessing procedure, while when the global variable has a second content, it indicates that the electronic device does not need to perform a preprocessing procedure.

[0150] It is understandable that if the first preset condition is met, the electronic device can set the global variable to the first content; however, if the first preset condition is not met, the electronic device can set the global variable to the second content.

[0151] It is understood that the first and second contents can be set according to actual needs, and their format can be numbers, letters, strings, etc., which is not limited in this application. For example, the first content can be true, and the second content can be false. As another example, the first content can be 1, and the second content can be 0.

[0152] In some embodiments of this application, if a first preset condition is met, the electronic device performs the same processing on the Nth frame image data during the process of updating the texture corresponding to the first surface texture object. In one possible implementation, if the first preset condition is met, the electronic device can set a global variable to a first content, and the first content includes only one parameter value, which can correspond to the same processing method. However, if the first preset condition is not met, the electronic device can set the global variable to a second content. As described above, when the global variable is the second content, it can indicate that the electronic device does not need to process the Nth frame image data and can directly bind the texture corresponding to the Nth frame image data to the first identifier. For example, the first content can be true, the second content can be false, and the processing method corresponding to true can be to increase brightness.

[0153] It is understood that the processing of the Nth frame image data by the electronic device can be set according to actual needs, and this application does not impose any limitations on this. For example, the processing of the Nth frame image data by the electronic device may include any one or more of video compression, adding effects, and color gamut conversion. Among them, adding effects may include any one or more of increasing / decreasing brightness, adding watermarks, setting high dynamic range imaging (HDRI / HDR) effects, etc. It is understood that when the added effect is to increase / decreasing brightness, the increased / decreasing brightness values ​​can be the same or different. In some embodiments of this application, increasing / decreasing brightness to different degrees can be different added effects.

[0154] In some embodiments of this application, if the first preset condition is met, when the electronic device determines the processing method for the Nth frame image data during the process of updating the texture corresponding to the first surface texture object, it does not need to determine the processing method based on the specific situation of meeting the first preset condition.

[0155] For example, satisfying the first preset condition may specifically include: the application to which the first control belongs is a preset application, that is, the first application is a preset application. The electronic device can determine the processing method for the Nth frame image data based on the identifier corresponding to the interface to be drawn by the electronic device triggered by the first operation on the first control.

[0156] For example, satisfying the first preset condition may specifically include: the first operation on the first control is a preset operation on a preset control. The electronic device can determine the processing method for the Nth frame image data based on the first application. For example, it can determine the processing method based on the category of the first application. Another example is determining the processing method based on whether the first application is in a preset list (e.g., if the first application is in the preset list, the electronic device uses the processing method corresponding to that preset list; if the first application is not in the preset list, the electronic device uses other processing methods). Yet another example is determining the processing method based on the package name of the first application.

[0157] In some embodiments of this application, if a first preset condition is met, during the process of updating the texture corresponding to the first surface texture object, the electronic device can determine the processing method for the Nth frame image data based on the specific situation of meeting the first preset condition, and then process the Nth frame image data based on the determined processing method. In this case, the processing method used by the electronic device for the Nth frame image data can be different when the first operation on the first control is different. In one possible implementation, if the first preset condition is met, the electronic device can set a global variable to a first content, and the first content can include multiple parameter values, which can correspond to different processing methods. For example, the first content can include 11, 22, and 33 (as shown below), and the second content can be 0. Wherein, when the first content is 11, the corresponding processing method can be setting an HDR effect; when the first content is 22, the corresponding processing method can be adding a watermark; and when the first content is 33, the corresponding processing method can be video compression. Similarly, if the first operation on the first control does not meet the first preset condition, the electronic device can set the global variable to the second content. As stated above, when the global variable is the second content, it can indicate that the electronic device does not need to process the Nth frame image data and can directly bind the texture corresponding to the Nth frame image data to the first identifier.

[0158] As described above, in some embodiments of this application, satisfying the first preset condition may specifically include: the application to which the first control belongs is a preset application, i.e., the first application is a preset application. In this case, the processing methods corresponding to different preset applications can be different. That is, if the first preset condition is satisfied, during the process of updating the texture corresponding to the first surface texture object, the electronic device can determine the processing method to be used for the Nth frame image data based on the first application. In other words, there is a correspondence between the preset application and the processing method used for the Nth frame image data, and the electronic device can determine the processing method to be used for the Nth frame image data based on this correspondence.

[0159] In one possible implementation, there is a correspondence between the identifier corresponding to the preset application (which can be simply referred to as the identifier of the preset application) and the processing method. If the identifier corresponding to the preset application includes the identifier corresponding to the first application, then the first application is the preset application. In this way, during the process of updating the texture corresponding to the first surface texture object, the electronic device can determine the processing method corresponding to the identifier corresponding to the first application and use that processing method to process the Nth frame image data.

[0160] For example, the identifier corresponding to the preset application can include 1, 2, and 3, which represent communication applications, video applications, and news applications, respectively. When the identifier is 1, the corresponding processing method can be setting an HDR effect. When the identifier is 2, the corresponding processing method can be adding a watermark. When the identifier is 3, the corresponding processing method can be video compression. In this case, when the identifier corresponding to the first application is 1, the electronic device can determine that the first application is the preset application. Then, during the process of updating the texture corresponding to the first surface texture object, the electronic device can obtain the context through the graphics library, set the HDR effect on the Nth frame image data, obtain the processed image data, and then upload the processed image data to the GPU, that is, bind the texture corresponding to the processed image data to the first identifier.

[0161] Furthermore, based on the above implementation, if the first preset condition is met, the electronic device can set global variables based on the identifier of a preset application. For example, when the identifier of the preset application is 1, the corresponding global variable is 11; similarly, when the identifier of the preset application is 2, the corresponding global variable is 22; and similarly, when the identifier of the preset application is 3, the corresponding global variable is 33. In this way, when the electronic device processes the Nth frame image data through the graphics library, it can determine the processing method for the Nth frame image data based on the parameter values ​​of the global variables.

[0162] As described above, in some embodiments of this application, satisfying the first preset condition may specifically include: the first operation on the first control is a preset operation on the preset control. In this case, the processing methods corresponding to different preset operations on the preset controls can be different. That is, if the first preset condition is satisfied, during the process of updating the texture corresponding to the first surface texture object, the electronic device can determine the processing method to be used for the Nth frame image data based on the first control and the first operation. In other words, there is a correspondence between the preset operation on the preset control and the processing method to be used for the Nth frame image data, and the electronic device can determine the processing method to be used for the Nth frame image data based on this correspondence.

[0163] For example, the first operation for the first control can be clicking the video thumbnail. In this case, the first preset condition is met. Correspondingly, during the process of updating the texture corresponding to the first surface texture object, the electronic device can determine that its corresponding processing method is to set the HDR effect. That is, the electronic device can obtain the context through the graphics library, set the HDR effect on the Nth frame image data, and then bind the image data obtained after setting the HDR effect to the first identifier. That is, the image data obtained after setting the HDR effect is uploaded to the GPU, so that the image data obtained after setting the HDR effect can be manipulated as a texture. This means that the texture obtained after setting the HDR effect is bound to the first identifier.

[0164] For example, the first operation for the first control can be clicking a video link. In this case, the first preset condition is met. Accordingly, during the process of updating the texture corresponding to the first surface texture object, the electronic device can determine that its corresponding processing method is video compression. That is, the electronic device can obtain the context through the graphics library, compress the Nth frame image data, and then bind the compressed image data with the first identifier. This means that the compressed image data is uploaded to the GPU, so that the compressed image data can be manipulated as a texture. This also means that the compressed texture is bound to the first identifier.

[0165] Furthermore, based on the above implementation, if the first preset condition is met, the electronic device can set global variables based on the first control and the first operation. For example, when the first operation on the first control is clicking a video thumbnail, the corresponding global variable is set to 11; similarly, when the first operation on the first control is clicking a video link, the corresponding global variable is set to 33. In this way, when the electronic device processes the Nth frame image data through the graphics library, it can determine the processing method for the Nth frame image data based on the parameter values ​​of the global variables.

[0166] As described above, in some embodiments of this application, satisfying the first preset condition may specifically include: the identifier corresponding to the interface to be drawn by the electronic device triggered by the first operation on the first control is a preset interface identifier. In this case, the processing methods corresponding to different preset interface identifiers can be different. That is, if the first preset condition is satisfied, during the process of updating the texture corresponding to the first surface texture object, the electronic device can determine the processing method to be used for the Nth frame image data based on the identifier corresponding to the interface to be drawn by the electronic device triggered by the first operation on the first control. In other words, there is a correspondence between the identifier corresponding to the interface to be drawn and the processing method used for the Nth frame image data, and the electronic device can determine the processing method to be used for the Nth frame image data based on this correspondence.

[0167] For example, when the identifier corresponding to the interface to be drawn by the electronic device is 101, the first preset condition is met. Accordingly, during the process of updating the texture corresponding to the first surface texture object, the electronic device can determine that its corresponding processing method is to increase the brightness.

[0168] For example, when the identifier corresponding to the interface to be drawn by the electronic device is 102, the first preset condition is met. Accordingly, during the process of updating the texture corresponding to the first surface texture object, the electronic device can determine that its corresponding processing method is video compression.

[0169] For example, when the identifier corresponding to the interface to be drawn by the electronic device is 103 triggered by the first operation of the first control, the first preset condition is met. Accordingly, during the process of updating the texture corresponding to the first surface texture object, the electronic device can determine that its corresponding processing method is to add a watermark.

[0170] Furthermore, based on the above implementation, if the first preset condition is met, the electronic device can set global variables based on the identifier corresponding to the interface to be drawn by the electronic device triggered by the first operation on the first control. For example, when the identifier corresponding to the interface to be drawn by the electronic device triggered by the first operation on the first control is 102, the corresponding global variable is set to 33; similarly, when the identifier corresponding to the interface to be drawn by the electronic device triggered by the first operation on the first control is 103, the corresponding global variable is set to 22. In this way, when the electronic device processes the Nth frame image data through the graphics library, it can determine the processing method for the Nth frame image data based on the parameter value of the global variable.

[0171] It is understandable that if the first operation on the first control does not meet the first preset condition, the electronic device can directly upload the Nth frame image data to the GPU without processing it during the process of updating the texture corresponding to the first surface texture object.

[0172] It is understood that during the process of updating the texture corresponding to the first surface texture object, the electronic device can directly upload the Nth frame image data to the GPU. Specifically, this may include the electronic device binding the Nth frame image data with a first identifier. It is understood that after the electronic device binds the Nth frame image data with the first identifier, the Nth frame image data can be manipulated as a texture. This means that the meaning of binding the Nth frame image data with the first identifier in this application is the same as the meaning of binding the texture corresponding to the Nth frame image data with the first identifier. In other words, if the first operation on the first control does not meet the first preset condition, during the process of updating the texture corresponding to the first surface texture object, the electronic device binds the Nth frame image data with the first identifier (or binds the texture corresponding to the Nth frame image data with the first identifier).

[0173] In some embodiments of this application, during the process of binding the Nth frame image data with the first identifier, the Nth frame image data is converted into a texture (such as the first texture mentioned above).

[0174] Optionally, in response to a first operation on the first control, if the electronic device recognizes that the current playback scene is a specific scene, the electronic device can create a first surface texture object through a codec component and a second surface texture object through a first application.

[0175] In some embodiments of this application, the electronic device can perform scene recognition based on the first preset condition mentioned above. That is, when the electronic device meets the first preset condition, it indicates that the current playback scene is a specific scene that requires preprocessing. In this case, the electronic device can execute a preprocessing process, that is, the electronic device can obtain the context from the image data decoded by the codec component (such as the Nth frame image data mentioned above), process it, and then bind the texture corresponding to the processed image data to the first identifier. When the electronic device does not meet the first preset condition, it indicates that the current playback scene does not require preprocessing. In this case, the electronic device can execute the original process, that is, the electronic device can directly bind the texture corresponding to the image data decoded by the codec component to the first identifier.

[0176] Of course, electronic devices can also use other scene recognition methods, and this application does not limit them.

[0177] In some embodiments of this application, when the first application adopts a preset playback mode, during the process of updating the texture corresponding to the first surface texture object, the electronic device can obtain the context through the graphics library, process the Nth frame image data to obtain the corresponding texture, and bind the texture to the first identifier. That is, when the first application adopts a preset playback mode, regardless of whether the first operation on the first control meets the first preset condition, the electronic device can perform the preprocessing process, namely, obtaining the context, processing the Nth frame image data, obtaining the corresponding texture, and binding the texture to the first identifier.

[0178] It should be noted that after the electronic device uploads the texture obtained after processing the Nth frame of image data to the GPU (that is, after binding the texture obtained after processing the Nth frame of image data with the first identifier), this texture can be stored in the first surface texture object. Subsequent modules can retrieve this texture in the first surface texture object based on the first identifier. In other words, binding the texture obtained after processing the Nth frame of image data with the first identifier means binding the texture obtained after processing the Nth frame of image data with the first surface texture object.

[0179] S105: The electronic device obtains the texture corresponding to the first surface texture object based on the first identifier through the second surface texture object, processes the texture, and binds the processed texture to the second identifier.

[0180] After the electronic device updates the texture corresponding to the first surface texture object (or the texture in the first surface texture object) based on the Nth frame image data, it can obtain the texture based on the first identifier through the second surface texture object, process the texture, and bind the processed texture to the second identifier. This also means that the processed texture is saved to the second surface texture object.

[0181] It is understandable that the second identifier is the identifier corresponding to the second surface texture object, which is used to upload the image data obtained by the second surface texture object to the GPU so that it can be manipulated as a texture, and is also used to identify the texture corresponding to the second surface texture object.

[0182] It is understandable that the second surface texture object is created by the electronic device through the first application, and the processing performed by the second surface texture object on the texture obtained from the first surface texture object is determined by the first application. In other words, the electronic device can determine the processing method adopted by the second surface texture object on the obtained texture through the first application.

[0183] In some embodiments of this application, the second surface texture object may not process the texture obtained from the first surface texture object, but instead directly draw the Nth frame image data of the first video based on the texture obtained from the first surface texture object.

[0184] S106: The electronic device obtains the texture corresponding to the second surface texture object based on the second identifier through the first application, draws the Nth frame image data based on the texture corresponding to the second surface texture object, and displays the drawn Nth frame image data through the display screen.

[0185] After the electronic device updates the texture corresponding to the second surface texture object (or the texture in the second surface texture object), it can obtain the texture corresponding to the second surface texture object through the first application based on the second identifier, then draw the Nth frame image data based on the texture corresponding to the second surface texture object, and display the drawn Nth frame image data on the display screen.

[0186] It should be noted that, as mentioned above, each time the electronic device decodes and obtains a frame of image data (i.e., the decoded image data is updated), the electronic device can execute steps S104-S106. Furthermore, when the electronic device executes step S106, it displays the updated frame of image data involved in step S104. In other words, during the execution of step S103, after decoding the Nth frame of image data, the electronic device can execute steps S104-S106, and when executing step S106, the electronic device displays the Nth frame of the first video. Alternatively, it can be understood that during the execution of step S103, the electronic device can continuously execute steps S104-S106.

[0187] It should be noted that the encoding / decoding components and graphics libraries involved in steps S102-S104 are components or modules in the operating system of the electronic device, and their names are merely examples provided in this application, and this application does not impose any limitations on them. Of course, the electronic device can also implement the steps performed by the encoding / decoding components and graphics libraries in the above steps S102-S104 through other components or modules of the operating system.

[0188] It is understood that step S101 is an optional step. When the electronic device executes step S102 without performing step S101, it can respond to the first operation. When the first application adopts a preset playback mode, the electronic device creates a first surface texture object through the codec component and creates a second surface texture object through the first application.

[0189] The following is combined with Figure 2 The software structure of the electronic device shown is used to illustrate a specific implementation of the above embodiments.

[0190] Please see Figure 5 , Figure 5 A flowchart illustrating yet another video playback method provided in this application embodiment. Figure 5 The first application shown can be Figure 2 The applications in the application layer shown above can be referred to for relevant descriptions, and will not be elaborated upon here. Figure 5 The TextureView object shown can be based on Figure 2 The object created by the TextureView shown. Figure 5 The first and second SurfaceTexture objects shown can be based on Figure 2 The objects created by the SurfaceTexture shown are examples of objects created by SurfaceTexture. The functions that SurfaceTexture can perform are the same as those of the first SurfaceTexture object and the second SurfaceTexture object. However, the specific functions implemented by the first SurfaceTexture object and the second SurfaceTexture object differ in this embodiment of the application; their relevant descriptions can be found above and will not be elaborated upon here. It is understood that... Figure 5 The video playback method shown may include, but is not limited to, the following steps:

[0191] S201: In response to the first operation, the first application notifies MediaCodec to decode the first video.

[0192] In some embodiments of this application, as shown in step S101, the display screen of the electronic device may display a first interface of a first application, and the first interface may include a first control. As shown in step S102, the first operation is a first operation on the first control. In this case, in response to the first operation on the first control (e.g., clicking the first control), the first application may notify MediaCodec to decode the first video.

[0193] Of course, as mentioned above, the first operation can also be other touch, voice control, gesture operation, etc., and this application does not limit it.

[0194] S202: When the first application plays the first video based on TextureView, MediaCodec creates the first SurfaceTexture object.

[0195] After the first application notifies MediaCodec to decode the first video, if the first application plays the video based on TextureView (or the first application uses a playback method based on TextureView), then MediaCodec can create a first SurfaceTexture object. It can be understood that MediaCodec is a component of the encoding / decoding components mentioned above. The specific implementation of step S202 can be found in step S102, and will not be repeated here.

[0196] S203: In response to the first operation, if the first application is playing a video based on a TextureView, the first application creates a TextureView object.

[0197] In response to the first operation, if the first application is playing a video based on a TextureView, the first application can first create a TextureView object.

[0198] S204: The TextureView object determines the first identifier corresponding to the first SurfaceTexture object and sends the first identifier to the first SurfaceTexture object.

[0199] After the first application creates a TextureView object, the TextureView object can determine the first identifier corresponding to the first SurfaceTexture object and send the first identifier to the first SurfaceTexture object.

[0200] Correspondingly, the first SurfaceTexture object can receive the first identifier sent by the TextureView object.

[0201] S205: The TextureView object creates a second SurfaceTexture object and determines a second identifier corresponding to the second SurfaceTexture object.

[0202] After the first application creates a TextureView object, the TextureView object can further create a second SurfaceTexture object and determine a second identifier corresponding to the second SurfaceTexture object.

[0203] It is understood that this application does not restrict the order in which steps S204 and S205 are performed.

[0204] S206: MediaCodec decodes the first video to obtain image data.

[0205] After the first application notifies MediaCodec to decode the first video, MediaCodec can decode the first video to obtain image data. Specifically, MediaCodec can decode the first video to obtain multiple frames of image data.

[0206] S207: When the first SurfaceTexture object detects an update in the image data, it notifies the rendering module to update the texture corresponding to the first SurfaceTexture object.

[0207] The first SurfaceTexture object can listen to the MediaCodec decoding process, specifically the image data decoded by MediaCodec. If MediaCodec decodes new image data, the first SurfaceTexture object can detect this update. Upon detecting an image data update, the first SurfaceTexture object can notify the rendering module to update the texture corresponding to its own texture.

[0208] It is understandable that the updated image data detected by the first SurfaceTexture object can be the decoded Nth frame image data mentioned above. The description and specific implementation of the Nth frame image data involved in steps S104-S106 can be found in steps S207-S213.

[0209] In some embodiments of this application, the first SurfaceTexture object can listen for available image data (or frame availability event) using the setOnFrameAvailableListener() method, i.e., setting a listener to monitor the image data decoded by MediaCodec. When new image data is decoded by MediaCodec (or the image data decoded by MediaCodec is updated, or the image data decoded by MediaCodec is available), the first SurfaceTexture object can trigger the callback method of the aforementioned method (or the callback method of the aforementioned listener). In this case, the first SurfaceTexture object can notify the rendering module to update the texture corresponding to the first SurfaceTexture object.

[0210] In some embodiments of this application, the first SurfaceTexture object can update the texture corresponding to the first SurfaceTexture object using the updateTexImage() method. During the execution of the updateTexImage() method of the first SurfaceTexture object, the EGLConsumer::updateTexImage method is further called, that is, the texture corresponding to the first SurfaceTexture object is ultimately updated through the graphics library. As shown below, during the call to the EGLConsumer::updateTexImage method, the drawing and rendering module can execute steps S208-S209.

[0211] Understandably, during the process of the first SurfaceTexture object calling the EGLConsumer::updateTexImage method, it first obtains an available buffer item (BufferItem) from the buffer queue (BufferQueue) through the buffer queue consumer (BufferQueueConsumer), which includes the graphics buffer (GraphicBuffer). Then, it creates an EglImage and EGLImageKHR based on the GraphicBuffer. Finally, it updates the texture content based on the EGLImageKHR. In one possible implementation, the first SurfaceTexture object can generate a texture content EGLImageKHR object and obtain the current display content GraphicBuffer (such as updated image data) through ConsumerBase::acquireBufferLocked. Then, it generates an EGLImageKHR image from this GraphicBuffer and binds the EGLImageKHR image to a GL_TEXTURE_EXTERNAL_OES type texture through glEGLImageTargetTexture2DOES. This allows the texture corresponding to the first SurfaceTexture object to be updated.

[0212] S208: The drawing and rendering module obtains the context.

[0213] After the first SurfaceTexture object notifies the drawing and rendering module to update the texture corresponding to the first SurfaceTexture object, the drawing and rendering module can obtain the current drawing and rendering context (which can be simply referred to as the context) so that the subsequent binding of texture resources and identifiers based on OpenGL instructions can proceed smoothly. Otherwise, the subsequent binding of texture resources and identifiers may be affected.

[0214] S209: The rendering module processes the updated image data to obtain the texture corresponding to the processed image data.

[0215] After the drawing and rendering module obtains the current drawing and rendering context, it can process the updated image data detected by the first SurfaceTexture object to obtain the texture corresponding to the processed image data.

[0216] S210: The rendering module binds the texture to the first identifier.

[0217] After the rendering module obtains the texture corresponding to the processed image, it can bind the texture to the first identifier. In this way, the texture can be stored in the first SurfaceTexture object. It should be noted that after the rendering module binds the texture to the first identifier, it signifies an update to the texture corresponding to the first SurfaceTexture object, or an update to the texture within the first SurfaceTexture object.

[0218] It is understood that the specific implementation of steps S209-S210 can be referred to the relevant description of step S104, and this application will not repeat it here.

[0219] S211: When the second SurfaceTexture object detects an update to the texture corresponding to the first SurfaceTexture object, it obtains the texture corresponding to the first SurfaceTexture object based on the first identifier.

[0220] After the rendering module binds the texture corresponding to the processed image data to the first identifier, the second SurfaceTexture object can detect texture updates corresponding to the first SurfaceTexture object. In this case, the second SurfaceTexture object can obtain the texture corresponding to the first SurfaceTexture object based on the first identifier. It can be understood that since the second SurfaceTexture object is created by a TextureView object, and the first identifier is determined by the TextureView object, the second SurfaceTexture object can obtain the identifier corresponding to the first SurfaceTexture object, i.e., the first identifier, and thus obtain the updated texture of the first SurfaceTexture object based on the first identifier.

[0221] In some embodiments of this application, the second SurfaceTexture object can also listen to the updated texture of the first SurfaceTexture object through the setOnFrameAvailableListener() method. The specific implementation method can be referred to above, and will not be repeated here.

[0222] S212: The second SurfaceTexture object processes the acquired texture and binds the processed texture to the second identifier.

[0223] After the second SurfaceTexture object obtains the updated texture from the first SurfaceTexture object, it can process the texture and bind the processed texture to the second identifier. It should be noted that after the second SurfaceTexture object binds the processed texture to the second identifier, it signifies a texture update corresponding to the second SurfaceTexture object, or a texture update within the second SurfaceTexture object.

[0224] In some embodiments of this application, the second SurfaceTexture object can update the texture corresponding to the second SurfaceTexture object through the updateTexImage() method. The specific implementation method can be referred to above, and will not be repeated here.

[0225] It is understood that this application does not limit the processing performed by the second SurfaceTexture object on the acquired texture. In some embodiments of this application, the second SurfaceTexture object may not process the acquired texture and may directly bind the acquired texture to the second identifier.

[0226] It is understood that the specific implementation of step S212 can be referred to the relevant description of step S105, and this application will not repeat it here.

[0227] S213: The first application obtains the texture corresponding to the second SurfaceTexture object based on the second identifier, and draws the image data to be displayed based on the texture.

[0228] After the texture corresponding to the second SurfaceTexture object is updated, the first application can obtain the texture corresponding to the second SurfaceTexture object based on the second identifier, and draw the image data to be displayed based on that texture. Further, the image compositing system (SurfaceFlinger) can composite the drawn content to obtain the image data to be displayed. Further, the image compositing system can send the image data to be displayed to the display driver. Further, the display driver can control the display screen to display the image data. It can be understood that the relevant steps of image data drawing, rendering, compositing, and display are... Figure 5 The specific implementation method is not shown in the document, but can be found in the relevant technical documents. This application will not elaborate on it here.

[0229] Understandably, since the second SurfaceTexture object is created by the TextureView object, and the TextureView object is created by the first application, and the second identifier is determined by the TextureView object, the first application can obtain the identifier corresponding to the second SurfaceTexture object, i.e., the second identifier, and thus obtain the updated texture of the second SurfaceTexture object based on the second identifier.

[0230] In some embodiments of this application, the electronic device can be responsible for drawing the image data to be displayed (such as the Nth frame image data of the first video) through the main thread (or UI thread) of the first application. In this case, the first application can obtain the texture corresponding to the second SurfaceTexture object based on the second identifier, and draw the image data to be displayed based on the texture through its main thread.

[0231] In some other embodiments of this application, the electronic device may use a rendering thread to draw the image data to be displayed (such as the Nth frame image data of a first video). In this case, the first application may obtain the texture corresponding to the second SurfaceTexture object based on the second identifier, and the rendering thread may obtain the first texture from the first application and then draw the Nth frame image data based on the first texture.

[0232] It is understandable that if the first SurfaceTexture object detects that the decoded new image data is the Nth frame image data, the image data to be displayed mentioned in step S213 can be the Nth frame image data mentioned above (as shown in step S106).

[0233] It should be noted that, as Figure 5As shown, the steps involving the first application (e.g., steps S203, S211, S212 and S213) are controlled by the first application, i.e. controlled by the application side, while the steps involving the codec component and the drawing and rendering module in the graphics library (e.g., steps S202, S206, S207-S210) are controlled by the operating system of the electronic device, i.e. controlled by the system side.

[0234] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A video playback method, characterized in that, The method is applied to an electronic device equipped with a display screen, and the method includes: In response to the first operation, the first video is decoded; the first operation is used to trigger the first application to play the first video. If the preset conditions are met, after decoding the first video to obtain the Nth frame image data, the operating system processes the Nth frame image data to obtain the first texture; N is a positive integer; The Nth frame image data is drawn based on the first texture using the first application; The Nth frame image data, after being drawn, is displayed on the screen.

2. The method as described in claim 1, characterized in that, The condition of satisfying the preset conditions includes: playing the first video using a preset playback method.

3. The method as described in claim 2, characterized in that, The conditions for satisfying the preset conditions also include any one or more of the following: the first application is a preset application, the first operation is a preset operation, or the interface identifier to be drawn corresponding to the first operation is a preset interface identifier.

4. The method according to any one of claims 1-3, characterized in that, The processing of the Nth frame image data includes: The processing method is determined based on the first application, and the Nth frame image data is processed based on the processing method.

5. The method according to any one of claims 1-3, characterized in that, The method further includes responding prior to the first operation: The first interface is displayed, which is the user interface of the first application. The first interface includes a first control. The first operation is an operation performed on the first control. The processing of the Nth frame image data includes: The processing method is determined based on the first control and the first operation, and the Nth frame image data is processed based on the processing method.

6. The method according to any one of claims 1-3, characterized in that, The processing of the Nth frame image data includes: Determine the interface identifier to be drawn corresponding to the first operation; The processing method is determined based on the interface identifier to be drawn corresponding to the first operation, and the Nth frame image data is processed based on the processing method.

7. The method according to any one of claims 4-6, characterized in that, The processing method includes any one or more of the following: video compression, adding effects, or color gamut conversion; The added effects include any one or more of the following: increasing / decreasing brightness, adding a watermark, or setting a high dynamic range (HDR) imaging effect.

8. The method as described in claim 2 or 3, characterized in that, The method further includes: In response to the first operation, when the preset playback mode is adopted, a first surface texture object is created through the operating system, and a second surface texture object is created through the first application; The step of processing the Nth frame image data through the operating system to obtain the first texture includes: The Nth frame image data is processed by the graphics renderer of the operating system, the first texture is generated based on the processed Nth frame image data, and the texture corresponding to the first surface texture object is updated to the first texture. The step of drawing the Nth frame image data based on the first texture using the first application includes: The first texture corresponding to the first surface texture object is obtained through the first application, and the texture corresponding to the second surface texture object is updated to the first texture; or, the first texture is processed to obtain the second texture, and the texture corresponding to the second surface texture object is updated to the second texture. The Nth frame image data is drawn based on the texture corresponding to the second surface texture object.

9. An electronic device, characterized in that, The electronic device includes a display screen, one or more memories, and one or more processors; the display screen is coupled to the one or more memories and the one or more processors, the display screen is used to display images, the memory is used to store computer program code, the computer program code includes computer instructions, and the processor calls the computer instructions to perform the method of any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Used to store computer instructions that, when executed on an electronic device, cause the electronic device to perform the method described in any one of claims 1-8.

Images