Video playing method and related device

Abstract

The invention discloses a video playing method and a related device, and the method comprises the steps: obtaining a plurality of video frames based on a to-be-played video, determining the size adaptation information in a current adaptive mode based on the current adaptive mode, the size information of the plurality of video frames, and the size information of a playing view, and playing the video. The size adaptation information comprises position information of to-be-displayed areas in the multiple video frames and position information of mapping areas in texture objects corresponding to the playing view, and the to-be-displayed areas of the multiple video frames are used for being mapped to the mapping areas respectively. And loading the video frame data of the plurality of video frames to the texture object in sequence according to the playing sequence of the plurality of video frames on the basis of the size adaptation information, and continuously drawing the data of the texture object to the drawing surface object corresponding to the playing view so as to perform video playing by using the playing view, thereby realizing size adaptation between the video frames and the playing view, and improving the video playing efficiency. The degree of freedom of size adaptation of the to-be-played video and the playing view is improved, and meanwhile the excellent performance of the playing view is guaranteed.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular, to a video playing method and related device. Background Art

[0002] Currently, there is a need to play videos in an application, such as playing a title video when the application starts, playing user-created short videos, playing film and television resources, etc. The size of the video to be played is usually fixed. However, the required size for video playing is often not fixed. For example, different devices may have different screen sizes, and the corresponding size to be displayed is also different.

[0003] Currently, an audio-video playing component can be used to play videos. However, there is a logic for final width and height adaptation based on the width and height of its own view and the width and height of the video inside the audio-video playing component, which is usually relatively single, and there is a problem of low freedom in video width and height adaptation. Summary of the Invention

[0004] To solve the above technical problems, this application provides a video playing method and related device. Based on the region selection logic defined by the size adaptation information in the current adaptive mode, texture loading of video frames and drawing of texture objects are performed, thereby realizing the size adaptation between video frames and drawing surface objects, improving the freedom of size adaptation between the video to be played and the playing view, and ensuring the excellent performance of the playing view at the same time.

[0005] The embodiments of this application disclose the following technical solutions:

[0006] On the one hand, this application provides a video playing method, and the method includes:

[0007] Obtain multiple video frames based on the video to be played;

[0008] Based on the current adaptive mode, the size information of the multiple video frames, and the size information of the texture object, determine the size adaptation information in the current adaptive mode; the size information of the texture object is determined according to the size information of the playing view corresponding to the texture object; the current adaptive mode is one of multiple adaptive modes; the size adaptation information includes the position information of the area to be displayed in the multiple video frames and the position information of the mapped area in the texture object;

[0009] Based on the size adaptation information, according to the playing order of the multiple video frames, sequentially load the video frame data of the multiple video frames into the texture object, and the areas to be displayed in the multiple video frames are respectively used to map to the mapped area;

[0010] Continuously draw the data of the texture object onto the drawing surface object corresponding to the playback view to play the video using the playback view.

[0011] Optionally, determining the size adaptation information in the current adaptive mode based on the current adaptive mode, the size information of the multiple video frames, and the size information of the texture object includes:

[0012] If the current adaptive mode includes full-screen display of the picture, determine the position information of the mapping area according to the entire area of the texture object;

[0013] Determine the position information of the area to be displayed according to the size information of the multiple video frames and the size information of the mapping area, so that the area to be displayed and the mapping area have the same aspect ratio.

[0014] Optionally, determining the position information of the area to be displayed according to the size information of the multiple video frames and the size information of the mapping area includes:

[0015] Determine the position information of the first feature point in the area to be displayed according to the current adaptive mode; if the current adaptive mode includes displaying the first vertex side part, the first feature point is the first vertex, and the position information of the first vertex is determined according to the position information of the first vertex of the multiple video frames; if the current adaptive mode includes displaying the central part, the first feature point is the center point of the area, and the position information of the center point of the area is determined according to the position information of the center point of the multiple video frames;

[0016] Determine the size of the first side of the area to be displayed and the size of the second side adjacent to the first side according to the size information of the multiple video frames and the size information of the mapping area;

[0017] Determine the position information of the area to be displayed according to the position information of the first feature point, the size of the first side, and the size of the second side.

[0018] Optionally, determining the size of the first side of the area to be displayed and the size of the second side adjacent to the first side according to the size information of the multiple video frames and the size information of the mapping area includes:

[0019] If it is determined according to the size information of the multiple video frames and the size information of the mapping area that the aspect ratio of the multiple video frames is greater than the aspect ratio of the mapping area, use the height of the multiple video frames as the size of the first side of the area to be displayed; determine the size of the second side adjacent to the first side according to the aspect ratio of the mapping area and the size of the first side;

[0020] If it is determined, according to the size information of the multiple video frames and the size information of the mapping area, that the aspect ratio of the multiple video frames is smaller than the aspect ratio of the mapping area, the width of the multiple video frames is used as the size of the second side of the area to be displayed; according to the aspect ratio of the mapping area and the size of the second side, the size of the first side adjacent to the second side is determined.

[0021] Optionally, if the current adaptive mode corresponds to full-screen display of the picture, the loading of the video frame data of the multiple video frames into the texture object in sequence according to the playing order of the multiple video frames based on the size adaptation information includes:

[0022] Based on the position information of the area to be displayed, the multiple video frames are intercepted in sequence according to the playing order of the multiple video frames to obtain the images to be displayed;

[0023] The image data of the images to be displayed is loaded into the texture object in sequence.

[0024] Optionally, the determination of the size adaptation information in the current adaptive mode based on the current adaptive mode, the size information of the multiple video frames, and the size information of the texture object includes:

[0025] If the current adaptive mode includes full-content display, the position information of the area to be displayed is determined according to the entire area of the multiple video frames;

[0026] According to the size information of the area to be displayed and the size information of the texture object, the position information of the mapping area is determined so that the area to be displayed and the mapping area have the same aspect ratio.

[0027] Optionally, the determination of the position information of the mapping area according to the size information of the area to be displayed and the size information of the texture object includes:

[0028] According to the current adaptive mode, the position information of the second feature point in the mapping area is determined; if the current adaptive mode includes second-vertex-side display, the second feature point is the second vertex, and the position information of the second vertex is determined according to the position information of the second vertex of the texture object; if the current adaptive mode includes centered display, the second feature point is the center point of the area, and the position information of the center point of the area is determined according to the position information of the center point of the texture object;

[0029] According to the size information of the area to be displayed and the size information of the texture object, the size of the third side of the mapping area and the size of the fourth side adjacent to the third side are determined;

[0030] Determine the position information of the mapping area according to the position information of the second feature point, the size of the third side, and the size of the fourth side.

[0031] Optionally, the determining the size of the third side of the mapping area and the size of the fourth side adjacent to the third side according to the size information of the area to be displayed and the size information of the texture object includes:

[0032] If it is determined according to the size information of the area to be displayed and the size information of the texture object that the aspect ratio of the area to be displayed is greater than the aspect ratio of the texture object, use the width of the texture object as the size of the third side of the mapping area; determine the size of the fourth side adjacent to the third side according to the aspect ratio of the area to be displayed and the size of the third side;

[0033] If it is determined according to the size information of the area to be displayed and the size information of the texture object that the aspect ratio of the area to be displayed is less than the aspect ratio of the texture object, use the height of the texture object as the size of the fourth side of the mapping area; determine the size of the third side adjacent to the fourth side according to the aspect ratio of the area to be displayed and the size of the fourth side.

[0034] Optionally, the method further includes:

[0035] Add the playback view to the view tree to obtain a drawing surface object corresponding to the playback view; the size information of the drawing surface object is determined according to the size information of the playback view;

[0036] Create a texture object according to the drawing surface object corresponding to the playback view; the size information of the texture object is determined according to the size information of the drawing surface object.

[0037] Optionally, the playback view is a texture view, and the method further includes:

[0038] If a display transformation operation for the view tree is obtained, adjust the display parameters of the playback view according to the display transformation operation.

[0039] Optionally, the method further includes:

[0040] During the video playback using the playback view, in response to a mode switching operation including a target mode, update the current adaptive mode according to the target mode, where the target mode is one of the multiple adaptive modes.

[0041] On the other hand, the present application provides a video playback device, and the device includes:

[0042] A video frame acquisition unit for obtaining multiple video frames based on a video to be played;

[0043] A size adaptation information determination unit for determining size adaptation information in the current adaptation mode based on the current adaptation mode, the size information of the multiple video frames, and the size information of the texture object; the size information of the texture object is determined according to the size information of the playback view corresponding to the texture object; the current adaptation mode is one of multiple adaptation modes; the size adaptation information includes the position information of the area to be displayed in the multiple video frames and the position information of the mapped area in the texture object;

[0044] A texture loading unit for sequentially loading the video frame data of the multiple video frames into the texture object based on the size adaptation information according to the playback order of the multiple video frames, and the areas to be displayed in the multiple video frames are used to be respectively mapped to the mapped areas;

[0045] A texture drawing unit for continuously drawing the data of the texture object onto the drawing surface object corresponding to the playback view to play the video using the playback view.

[0046] Optionally, the size adaptation information determination unit includes:

[0047] A first position determination unit for, if the current adaptation mode includes full-screen display of the picture, determining the position information of the mapped area according to the entire area of the texture object;

[0048] A second position determination unit for determining the position information of the area to be displayed according to the size information of the multiple video frames and the size information of the mapped area, so that the area to be displayed and the mapped area have the same aspect ratio.

[0049] Optionally, the second position determination unit includes:

[0050] A first feature point position determination unit for determining the position information of the first feature point in the area to be displayed according to the current adaptation mode; if the current adaptation mode includes displaying a part of the first vertex side, the first feature point is the first vertex, and the position information of the first vertex is determined according to the position information of the first vertex of the multiple video frames; if the current adaptation mode includes displaying the central part, the first feature point is the center point of the area, and the position information of the center point of the area is determined according to the position information of the center point of the multiple video frames;

[0051] A first size determination unit for determining the size of the first side of the area to be displayed and the size of the second side adjacent to the first side according to the size information of the multiple video frames and the size information of the mapped area;

[0052] A to-be-displayed area determination subunit, configured to determine the position information of the to-be-displayed area according to the position information of the first feature point, the size of the first side, and the size of the second side.

[0053] Optionally, the first size determination unit is specifically configured to:

[0054] If it is determined according to the size information of the multiple video frames and the size information of the mapping area that the aspect ratio of the multiple video frames is greater than the aspect ratio of the mapping area, take the height of the multiple video frames as the size of the first side of the to-be-displayed area; determine the size of the second side adjacent to the first side according to the aspect ratio of the mapping area and the size of the first side;

[0055] If it is determined according to the size information of the multiple video frames and the size information of the mapping area that the aspect ratio of the multiple video frames is less than the aspect ratio of the mapping area, take the width of the multiple video frames as the size of the second side of the to-be-displayed area; determine the size of the first side adjacent to the second side according to the aspect ratio of the mapping area and the size of the second side.

[0056] Optionally, if the current adaptive mode corresponds to full-screen display of the picture, the texture loading unit includes:

[0057] An adaptive module, configured to sequentially intercept the multiple video frames to obtain to-be-displayed images based on the position information of the to-be-displayed area and according to the playing order of the multiple video frames;

[0058] A texture loading subunit, configured to sequentially load the image data of the to-be-displayed images into the texture object.

[0059] Optionally, the size adaptation information determination unit includes:

[0060] A third position determination unit, configured to, if the current adaptive mode includes full-content display, determine the position information of the to-be-displayed area according to the entire area of the multiple video frames;

[0061] A fourth position determination unit, configured to determine the position information of the mapping area according to the size information of the to-be-displayed area and the size information of the texture object, so that the to-be-displayed area and the mapping area have the same aspect ratio.

[0062] Optionally, the fourth position determination unit includes:

[0063] A second feature point position determination unit, configured to determine the position information of a second feature point in the mapping area according to the current adaptive mode; if the current adaptive mode includes second vertex side display, the second feature point is the second vertex, and the position information of the second vertex is determined according to the position information of the second vertex of the texture object; if the current adaptive mode includes centered display, the second feature point is the center point of the area, and the position information of the center point of the area is determined according to the position information of the center point of the texture object;

[0064] A second size determination unit, configured to determine the size of the third side of the mapping area and the size of the fourth side adjacent to the third side according to the size information of the area to be displayed and the size information of the texture object;

[0065] A mapping area determination subunit, configured to determine the position information of the mapping area according to the position information of the second feature point, the size of the third side, and the size of the fourth side.

[0066] Optionally, the second size determination unit is specifically configured to:

[0067] If it is determined according to the size information of the area to be displayed and the size information of the texture object that the aspect ratio of the area to be displayed is greater than the aspect ratio of the texture object, the width of the texture object is used as the size of the third side of the mapping area; according to the aspect ratio of the area to be displayed and the size of the third side, the size of the fourth side adjacent to the third side is determined;

[0068] If it is determined according to the size information of the area to be displayed and the size information of the texture object that the aspect ratio of the area to be displayed is less than the aspect ratio of the texture object, the height of the texture object is used as the size of the fourth side of the mapping area; according to the aspect ratio of the area to be displayed and the size of the fourth side, the size of the third side adjacent to the fourth side is determined.

[0069] Optionally, the device further includes:

[0070] A playback view adding unit, configured to add the playback view in the view tree to obtain a drawing surface object corresponding to the playback view; the size information of the drawing surface object is determined according to the size information of the playback view;

[0071] A texture object creation unit, configured to create a texture object according to the drawing surface object corresponding to the playback view; the size information of the texture object is determined according to the size information of the drawing surface object.

[0072] Optionally, the playback view is a texture view, and the device further includes:

[0073] A display parameter adjustment unit, configured to, if a display transformation operation for the view tree is obtained, adjust the display parameters of the playback view according to the display transformation operation.

[0074] Optionally, the apparatus further includes:

[0075] An adaptive mode switching unit, configured to, during the process of playing a video using the playback view, in response to a mode switching operation including a target mode, update the current adaptive mode according to the target mode, where the target mode is one of the multiple adaptive modes.

[0076] On the other hand, the present application provides a computer device, where the device includes a processor and a memory:

[0077] The memory is configured to store a computer program and transmit the computer program to the processor;

[0078] The processor is configured to execute the video playback method described in the above aspect according to the instructions in the computer program.

[0079] On the other hand, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium is configured to store a computer program, and the computer program is used to execute the video playback method described in the above aspect.

[0080] On the other hand, an embodiment of the present application provides a computer program product including a computer program, which, when running on a computer device, causes the computer device to execute the video playback method.

[0081] It can be seen from the above technical solutions that multiple video frames can be obtained based on the video to be played. Based on the current adaptive mode, the size information of the multiple video frames, and the size information of the playback view, the size adaptation information in the current adaptive mode can be determined. The size information of the texture object is determined according to the size of the playback view corresponding to the texture object. The current adaptive mode is one of the multiple adaptive modes. The size adaptation information includes the position information of the area to be displayed in the multiple video frames and the position information of the mapped area in the texture object corresponding to the playback view, where the areas to be displayed in the multiple video frames are respectively mapped to the mapped area. It can be understood that when the area to be displayed is smaller than the overall area of the video frame, the mapping process is equivalent to intercepting the multiple video frames, and when the mapped area is smaller than the overall area of the texture object, the mapping process is equivalent to selecting an area of the texture object. Therefore, through the size adaptation information in the current adaptive mode, the area selection logic between the video frames and the texture object in the current adaptive mode can be defined, and the size adaptation between the video frames and the texture object in the current adaptive mode can be achieved.

[0082] After that, based on the size adaptation information, according to the playback order of multiple video frames, the video frame data of multiple video frames can be loaded into the texture object in sequence, and then the data of the texture object can be continuously drawn onto the drawing surface object corresponding to the playback view, so as to play the video using the playback view. That is to say, based on the region selection logic defined by the size adaptation information in the current adaptive mode, the texture loading of video frames and the drawing of the texture object can be performed. That is, through the size adaptation between the video frame and the texture object, the size adaptation between the video frame and the playback view is achieved, so that the area to be displayed of the video frame can be displayed within the video playback area corresponding to the mapped area in the playback view. In this way, the problem of size adaptation between the video to be played and the playback view is transformed into the size adaptation between the video frame and the texture object during the texture loading process. Different adaptive modes can correspond to different size adaptation logics between the video frame and the texture object, greatly improving the freedom of size adaptation between the video to be played and the playback view. Compared with the size adaptation between the video to be played and the playback view, the adaptation efficiency between the image frame and the texture object is higher and the real-time performance is better. During the playback process, size adaptation can be achieved without adjusting the playback view, and no additional memory occupation will occur. In this way, while improving the freedom of size adaptation between the video to be played and the playback view, the excellent performance of the playback view is ensured. Description of the Drawings

[0083] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0084] Figure 1 Schematic diagram of an application scenario of a video playback method provided by an embodiment of the present application;

[0085] Figure 2 Schematic diagram of a video playback process provided by an embodiment of the present application;

[0086] Figure 3 Another schematic diagram of a video playback process provided by an embodiment of the present application;

[0087] Figure 4 Flowchart of a video playback method provided by an embodiment of the present application;

[0088] Figure 5 Screenshot schematic diagram of a video playback provided by an embodiment of the present application;

[0089] Figure 6 Schematic diagram of a video playback framework provided by an embodiment of the present application;

[0090] Figure 7 , Figure 8 and Figure 9 are schematic diagrams of various implementation manners for full-screen display of a screen in an embodiment of the present application;

[0091] Figure 10 and Figure 11 are schematic diagrams of various implementation manners for full-content display in an embodiment of the present application;

[0092] Figure 12 is another schematic diagram of a video playback framework provided by an embodiment of the present application;

[0093] Figure 13 is a structural block diagram of a video playback device provided by an embodiment of the present application;

[0094] Figure 14 is a structural diagram of a terminal device provided by an embodiment of the present application;

[0095] Figure 15 is a structural diagram of a server provided by an embodiment of the present application. Specific embodiments

[0096] The embodiments of the present application will be described below with reference to the accompanying drawings.

[0097] Currently, a video can be played by using an audio-video playback component. However, there is a logic for final width and height adaptation based on the width and height of its own view and the width and height of the video inside the audio-video playback component, which is usually relatively single, and there is a problem of low freedom in video width and height adaptation.

[0098] To solve the above technical problems, the present application provides a video playback method and related device. Based on the region selection logic defined by the size adaptation information in the current adaptive mode, texture loading of video frames and drawing of texture objects are performed, thereby realizing size adaptation between video frames and drawing surface objects, improving the freedom of size adaptation between the video to be played and the playback view, and ensuring excellent performance of the playback view at the same time.

[0099] The video playback method provided by the embodiments of the present application can be implemented by a computer device, which can be a terminal device or a server. Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. The terminal device includes but is not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc. The terminal device and the server can be directly or indirectly connected through wired or wireless communication methods, and the present application does not make any restrictions here.

[0100] To facilitate understanding of the technical solution provided in this application, next, a video playing method provided in an embodiment of this application will be introduced in combination with an actual application scenario.

[0101] Figure 1 FIG. shows a schematic diagram of an application scenario of a video playing method provided in an embodiment of this application. In this scenario, it includes a server 20 and a terminal device 10. An application program for video playing is installed in the terminal device 10. The server 20 corresponding to this application program and the terminal device 10 can interact through a network. The terminal device 10 may have a playing area 100. The server 20 or the terminal device 10 can be used as the aforementioned computer device to perform video playing operations according to video playing tasks. The terminal device 10 is used to interact with the user, obtain video playing tasks, and display videos in the playing area 100. The following will take the terminal device 10 as an example of the aforementioned computer device for illustration.

[0102] Based on the video to be played, the terminal device 10 can obtain multiple video frames. Based on the current adaptive mode, the size information of multiple video frames, and the size information of the playing view, the size adaptation information in the current adaptive mode can be determined. The size information of the texture object is determined according to the size of the playing view corresponding to the texture object. The current adaptive mode is one of multiple adaptive modes. The size adaptation information includes the position information of the area to be displayed in multiple video frames and the position information of the mapped area in the texture object corresponding to the playing view, where the areas to be displayed in multiple video frames are used to be mapped to the mapped area respectively. It can be understood that when the area to be displayed is smaller than the overall area of the video frame, the mapping process is equivalent to intercepting multiple video frames. When the mapped area is smaller than the overall area of the texture object, the mapping process is equivalent to making an area selection for the texture object. Therefore, through the size adaptation information in the current adaptive mode, the area selection logic between the video frame and the texture object in the current adaptive mode can be defined, and the size adaptation between the video frame and the texture object in the current adaptive mode can be achieved.

[0103] After that, based on the size adaptation information, the terminal device 10 can sequentially load the video frame data of multiple video frames into a texture object according to the playback order of the multiple video frames, and then continuously draw the data of the texture object onto the drawing surface object corresponding to the playback view, so as to play the video using the playback view. That is to say, based on the region selection logic defined by the size adaptation information in the current adaptive mode, texture loading of video frames and drawing of texture objects can be performed. That is, through the size adaptation between the video frames and the texture object, the size adaptation between the video frames and the playback view is achieved, so that the area to be displayed of the video frames can be displayed within the video playback area corresponding to the mapped area in the playback view. In this way, the size adaptation problem between the video to be played and the playback view is transformed into the size adaptation between the video frames and the texture object during the texture loading process. Different adaptive modes can correspond to different size adaptation logics between the video frames and the texture object, greatly improving the degree of freedom of the size adaptation between the video to be played and the playback view. Compared with the size adaptation between the video to be played and the playback view, the adaptation efficiency between the image frames and the texture object is higher and the real-time performance is better. During the playback process, size adaptation can be achieved without adjusting the playback view, and no additional memory occupation will occur. In this way, while improving the degree of freedom of the size adaptation between the video to be played and the playback view, the excellent performance of the playback view is ensured.

[0104] For ease of explanation, the related concepts involved in the embodiments of the present application are introduced below.

[0105] An operating system can be installed in the terminal device. Android is an open-source operating system based on the Linux kernel developed by Google and is mainly used for mobile devices such as smartphones and tablets. Since its first release in 2008, Android has become one of the most widely used mobile operating systems in the world. The Android operating system has many features, such as a large application ecosystem, a convenient operation interface, regular new version releases that bring new functions and security improvements, and provides powerful development tools and resources to support Android application development. Among them, the most crucial thing is that the Android system is open-source, which means that developers can freely view, modify, and distribute its source code. This openness makes the Android system very flexible and can be widely customized and applied to various devices.

[0106] The operating system can provide views for developers to develop user interfaces. In the development of the Android system, the native View interface framework is the basis for building user interfaces. Android provides a rich set of View and ViewGroup (view group) classes for creating and managing user interface components. View is the base class of all user interface (UI) components in Android and is the basic element that makes up the user interface. It represents a rectangular area on the screen, can draw content, and handle user interactions. For example, buttons, text boxes, etc. are all subclasses of View. ViewGroup is a special View that can contain other Views (including other ViewGroups). ViewGroup is the base class of all layout classes, and common subclasses include LinearLayout, RelativeLayout, FrameLayout, ConstraintLayout, etc.

[0107] A view (View) can include a playback view for multimedia playback, such as playing static images, dynamic images, or videos, etc. In the Android system, playback views can include TextureView, SurfaceView, VideoView, etc. These playback views are all subclasses of View provided by the Android system. The following introduces these three playback views and other related concepts involved.

[0108] First of all, TextureView is a view class provided by Android for displaying dynamic images. It is essentially an ordinary View and can be added to the view hierarchy like an ordinary View. TextureView can provide an independent drawing surface (Surface) object. Developers can fill the drawing content into this drawing surface object through operations such as graphics rendering APIs (Application Programming Interfaces), video decoding, etc., so as to use TextureView for dynamic image playback. TextureView does not create a separate window but is drawn in the ordinary View hierarchy, so it can be transformed, animated, and cropped together with other Views.

[0109] Among them, Surface (drawing surface) is a very important class in Android development. It is an abstract drawing surface that provides a buffer for direct drawing. It allows drawing operations to be performed on the provided drawing surface, provides a specific carrier for upper-layer drawing operations, and provides underlying drawing support for Views. Surface is usually used in scenarios that require direct access to the display buffer, such as video playback, game rendering, and camera preview. The Surface class itself does not perform any drawing operations, but provides an interface that allows other components (such as Canvas (drawing board), OpenGL (Open Graphics Library), or MediaCodec (media codec), etc.) to draw on it.

[0110] During the process of drawing the user interface through Surface, the user interface can be mapped to the Surface object, and its content can be presented within the Surface. This is equivalent to filling the content of the user interface by drawing on the Surface object. Taking the View in the user interface as an example, the drawing process of the View ultimately occurs on the Surface. When the View needs to display content, it will perform drawing operations through the canvas provided by the Surface, drawing the appearance, content, etc. of the View onto the memory area represented by the Surface, and then the system will display the content of the Surface on the screen.

[0111] Other components such as OpenGL is a cross-platform graphics API designed specifically for embedded systems such as mobile devices, tablets, and game consoles. It provides efficient 2D and 3D graphics rendering capabilities. OpenGL is widely used in mobile operating systems such as Android and iOS for developing high-performance graphics applications and games. It consists of a series of functions and tools that provide developers with powerful graphics processing capabilities, enabling them to create highly realistic and interactive graphics applications. In OpenGL, a texture is image data used to map onto the surface of 3D graphics, and a texture object is used to store and manage the corresponding texture data. Textures can greatly enhance the visual effects of graphics, making the rendered objects look more realistic and delicate. Surface can be used as a rendering object for OpenGL. OpenGL provides rich APIs to load, bind, and manipulate texture data. After the image data is uploaded to a texture ID, it becomes the texture data corresponding to that texture ID, and this ID can be used to manipulate the texture data to transform the image or draw the image onto the screen.

[0112] Other components such as MediaCodec are lower-level audio and video encoding and decoding APIs provided by the Android platform for accessing underlying media encoding and decoding functions. It provides the functions of encoding and decoding audio and video data, enabling developers to implement efficient media processing in Android applications. Through MediaCodec, developers can encode raw audio or video data in a specific encoding format for storage or transmission, or decode the received encoded data to restore it to a playable audio or video stream, thus realizing functions such as video playback, recording, and live streaming. MediaCodec is implemented in C++ at the bottom layer, can directly use the hardware codec, provides efficient audio and video processing capabilities, and greatly improves the efficiency of video processing; MediaCodec supports a variety of common audio and video formats, such as H.264, AAC, MP4, etc.; MediaCodec has low latency and is suitable for real-time audio and video processing. In addition, MediaCodec is also suitable for custom audio and video decoding and rendering players.

[0113] Secondly, SurfaceView is also a view class in Android for displaying graphics and videos, providing an independent drawing surface (Surface) object that allows the application to perform drawing operations on an independent drawing surface. SurfaceView draws by creating an independent window, so it can draw in a background thread without affecting the performance of the main thread and has high drawing performance. This makes SurfaceView very suitable for scenarios that require high-performance drawing, such as video playback, game rendering, and camera preview, etc., and it can achieve real-time updates and high frame rate displays. Different from TextureView, SurfaceView cannot be transformed, animated, and cropped together with other views. Figure 1 together for transformation, animation, and cropping.

[0114] Among them, the main thread in the Android system is also called the UI thread, which is responsible for handling user interface operations, event handling threads, and managing other components, etc. User interface operations such as updating views, UI operations, etc., UI operations such as click, touch, and other user input events, and other components such as activities (Activity), services (Service), etc. Activity is the basic component in an Android application for implementing the user interface, providing a display container and context environment for Views and ViewGroups. An Activity can include one or more Views, and these Views together constitute the user interface of the Activity. For example, an Activity for a login interface may contain multiple Views such as a username input box, a password input box, and a login button.

[0115] Again, VideoView is a view class provided by Android specifically for playing videos. As an audio and video playback component of Android, it is mainly applied to video playback scenarios. It is a component that inherits from the Android SurfaceView and is a wrapper of SurfaceView. VideoView encapsulates the basic functions of video playback, enabling developers to easily integrate video playback functions into their applications without having to handle the underlying media decoding and rendering details, thus simplifying the video playback operation. Essentially, VideoView is a View and can be directly added to the layout file like a normal View and combined with other Views. Its display position and size are determined by the layout parameters. VideoView internally integrates a media player (MediaPlayer) and can directly play local video files or network video streams. VideoView is suitable for simple video playback scenarios. For some complex video processing requirements, it may be necessary to use MediaPlayer, SurfaceView, or TextureView to implement.

[0116] Among them, MediaPlayer is a class provided by Android for playing audio and video. It provides complete media playback functions and is the core underlying API for playing videos at the Java layer of the Android platform. The video playback capabilities encapsulated by VideoView are implemented based on MediaPlayer. MediaPlayer itself does not have a display function and is usually used in conjunction with view components such as SurfaceView or TextureView to display video images. Developers need to associate MediaPlayer with these view components to render the video data onto the corresponding views. Although this method increases the development complexity, it also provides more flexibility. For example, custom video rendering effects can be achieved.

[0117] On the Android platform, the simplest way to play a video is to directly use the VideoView component provided by the system. VideoView is highly encapsulated based on MediaPlayer, making it very convenient for users to play videos. Refer to Figure 2 As shown, it is a schematic diagram of a video playback process provided by an embodiment of this application. VideoView inherits from SurfaceView and internally integrates a media player (MediaPlayer) and a media codec (MediaCodec). The video to be played calls MediaCodec for encoding and decoding through the VideoView component and calls MediaPlayer for playback, thereby outputting the video image.

[0118] However, in this playback method, there is a logic inside the VideoView for final width and height adaptation based on its own width and height and the width and height of the video. Developers cannot customize or extend it and can only follow the width and height adaptation mode defined by the VideoView. In the logic of width and height adaptation, usually the width and height of the video are scaled proportionally until the width or height meets the preset width and height of the VideoView, and the remaining video area is left with black borders; or the width and height of the VideoView are set to the width and height of the video, and the width and height of the VideoView are scaled proportionally to make it fit the drawable area size. That is to say, the VideoView will first ensure that the video picture itself is not stretched or compressed, and at the same time scale the video or the VideoView proportionally until the width or height fills the width or height of the drawable area of the VideoView. Whether there are black borders or the retention of video content during the scaling of the video or the VideoView is determined according to the single width and height adaptation logic preset inside the VideoView.

[0119] In another video playback solution, a custom TextureView is used to obtain a video playback container for video playback, and the width and height of the TextureView are adjusted for width and height adaptation, enabling custom width and height adaptation. Refer to Figure 3 As shown, it is another schematic diagram of the video playback process provided by the embodiment of the present application. The TextureView provides a surface to the underlying layer to draw the video through the media player on it. During the video playback process, the media codec (MediaCodec) decodes and encodes the video to be played, and the media player (MediaPlayer) provides a video playback API for the TextureView, so as to display the video picture through the TextureView. In this mode, the content provided by the media player always fills the entire TextureView (covering the entire Surface). If the width and height of the TextureView are the same as the width and height of the video, the video picture is not stretched or compressed; otherwise, the video picture shows stretching or compression performance.

[0120] A width and height calculation module is set in the TextureView. The width and height calculation module calculates and adjusts the width and height of the TextureView according to the desired adaptation mode to ensure that the width and height ratio of the TextureView is the same as that of the video, and scales the TextureView to meet the actual playback requirements, thus ensuring the video playback effect. Taking common full-screen playback as an example, it is necessary to scale the width and height values of the TextureView according to the width and height ratio of the video until the TextureView can completely fill the screen.

[0121] However, in some video material scenarios, it is easy to cause a large scaling ratio of the size of the TextureView itself. For example, in the video playback scenario of the TextureView in landscape mode, when adapting in a full-screen mode, if the video material is in portrait orientation, it will cause the TextureView to be stretched several times to make the picture fill the screen. Since the TextureView itself has a large size and includes a large number of pixels, it will cause a large memory occupancy problem. At the same time, such a solution cannot handle the problem of switching between different adaptation modes well (it is necessary to recalculate and set the width and height of the TextureView, which will cause the picture to jitter).

[0122] Figure 4 The flowchart of a video playback method provided by an embodiment of the present application is shown. In this embodiment, the terminal device is used as the aforementioned computer device for illustration. The video playback method may include the following steps S101 - S104.

[0123] S101, obtaining a plurality of video frames based on the video to be played.

[0124] In the embodiment of the present application, the video to be played refers to the video that needs to be played in the application, such as self-made short video playback, film and television resource playback, title video playback when the application starts, etc. The application can be an application under various operating system platforms, such as an application on the Android system platform, etc. Refer to Figure 5 As shown, it is a schematic screenshot of a video playback provided by an embodiment of the present application. The video is the title video playback when the application starts and is fully displayed on the display screen.

[0125] The playback of the video to be played needs to be matched with the playback area of the terminal device. The playback area refers to the area on the display screen of the terminal device used for playing the video. In the case of playing the video through the playback view, the playback area is the area corresponding to the visible area of the playback view on the screen, and the size of the visible area of the playback view is the same as the size of the playback area. It should be noted that in this article, the invisible area of the playback view is not considered, and the size of the visible area of the playback view is briefly described as the size of the playback view. In the case where the playback view still has an invisible area, the total size of the playback view is the sum of the size of the aforementioned playback view and the size of the invisible area.

[0126] The size of the visible area of the playback region is related to the screen size of the terminal device and also to the playback requirements in different scenarios. The playback requirements include, for example, full-screen display or centered display, etc. When in full-screen display, the playback region is the entire display screen area of the terminal device. Due to the diversification of the screen sizes of terminal devices and the diversification of playback requirements, the sizes of the playback region and the video to be played are often different. For these complex scenarios of different terminal devices and playback requirements, adapting the size of the video to be played to the playback region is beneficial to improving the display quality. How the video content is mapped into the playback region is the main problem to be solved by the width and height adaptation of the video player.

[0127] Before playing the video to be played, multiple video frames can be obtained based on the video to be played. Multiple video frames often have the same aspect ratio and have a certain playback order. During the playback of the video, multiple video frames can be continuously obtained to perform subsequent operations in real time, and then the video is played through the video frames. Compared with using the video playback API encapsulated by the platform, to implement a highly customizable playback function in the embodiments of the present application, development needs to be carried out based on the underlying audio and video interfaces. Specifically, the video to be played can be decoded to obtain multiple video frames. For example, the decoding logic can be executed through a media codec to decode the video to be played to obtain multiple video frames. The media codec can be implemented through the MediaCodec module of the Android platform.

[0128] Reference Figure 6 As shown, it is a schematic diagram of a video playback framework provided by an embodiment of the present application. The video playback framework can include an upper layer of the player and a media codec. The upper layer of the player obtains the video to be played. The media codec, such as the MediaCodec module of the Android platform, is used to decode the video to be played to obtain multiple video frames, and continuously outputs the video frame data of multiple video frames for use. The output order of the video frame data of multiple video frames can be the same as the playback order of multiple video frames, or there can be a small difference.

[0129] The video frame data of multiple video frames is usually represented in formats such as YUV or RGB. Multiple video frames have size information, and the size information can include the width and height of multiple video frames, denoted as w v and h v .

[0130] S102. Determine the size adaptation information in the current adaptation mode based on the current adaptation mode, the size information of multiple video frames, and the size information of the texture object.

[0131] In the embodiments of the present application, the current adaptive mode can be determined. The current adaptive mode is one of multiple adaptive modes, which can be a preset adaptive mode or an adaptive mode corresponding to a mode switching operation. Multiple adaptive modes are built into the playback component, enabling quick application and switching of adaptive modes. From the perspective of screen filling, the multiple adaptive modes can include two types: full-screen display of the video and overview display of the content. Different screen filling angles can meet different visual effect requirements. Of course, the multiple adaptive modes can also include a custom mode, which can directly define the corresponding size adaptation information and will not be elaborated here.

[0132] In actual operation, in the case where no mode switching operation is obtained, the current adaptive mode can be a preset adaptive mode, that is, the default adaptive mode. If a mode switching operation is obtained during video playback and the mode switching operation corresponds to a target mode, the current adaptive mode can be updated according to the target mode, and the target mode is also one of the multiple adaptive modes. In this way, by building in multiple adaptive modes, video playback in various adaptive modes can be achieved, and switching between different adaptive modes can be performed, improving the freedom of size adaptation. Among them, the current adaptive mode and the target mode can be the same or different. If the current adaptive mode and the target mode are the same, the current adaptive mode is not updated or updated to the target mode, and the adaptive mode remains unchanged. If the current adaptive mode and the target mode are different, the current adaptive mode is updated to the target mode, that is, the adaptive mode has changed.

[0133] Among them, full-screen display of the video means that the entire video screen fills the playback area. If the size of the original video screen is different from the size of the playback area, the size of the original video screen can be scaled proportionally until the video screen fills the playback area. If the playback area is the entire screen, the video screen fills the entire screen, that is, the video screen is displayed full screen. At this time, the size of the playback view is also the size of the entire screen.

[0134] Reference Figure 7 、 Figure 8 and Figure 9 As shown in, are schematic diagrams of multiple implementation methods for full-screen display of the video in the embodiments of the present application. As shown in Figure 7 , if the aspect ratio of the original video screen is the same as that of the playback area, the video screen can be scaled proportionally to exactly fill the playback area. As shown in Figure 8 and Figure 9 , if the aspect ratio of the original video screen is different from that of the playback area, the video screen can be scaled proportionally and then cropped. The remaining part of the video screen after cropping has the same size as the playback area, so that the intercepted part fills the playback area.

[0135] Full-content display means that after the video picture is scaled, its complete content needs to be displayed within the playback area. In this way, when the size of the original video picture is inconsistent with the size of the playback area, blank spaces can be left on the left and right or up and down, and the blank spaces appear black, that is, the black edges in the video playback scene. If the playback area is the entire screen, the entire screen playback area is divided into a first area with a picture and a second area without a picture. The first area corresponds to the video playback area in the playback view, and the second area corresponds to the pixel-free area in the playback view, presenting as a black area.

[0136] Reference Figure 10 And Figure 11 As shown, it is a schematic diagram of various implementation methods of full-content display in the embodiments of the present application. Figure 7 What is shown can also be used as an implementation method of full-content display. Reference Figure 7 As shown, if the aspect ratio of the original video picture is the same as that of the playback area, the video picture can be scaled proportionally, so that it can exactly cover the playback area without losing the video picture; reference Figure 10 And Figure 11 As shown, if the aspect ratio of the original video picture is different from that of the playback area, after the video picture is scaled proportionally, the entire video picture can be mapped to the first area in the playback area, and the positions outside the first area are left blank as the second area.

[0137] Among them, the full-screen display of the picture can include at least two types: displaying the side part of the first vertex and displaying the central part. The first vertex side can include the upper left vertex side, the lower left vertex side, the upper right vertex side, and the lower right vertex side. Reference Figure 8 As shown, when the aspect ratio of the video picture is greater than that of the playback area, the height direction of the video picture can be fully displayed to retain more video content, and the width direction is intercepted. Then, displaying the upper left vertex side and the lower left vertex side actually means displaying the left part and removing the right part, and displaying the lower right vertex side and the upper right vertex side actually means displaying the right part and removing the left part. In the case of displaying the central part, both the left part and the right part are removed at the same time; reference Figure 9 As shown, when the aspect ratio of the video picture is less than that of the playback area, the width direction of the video picture can be fully displayed to retain more video content, and the height direction is intercepted. Then, displaying the upper left vertex side actually means displaying the upper part and removing the lower part, and displaying the lower right vertex side actually means displaying the lower part and removing the upper part. In the case of displaying the central part, both the upper part and the lower part are removed at the same time. Of course, in other embodiments where it is not necessary to retain more video content, the video picture can be intercepted in both the width and height directions, and the display of each first vertex side corresponds to the display of different areas to be displayed in the video picture.

[0138] The full-content display can include at least two types: display on the second vertex side and centered display. The second vertex side includes the upper-left vertex side, the lower-left vertex side, the upper-right vertex side, and the lower-right vertex side. Refer to Figure 10 As shown, when the aspect ratio of the video frame is greater than the aspect ratio of the playback area, the width direction of the playback area can be filled to retain a larger video playback area. In fact, the upper-left vertex side and the upper-right vertex side displays are both upper-side displays with the lower side left blank, and the lower-right vertex side and the lower-left vertex side displays are both lower-side displays with the upper side left blank. The centered display actually has blanks on both the upper side and the lower side. Refer to Figure 11 As shown, when the aspect ratio of the video frame is less than the aspect ratio of the playback area, the height direction of the playback area can be filled to retain a larger video playback area. In fact, the upper-left vertex side and the lower-left vertex side displays are both left-side displays with the right side left blank, and the lower-right vertex side and the upper-right vertex side displays are both right-side displays with the left side left blank. The centered display actually has blanks on both the left side and the right side. Of course, in other embodiments where it is not necessary to retain a larger video playback area, the width and height of the playback area can be selected, and the display on each first vertex side respectively corresponds to the display of different first regions of the video frame in the playback area.

[0139] Before video playback, a playback view for video playback can also be added to the view tree, thereby obtaining a drawing surface (Surface) object corresponding to the playback view. According to the drawing surface object corresponding to the playback view, a texture object is created. The size information of the drawing surface object can be determined according to the size information of the playback view corresponding to the texture object. For example, the size information of the drawing surface object is determined according to the size information of the playback view, and the size information of the texture object is determined according to the size information of the drawing surface object. The texture object is used to store texture data. The texture object has an object identifier (ID). Through the object identifier of the texture object, the texture object can be operated and called, so as to operate and call the texture data stored in the texture object. In this way, before video playback, the playback view, the drawing surface object provided by the playback view, and the texture object created accordingly are obtained, which is equivalent to constructing a framework for video playback and providing a basis for the playback of the video to be played.

[0140] The playback view is a view that can provide a drawing surface object, and can be a texture view or a surface view. The texture object can be, for example, a texture object in OpenGL, that is, the OpenGL rendering logic is introduced during video playback. Refer to Figure 6 As shown, through the OpenGL thread, the video frame data can be loaded into the texture object. Then the video frame data is used as the texture data corresponding to the texture object, which can also be called a texture image. The texture object can be created through the functions of OpenGL.

[0141] The texture object may have size information, which determines the size information of the texture image. The size information includes the width and height of the texture object, denoted as w s and h s . The texture object and the playback view may have the same aspect ratio, and their sizes may be the same or in a certain proportion. The size information of the playback view corresponding to the texture object may include the width and height, denoted as w t and h t .

[0142] In actual operation, the aspect ratios of multiple video frames are usually different from the aspect ratio of the playback area. When playing through the playback view and the size of the playback view is equal to the size of the playback area, the aspect ratios of multiple video frames are usually different from the aspect ratio of the playback view, so the aspect ratios of multiple video frames are also different from the aspect ratio of the texture object. How to map multiple video frames to the texture object determines how they are mapped into the playback area. Therefore, different adaptive modes can correspond to different size adaptation information.

[0143] Specifically, based on the current adaptive mode, the size information of multiple video frames, and the size information of the texture object, the size adaptation information in the current adaptive mode can be determined. The size adaptation information includes the position information of the area to be displayed in multiple video frames and the position information of the mapped area in the texture object. The areas to be displayed in multiple video frames are used to be mapped to the mapped areas respectively. There can be corresponding determination methods for the size video information for different adaptive modes. The determination process of the size adaptation information in different adaptive modes is referred to the subsequent description.

[0144] It can be understood that when the area to be displayed is smaller than the overall area of the video frame, it is equivalent to intercepting multiple video frames respectively during the mapping process. When the mapped area is smaller than the overall area of the texture object, it is equivalent to selecting an area of the texture object during the mapping process. Therefore, it is equivalent to defining the area selection logic of the video frame and the texture object in the current adaptive mode through the size adaptation information in the current adaptive mode. The size ratio between the two determines the scaling method of multiple video frames, so as to achieve the size adaptation between the video frame and the texture object in the current adaptive mode.

[0145] In the embodiments of the present application, a width and height adaptive module can also be set to determine the size matching information in the current adaptive mode. Refer to Figure 12 As shown, it is another schematic diagram of the video playback framework provided by the embodiments of the present application. A width and height adaptive module can be set in the OpenGL thread to solve the problem of video width and height adaptation. It can determine the size matching information in the current adaptive mode for use by the OpenGL thread during the texture loading process.

[0146] The width-height adaptive module can also provide an interface for setting the adaptive mode to the upper layer of the player for selection. Based on this interface, the user can switch the adaptive mode through mode switching operations and determine the size adaptation information in the current adaptive mode. After the current adaptive mode is updated, the playback of the next frame will take effect. That is, during the video playback process, the adaptive mode can be adjusted dynamically in real time. This playback component obtains the ability to support dynamic adjustment of the width-height adaptive mode, enabling interactive video playback, and the adaptive mode can be dynamically modified based on the user's behavior during the playback process.

[0147] Among them, the mode switching operation can include a triggering operation on the playback area. Different mode switching operations can correspond to different target modes, and the corresponding relationship between the mode switching operation and the target mode can be set according to the actual situation. The mode switching operation is, for example, a triggering operation on the mode switching control, or a triggering operation on the first area with video footage in the playback area, or a triggering operation on the second area without video footage in the playback area, etc.

[0148] For example, the triggering operation on the first area in the playback area can correspond to the adaptive mode of full-content display, and the triggering operation on the second area without video footage in the playback area can correspond to the adaptive mode of full-screen display; or, the first triggering operation on the first area with video footage in the playback area corresponds to switching the display from the upper left vertex side to the central part, and the second triggering operation corresponds to switching the display from the central part to the lower right vertex side, etc.

[0149] S103. Based on the size adaptation information, according to the playback order of multiple video frames, sequentially load the video frame data of multiple video frames into the texture object.

[0150] After determining the size adaptation information, the video frame data of multiple video frames can be sequentially loaded into the texture object according to the playback order of multiple video frames. The video frame data loaded into the texture object is used as the texture data corresponding to the texture object, which can also be called the texture image or the data of the texture object. Specifically, the video frame data of multiple video frames can be sequentially loaded into the memory space corresponding to the texture object. In this way, the video frames loaded into the texture object can perform subsequent operations for display. The areas to be displayed of multiple video frames are used to be mapped to the mapping area respectively. When the area to be displayed is smaller than the overall area of the video frame, the mapping process is equivalent to intercepting multiple video frames. When the mapping area is smaller than the overall area of the texture object, the mapping process is equivalent to selecting an area of the texture object.

[0151] The video frame data of a video frame can be loaded into a texture object through a function, such as the glTexImage2D function. During the process of loading the video frame data of a video frame into a texture object, the mapping area and the area to be displayed can be defined through a function; alternatively, based on the position information of the area to be displayed and according to the playback order of multiple video frames, multiple video frames can be intercepted in sequence to obtain the image to be displayed, and the image data of the image to be displayed is loaded into the texture object in sequence. At this time, the mapping area can be defined through a function, and the image to be displayed is used to define the area to be displayed; alternatively, the area to be displayed can be defined through a function, and the mapping area is default to the entire area of the texture object; or, the mapping area is default to the entire area of the texture object, and the area to be displayed is the entire area of the image to be displayed or the entire area of the video frame, then the function can not define the mapping area and the area to be displayed. Among them, intercepting multiple video frames to obtain the image to be displayed can be implemented through a width and height adaptive module, and the image data of the image to be displayed is called and loaded into the texture object, which can simplify the parameter configuration of the texture loading process.

[0152] Before loading the video frame data of multiple video frames into a texture object, the video frame data of multiple video frames can be converted to a texture data format suitable for OpenGL processing. This may involve operations such as color space conversion and data format adjustment. For example, converting the YUV format to the RGB format and arranging the data into a texture image format that OpenGL can recognize, etc.

[0153] S104, continuously draw the data of the texture object to the drawing surface object corresponding to the playback view to play the video using the playback view.

[0154] In the embodiment of the present application, after loading the video frame data of a video frame into a texture object, the data of the texture object can be continuously drawn to the drawing surface object corresponding to the playback view, which is equivalent to using the texture image of the texture object as the texture map of the drawing surface object, so that the playback view has video content. In the case where multiple video frames are loaded into the texture object in sequence according to the playback order, the playback view sequentially displays the content of the area to be displayed of each video frame, realizing the playback of the video to be played.

[0155] That is to say, based on the region selection logic defined by the size adaptation information in the current adaptive mode, texture loading of the video frame and drawing of the texture object can be performed. That is, through the size adaptation between the video frame and the texture object, the size adaptation between the video frame and the drawing surface object is achieved, enabling the area to be displayed of the video frame to be shown within the video playback area corresponding to the mapping area of the playback view. In this way, the problem of size adaptation between the video to be played and the playback view is transformed into the size adaptation between the video frame and the texture object during the texture loading process. Different adaptive modes can correspond to different size adaptation logics between the video frame and the texture object, greatly enhancing the degree of freedom of size adaptation between the video to be played and the playback view. Moreover, compared with the size adaptation between the video and the view, the adaptation efficiency between the image frame and the texture object is higher and the real-time performance is better. During the playback process, size adaptation can be achieved without adjusting the playback view, without generating additional memory occupancy, improving the degree of freedom of size adaptation between the video to be played and the playback view while ensuring the excellent performance of the playback view.

[0156] During the process of drawing the data of the texture object onto the drawing surface object corresponding to the playback view, sampling can be performed from the texture image of the texture object through the texture coordinates of the points of the drawing surface object to obtain the mapped image of the drawing surface object. The mapped image is displayed on the display device through the playback view, realizing the video picture being shown on the screen, and the user can then see the video playback picture.

[0157] Reference Figure 6 and Figure 12 As shown, the drawing surface object serves as the display container for the video playback picture. It is provided by the playback view and can be used as the rendering and drawing window of OpenGL. In this way, OpenGL and MediaCodec, combined with Surface, constitute a video playback framework, obtaining a set of video playback components with high efficiency, high degree of freedom, and video width and height self-adaptation, realizing a video player with width and height self-adaptation on the Android platform. In this video playback component, MediaCodec realizes decoding, and OpenGL and Surface realize rendering and showing on the screen, enabling the user to see the video picture. The playback view is equivalent to providing the basic environment for OpenGL operations, including creating and managing the OpenGL context.

[0158] Based on the above description, the overall operation logic of the video playback process includes: adding a playback view to the view tree to provide a drawing surface object through the playback view; in the OpenGL thread, initializing the rendering window through the drawing surface object and creating a texture object; decoding the video to be played through MediaCodec to obtain multiple video frames and continuously obtaining the decoded video frame data; uploading the video frame data of multiple video frames to the texture object according to the size matching information corresponding to the current adaptive mode; the OpenGL thread continuously draws the data of the texture object to the drawing window (i.e., the drawing surface object) to display the video frame on the screen. The decoding, uploading of video frame data to the texture object, and drawing of the texture object data to the drawing window are executed in a loop until the video playback is completed. During the process of uploading the video frame data to the texture object, the size adaptation between the video frame and the texture object is realized, and this size adaptation can be executed in real time and adjusted according to requirements in real time.

[0159] The parameters of the playback view can be customized. The playback view is displayed on the screen and can be used as the playback area on the screen. In the case of full-screen display, the size of the playback view can be the screen size; in the case of non-full-screen display, the size of the playback view is smaller than the screen size. In this way, the size of the playback view can be determined according to the size of the playback area, the size of the drawing surface object can be determined according to the size of the playback view, and the size of the texture object can be determined, so as to realize the size adaptation between the video frame and the playback area through the size adaptation between the texture object and the video frame.

[0160] In the embodiment of the present application, if the playback view is a texture view and the playback view is added to the view tree, then if a display transformation operation for the view tree is obtained, the display parameters of the playback view can be adjusted according to the display transformation operation. The display parameters include position parameters, size parameters, etc., and are used to perform transformation operations such as rotation, scaling, and translation on the playback view, that is, the playback view can be transformed, animated, or cropped together with other View interfaces, enriching the form of video playback.

[0161] In the embodiment of the present application, if the playback view is a surface view, the operation of continuously drawing the texture object to the drawing surface object corresponding to the playback view can be executed by a background thread other than the main thread, without occupying the main thread, reducing the main thread lag, and improving the playback performance.

[0162] The determination process of the size adaptation information in different adaptive modes is introduced below.

[0163] The size matching information includes the position information of the display area to be displayed of multiple video frames and the position information of the mapping area of the texture object, and there is a mapping relationship between the two. In this way, based on the mapping relationship between the texture object and the drawing surface object, the display area to be displayed can be mapped to the drawing surface object.

[0164] As a possible implementation, if the current adaptive mode includes full-screen display of the picture, indicating that the entire area of the texture object is occupied, the position information of the mapping area can be determined according to the entire area of the texture object. For example, the position information of the entire area of the texture object can be used as the position information of the mapping area. Then, according to the size information of multiple video frames and the size information of the mapping area, the position information of the area to be displayed is determined, so that the area to be displayed and the mapping area have the same aspect ratio, ensuring the proportional playback of video frames in the full-screen display mode and guaranteeing a good display effect of the video. It should be noted that the entire area of the texture object is the area that has a mapping relationship with the entire visible area of the playback view. Therefore, when the entire area of the texture object is occupied, it means that the entire visible area of the drawing surface object is occupied, which is equivalent to the entire visible area of the playback view being occupied.

[0165] In the process of determining the position information of the area to be displayed, the position information of the first feature point in the area to be displayed can be determined according to the current adaptive mode; according to the size information of multiple video frames and the size information of the mapping area, the size of the first side of the video display area and the size of the second side adjacent to the first side are determined; according to the position information of the first feature point, the size of the first side and the size of the second side, the position information of the area to be displayed is determined. In this way, with the first feature point as the anchor point, the area to be displayed can be constructed according to the size of the first side and the size of the second side, and then the position information of the area to be displayed is determined, improving the accuracy of determining the position information.

[0166] Among them, if the current adaptive mode includes displaying a part of the first vertex side, the first feature point is the first vertex, and the position information of the first vertex is determined according to the position information of the first vertex of multiple video frames. The first vertex side can include the upper left vertex side, the lower left vertex side, the upper right vertex side, and the lower right vertex side; if the current adaptive mode includes displaying the central part, the first feature point is the center point of the area, and the position information of the center point of the area is determined according to the position information of the center points of multiple video frames.

[0167] In the process of determining the sizes of the first side and the second side of the area to be displayed, if it is determined according to the size information of multiple video frames and the size information of the mapping area that the aspect ratio of multiple video frames is equal to the aspect ratio of the mapping area, the height of multiple video frames is used as the size of the first side of the area to be displayed, and the width of multiple video frames is used as the size of the second side adjacent to the first side. That is, the entire area of each of multiple video frames is used as the area to be displayed.

[0168] Let the width and height of multiple video frames be denoted as w v and h v , and the aspect ratio r v = w v / hv , the entire area of multiple video frames can be represented as (0,0)-(w v ,h v ); Denote the width and height of the playback view as w t and h t , and the aspect ratio r t = w t / h t , then the playback view can be represented as (0,0)-(w t ,h t ); Denote the width and height of the texture object corresponding to the playback view as w s and h s , and the aspect ratio r s = w s / h s , then the entire area of the texture object can be represented as (0,0)-(w s ,h s ). After determining the mapping area of the texture object, it can be mapped to the video playback area of the playback view. Thus, after the texture object loads the video frames, video display can be performed within the video playback area of the playback view. The aspect ratios of the playback view and the texture object can be equal, that is, r t = w t / h t = w s / h s = r s .

[0169] In the scenario where the screen is fully covered for display, the entire area of the texture object serves as the mapping area. Then, the width and height of the mapping area are respectively the width and height of the texture object, which are w s and h s , and the aspect ratio is r s . The mapping area can be represented as (0,0)-(w s ,h s ). The video playback area corresponding to the mapping area in the playback view can be represented as (0,0)-(w t ,h t ), which is the entire area of the playback view.

[0170] Specifically, as shown in Figure 7 , if it is determined that the aspect ratio of multiple video frames is equal to the aspect ratio of the mapping area (r s = r v ), then the height h v of multiple video frames can be used as the dimension of the first side, and the width w v of multiple video frames can be used as the dimension of the second side. It can be determined that the area to be displayed is the entire area of each of the multiple video frames, represented as (0,0)-(w v ,hv )。That is, map the area to be displayed (0, 0)-(w v , h v ) to the mapped area (0, 0)-(w s , h s ). The video playing area in the corresponding playing view is (0, 0)-(w t , h t ).

[0171] In the process of determining the sizes of the first side and the second side of the area to be displayed, if it is determined according to the size information of multiple video frames and the size information of the mapped area that the aspect ratio of multiple video frames is greater than the aspect ratio of the mapped area, then the height directions of multiple video frames can be fully displayed to retain more video content, and they are intercepted in their width directions. That is, the height of multiple video frames is used as the size of the first side of the area to be displayed. Then, according to the aspect ratio of the mapped area and the size of the first side, the size of the second side adjacent to the first side is determined. Since the aspect ratios of the mapped area and the area to be displayed are equal, it is equivalent to using the aspect ratio of the mapped area as the aspect ratio of the area to be displayed. Then, according to the aspect ratio of the area to be displayed and the size of the first side, the size of the second side is determined. In this way, the largest area is determined from multiple video frames as the area to be displayed that meets the aspect ratio requirements, and the content of multiple video frames is retained to the greatest extent.

[0172] Specifically, if it is determined that the aspect ratio of multiple video frames is greater than the aspect ratio of the mapped area (r v > r s ), then the height h v of multiple video frames can be used as the size of the first side, that is, the height range of the area to be displayed is 0 - h v , and the aspect ratio r s of the mapped area is used as the aspect ratio of the area to be displayed. Then, the size of the second side can be denoted as h v r s .

[0173] Refer to Figure 8 shown. If the current adaptive mode includes displaying the first vertex side part, and the first vertex side part is the upper left vertex side, then the position of the first vertex of multiple video frames is (0, 0), which can be used as the first feature point in the area to be displayed, that is, the position of the first vertex. According to the position (0, 0) of the first vertex of the area to be displayed, as well as the size h v of the first side and the size h v r s of the second side, the area to be displayed can be determined as (0, 0)-(r s h v , h v ). That is, map the area to be displayed (0, 0)-(rs h v , h v ) is mapped to the mapping area (0, 0)-(w s , h s ). The video playing area in the corresponding playing view is (0, 0)-(w t , h t ).

[0174] Reference Figure 8 As shown, if the current adaptive mode includes displaying the first vertex side part, and the first vertex side part is the lower right vertex side, the position of the first vertex of multiple video frames is (w v , h v ), which can be used as the first feature point in the area to be displayed, that is, the position of the first vertex. According to the position of the first vertex of the area to be displayed (w v , h v ), and the first side dimension h v and the second side dimension h v r s , the area to be displayed can be determined as (w v - h v r s , 0)-(w v , h v ). That is, the area to be displayed (w v - h v r s , 0)-(w v , h v ) is mapped to the mapping area (0, 0)-(w s , h s ). The video playing area in the corresponding playing view is (0, 0)-(w t , h t ).

[0175] Reference Figure 8 As shown, if the current adaptive mode includes displaying the central part, the position of the first vertex of multiple video frames is the central position (w v / 2, h v / 2), which can be used as the first feature point of the area to be displayed, that is, the position of the center point of the area. According to the position of the center of the area to be displayed (w v / 2, h v / 2), and the first side dimension h v and the second side dimension h v r s , the area to be displayed can be determined as (w v / 2 - h v r s / 2, 0)-(wv / 2 + h v r s / 2,h v ). Area to be displayed (w v / 2-h v r s / 2,0)-(w v / 2 + h v r s / 2,h v ) is mapped to the mapping area (0,0)-(w s ,h s ). The corresponding video playback area in the playback view is (0,0)-(w t ,h t ).

[0176] In the process of determining the size of the first side and the second side of the area to be displayed, if it is determined that the aspect ratio of the multiple video frames is smaller than the aspect ratio of the mapping area based on the size of the multiple video frames and the size information of the mapping area, then the width direction of the multiple video frames can be fully displayed to retain more video content, and they are cut in the height direction, that is, the width of the multiple video frames is used as the size of the second side of the area to be displayed, and then the size of the first side adjacent to the second side is determined based on the aspect ratio of the mapping area and the size of the second side. Among them, the aspect ratio of the mapping area and the area to be displayed is equal, so it is equivalent to using the aspect ratio of the mapping area as the aspect ratio of the area to be displayed, and then determining the size of the first side based on the aspect ratio of the area to be displayed and the size of the second side. In this way, the largest area is determined from the multiple video frames as the area to be displayed that meets the aspect ratio requirements, and the content of the multiple video frames is retained to the greatest extent.

[0177] Specifically, if it is determined that the aspect ratio of the plurality of video frames is smaller than the aspect ratio of the mapping region (r v <r s ), then the width w of multiple video frames can be v As the size of the second side, the width of the area to be displayed ranges from 0 to w v , the aspect ratio r of the mapping area s As the aspect ratio of the area to be displayed, the size of the first side can be recorded as w v / r s .

[0178] refer to Figure 9As shown, if the current adaptive mode includes displaying the first vertex side part and the first vertex side part is the upper left vertex side, the position of the first vertex of multiple video frames is (0, 0), which can be used as the first feature point in the area to be displayed, that is, the position of the first vertex. According to the position (0, 0) of the first vertex of the area to be displayed and the first side dimension w v / r s and the second side dimension w v , the area to be displayed can be determined as (0, 0)-(w v , w v / r s ). That is, the area to be displayed (0, 0)-(w v , w v / r s ) is mapped to the mapping area (0, 0)-(w s , h s ). The video playing area in the corresponding playing view is (0, 0)-(w t , h t ).

[0179] Reference Figure 9 As shown, if the current adaptive mode includes displaying the first vertex side part and the first vertex side part is the lower right vertex side, the position of the first vertex of multiple video frames is (w v , h v ), which can be used as the first feature point in the area to be displayed, that is, the position of the first vertex. According to the position (w v , h v ) of the first vertex of the area to be displayed and the first side dimension w v / r s and the second side dimension w v , the area to be displayed can be determined as (0, h v -w v / r s )-(w v , h v ). That is, the area to be displayed (0, h v -w v / r s )-(w v , h v ) is mapped to the mapping area (0, 0)-(w s , h s ). The video playing area in the corresponding playing view is (0, 0)-(w t , h t ).

[0180] Reference Figure 9As shown, if the current adaptive mode includes displaying the central part, the position of the first vertex of multiple video frames is the central position (w v / 2, h v / 2), which can be used as the first feature point of the area to be displayed, that is, the position of the center point of the area. According to the position of the center of the area to be displayed (w v / 2, h v / 2), and the first side dimension w v / r s and the second side dimension w v , the area to be displayed can be determined as (0, h v / 2 - w v / r s / 2)-(w v , h v / 2 + w v / r s / 2). That is, the area to be displayed (0, h v / 2 - w v / r s / 2)-(w v , h v / 2 + w v / r s / 2) is mapped to the mapping area (0, 0)-(w s , h s ). The video playing area in the corresponding playing view is (0, 0)-(w t , h t ).

[0181] As another possible implementation, if the current adaptive mode includes full content preview display, it means that it is necessary to ensure that all the content in the video frames is displayed first. Then, the position information of the area to be displayed is determined according to the entire area of multiple video frames. For example, the position information of the entire area of multiple video frames is used as the position information of the area to be displayed. Then, according to the size information of the area to be displayed and the size information of the texture object, the position information of the mapping area is determined to make the area to be displayed and the mapping area have the same aspect ratio, ensuring the proportional playing of video frames in the full content preview display mode and ensuring a good display effect of the video.

[0182] In the process of determining the position information of the mapping area, the position information of the second feature point in the mapping area can be determined according to the current adaptive mode; according to the size information of the area to be displayed and the size information of the texture object, the size of the third side of the mapping area and the size of the fourth side adjacent to the third side are determined; according to the position information of the second feature point, the size of the third side and the size of the fourth side, the position information of the mapping area is determined. In this way, with the second feature point as the anchor point, the mapping area can be constructed according to the size of the third side and the size of the fourth side, and then the position information of the mapping area is determined, improving the accuracy of determining the position information.

[0183] Among them, if the current adaptive mode includes second vertex side display, the second feature point is the second vertex, and the position information of the second vertex is determined according to the position information of the second vertex of the texture object. The second vertex side can include the upper left vertex side, the lower left vertex side, the upper right vertex side, and the lower right vertex side; if the current adaptive mode includes centered display, the second feature point is the center point of the area, and the position information of the center point of the area is determined according to the position information of the center point of the texture object.

[0184] In the process of determining the third side and the fourth side of the mapping area, if it is determined that the aspect ratio of the area to be displayed is equal to the aspect ratio of the texture object, the width of the texture object is used as the size of the third side of the mapping area; the height of the texture object is used as the size of the fourth side adjacent to the third side. That is, the entire area of the mapping area is used as the mapping area.

[0185] In the scenario where the screen is fully displayed, the area to be displayed is the entire area of each of multiple video frames. Then the width and height of the area to be displayed are the width and height of the multiple video frames, which are w v and h v , and the aspect ratio is r v . The area to be displayed can be expressed as (0,0)-(w v ,h v ).

[0186] Specifically, as shown in Figure 7 , if it is determined that the aspect ratio of the area to be displayed is equal to the aspect ratio of the texture object (r s = r v ), then the width w s of the texture object can be used as the size of the third side of the mapping area; the height h s of the texture object can be used as the size of the fourth side adjacent to the third side. The mapping area is the entire area of the texture object, expressed as (0,0)-(w s ,h s ). That is, the area to be displayed (0,0)-(w v ,h v)Mapped to the mapping area (0, 0)-(w s , h s ). The video playing area in the corresponding playing view is (0, 0)-(w t , h t ).

[0187] In the process of determining the third side and the fourth side of the mapping area, if it is determined according to the sizes of multiple video frames and the size information of the mapping area that the aspect ratio of the area to be displayed is greater than the aspect ratio of the texture object, the width direction of the playing view can be filled to retain a larger video playing area, that is, the width of the texture object is used as the size of the third side of the mapping area; according to the aspect ratio of the area to be displayed and the size of the third side, the size of the fourth side adjacent to the third side is determined. Among them, the aspect ratios of the mapping area and the area to be displayed are equal, so it is equivalent to using the aspect ratio of the area to be displayed as the aspect ratio of the mapping area, and then determining the size of the fourth side according to the aspect ratio of the mapping area and the size of the third side. In this way, the largest area is determined from the texture object as the mapping area that meets the aspect ratio requirements, ensuring the display area to the greatest extent.

[0188] Specifically, if it is determined that the aspect ratio of multiple video frames is greater than the aspect ratio of the texture object (r v > r s ), then the width w s of the texture object can be used as the size of the third side of the mapping area, that is, the width range of the texture object is 0 - w s , and the aspect ratio r v of the area to be displayed is used as the aspect ratio of the mapping area, then the size of the fourth side can be denoted as w s / r v .

[0189] Refer to Figure 10 As shown, if the current adaptive mode includes displaying the second vertex side part, and the second vertex side part is the upper left vertex side, the position of the second vertex of the texture object is (0, 0), which can be used as the second feature point in the mapping area, that is, the position of the second vertex. According to the position (0, 0) of the second vertex of the mapping area, as well as the size w s of the third side and the size w s / r v of the fourth side, the mapping area can be determined as (0, 0)-(w s , w s / r v ), that is, mapping the area to be displayed (0, 0)-(w v , h v ) to the mapping area (0, 0)-(w s , w s / r v), the video playing area in the corresponding playing view can be expressed as (0,0)-(w t ,w s / r v ).

[0190] Refer to Figure 10 As shown, if the current adaptive mode includes displaying the second vertex side part, and the second vertex side part is the lower right vertex side, the position of the second vertex of the texture object is (w s ,h s ), which can be used as the second feature point in the mapping area, that is, the position of the second vertex. According to the position of the second vertex of the mapping area (w s ,h s ), and the third side dimension w s and the fourth side dimension w s / r v , the mapping area can be determined as (0,h s -w s / r v )-(w s ,h s ), that is, mapping the area to be displayed (0,0)-(w v ,h v ) to the mapping area (0,h s -w s / r v )-(w s ,h s ), and the video playing area in the corresponding playing view can be expressed as (0,h t -w t / r v )-(w t ,h t ).

[0191] Refer to Figure 10 As shown, if the current adaptive mode includes displaying the central part, the position of the second vertex of the texture object is the central position (w s / 2,h s / 2), which can be used as the second feature point in the mapping area, that is, the position of the center point of the area. According to the position of the center of the mapping area (w s / 2,h s / 2), and the third side dimension w s and the fourth side dimension w s / r v , the mapping area can be determined as (0,h s / 2-w s / r v / 2)-(w s ,h s / 2+ws / r v / 2), that is, map the to-be-displayed area (0, 0)-(w v , h v ) to the mapped area (0, h s / 2 - w s / r v / 2)-(w s , h s / 2 + w s / r v / 2), and the video playing area in the corresponding playing view can be expressed as (0, h t / 2 - w t / r v / 2)-(w t , h t / 2 + w t / r v / 2).

[0192] In the process of determining the third side and the fourth side of the mapped area, if it is determined that the aspect ratio of the to-be-displayed area is less than the aspect ratio of the texture object according to the size information of multiple video frames and the size of the mapped area, the height direction of the playing view can be filled to retain a larger video playing area, that is, use the height of the texture object as the size of the fourth side of the mapped area; according to the aspect ratio of the to-be-displayed area and the size of the fourth side, determine the size of the third side adjacent to the fourth side. Among them, the aspect ratios of the mapped area and the to-be-displayed area are equal, so it is equivalent to using the aspect ratio of the to-be-displayed area as the aspect ratio of the mapped area, and then determining the size of the third side according to the aspect ratio of the mapped area and the size of the fourth side. In this way, the largest area is determined from the texture object as the mapped area that meets the aspect ratio requirements, ensuring the display area to the greatest extent.

[0193] Specifically, if it is determined that the aspect ratio of the to-be-displayed area is greater than the aspect ratio of the texture object (r v <r s ), then the height h s of the texture object can be used as the size of the fourth side of the mapped area, that is, the height range of the texture object is 0 - h s , and the aspect ratio r v of the to-be-displayed area is used as the aspect ratio of the mapped area, then the size of the third side can be denoted as r v h s .

[0194] Reference Figure 11As shown, if the current adaptive mode includes displaying the second vertex side part, and the second vertex side part is the upper left vertex side, the position of the second vertex of the texture object is (0, 0), which can be used as the second feature point in the mapping area, that is, the position of the second vertex. According to the position (0, 0) of the second vertex of the mapping area, and the third side dimension r v h s and the fourth side dimension h s , the mapping area can be determined as (0, 0)-(r v h s ,h s ), that is, mapping the area to be displayed (0, 0)-(w v ,h v ) to the mapping area (0, 0)-(r v h s ,h s ). The video playback area in the corresponding playback view can be expressed as (0, 0)-(r v h t ,h t ).

[0195] Refer to Figure 11 As shown, if the current adaptive mode includes displaying the second vertex side part, and the second vertex side part is the lower right vertex side, the position of the second vertex of the texture object is (w s ,h s ), which can be used as the second feature point in the mapping area, that is, the position of the second vertex. According to the position (w s ,h s ) of the second vertex of the mapping area, and the third side dimension r v h s and the fourth side dimension h s , the mapping area can be determined as (w s -r v h s ,0)-(w s ,h s ), that is, mapping the area to be displayed (0, 0)-(w v ,h v ) to the mapping area (w s -r v h s ,0)-(w s ,h s ). The video playback area in the corresponding playback view can be expressed as (w t -r v h t ,0)-(w t ,h t ).

[0196] Refer toFigure 11 As shown, if the current adaptive mode includes displaying the central part, the position of the second vertex of the texture object is the central position (w s / 2, h s / 2), which can be used as the second feature point of the mapping area, that is, the position of the center point of the area. According to the position of the center of the mapping area (w s / 2, h s / 2), and the third side dimension r v h s and the fourth side dimension h s , the mapping area can be determined as (w s / 2 - r v h s / 2, 0) - (w s / 2 + r v h s / 2, h s ), that is, mapping the area to be displayed (0, 0) - (w v , h v ) to the mapping area (w s / 2 - r v h s / 2, 0) - (w s / 2 + r v h s / 2, h s ), and the video playing area in the corresponding playing view can be expressed as (w t / 2 - r v h t / 2, 0) - (w t / 2 + r v h t / 2, h t ).

[0197] Based on a video playing method provided by an embodiment of the present application, an embodiment of the present application further provides a video playing device. Referring to Figure 13 shown, it is a structural block diagram of a video playing device provided by an embodiment of the present application. The video playing device 1300 includes:

[0198] A video frame acquisition unit 1301, configured to obtain multiple video frames based on the video to be played;

[0199] A size adaptation information determination unit 1302, configured to determine size adaptation information in the current adaptation mode based on the current adaptation mode, size information of the multiple video frames, and size information of the texture object; the size information of the texture object is determined according to size information of a playback view corresponding to the texture object; the current adaptation mode is one of multiple adaptation modes; the size adaptation information includes position information of a region to be displayed in the multiple video frames and position information of a mapped region in the texture object;

[0200] A texture loading unit 1303, configured to sequentially load video frame data of the multiple video frames into the texture object based on the size adaptation information according to the playback order of the multiple video frames, where regions to be displayed in the multiple video frames are respectively used to be mapped to the mapped region;

[0201] A texture drawing unit 1304, configured to continuously draw data of the texture object onto a drawing surface object corresponding to the playback view, so as to play a video by using the playback view.

[0202] Optionally, the size adaptation information determination unit includes:

[0203] A first position determination unit, configured to, if the current adaptation mode includes full-screen display, determine position information of the mapped region according to the entire region of the texture object;

[0204] A second position determination unit, configured to determine position information of the region to be displayed according to size information of the multiple video frames and size information of the mapped region, so that the region to be displayed and the mapped region have the same aspect ratio.

[0205] Optionally, the second position determination unit includes:

[0206] A first feature point position determination unit, configured to determine position information of a first feature point in the region to be displayed according to the current adaptation mode; if the current adaptation mode includes displaying a first vertex side part, the first feature point is a first vertex, and the position information of the first vertex is determined according to position information of the first vertex of the multiple video frames; if the current adaptation mode includes displaying a central part, the first feature point is a region center point, and the position information of the region center point is determined according to position information of the center point of the multiple video frames;

[0207] A first size determination unit, configured to determine the size of a first side edge of the region to be displayed and the size of a second side edge adjacent to the first side edge according to size information of the multiple video frames and size information of the mapped region;

[0208] A to-be-displayed area determination subunit, configured to determine the position information of the to-be-displayed area according to the position information of the first feature point, the size of the first side, and the size of the second side.

[0209] Optionally, the first size determination unit is specifically configured to:

[0210] If it is determined according to the size information of the multiple video frames and the size information of the mapping area that the aspect ratio of the multiple video frames is greater than the aspect ratio of the mapping area, take the height of the multiple video frames as the size of the first side of the to-be-displayed area; determine the size of the second side adjacent to the first side according to the aspect ratio of the mapping area and the size of the first side;

[0211] If it is determined according to the size information of the multiple video frames and the size information of the mapping area that the aspect ratio of the multiple video frames is less than the aspect ratio of the mapping area, take the width of the multiple video frames as the size of the second side of the to-be-displayed area; determine the size of the first side adjacent to the second side according to the aspect ratio of the mapping area and the size of the second side.

[0212] Optionally, if the current adaptive mode corresponds to full-screen display of the picture, the texture loading unit includes:

[0213] An adaptive module, configured to sequentially intercept the multiple video frames to obtain to-be-displayed images based on the position information of the to-be-displayed area and according to the playing order of the multiple video frames;

[0214] A texture loading subunit, configured to sequentially load the image data of the to-be-displayed images into the texture object.

[0215] Optionally, the size adaptation information determination unit includes:

[0216] A third position determination unit, configured to, if the current adaptive mode includes full-content preview display, determine the position information of the to-be-displayed area according to the entire area of the multiple video frames;

[0217] A fourth position determination unit, configured to determine the position information of the mapping area according to the size information of the to-be-displayed area and the size information of the texture object, so that the to-be-displayed area and the mapping area have the same aspect ratio.

[0218] Optionally, the fourth position determination unit includes:

[0219] The second feature point position determination unit is configured to determine the position information of the second feature point in the mapping area according to the current adaptive mode; if the current adaptive mode includes second vertex side display, the second feature point is the second vertex, and the position information of the second vertex is determined according to the position information of the second vertex of the texture object; if the current adaptive mode includes centered display, the second feature point is the center point of the area, and the position information of the center point of the area is determined according to the position information of the center point of the texture object;

[0220] The second dimension determination unit is configured to determine the dimension of the third side of the mapping area and the dimension of the fourth side adjacent to the third side according to the dimension information of the area to be displayed and the dimension information of the texture object;

[0221] The mapping area determination sub-unit is configured to determine the position information of the mapping area according to the position information of the second feature point, the dimension of the third side, and the dimension of the fourth side.

[0222] Optionally, the second dimension determination unit is specifically configured to:

[0223] If it is determined according to the dimension information of the area to be displayed and the dimension information of the texture object that the aspect ratio of the area to be displayed is greater than the aspect ratio of the texture object, the width of the texture object is used as the dimension of the third side of the mapping area; according to the aspect ratio of the area to be displayed and the dimension of the third side, the dimension of the fourth side adjacent to the third side is determined;

[0224] If it is determined according to the dimension information of the area to be displayed and the dimension information of the texture object that the aspect ratio of the area to be displayed is less than the aspect ratio of the texture object, the height of the texture object is used as the dimension of the fourth side of the mapping area; according to the aspect ratio of the area to be displayed and the dimension of the fourth side, the dimension of the third side adjacent to the fourth side is determined.

[0225] Optionally, the device further includes:

[0226] The playback view adding unit is configured to add the playback view in the view tree to obtain the drawing surface object corresponding to the playback view; the dimension information of the drawing surface object is determined according to the dimension information of the playback view;

[0227] The texture object creation unit is configured to create a texture object according to the drawing surface object corresponding to the playback view; the dimension information of the texture object is determined according to the dimension information of the drawing surface object.

[0228] Optionally, the playback view is a texture view, and the device further includes:

[0229] A display parameter adjustment unit, configured to, if a display transformation operation for the view tree is obtained, adjust the display parameters of the playback view according to the display transformation operation.

[0230] Optionally, the device further includes:

[0231] An adaptive mode switching unit, configured to, during the process of playing a video using the playback view, in response to a mode switching operation including a target mode, update the current adaptive mode according to the target mode, where the target mode is one of the multiple adaptive modes.

[0232] It can be seen from the above technical solution that multiple video frames can be obtained based on the video to be played. Based on the current adaptive mode, the size information of the multiple video frames, and the size information of the playback view, the size adaptation information in the current adaptive mode can be determined. The size information of the texture object is determined according to the size of the playback view corresponding to the texture object. The current adaptive mode is one of the multiple adaptive modes. The size adaptation information includes the position information of the area to be displayed in the multiple video frames and the position information of the mapped area in the texture object corresponding to the playback view, where the areas to be displayed in the multiple video frames are respectively mapped to the mapped area. It can be understood that when the area to be displayed is smaller than the overall area of the video frame, the mapping process is equivalent to intercepting the multiple video frames. When the mapped area is smaller than the overall area of the texture object, the mapping process is equivalent to making a region selection for the texture object. Therefore, through the size adaptation information in the current adaptive mode, the region selection logic between the video frame and the texture object in the current adaptive mode can be defined, and the size adaptation between the video frame and the texture object in the current adaptive mode can be achieved.

[0233] After that, based on the size adaptation information, according to the playback order of multiple video frames, the video frame data of multiple video frames can be loaded into the texture object in sequence, and then the data of the texture object can be continuously drawn onto the drawing surface object corresponding to the playback view, so as to play the video using the playback view. That is to say, based on the region selection logic defined by the size adaptation information in the current adaptive mode, the texture loading of video frames and the drawing of the texture object can be performed. That is, through the size adaptation between the video frame and the texture object, the size adaptation between the video frame and the playback view is achieved, so that the area to be displayed of the video frame can be displayed within the video playback area corresponding to the mapped area in the playback view. In this way, the size adaptation problem between the video to be played and the playback view is transformed into the size adaptation between the video frame and the texture object during the texture loading process. Different adaptive modes can correspond to different size adaptation logics between the video frame and the texture object, greatly improving the freedom of size adaptation between the video to be played and the playback view. Compared with the size adaptation between the video to be played and the playback view, the adaptation efficiency between the image frame and the texture object is higher and the real-time performance is better. During the playback process, size adaptation can be achieved without adjusting the playback view, and no additional memory occupancy will be generated. In this way, while improving the freedom of size adaptation between the video to be played and the playback view, the excellent performance of the playback view is ensured.

[0234] The embodiment of the present application also provides a computer device, which is the computer device described above, and may include a terminal device or a server. The foregoing video playback device may be configured in this computer device. The computer device will be introduced below with reference to the accompanying drawings.

[0235] If the computer device is a terminal device, please refer to Figure 14 As shown, the embodiment of the present application provides a terminal device. Taking the terminal device as a mobile phone as an example:

[0236] Figure 14 The block diagram of a part of the structure of the mobile phone related to the terminal device provided by the embodiment of the present application is shown. Refer to Figure 14 , the mobile phone includes: a Radio Frequency (RF) circuit 1410, a memory 1420, an input unit 1430, a display unit 1440, a sensor 1450, an audio circuit 1460, a Wireless Fidelity (WiFi) module 1470, a processor 1480, and a power supply 1490, etc. Those skilled in the art can understand that Figure 14 the structure of the mobile phone shown in

[0237] does not constitute a limitation to the mobile phone, and may include more or fewer components than shown, or combine some components, or have different component arrangements. Figure 14 The following will specifically introduce each component of the mobile phone in combination with

[0238] The RF circuit 1410 can be used for receiving and transmitting information or signals during communication. Specifically, after receiving the downlink information from the base station, it is processed by the processor 1480. Additionally, it transmits the uplink data to the base station.

[0239] The memory 1420 can be used to store software programs and modules. The processor 1480 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 1420. The memory 1420 mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, image playback function, etc.); the data storage area can store the data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory 1420 can include high-speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage devices.

[0240] The input unit 1430 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the mobile phone. Specifically, the input unit 1430 can include a touch panel 1431 and other input devices 1432.

[0241] The display unit 1440 can be used to display the information input by the user or the information provided to the user, as well as various menus of the mobile phone. The display unit 1440 can include a display panel 1441.

[0242] The mobile phone can also include at least one sensor 1450, such as a light sensor, a motion sensor, and other sensors.

[0243] The audio circuit 1460, the speaker 1461, and the microphone 1462 can provide an audio interface between the user and the mobile phone.

[0244] WiFi belongs to short-range wireless transmission technology. The mobile phone can help users receive and send emails, browse the web, and access streaming media through the WiFi module 1470, which provides users with wireless broadband Internet access.

[0245] The processor 1480 is the control center of the mobile phone, connecting various parts of the entire mobile phone using various interfaces and circuits. By running or executing the software programs and / or modules stored in the memory 1420, and by calling the data stored in the memory 1420, it executes various functions of the mobile phone and processes data.

[0246] The mobile phone also includes a power supply 1490 (such as a battery) for powering each component.

[0247] In this embodiment, the processor 1480 included in the terminal device further has the following functions:

[0248] Obtain multiple video frames based on the video to be played;

[0249] Based on the current adaptive mode, the size information of the multiple video frames, and the size information of the texture object, determine the size adaptation information in the current adaptive mode; the size information of the texture object is determined according to the size information of the playback view corresponding to the texture object; the current adaptive mode is one of multiple adaptive modes; the size adaptation information includes the position information of the area to be displayed in the multiple video frames and the position information of the mapped area in the texture object;

[0250] Based on the size adaptation information, according to the playback order of the multiple video frames, sequentially load the video frame data of the multiple video frames into the texture object, and the areas to be displayed in the multiple video frames are respectively used to map to the mapped areas;

[0251] Continuously draw the data of the texture object to the drawing surface object corresponding to the playback view to play the video using the playback view.

[0252] If the computer device is a server, an embodiment of the present application further provides a server. Please refer to Figure 15 as shown Figure 15 is a structural diagram of the server 1500 provided by the embodiment of the present application. The server 1500 may vary greatly due to configuration or performance differences and may include one or more processors 1522, such as a central processing unit (Central Processing Units, abbreviated as CPU), a memory 1532, and one or more storage media 1530 (such as one or more mass storage devices) for storing application programs 1542 or data 1544. Among them, the memory 1532 and the storage media 1530 can be transient storage or persistent storage. The program stored in the storage media 1530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Further, the processor 1522 may be set to communicate with the storage media 1530 and execute a series of instruction operations in the storage media 1530 on the server 1500.

[0253] The server 1500 may further include one or more power supplies 1526, one or more wired or wireless network interfaces 1550, one or more input / output interfaces 1558, and / or one or more operating systems 1541, such as Windows Server TM , Mac OS X TM , UnixTM , Linux TM , FreeBSD TM and so on.

[0254] The steps performed by the server in the above embodiments may be based on Figure 15 the server structure shown.

[0255] In addition, an embodiment of the present application further provides a computer-readable storage medium for storing a computer program for executing the method provided in the above embodiment.

[0256] An embodiment of the present application further provides a computer program product including a computer program, which, when running on a computer device, causes the computer device to execute the method provided in the above embodiment.

[0257] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to a computer program. The foregoing computer program can be stored in a computer-readable storage medium. When the computer program is executed, it performs the steps including the above method embodiments; and the foregoing computer-readable storage medium can be at least one of the following media: read-only memory (abbreviation: ROM), RAM, magnetic disk, or optical disk, etc., which can store computer programs.

[0258] It should be noted that the embodiments in this specification are all described in a progressive manner. For the same or similar parts between the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, reference can be made to the partial description of the method embodiments. The device and system embodiments described above are only illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0259] As described above, it is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed by the present application should be covered within the protection scope of the present application. Moreover, based on the implementation manners provided in the above aspects of the present application, further combinations can be made to provide more implementation manners. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A video playback method, characterized in that: The method comprises: Obtain multiple video frames based on the video to be played; Based on the current adaptive mode, the size information of the multiple video frames and the size information of the texture object, the size adaptation information under the current adaptive mode is determined; the size information of the texture object is determined according to the size information of the playback view corresponding to the texture object; the current adaptive mode is one of multiple adaptive modes; the size adaptation information includes the position information of the to-be-displayed area in the multiple video frames and the position information of the mapping area in the texture object; Based on the size adaptation information, and according to the playback order of the multiple video frames, sequentially loading the video frame data of the multiple video frames into the texture object, and the to-be-displayed areas of the multiple video frames are used to be respectively mapped to the mapping areas; The data of the texture object is continuously drawn to the drawing surface object corresponding to the playback view, so as to play the video using the playback view.

2. The method according to claim 1, characterized in that The determining, based on the current adaptive mode, the size information of the plurality of video frames and the size information of the texture object, the size adaptation information under the current adaptive mode comprises: If the current adaptive mode includes full screen display, determining the position information of the mapping area according to the entire area of ​​the texture object; The position information of the area to be displayed is determined according to the size information of the multiple video frames and the size information of the mapping area, so that the area to be displayed and the mapping area have the same aspect ratio.

3. The method according to claim 2, characterized in that The determining, according to the size information of the plurality of video frames and the size information of the mapping area, the position information of the to-be-displayed area comprises: According to the current adaptive mode, position information of a first feature point in the area to be displayed is determined; if the current adaptive mode includes displaying a first vertex side portion, the first feature point is a first vertex, and the position information of the first vertex is determined according to the position information of the first vertices of the multiple video frames; if the current adaptive mode includes displaying a central portion, the first feature point is a region center point, and the position information of the region center point is determined according to the position information of the central points of the multiple video frames; Determine, according to the size information of the multiple video frames and the size information of the mapping area, the size of a first side of the to-be-displayed area and the size of a second side adjacent to the first side; The position information of the to-be-displayed area is determined according to the position information of the first feature point, the size of the first side edge, and the size of the second side edge.

4. The method according to claim 3, characterized in that The determining, according to the size information of the plurality of video frames and the size information of the mapping area, the size of the first side of the to-be-displayed area and the size of the second side adjacent to the first side, comprises: If it is determined, based on the size information of the multiple video frames and the size information of the mapping area, that the aspect ratio of the multiple video frames is greater than the aspect ratio of the mapping area, the heights of the multiple video frames are used as the size of the first side of the to-be-displayed area; and the size of the second side adjacent to the first side is determined based on the aspect ratio of the mapping area and the size of the first side; If it is determined, based on the size information of the multiple video frames and the size information of the mapping area, that the aspect ratio of the multiple video frames is smaller than the aspect ratio of the mapping area, the widths of the multiple video frames are used as the size of the second side of the area to be displayed; and based on the aspect ratio of the mapping area and the size of the second side, the size of the first side adjacent to the second side is determined.

5. The method according to claim 2, characterized in that: If the current adaptive mode corresponds to full display of the screen, the step of sequentially loading the video frame data of the multiple video frames into the texture object based on the size adaptation information and according to the playback order of the multiple video frames includes: Based on the position information of the to-be-displayed area, and according to the playback order of the multiple video frames, the multiple video frames are sequentially intercepted to obtain the to-be-displayed image; The image data of the image to be displayed is loaded into the texture object in sequence.

6. The method according to claim 1, characterized in that The determining, based on the current adaptive mode, the size information of the plurality of video frames and the size information of the texture object, the size adaptation information under the current adaptive mode comprises: If the current adaptive mode includes full content display, determining the position information of the to-be-displayed area according to the entire area of ​​the plurality of video frames; The position information of the mapping area is determined according to the size information of the area to be displayed and the size information of the texture object, so that the area to be displayed and the mapping area have the same aspect ratio.

7. The method according to claim 6, characterized in that The determining the position information of the mapping area according to the size information of the area to be displayed and the size information of the texture object includes: According to the current adaptive mode, determining the position information of the second feature point in the mapping area; if the current adaptive mode includes second vertex side display, the second feature point is the second vertex, and the position information of the second vertex is determined according to the position information of the second vertex of the texture object; if the current adaptive mode includes center display, the second feature point is the area center point, and the position information of the area center point is determined according to the position information of the center point of the texture object; Determining the size of a third side of the mapping area and the size of a fourth side adjacent to the third side according to the size information of the area to be displayed and the size information of the texture object; The position information of the mapping area is determined according to the position information of the second feature point, the size of the third side and the size of the fourth side.

8. The method according to claim 7, characterized in that The determining, according to the size information of the area to be displayed and the size information of the texture object, the size of the third side of the mapping area and the size of the fourth side adjacent to the third side, comprises: If it is determined that the aspect ratio of the area to be displayed is greater than the aspect ratio of the texture object according to the size information of the area to be displayed and the size information of the texture object, the width of the texture object is used as the size of the third side of the mapping area; and the size of the fourth side adjacent to the third side is determined according to the aspect ratio of the area to be displayed and the size of the third side; If it is determined, based on the size information of the area to be displayed and the size information of the texture object, that the aspect ratio of the area to be displayed is smaller than the aspect ratio of the texture object, the height of the texture object is used as the size of the fourth side of the mapping area; and the size of the third side adjacent to the fourth side is determined based on the aspect ratio of the area to be displayed and the size of the fourth side.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Adding the playback view in the view tree to obtain a drawing surface object corresponding to the playback view; the size information of the drawing surface object is determined according to the size information of the playback view; A texture object is created according to the drawing surface object corresponding to the playback view; and size information of the texture object is determined according to size information of the drawing surface object.

10. The method according to claim 9, characterized in that The playback view is a texture view, and the method further includes: If a display transformation operation for the view tree is obtained, the display parameters of the playback view are adjusted according to the display transformation operation.

11. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: In the process of playing the video using the play view, in response to a mode switching operation corresponding to a target mode, the current adaptive mode is updated according to the target mode, and the target mode is one of the multiple adaptive modes.

12. A video playback device, characterized in that: The device comprises: A video frame acquisition unit, used to obtain multiple video frames based on the video to be played; a size adaptation information determining unit, configured to determine the size adaptation information under the current adaptive mode based on the current adaptive mode, the size information of the plurality of video frames, and the size information of the texture object; the size information of the texture object is determined according to the size information of the playback view corresponding to the texture object; the current adaptive mode is one of the plurality of adaptive modes; the size adaptation information includes the position information of the to-be-displayed area in the plurality of video frames and the position information of the mapping area in the texture object; A texture loading unit, configured to load the video frame data of the plurality of video frames into the texture object in sequence based on the size adaptation information and according to the playback order of the plurality of video frames, wherein the to-be-displayed areas of the plurality of video frames are respectively mapped to the mapping areas; The texture drawing unit is used to continuously draw the data of the texture object to the drawing surface object corresponding to the playback view, so as to use the playback view to play the video.

13. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is used to execute the video playback method described in any one of claims 1-11 according to the instructions in the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the video playback method described in any one of claims 1-11.

15. A computer program product comprising a computer program, characterized in that When the method is run on a computer device, the computer device executes the video playback method described in any one of claims 1 to 11.

Images