Video data control method and device and electronic equipment
Patent Information
- Application Number
- CN202380085856.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-08
- Publication Date
- 2025-07-22
AI Technical Summary
When playing videos on Android system applications, the size and amount of memory input by the decoder is difficult to predict, resulting in inappropriate memory settings, frame drops or memory waste.
A video data control method is proposed to realize on-demand allocation and closed-loop control of actual memory by optimizing memory pool design. The method includes obtaining continuous frame data and custom handle memory, and dynamically allocating actual memory according to the actual size of continuous frame data, ensuring normal video playback and reducing memory waste.
By dynamically allocating memory, memory usage is optimized, frame loss during video playback is avoided, memory waste is reduced, and memory utilization efficiency of the system is improved.
Smart Images
Figure CN120359760A_ABST
Abstract
Description
Video data control method, device and electronic equipment Technical Field
[0001] The present application relates to the field of memory optimization technology, and in particular to a video data control method and device and an electronic device. Background Art
[0002] When playing videos on Android apps, multiple decoder input memories are allocated, and these memory inputs are recycled throughout the playback process. The size and quantity of decoder input memories are determined by the vendor adaptation layer of the SoC (System on Chip). During this process, the adaptation layer struggles to predict the specific memory usage required for video playback, leading to inconsistencies. For example, improperly sized memory can lead to choppy video playback and frame drops, while overly large memory sizes or excessive memory counts can lead to excessive memory usage and waste.
[0003] Summary of the Invention
[0004] This application aims to solve at least one of the technical problems existing in the prior art.
[0005] To this end, one purpose of this application is to propose a video data control method that optimizes the memory pool design, realizes on-demand allocation and closed-loop control of actual memory, and reduces the problem of memory waste while ensuring normal video playback.
[0006] Therefore, the second object of this application is to provide a video data control device.
[0007] To this end, the third object of this application is to provide an electronic device.
[0008] In order to achieve the above-mentioned purpose, an embodiment of the first aspect of the present application proposes a video data control method, which includes: obtaining continuous frame data and custom handle memory; allocating actual memory to the custom handle memory according to the continuous frame data memory; and decoding the continuous frame data according to the actual memory.
[0009] According to the video data control method of the embodiment of the present application, by obtaining continuous frame data and custom handle memory, actual memory is allocated to the custom handle memory according to the different memory sizes of the continuous frame data, and the continuous frame data is decoded according to the actual memory. By allocating actual memory according to the actual size of the continuous frame data during actual use, the memory pool design is optimized, and on-demand allocation and closed-loop control of the actual memory are realized, thereby reducing the problem of memory waste while ensuring normal playback of the video.
[0010] In some embodiments, allocating actual memory to the custom handle memory according to the continuous frame data memory includes: acquiring a first custom handle memory; and allocating actual memory to the first custom handle memory according to the memory of the continuous frame data.
[0011] In some embodiments, decoding the continuous frame data according to the actual memory includes: writing the continuous frame data into the actual memory to obtain encrypted continuous frame data; decrypting the encrypted continuous frame data to obtain an actual frame data address; and decoding the continuous frame data according to the actual frame data address.
[0012] In some embodiments, decoding the continuous frame data according to the actual frame data address includes: acquiring a second custom handle memory; and decoding the continuous frame data according to the second custom handle memory and the actual frame data address.
[0013] In some embodiments, decoding the continuous frame data according to the second custom handle memory and the actual frame data address includes: obtaining the continuous frame data corresponding to the actual frame address; and decoding the continuous frame data corresponding to the actual frame address in the second custom handle memory.
[0014] In some embodiments, after decoding the continuous frame data according to the actual memory, the method further includes: after determining that decoding of the current frame data is completed, sending the available custom handle in the custom handle memory.
[0015] In some embodiments, sending the available custom handle in the custom handle memory includes: predicting the size of the memory required to release the next frame data in the process of releasing the actual memory; and delaying sending the available custom handle when the required released memory exceeds the maximum allocatable memory.
[0016] In some embodiments, before predicting the release of actual memory, the method further includes: determining the order in which the continuous frame data releases memory; and releasing the actual memory in sequence according to the order.
[0017] In order to achieve the above-mentioned purpose, an embodiment of the second aspect of the present application proposes a video data control device, which includes: an acquisition module for acquiring continuous frame data and custom handle memory; an allocation module for allocating actual memory for the continuous frame data according to the custom handle memory; and a decoding module for decoding the continuous frame data according to the actual memory.
[0018] According to the video data control device of the embodiment of the present application, by obtaining continuous frame data and custom handle memory, actual memory is allocated to the custom handle memory according to the different memory sizes of the continuous frame data, and the continuous frame data is decoded according to the actual memory. By allocating actual memory according to the actual size of the continuous frame data during actual use, the memory pool design is optimized, and on-demand allocation and closed-loop control of the actual memory are realized, thereby reducing the problem of memory waste while ensuring normal playback of the video.
[0019] In order to achieve the above-mentioned purpose, an embodiment of the third aspect of the present application provides an electronic device, which includes: the video data control device described in the above-mentioned embodiment.
[0020] According to the electronic device of the application embodiment, by obtaining continuous frame data and custom handle memory, actual memory is allocated to the custom handle memory according to the different memory sizes of the continuous frame data, and the continuous frame data is decoded according to the actual memory. By allocating actual memory according to the actual size of the continuous frame data during actual use, the memory pool design is optimized, and on-demand allocation and closed-loop control of the actual memory are realized, thereby reducing the problem of memory waste while ensuring normal playback of the video.
[0021] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0023] FIG1 is a flow chart of a video data control method according to an embodiment of the present application;
[0024] FIG2 is a flow chart of a video data control method according to an embodiment of the present application;
[0025] FIG3 is a result block diagram of a video data control apparatus according to an embodiment of the present application.
[0026] FIG4 is a structural block diagram of an electronic device according to an embodiment of the present application.
[0027] Reference numerals: video data control device 2; acquisition module 21; distribution module 22; decoding module 23; electronic device 3. DETAILED DESCRIPTION
[0028] The following describes in detail embodiments of the present application, and the embodiments described with reference to the accompanying drawings are exemplary.
[0029] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.
[0030] The video data control method according to an embodiment of the present application will be described below with reference to FIG1 and FIG2 .
[0031] As shown in FIG1 , the data control method of the embodiment of the present application includes at least steps S1 to S3 .
[0032] Step S1, obtaining continuous frame data and custom handle memory.
[0033] The continuous frame data is data with known memory size and sequence during video playback. Since the size of the continuous frame data is known during actual use, memory allocation can be performed based on the required memory of the continuous frame data.
[0034] Since the memory is arranged sequentially, continuous frame data are densely arranged together. When there is insufficient free space at the end, it will loop back to the head of the memory pool, and the size of the continuous frame data fluctuates greatly, that is, some frame data is larger and some frame data is smaller; the custom handle memory is a specific handle memory stored in the Android native handle. It serves as an identifier in the memory allocation process. When the custom handle memory is identified, the actual memory corresponding to the continuous frame data is allocated to it.
[0035] In an embodiment, at a certain moment in the video playback, continuous frame data is obtained, and by calling a dedicated API (Application Programming Interface), the custom handle memory stored in the Android native handle is obtained. By obtaining continuous frame data and custom handle memory, data support is provided for subsequent memory usage.
[0036] Step S2: allocating actual memory to the custom handle memory according to the continuous frame data memory.
[0037] In an embodiment, after obtaining continuous frame data and custom handle memory, a hook function is added when APK (Android Package) sends a frame of data to the decoder. In the hook function, the memory size of the continuous frame data can be obtained, and the custom handle memory in the Android native handle can be identified. According to the memory size of the continuous frame data, actual memory is allocated to the custom handle memory, and then the frame sending call is continued to be executed to complete the writing of the continuous frame data. In actual use, actual memory is allocated to the custom handle according to the memory size of the continuous frame data to store the continuous data frame, so that the memory size is adapted to the size of the continuous frame data, thereby optimizing the memory pool design and realizing on-demand allocation of memory, thereby reducing the problem of memory waste.
[0038] Step S3: Decode the continuous frame data according to the actual memory.
[0039] In an embodiment, after obtaining the actual memory, the decoder recognizes that the actual memory is in Android native handle mode, extracts the custom handle memory stored in the Android native handle, and obtains the real frame data address based on the continuous frame data and the actual memory, decodes the continuous frame data corresponding to the real frame data address, and realizes closed-loop control of the actual memory to ensure normal playback of the video.
[0040] According to the video data control method of the embodiment of the present application, by obtaining continuous frame data and custom handle memory, actual memory is allocated to the custom handle memory according to the different memory sizes of the continuous frame data, and the continuous frame data is decoded according to the actual memory. By allocating actual memory according to the actual size of the continuous frame data during actual use, the memory pool design is optimized, and on-demand allocation and closed-loop control of the actual memory are realized, thereby reducing the problem of memory waste while ensuring normal playback of the video.
[0041] In some embodiments, when allocating actual memory to the custom handle memory according to the continuous frame data memory, a first custom handle memory is acquired; and actual memory is allocated to the first custom handle memory according to the continuous frame data memory.
[0042] In an embodiment, actual memory is allocated for continuous frame data. For example, when the decoder receives frame data sent by the APK, a hook function is added. In the hook function, the memory size of the continuous frame data can be obtained. At this time, a dedicated API is called to extract the first custom handle memory stored in the Android native handle, and actual memory is allocated to the first custom handle memory based on the memory of the continuous frame data, avoiding the problem of excessive actual memory occupation and memory waste caused by the use of the native buffer.
[0043] For example, the continuous frame data consists of four frames of data, and their memory sizes are 3MB, 1MB, 1MB, and 2MB respectively. The first custom handle memory, i.e., four custom handle memories, is obtained. According to the memory 3MB, 1MB, 1MB, and 2MB of the continuous frame data, the actual memory allocated for the first custom handle memory, i.e., four custom handle memories, is 3MB, 1MB, 1MB, and 2MB respectively. The actual total memory occupied by playing the video is 7MB; if four native buffers are used, the memory of each native buffer is set to 5MB, and the actual total memory occupied by playing the video is 20MB, which greatly reduces the memory.
[0044] In some embodiments, decoding the continuous frame data according to the actual memory includes: writing the continuous frame data into the actual memory to obtain encrypted continuous frame data; decrypting the encrypted continuous frame data to obtain the actual frame data address; and decoding the continuous frame data according to the actual frame data address.
[0045] In an embodiment, after obtaining the actual memory, the continuous frame data is written into the actual memory to obtain encrypted continuous frame data; the decoder decrypts the encrypted continuous frame data and converts it into an actual frame data address to decode the continuous frame data according to the actual frame data address to ensure normal playback of the video.
[0046] In some embodiments, decoding the continuous frame data according to the actual frame data address includes: acquiring a second custom handle memory; and decoding the continuous frame data according to the second custom handle memory and the actual frame data address.
[0047] In an embodiment, after obtaining the actual frame data address, a dedicated API is called to extract the second custom handle memory stored in the Android native handle, so as to decode the continuous frame data according to the second custom handle memory and the actual frame data address to ensure normal playback of the video.
[0048] In some embodiments, decoding continuous frame data according to the second custom handle memory and the actual frame data address includes: obtaining continuous frame data corresponding to the actual frame address; and decoding the continuous frame data corresponding to the actual frame address in the second custom handle memory.
[0049] In an embodiment, after obtaining the actual frame data address and the second custom handle memory, the continuous frame data corresponding to the actual frame address is obtained, and the continuous frame data corresponding to the actual frame address is decoded in the second custom handle memory to ensure normal playback of the video.
[0050] In some embodiments, after decoding the continuous frame data according to the actual memory, the method further includes: after determining that the decoding of the current frame data is completed, sending the available custom handle in the custom handle memory.
[0051] In an embodiment, after determining that the decoding of the current frame data is completed, the available custom handle in the custom handle memory, that is, the free handle, is sent, and then the upper layer is notified that the actual memory is used up, completing the closed-loop use of the memory, that is, each time the APK sends a frame of data to the decoder, the custom handle memory allocates actual memory; each time a frame of data is decoded, the custom handle memory releases the actual memory and sends it to the APK for use in the next data transmission.
[0052] In some embodiments, sending the available custom handle in the custom handle memory includes: predicting the size of the memory required to release the next frame data in the process of releasing the actual memory; and delaying sending the available custom handle when the required released memory exceeds the maximum allocatable memory.
[0053] In an embodiment, after the decoding of the current frame data is completed, the size of the memory required to release the next frame data in the process of releasing the actual memory is predicted according to the embedded algorithm. When the required released memory exceeds the maximum allocable memory, flow control is triggered to control the return of the native buffer, and delay the sending of the available custom handle, that is, the idle handle, to control the data flow rate. By introducing a prediction mechanism, it is prevented that when the data volume of consecutive frame data is large, the actual memory exceeds the total amount of the memory pool and causes playback problems.
[0054] In some embodiments, before predicting the release of actual memory, the method further includes: determining the order in which the memory of consecutive frame data is released; and releasing the actual memory in sequence according to the order.
[0055] In an embodiment, before predicting the release of actual memory, the order in which the memory of consecutive frame data is released is determined, for example, in chronological order, so that the actual memory is released in sequence, that is, the actual memory allocated first is released first; if the actual memory is not released in sequence, the release of the actual memory is not allowed.
[0056] This application can increase the number of native buffers while keeping the actual total memory usage of the video being played the same. For example, it can be set to 32, and the maximum tolerable network interruption time is 533ms, to avoid having only four native buffers during playback. When the network fluctuates, the maximum tolerable network interruption time is 4*1000 / frame rate (ms). For example, for a 60fps video, the maximum tolerable network interruption time is about 67ms. If this time is exceeded, the video will become stuck and frames will be dropped.
[0057] The video data control method according to an embodiment of the present application is described below with reference to FIG2 .
[0058] As shown in FIG. 2 , the video data control method according to the embodiment of the present application at least includes steps S11 to S18 .
[0059] Step S11, obtaining continuous frame data and custom handle memory.
[0060] Step S12, obtaining a first custom handle memory; allocating actual memory to the first custom handle memory according to the memory of the continuous frame data.
[0061] Step S13, writing the continuous frame data into the actual memory to obtain encrypted continuous frame data; decrypting the encrypted continuous frame data to obtain the actual frame data address.
[0062] Step S14: Obtain a second custom handle memory.
[0063] Step S15, obtaining continuous frame data corresponding to the actual frame address; decoding the continuous frame data corresponding to the actual frame address in the second custom handle memory.
[0064] Step S16: Determine whether the decoding of the current frame data is completed.
[0065] Step S17, determining the order of releasing the memory of the continuous frame data; releasing the actual memory in sequence according to the order.
[0066] Step S18, predicting the size of the memory required to be released for the next frame data in the process of releasing the actual memory; when the required released memory exceeds the maximum allocatable memory, delaying sending the available custom handle.
[0067] According to the video data control method of the embodiment of the present application, by obtaining continuous frame data and custom handle memory, actual memory is allocated to the custom handle memory according to the different memory sizes of the continuous frame data, and the continuous frame data is decoded according to the actual memory. By allocating actual memory according to the actual size of the continuous frame data during actual use, the memory pool design is optimized, and on-demand allocation and closed-loop control of the actual memory are realized, thereby reducing the problem of memory waste while ensuring normal playback of the video.
[0068] The video data control device 2 according to an embodiment of the present application will be described below with reference to FIG. 3 .
[0069] As shown in FIG3 , the video data control device 2 of the embodiment of the present application includes: an acquisition module 21 , an allocation module 22 and a decoding module 23 , wherein:
[0070] The acquisition module 21 is used to acquire continuous frame data and custom handle memory; the allocation module 22 is used to allocate actual memory for the continuous frame data according to the custom handle memory; and the decoding module 23 is used to decode the continuous frame data according to the actual memory.
[0071] In an embodiment, the acquisition module 21 acquires continuous frame data at a certain moment during video playback and, by calling a dedicated API (Application Programming Interface), acquires custom handle memory stored in the Android native handle. Acquiring the continuous frame data and custom handle memory provides data support for subsequent memory usage. The continuous frame data is data whose memory size and sequence are known during video playback. Since the size of the continuous frame data is known during actual use, memory allocation can be performed based on the required memory for the continuous frame data.
[0072] Since the memory is arranged sequentially, continuous frame data are densely arranged together. When there is insufficient free space at the end, it will loop back to the head of the memory pool, and the size of the continuous frame data fluctuates greatly, that is, some frame data is larger and some frame data is smaller; the custom handle memory is a specific handle memory stored in the Android native handle. It serves as an identifier in the memory allocation process. When the custom handle memory is identified, the actual memory corresponding to the continuous frame data is allocated to it.
[0073] After the allocation module 22 obtains the continuous frame data and the custom handle memory, a hook function is added when the APK (Android Package) sends a frame of data to the decoder. In the hook function, the memory size of the continuous frame data can be obtained, and the custom handle memory in the Android native handle can be identified. According to the memory size of the continuous frame data, actual memory is allocated to the custom handle memory, and then the frame sending call is continued to be executed to complete the writing of the continuous frame data. In actual use, actual memory is allocated to the custom handle according to the memory size of the continuous frame data to store the continuous data frame, so that the memory size is adapted to the size of the continuous frame data, thereby optimizing the memory pool design and realizing on-demand allocation of memory, thereby reducing the problem of memory waste.
[0074] After the decoding module 23 obtains the actual memory, the decoder recognizes that the actual memory is in Android native handle mode, extracts the custom handle memory stored in the Android native handle, and obtains the real frame data address based on the continuous frame data and the actual memory, decodes the continuous frame data corresponding to the real frame data address, and realizes closed-loop control of the actual memory to ensure normal playback of the video.
[0075] According to the video data control device 2 of the embodiment of the present application, by acquiring continuous frame data and custom handle memory, actual memory is allocated to the custom handle memory according to the different memory sizes of the continuous frame data, and the continuous frame data is decoded according to the actual memory. By allocating actual memory according to the actual size of the continuous frame data during actual use, the memory pool design is optimized, and on-demand allocation and closed-loop control of the actual memory are realized, thereby reducing the problem of memory waste while ensuring normal playback of the video.
[0076] The electronic device 3 according to an embodiment of the present application is described below with reference to FIG. 4 .
[0077] As shown in FIG4 , the electronic device 3 according to the embodiment of the present application includes the video data control device 2 according to the above embodiment.
[0078] According to the electronic device 3 of the application embodiment, by obtaining continuous frame data and custom handle memory, actual memory is allocated to the custom handle memory according to the different memory sizes of the continuous frame data, and the continuous frame data is decoded according to the actual memory. By allocating actual memory according to the actual size of the continuous frame data during actual use, the memory pool design is optimized, and on-demand allocation and closed-loop control of the actual memory are realized, thereby reducing the problem of memory waste while ensuring normal playback of the video.
[0079] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "example," "specific example," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.
[0080] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.
Claims
1. A video data control method, characterized in that, comprising: obtaining continuous frame data and a custom handle memory; allocating actual memory for the custom handle memory according to the continuous frame data memory; decoding the continuous frame data according to the actual memory.
2. The video data control method according to claim 1, characterized in that, allocating actual memory for the custom handle memory according to the continuous frame data memory, comprising: obtaining a first custom handle memory; allocating actual memory for the first custom handle memory according to the memory of the continuous frame data.
3. The video data control method according to claim 1, characterized in that, decoding the continuous frame data according to the actual memory, comprising: writing the continuous frame data into the actual memory to obtain encrypted continuous frame data; decrypting the encrypted continuous frame data to obtain an actual frame data address; decoding the continuous frame data according to the actual frame data address.
4. The video data control method according to claim 3, characterized in that, decoding the continuous frame data according to the actual frame data address, comprising: obtaining a second custom handle memory; decoding the continuous frame data according to the second custom handle memory and the actual frame data address.
5. The video data control method according to claim 4, characterized in that, decoding the continuous frame data according to the second custom handle memory and the actual frame data address, comprising: obtaining the continuous frame data corresponding to the actual frame address; decoding the continuous frame data corresponding to the actual frame address in the second custom handle memory.
6. The video data control method according to claim 1, characterized in that, after decoding the continuous frame data according to the actual memory, further comprising: after determining that the decoding of the current frame data is completed, sending the available custom handles in the custom handle memory.
7. The video data control method according to claim 6, characterized in that, sending the available custom handles in the custom handle memory, comprising: predicting the size of the memory to be released for the next frame data during the process of releasing the actual memory; when the memory to be released exceeds the maximum allocable memory, delaying the sending of the available custom handles.
8. The video data control method according to claim 7, characterized in that, before predicting the release of the actual memory, further comprising: determining the order of releasing the memory of the continuous frame data; releasing the actual memory sequentially according to the order.
9. A video data control device, characterized in that, comprising: an obtaining module for obtaining continuous frame data and a custom handle memory; an allocating module for allocating actual memory for the continuous frame data according to the custom handle memory; a decoding module for decoding the continuous frame data according to the actual memory.
10. An electronic device, characterized in that, comprising: the video data control device according to claim 9.