Android mobile terminal video cache optimization method and system

Abstract

The invention relates to the technical field of video processing, and particularly discloses an Android mobile terminal video cache optimization method and system. The method comprises the following steps: establishing a webSocket connection between a streaming media data receiving end and a streaming media server, and storing received data into a Linked Block Queue queue by using a data analysis thread; firstly writing queue data into a temporary memory display cache in real time, and then storing the analyzed data into a mobile terminal disk according to a queue sequence; a reading thread reads and analyzes the audio and video data in sequence and distributes the audio and video data to a corresponding decoder, and the decoded data is presented by a renderer; according to the invention, by optimizing data processing, cache management and a play control mechanism, the stable play speed of the end side, continuous play when the network is not good, play without interruption at the server end and accurate and synchronous audio and picture are ensured, so that the video play performance of the mobile end is improved, and smooth, stable and high-quality watching experience is provided for a user.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing, and in particular to an Android mobile terminal video cache optimization method and system. Background Art

[0002] With the rapid development of mobile Internet, mobile video playback has become an important way for people to obtain information and entertainment. However, in this field, there are still many problems in the existing technology that need to be solved.

[0003] 1. Streaming media services and network-related issues; Impact of streaming stability on playback speed: When streaming services transmit data, it is common for streaming to be unstable. This may be due to uneven resource allocation on the server side, interference factors in the network transmission link, etc. When streaming is unstable, the rate at which the mobile terminal receives data fluctuates, which directly leads to the end-side playback speed being fast and slow. For example, when watching online videos, users may frequently encounter situations where the video playback speeds up and slows down, which seriously interferes with the normal viewing rhythm and reduces the user experience.

[0004] Playback difficulties caused by unstable network: When users are in a mobile environment, the network conditions are complex and changeable. Weak network environments (such as poor signal coverage, network congestion, etc.) or sudden network disconnections often occur. Under existing technologies, when encountering such network problems, video playback often gets into trouble. On the one hand, network instability will slow down or even stagnate the video loading speed, causing the video to be in a buffering state for a long time and unable to play normally; on the other hand, once the network is interrupted, if there is no effective caching mechanism, the video will stop playing directly, and users can only wait for the network to recover and reload, which consumes a lot of time and traffic.

[0005] 2. Data integrity and playback anomalies; Data packet loss and damaged playback integrity: During network transmission, data packet loss is inevitable due to signal interference, network equipment failure, etc. In video playback, the lost data may contain key video frames or audio clips, which will cause the video to be incomplete and unable to play normally. For example, users may suddenly encounter a freeze in the video while watching it, and then the video cannot be played directly. This is often the result of data packet loss causing video file damage.

[0006] Cache and playback white screen problem: Although some existing technologies try to alleviate playback problems through caching, the caching mechanism is imperfect. When the cache data is insufficient or the cache is not managed properly, a white screen may still appear during playback. This is because when the data required for playback is not loaded in time or the cache is invalid, the player cannot obtain valid picture data, resulting in a blank screen and affecting user viewing.

[0007] III. Defects of existing technologies in terms of playback quality The problem of audio and video being out of sync: Existing technologies are deficient in the synchronous processing of audio and video data. During video playback, the decoding and transmission speeds of audio and video are inconsistent, which can easily lead to the situation where the sound and picture are out of sync. For example, when watching a movie or making a video call, the time difference between the sound and the picture can cause the audience to feel obviously uncomfortable, seriously affecting their understanding and perception of the video content.

[0008] Abnormal playback speed: Due to insufficient optimization of data processing logic, the video playback speed may exceed the normal range in some cases. This abnormal playback speed will cause the screen to switch quickly, making it difficult for users to keep up with the rhythm of the video and unable to watch the video content normally. This problem is particularly prominent when watching teaching videos, plot videos, and other scenarios that require a normal rhythm.

[0009] Playback freezes due to frame loss: Data packet loss or improper processing will not only affect the integrity of the video, but also cause frame loss. Lost video frames make it impossible to play the picture continuously and smoothly, resulting in freezes, which destroys the continuity of the video and reduces the visual effect and viewing value of the video.

[0010] To sum up, the existing mobile video playback technology has many limitations in terms of unstable streaming media services, unstable networks, and data processing. There is an urgent need for an innovative technology that can effectively solve these problems to improve the stability, integrity and quality of mobile video playback and meet users' growing demand for high-quality video viewing. Summary of the invention

[0011] In view of the shortcomings of the current related existing technologies, the present invention provides an Android mobile video cache optimization method, which aims to solve the problems of unstable streaming media services causing fluctuations in playback speed, difficulty in loading and playing when the user has a weak network or is disconnected, server-side interruption of streaming and client-side non-buffered playback, and audio and video asynchrony. By optimizing data processing, cache management and playback control mechanisms, it is ensured that the end-side playback speed is stable, playback can be resumed when the network is poor, playback is not interrupted when the server is disconnected, and audio and video are accurately synchronized, thereby improving the mobile video playback performance and providing users with a smooth, stable, and high-quality viewing experience, so as to overcome the defects of the existing technology and meet the user's video viewing needs in a complex network environment.

[0012] To achieve the above object, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides an Android mobile terminal video cache optimization method, which at least comprises the following steps: Step S1, establishing data connection and parsing: the streaming media receiving data terminal establishes a webSocket connection with the streaming media server, and adopts a multi-threaded and read-write separation architecture to perform stream parsing; stream parsing, combining the read-write thread, decoder and rendering according to the playback process; using the data parsing thread to parse the received data, and storing the parsed data in a LinkedBlockingQueue queue with first-in-first-out and non-interfering access; Step S2, data caching operation: using the RandomAccessFile feature, the data stored in the LinkedBlockingQueue queue is written into the temporary memory of the RandomAccessFile in real time, and displayed and cached, waiting for the next operation; By using the RandomAccessFile specific method attributes and its LinkedBlockingQueue class in combination, the parsed data is written into the storage file in the order of the queue in the data, and the data is saved on the disk of the mobile terminal; Step S3, read data and decode and render: read the data in the order of the queue through the reading thread, parse the read audio and video data, send the audio and video data to the Android decoder to parse the data; send the parsed data to the Android renderer; Step S4: The mobile terminal implements audio and video synchronization control.

[0013] Furthermore, the read audio and video data is parsed to determine whether it is video data, and if so, it is sent to the video decoder; if not, it is sent to the audio decoder, and after decoding, the audio and video are played synchronously; Furthermore, if the data is video data, parse the NAL unit according to the H.264 / H.265 standard, identify the NALU type, length and related fields through byte scanning and bit operations, and use dynamic buffer expansion technology to accumulate data when the buffer data is insufficient; extract SPS / PPS information, obtain information by traversing the NAL unit, and set the MediaFormat parameters when creating the decoding; configure MediaCodec for decoding, optimize the configuration according to the video resolution, frame rate, bit rate and color space parameters, and use the thread pool to manage thread resources after starting the decoder thread; when the decoder completes a frame of video processing, it triggers a callback event, checks the output buffer status and calls the releaseOutputBuffer method to release the buffer and sets the parameter to true to present the frame, and processes all frames in a loop.

[0014] Furthermore, if the data is audio data, an audio feature extraction algorithm is used to parse the audio data and determine the type, including one or more types of AAC, G711A and G711U, and a corresponding decoder is matched and called from a pre-stored audio decoder library for decoding and playback, and playback parameters are optimized according to audio characteristics during the initialization phase of the audio player; Initialize the audio player for different types, and use different decoders to decode and play the audio; while obtaining audio and video, it is necessary to control the audio and video synchronization through timestamps to maintain stable playback.

[0015] Furthermore, once the existence of NALU is confirmed in the parsing of NAL unit, the next task is to extract specific NAL unit information from the original byte sequence; the NAL unit information includes identifying the NALU type, length and one or more of the related fields; according to the H.264 / H.265 standard, NALU consists of a one-byte header information and subsequent load data, where the header contains a prohibition bit, a reference level indication and a NAL unit type.

[0016] Furthermore, the SPS / PPS extraction step includes the following: SPS and PPS, as key information for initializing the decoder, need to be accurately extracted from the NAL unit; by traversing the NAL unit and checking its type field, these special types of NAL units are identified, and the corresponding SPS and PPS related information are obtained, and the corresponding parameters are set during the decoding process.

[0017] Furthermore, the steps to configure MediaCodec for decoding include the following: MediaCodecAPI enables developers to directly manipulate the codec to achieve efficient decoding of video streams. To ensure that the decoder can accurately interpret the input data, necessary configuration information, including SPS and PPS data, must be provided in advance. The MediaFormat class is used to define the media format. It accepts parameters including width, height, and additional encoding settings. By calling the MediaCodec.configure function, the MediaFormat instance is passed to the decoder, and the output target Surface is specified. The decoded image can be rendered to the interface. In addition, the decoder thread needs to be started to prepare it for receiving input.

[0018] Furthermore, in the step of displaying the video frame, when the decoder completes processing a frame of video data, it triggers a corresponding callback event to notify that new output is available; at this time, the status of the output buffer should be checked, and the releaseOutputBuffer function should be called to release the buffer, and the parameter should be set to true to indicate that the frame should be presented immediately; this process is completed in a loop until all frames are processed.

[0019] The second aspect of the present invention provides an Android mobile terminal video cache optimization system, which is used to implement the above method, and the system at least includes: The streaming media receiving module is used to establish a webSocket connection with the streaming media server. The module has a built-in multi-threaded scheduling and management sub-module to control the stream parsing process under the multi-threaded and read-write separation architecture, and transmit the parsed data to the subsequent processing link; The cache management module uses the RandomAccessFile feature to write data from the LinkedBlockingQueue queue to temporary memory to ensure the timeliness of the display cache, and writes the parsed data to the mobile disk in a persistent manner according to the queue order. It has a storage monitoring and optimization submodule to allocate storage resources based on disk space and read and write frequency; The integrated data reading and decoding rendering module reads data from the queue in sequence and distributes it to the corresponding video decoder or audio decoder according to the data type. After the decoder completes the task, the rendering module will parse the data and present it on the mobile screen.

[0020] Furthermore, it also includes: The audio and video synchronization coordination module tracks the video frame rate and audio sampling rate in real time to generate dynamic timestamps. It adjusts the audio and video timestamps based on the network conditions and the time consumed by each link in data processing to achieve audio and video synchronization when playing videos on mobile devices.

[0021] The decoder adaptation module, for video data, is based on the H.264 / H.265 standard parsing requirements, has a built-in NAL unit parsing sub-module, uses byte scanning and bit operations to identify NALU key information, and triggers a dynamic buffer expansion mechanism when the data buffer is insufficient; for SPS / PPS extraction, MediaCodec configuration decoding and other links, there are corresponding adaptation sub-modules to ensure efficient and accurate video decoding; for audio data, the type is determined based on audio features, the decoder is matched and called from the pre-stored decoder library, and the playback parameters are optimized when the audio player is initialized to improve the decoding and playback performance.

[0022] The present invention adopts the above technical solution, which has at least the following beneficial effects: 1. Improve data transmission and processing efficiency In the data connection and analysis stage, the webSocket connection established between the streaming media receiving end and the streaming media server greatly enhances the real-time and stability of data transmission compared to traditional connection methods, and effectively reduces unnecessary delays and communication failures. The multi-threaded and read-write separation architecture allows the stream analysis work to proceed in parallel, and each thread performs its own duties and cooperates in an orderly manner. The semaphore or lock mechanism is used to ensure smooth data flow, significantly improving the data processing speed and gaining valuable time for subsequent links. The data analysis thread will accurately analyze the received data and store it in a LinkedBlockingQueue queue that is first-in, first-out, and does not interfere with each other, further ensuring that the data enters the subsequent process in an orderly manner to avoid confusion and blockage.

[0023] In the data cache operation phase, with the help of the RandomAccessFile feature, the queue data is first quickly written to the temporary memory to achieve display cache to meet the user's instant viewing needs, and then the parsed data is efficiently and persistently written to the mobile disk in the queue order. In this process, the storage monitoring and optimization submodule dynamically allocates resources based on disk space and read and write frequency, which not only ensures data integrity but also avoids waste of storage resources, making the overall cache process efficient and reliable, and reducing playback jams caused by cache problems.

[0024] 2. Optimize video decoding and rendering quality For video data, after reading, the NAL unit is parsed strictly according to the H.264 / H.265 standard, and the NALU type, length and related fields are accurately identified using byte scanning and bit operations. When the buffer data is insufficient, the dynamic buffer expansion technology is automatically enabled to accumulate data to ensure that the parsing work is coherent and accurate, providing a solid foundation for subsequent decoding. When extracting SPS / PPS information, key information is quickly located by traversing the NAL unit, and the MediaFormat parameters are reasonably set when creating the decoding, and MediaCodec is used for efficient decoding. When configuring MediaCodec, full consideration is given to optimizing the configuration of parameters such as video resolution, frame rate, bit rate and color space. After starting the decoder thread, the thread pool is used to scientifically manage thread resources, so that the video decoding process is fast and stable, and high-quality images are output.

[0025] In the display step of the decoded video frame, the system can promptly respond to the callback event after the decoder completes processing of a frame, quickly check the output buffer status and call the releaseOutputBuffer method to release the buffer, and set the parameter to true to indicate that the frame should be presented immediately. The whole process is completed in an orderly manner within the loop, ensuring smooth video playback without screen tearing, freezes, etc., bringing an immersive viewing experience to users.

[0026] 3. Accurately adapt to audio processing needs For audio data, the audio feature extraction algorithm is used to parse and accurately determine the type, including common AAC, G711A and G711U, etc., and the corresponding decoder is matched and called from the pre-stored audio decoder library for decoding and playback, which is highly targeted. During the initialization stage of the audio player, the playback parameters such as sampling rate, number of channels, volume gain, etc. are finely optimized according to the audio characteristics to ensure that different types of audio can be played with the best effect and restore clear and full sound.

[0027] The audio and video synchronization is controlled by timestamps, and the audio and video timestamps are adjusted in real time in close combination with network delays and the time spent on data processing. The feedback control loop is used to continuously monitor and correct synchronization deviations. In the complex and changeable mobile network environment and system load, the accurate synchronization of audio and video is always guaranteed to avoid viewing troubles caused by the sound and picture being out of sync, and the user's satisfaction with watching videos is improved in all aspects.

[0028] 4. Enhance the overall adaptability and scalability of the system The designed mobile terminal system architecture is clear, and each module has a clear division of labor and cooperates with each other. The streaming media receiving module, cache management module, data reading and decoding rendering module, audio and video synchronization coordination module, decoder adaptation module, etc. work closely together to form a complete ecosystem. Whether facing different streaming media servers, diverse video and audio encoding formats, or changing mobile terminal hardware performance and network conditions, the system can respond flexibly, and through the adaptive adjustment and optimization of internal modules, it ensures the stability and efficiency of video caching and playback.

[0029] For example, the decoder adaptation module shows strong compatibility with various analysis requirements of video data and various types of audio data judgment and decoding processing. If new video and audio standards or formats appear in the future, it only needs to be moderately expanded and upgraded in the corresponding modules to quickly integrate new technologies, reduce R&D costs and time cycles, extend product life cycles, and enable the present invention to always remain competitive in the rapidly developing mobile multimedia field. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 It is a flow chart of the Android mobile terminal video cache optimization method of the present invention; Figure 2It is a schematic diagram of an embodiment of video caching optimization for an Android mobile terminal of the present invention. DETAILED DESCRIPTION

[0032] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0033] Embodiment 1

[0034] See also Figure 1 and Figure 2 As shown, the first aspect of this embodiment provides an Android mobile terminal video cache optimization method, which at least includes the following steps: Step S1, establishing data connection and parsing: the streaming media receiving data terminal establishes a webSocket connection with the streaming media server, and adopts a multi-threaded and read-write separation architecture to perform stream parsing; stream parsing, combining the read-write thread, decoder and rendering according to the playback process; using the data parsing thread to parse the received data, and storing the parsed data in a LinkedBlockingQueue queue with first-in-first-out and non-interfering access; Step S2, data caching operation: using the RandomAccessFile feature, the data stored in the LinkedBlockingQueue queue is written into the temporary memory of the RandomAccessFile in real time, and displayed and cached, waiting for the next operation; By using the RandomAccessFile specific method attributes and its LinkedBlockingQueue class in combination, the parsed data is written into the storage file in the order of the queue in the data, and the data is saved on the disk of the mobile terminal; Step S3, read data and decode and render: read the data in the order of the queue through the reading thread, parse the read audio and video data, send the audio and video data to the Android decoder to parse the data; send the parsed data to the Android renderer; Step S4: The mobile terminal implements audio and video synchronization control.

[0035] Furthermore, the read audio and video data is parsed to determine whether it is video data, and if so, it is sent to the video decoder; if not, it is sent to the audio decoder, and after decoding, the audio and video are played synchronously; As an implementation method, in this embodiment, if the data is video data, the NAL unit is parsed according to the H.264 / H.265 standard, the NALU type, length and related fields are identified through byte scanning and bit operations, and the dynamic buffer expansion technology is used to accumulate data when the buffer data is insufficient; SPS / PPS information is extracted, the information is obtained by traversing the NAL unit, and the MediaFormat parameters are set when creating the decoding; MediaCodec is configured for decoding, and the configuration is optimized according to the video resolution, frame rate, bit rate and color space parameters. After starting the decoder thread, the thread pool is used to manage thread resources; when the decoder completes a frame of video processing, a callback event is triggered, the output buffer status is checked, and the releaseOutputBuffer method is called to release the buffer and the parameter is set to true to present the frame, and all frames are processed in a loop.

[0036] As an implementation mode, in this embodiment, if the data is audio data, an audio feature extraction algorithm is used to parse the audio data and determine the type, including one or more types of AAC, G711A and G711U, and a corresponding decoder is matched and called from a pre-stored audio decoder library for decoding and playback, and playback parameters are optimized according to audio characteristics during the initialization phase of the audio player; Initialize the audio player for different types, and use different decoders to decode and play the audio; while obtaining audio and video, it is necessary to control the audio and video synchronization through timestamps to maintain stable playback.

[0037] As an implementation method, once the existence of NALU is confirmed when parsing NAL unit in this embodiment, the next task is to extract specific NAL unit information from the original byte sequence; including but not limited to identifying NALU type, length and other related fields. According to the H.264 / H.265 standard, NALU consists of a byte of header information and subsequent load data, where the header contains a forbidden bit (forbidden_bit), a reference level indication (nal_reference_idc) and a NAL unit type (nal_unit_type). Especially for the two special NALUs SPS (Sequence Parameter Set) and PPS (Picture Parameter Set). H.264 / H.265 start code condition (0x00 00 00 01 or 0x00 0001), which marks the beginning of a new NALU. If the content in the current buffer is not enough to constitute a complete NALU, continue to accumulate until the condition is met.

[0038] As an implementation method, the SPS / PPS extraction step in this embodiment includes the following: SPS and PPS are key information for initializing the decoder and must be correctly extracted from the NAL unit. These special types of NAL units are found by traversing the NAL units and checking their type fields (a byte in the NAL unit header) to obtain the corresponding sps and pps related information, and set the corresponding parameters when creating the decoding, such as mediaFormat.setByteBuffer("csd-0",ByteBuffer.wrap(spsData)); mediaFormat.setByteBuffer("csd-1", ByteBuffer.wrap(ppsData)); As an implementation method, the steps of configuring MediaCodec for decoding in this embodiment include the following: MediaCodecAPI enables developers to directly manipulate the codec to achieve efficient decoding of video streams. To ensure that the decoder can accurately interpret the input data, necessary configuration information, including SPS and PPS data, must be provided in advance. The MediaFormat class is used to define the media format. It accepts parameters including width, height, and additional encoding settings. By calling the MediaCodec.configure function, the MediaFormat instance is passed to the decoder, and the output target Surface is specified. The decoded image can be rendered to the interface. In addition, the decoder thread needs to be started to prepare it for receiving input.

[0039] As an implementation method, in this embodiment, in the step of displaying video frames, when the decoder completes processing a frame of video data, it will trigger a corresponding callback event to notify that new output is available; at this time, the status of the output buffer should be checked, and the releaseOutputBuffer function should be called to release the buffer, and the parameter should be set to true to indicate that the frame should be presented immediately; this process is completed in a loop until all frames are processed.

[0040] Embodiment 2

[0041] like Figure 2 As shown, this embodiment provides an overall process implementation method for optimizing video cache on an Android mobile terminal: The overall process starts with the streaming media server and the client establishing a link through webSocket (ws). This method has higher real-time and stability than traditional connection methods, and can effectively reduce delays and communication problems. After the link is established, the client starts to read the original data stream and parse the audio and video data. The parsed data will be placed in the LinkedBlockingQueue queue, which adopts the first-in-first-out rule to ensure that storage and access do not interfere with each other, and ensure the order and stability of data processing.

[0042] After the data is stored in the queue, on the one hand, the characteristics of RandomAccessFile are used to write the data in the queue to the temporary memory in real time for display caching, waiting for subsequent operations; on the other hand, the parsed data will be written to the mobile phone disk storage file in the order of the data in the queue to achieve persistent storage of the data. When reading data, the reading thread will read the data in sequence according to the queue order and parse it again, and then process it separately according to the data type (video or audio). For video data, detailed NAL unit parsing, SPS / PPS extraction, MediaCodec configuration decoding and final video frame display will be performed; for audio data, the audio type (such as AAC, G711A, G711U) will be parsed and determined first, and then the audio player will be initialized and the corresponding decoding and playback operations will be performed for different types. In the whole process, the audio and video synchronization is controlled by timestamps to ensure stable playback.

[0043] Embodiment 3

[0044] The second aspect of the present invention provides an Android mobile terminal video cache optimization system, which is used to implement the above method, and the system at least includes: The streaming media receiving module is used to establish a webSocket connection with the streaming media server. The module has a built-in multi-threaded scheduling and management sub-module to control the stream parsing process under the multi-threaded and read-write separation architecture, and transmit the parsed data to the subsequent processing link; The cache management module uses the RandomAccessFile feature to write data from the LinkedBlockingQueue queue to temporary memory to ensure the timeliness of the display cache, and writes the parsed data to the mobile disk in a persistent manner according to the queue order. It has a storage monitoring and optimization submodule to allocate storage resources based on disk space and read and write frequency; The integrated data reading and decoding rendering module reads data from the queue in sequence and distributes it to the corresponding video decoder or audio decoder according to the data type. After the decoder completes the task, the rendering module will parse the data and present it on the mobile screen.

[0045] Furthermore, it also includes: an audio and video synchronization coordination module, which tracks the video frame rate and audio sampling rate in real time to generate dynamic timestamps, adjusts the audio and video timestamps based on network conditions and the time consumed by each link of data processing, and achieves audio and video synchronization when playing videos on mobile devices.

[0046] The decoder adaptation module, for video data, is based on the H.264 / H.265 standard parsing requirements, has a built-in NAL unit parsing sub-module, uses byte scanning and bit operations to identify NALU key information, and triggers a dynamic buffer expansion mechanism when the data buffer is insufficient; for SPS / PPS extraction, MediaCodec configuration decoding and other links, there are corresponding adaptation sub-modules to ensure efficient and accurate video decoding; for audio data, the type is determined based on audio features, the decoder is matched and called from the pre-stored decoder library, and the playback parameters are optimized when the audio player is initialized to improve the decoding and playback performance.

[0047] The present invention has many significant benefits and effectively solves many problems faced in practical applications: It can solve the problem of unstable streaming of streaming media services, avoid the sudden change of playback speed on the end side due to unstable streaming, ensure a smooth and orderly playback progress, and provide users with a smooth and stable viewing experience.

[0048] For situations where users are in a weak network (unstable network) or suddenly have no network, effective caching mechanisms and other technical means are used to avoid loading problems, allowing users to watch video content normally even in a poor network environment.

[0049] When the server suddenly loses streaming, the client can rely on the cached data to continue playing thanks to its perfect caching function, avoiding the embarrassing situation of viewing interruption due to no cache playback and ensuring viewing continuity.

[0050] It focuses on solving the common problem of audio and video being out of sync. Through precise timestamp control and coordinated cooperation of various links, it ensures that audio and video are always synchronized, greatly improving the user's overall experience of watching videos.

[0051] Although the embodiments of the present invention have been shown and described above, it will be appreciated that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, substitute and vary the above embodiments within the scope of the present invention.

Claims

1. Android mobile video cache optimization method, characterized by: At least the following steps are included: Step S1, establishing data connection and parsing: the streaming media receiving data terminal establishes a webSocket connection with the streaming media server, and adopts a multi-threaded and read-write separation architecture to perform stream parsing; stream parsing, combining the read-write thread, decoder and rendering according to the playback process; using the data parsing thread to parse the received data, and storing the parsed data in a LinkedBlockingQueue queue with first-in-first-out and non-interfering access; Step S2, data caching operation: using the RandomAccessFile feature, the data stored in the LinkedBlockingQueue queue is written into the temporary memory of the RandomAccessFile in real time, and displayed and cached, waiting for the next operation; By using the RandomAccessFile specific method attributes and its LinkedBlockingQueue class in combination, the parsed data is written into the storage file in the order of the queue in the data, and the data is saved on the disk of the mobile terminal; Step S3, read data and decode and render: read the data in the order of the queue through the reading thread, parse the read audio and video data, send the audio and video data to the Android decoder to parse the data; send the parsed data to the Android renderer; Step S4: The mobile terminal implements audio and video synchronization control.

2. The method according to claim 1, characterized in that: The read audio and video data is parsed to determine whether it is video data. If so, it is sent to the video decoder; if not, it is sent to the audio decoder for synchronous playback of audio and video after decoding.

3. The method according to claim 2, characterized in that: If the data is video data, parse the NAL unit according to the H.264 / H.265 standard, identify the NALU type, length and related fields through byte scanning and bit operations, and use dynamic buffer expansion technology to accumulate data when the buffer data is insufficient; extract SPS / PPS information, obtain information by traversing the NAL unit, and set the MediaFormat parameters when creating the decoding; configure MediaCodec for decoding, optimize the configuration according to the video resolution, frame rate, bit rate and color space parameters, and use the thread pool to manage thread resources after starting the decoder thread; when the decoder completes a frame of video processing, it triggers a callback event, checks the output buffer status and calls the releaseOutputBuffer method to release the buffer and sets the parameter to true to present the frame, and processes all frames in a loop.

4. The method according to claim 3, characterized in that: If the data is audio data, use an audio feature extraction algorithm to parse the audio data and determine the type, including one or more types of AAC, G711A and G711U, match and call the corresponding decoder from the pre-stored audio decoder library for decoding and playback, and optimize the playback parameters according to the audio characteristics during the audio player initialization stage; Initialize the audio player for different types, and use different decoders to decode and play the audio; while obtaining audio and video, it is necessary to control the audio and video synchronization through timestamps to maintain stable playback.

5. The method according to claim 3, characterized in that: Parsing the NAL unit Once the existence of NALU is confirmed, the next task is to extract the specific NAL unit information from the original byte sequence; the NAL unit information includes identifying the NALU type, length, and one or more of the related fields; according to the H.264 / H.265 standard, NALU consists of a one-byte header information and subsequent load data, where the header contains a prohibition bit, a reference level indication, and a NAL unit type.

6. The method according to claim 3, characterized in that: The SPS / PPS extraction steps include the following: SPS and PPS, as key information for initializing the decoder, need to be accurately extracted from the NAL unit; by traversing the NAL unit and checking its type field, these special types of NAL units are identified, and the corresponding SPS and PPS related information is obtained, and the corresponding parameters are set during the decoding process.

7. The method according to claim 3, characterized in that: The steps to configure MediaCodec for decoding include: MediaCodecAPI enables developers to directly manipulate the codec to achieve efficient decoding of video streams. To ensure that the decoder can accurately interpret the input data, necessary configuration information, including SPS and PPS data, must be provided in advance. The MediaFormat class is used to define the media format. It accepts parameters including width, height, and additional encoding settings. By calling the MediaCodec.configure function, the MediaFormat instance is passed to the decoder, and the output target Surface is specified. The decoded image can be rendered to the interface. In addition, the decoder thread needs to be started to prepare it for receiving input.

8. The method according to any one of claims 1 to 7, characterized in that: In the step of displaying video frames, when the decoder completes processing a frame of video data, it triggers the corresponding callback event to notify that new output is available; at this time, the status of the output buffer should be checked, and the releaseOutputBuffer function should be called to release the buffer, and the parameter should be set to true to indicate that the frame should be presented immediately; this process is completed in a loop until all frames are processed. 9.Android mobile video cache optimization system, characterized by: The system is used to implement the method described in any one of claims 1 to 8 above, and the system at least includes: The streaming media receiving module is used to establish a webSocket connection with the streaming media server. The module has a built-in multi-threaded scheduling and management sub-module to control the stream parsing process under the multi-threaded and read-write separation architecture, and transmit the parsed data to the subsequent processing link; The cache management module uses the RandomAccessFile feature to write data from the LinkedBlockingQueue queue to temporary memory to ensure the timeliness of the display cache, and writes the parsed data to the mobile disk in a persistent manner according to the queue order. It has a storage monitoring and optimization submodule to allocate storage resources based on disk space and read and write frequency; The integrated data reading and decoding rendering module reads data from the queue in sequence and distributes it to the corresponding video decoder or audio decoder according to the data type. After the decoder completes the task, the rendering module will parse the data and present it on the mobile screen.

10. The system according to claim 9, characterized in that: It also includes an audio-visual synchronization coordination module, which tracks the video frame rate and audio sampling rate in real time to generate dynamic timestamps, and adjusts the audio and video timestamps based on the network conditions and the time consumption of each link of data processing to achieve audio-visual synchronization when playing videos on mobile terminals; The decoder adaptation module, for video data, is based on the H.264 / H.265 standard parsing requirements, has a built-in NAL unit parsing sub-module, uses byte scanning and bit operations to identify NALU key information, and triggers a dynamic buffer expansion mechanism when the data buffer is insufficient; for SPS / PPS extraction and MediaCodec configuration decoding links, there are corresponding adaptation sub-modules to ensure efficient and accurate video decoding; for audio data, the type is determined based on audio features, the decoder is matched and called from the pre-stored decoder library, and the playback parameters are optimized when initializing the audio player to improve the decoding and playback performance.

Images