A memory management method for audio and video calls and a Linux embedded audio and video system

Abstract

The present invention discloses a memory management method for audio and video calls and a Linux embedded audio and video system, relating to the field of computer technology. The memory management method divides page-based memory management into large and small memory blocks. Large memory blocks are directly allocated, and small memory blocks are allocated according to each page as a node. When memory is requested, memory allocation is performed by moving the tail pointer of the small block. When the memory is allocated to the tail of the linked list or insufficient memory is left, a new storage page is allocated. An accelerated matching method is used herein, and the adjacent priority matching principle is utilized to reduce time complexity. All memory within the audio and video session is uniformly managed, allocated, and released, thereby reducing the risk of memory leaks and reducing system calls.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly to a memory management method for audio and video calls and a Linux embedded audio and video system. Background Art

[0002] With the increase in network bandwidth and the growing market demand for audio and video products, the demand for audio and video in embedded terminals is rapidly increasing. However, due to various cost and performance constraints, most embedded systems cannot directly use mainstream audio and video frameworks in terms of product performance and memory. These systems suffer from problems such as irrational memory pool allocation, frequent calls, low allocation efficiency, poor system call capabilities, and high leakage risks. Therefore, it is necessary to develop a method to manage memory calls during audio and video calls.

[0003] In addition, although there are many chip platforms for existing Linux embedded products, they are limited by chip flash and memory limitations, and there are deficiencies in memory management and functional module management strategies. Currently, there is no universal audio and video solution that supports various embedded devices and cannot meet the expansion of various specific needs. Summary of the Invention

[0004] The present invention provides a memory management method for audio and video calls, aiming to solve the above-mentioned problems existing in the prior art.

[0005] The present invention adopts the following technical solutions:

[0006] A memory management method for audio and video calls, comprising the following steps:

[0007] (1) Apply for memory;

[0008] (2) Determine whether the size of the requested memory is greater than a first threshold. If not, execute step (3); if so, allocate a large memory node and then execute step (8);

[0009] (3) Get the current node of the small block memory management list and get the head pointer byte;

[0010] (4) Determine whether the remaining memory size of the current node meets the requested memory size; if so, execute step (8); if not, execute step (5);

[0011] (5) The number of failed memory requests is increased by one; then, it is determined whether the number of failed memory requests is greater than or equal to a second threshold; if not, step (6) is executed; if so, step (7) is executed;

[0012] (6) Determine whether the current node is the end of the linked list; if not, point to the next node and then execute step (4); if so, execute step (7);

[0013] (7) Allocate new small memory nodes;

[0014] (8) Return the allocated address; (9) End.

[0015] Furthermore, before step (1), the method further includes: establishing a memory pool after receiving the first instruction.

[0016] Furthermore, the first instruction is: starting an audio and video session.

[0017] Furthermore, the memory pool adopts page storage, and the first threshold is the same as the page size of the page storage.

[0018] Furthermore, in step (2), allocating the large memory node specifically includes: first obtaining the small memory node through the method recorded in steps (3) to (8); then adding the obtained small memory node to the large memory management list; applying for the large memory; and returning the allocation address.

[0019] Furthermore, the second threshold is set to 3. The first threshold is set to 4K.

[0020] Furthermore, the step (7) specifically includes: allocating a new small memory node and adding it to the small memory management linked list; then, obtaining the current node of the new small memory node and updating it to the current node position.

[0021] A Linux embedded audio and video system comprises a memory management module. The memory management module uses any of the above-mentioned memory management methods for audio and video calls.

[0022] Furthermore, the system includes an interconnected component management module, a network management module, the memory management module, a high-performance lock-free queue module, an audio and video processing component, a cross-platform adaptation module and an audio and video acquisition module.

[0023] From the above description of the structure of the present invention, it can be seen that the present invention has the following advantages:

[0024] The memory management method of this invention divides paged memory into large and small blocks based on page size. Large blocks are allocated directly, while small blocks are allocated as nodes per page. When memory is requested, memory is allocated by moving the tail pointer of the small block until the end of the linked list is reached or insufficient space is left. A new storage page is allocated. Accelerated matching is used here, utilizing the adjacent-first matching principle to reduce time complexity. A second threshold is set to control the number of failures within a reasonable range. Unified management, allocation, and release of all memory within audio and video sessions reduces the risk of memory leaks, reduces system calls, and improves performance.

[0025] The present invention also discloses an audio and video system. First, using the component management module M1, users can modify configuration files and tailor functionality based on their product needs. Second, audio and video communication utilizes third-party interaction protocols such as SIP or RTSP to negotiate audio and video. Basic parameters are then transferred to the network management module M2 to connect the data between the two parties. Third, the interface definitions provided by the cross-platform adapter module ensure that hardware encoding and decoding and device control can function properly. By combining these three aspects, a complete audio and video communication system can be constructed. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flowchart of the method in the present invention.

[0027] Figure 2 This is a structural block diagram of the system in the present invention. DETAILED DESCRIPTION

[0028] Specific embodiments of the present invention will be described below with reference to the accompanying drawings.

[0029] like Figure 1 and Figure 2 As shown, a memory management method for audio and video calls includes the following steps:

[0030] (1) After receiving the first instruction, a memory pool is established. Preferably, the first instruction includes but is not limited to starting an audio or video session. The memory pool includes but is not limited to using paged storage, and the page size of the paged storage includes but is not limited to 4K, and can also be 2K, 8K, 16K, etc.

[0031] (2) Requesting memory. Specifically, when other modules such as the network management module, audio and video processing module, or audio and video acquisition module need to use memory, these modules submit a request for memory use to the memory management module based on their own needs. The memory management module then allocates addresses reasonably through the memory management method of the audio and video call to facilitate memory access.

[0032] (3) Determine whether the size of the requested memory is greater than the first threshold. If not, execute step (4); if so, allocate a large memory node and then execute step (9).

[0033] Preferably, the first threshold is the same as the page size of the paged storage; taking the page size of 4K as an example, the preferred value of the first threshold is 4K.

[0034] (4) Get the current node of the small block memory management list and get the head pointer byte.

[0035] (5) Determine whether the remaining memory size of the current node meets the requested memory size; if so, execute step (9); if not, execute step (6).

[0036] (6) The number of failed memory requests is increased by one; then, it is determined whether the number of failed memory requests is greater than or equal to a second threshold; if not, step (7) is executed; if so, step (8) is executed. Preferably, the second threshold is 3.

[0037] (7) Determine whether the current node is the tail of the linked list; if not, point to the next node and then execute step (4); if so, execute step (8).

[0038] (8) Allocate a new small memory node. This is specifically divided into: allocating a new small memory node, adding the new small memory node to the small memory management list; then, obtaining the current node of the new small memory node and updating it to the current node position.

[0039] (9) Return the allocated address.

[0040] (10) End.

[0041] Specifically, in the above step (2), allocating the large memory node specifically includes: first, obtaining the small memory node by using the method described in steps (3) to (8); then, adding the obtained small memory node to the large memory management list; then applying for the large memory, and then returning the allocated address.

[0042] After the memory address is allocated, the memory management module calls the memory according to the allocated address to provide storage functions for other modules.

[0043] Taking the first instruction to start an audio and video session as an example, after the memory management module receives the second instruction including but limited to "end the audio and video session", the memory management module will release the memory uniformly.

[0044] like Figure 1 and Figure 2 As shown, the present invention also discloses a Linux embedded audio and video system, including a memory management module, an interconnected component management module, a network management module, the memory management module, a high-performance lock-free queue module, an audio and video processing component, a cross-platform adaptation module and an audio and video acquisition module.

[0045] The component management module M1 is mainly used to support the plug-and-play management of components, support component registration and use chain management to support data transmission between components along the link, use a unified registration function and component external interface consistency, and support dynamic loading of some component dynamic libraries, thereby reducing runtime memory.

[0046] The network management module M2 primarily functions as network data transmission and reception, as well as session management. Session management primarily supports multiple simultaneous audio and video sessions. Preferably, the network management module M2 utilizes a high-performance single-threaded Reactor model for asynchronous network data transmission and reception, significantly improving concurrent data efficiency compared to traditional blocking data transmission and reception models.

[0047] Memory management module M3 uses the aforementioned memory management method for audio and video calls. This memory management module M3 builds upon the session management provided by network management module M2. Unlike traditional memory pools, memory allocations are retained throughout the entire session lifecycle, preventing interference between sessions and avoiding frequent memory allocation and release operations. Memory allocations are released uniformly at the end of each session. This eliminates the risk of memory leaks and significantly improves efficiency compared to traditional system calls.

[0048] The high-performance lock-free queue module M4 first queues network data received from the network management module M2, and then retrieves the data for processing by other processing modules. Due to multithreading issues, this high-performance lock-free queue module M4 uses assembly instructions to support atomic operations, ensuring that the queue can be managed in multithreaded environments without locking. Compared to traditional resource protection methods using mutex locks, this greatly improves data transfer efficiency and shortens data transmission latency.

[0049] The audio and video processing component M5 mainly includes a protocol component, a weak network component, an SRTP component, a ZRTP component, an audio 3A component, a facial recognition component, and a QR code component. As the core component of audio and video processing, it covers mainstream audio and video SRTP and ZRTP encryption functions, audio and video weak network countermeasures, audio processing includes echo duplexing, automatic gain, and a proprietary 3A algorithm for noise suppression. Video can be expanded to include QR code recognition, facial recognition, and other functions. The audio and video processing component M5 can provide a high-quality audio and video experience in weak network conditions, while also providing encryption protection for greater data security.

[0050] Cross-platform adaptation module M6, after the data flows through the audio and video processing component M5, it needs to perform platform layer encoding and decoding and other operations. The system dynamically loads the FFmpeg audio and video library for software encoding and decoding, and supports platform layer hardware encoding and decoding adaptation. The platform adaptation layer can give priority to hardware encoding and decoding to ensure encoding and decoding efficiency, and use FFmpeg software decoding to supplement the encoding and decoding formats that the hardware encoding and decoding cannot support, so that embedded devices can basically support the industry's mainstream audio and video encoding and decoding formats.

[0051] The audio and video capture module M7 is primarily used for audio and video acquisition on the transmitter side. After acquiring data from the camera, it typically first enters the driver program, making it unavailable to user programs. The M7 utilizes zero-copy and memory-mapping technologies to reduce the overhead of single data copies, significantly improving acquisition performance and enabling high-speed data transmission, thereby reducing acquisition latency.

[0052] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited to this. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A memory management method for audio and video calls, characterized in that: The following steps are involved: (1) After receiving a first instruction, a memory pool is established; the first instruction is to start an audio and video session, and the memory pool adopts page storage; (2) Apply for memory; (3) Determine whether the size of the memory requested is greater than a first threshold. If not, execute step (4); if so, allocate a large memory node. The specific steps of allocating the large memory node are: first, obtain a small memory node by the method described in steps (3) to (8); then add the obtained small memory node to the large memory management list; apply for large memory; and then execute step (9); (4) Get the current node of the small block memory management list and get the head pointer byte; (5) Determine whether the remaining memory size of the current node meets the requested memory size; if so, execute step (9); if not, execute step (6); (6) The number of failed memory requests is increased by one; then, it is determined whether the number of failed memory requests is greater than or equal to a second threshold; if not, step (7) is executed; if so, step (8) is executed; (7) Determine whether the current node is the tail of the linked list; if not, point to the next node and then execute step (5); if so, execute step (8); (8) Allocate a new small memory node; (9) Return the allocated address; (10) End; The memory management method for audio and video calls is applied to a Linux embedded audio and video system, which includes a network management module and a memory management module. The memory management module uses the memory management method for audio and video calls to establish a memory pool based on the session management of the network management module. The memory application will be retained throughout the entire session life cycle, and the memory will not be dynamically released during the session. The memory will be released uniformly at the end of the session.

2. The memory management method for audio and video calls according to claim 1, characterized in that: The memory pool adopts page storage, and the first threshold is the same as the page size of the page storage.

3. The memory management method for audio and video calls according to claim 1, wherein: The first threshold value is 4K.

4. The memory management method for audio and video calls according to claim 1, wherein: The second threshold value is 3.

5. The memory management method for audio and video calls according to claim 1, characterized in that: The step (8) specifically includes: allocating a new small memory node and adding it to the small memory management linked list; then, obtaining the current node of the new small memory node and updating it to the current node position.

6. A Linux embedded audio and video system, characterized by: It includes a memory management module, which uses the memory management method for audio and video calls as described in any one of claims 1 to 5.

7. A Linux embedded audio and video system according to claim 6, characterized in that: It includes an interconnected component management module, a network management module, a memory management module, a high-performance lock-free queue module, an audio and video processing component, a cross-platform adaptation module and an audio and video acquisition module.

Images