Multi-terminal-oriented distributed audio and video adaptive production system

Abstract

The invention, which relates to the technical field of audio and video production, discloses a multi-terminal-oriented distributed audio and video adaptive production system comprising a distributed storage cluster, an audio and video processing node, a cloud scheduling center, a sensing module, a content analysis module and a client. The storage cluster uses erasure codes, Ceph and an enhanced data redundancy rate formula to ensure data reliability; the sensing module collects terminal information to form a device feature vector, and the content analysis module generates a content feature vector by using an algorithm; the cloud scheduling center allocates tasks according to the comprehensive processing capability adaptation factor; a client interface is adaptive, multi-thread efficiency improvement is achieved, interaction with a cloud scheduling center is achieved, and the requirements of different terminal users are met. Data security is guaranteed through distributed storage, cost is reduced through hierarchical storage, efficiency is improved through cloud scheduling optimization task allocation, personalized production is achieved through intelligent perception and content analysis, system stability is guaranteed through adaptive coding and multi-terminal adaption, real-time monitoring is achieved, interfaces are opened to expand ecology, and audio and video production experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of audio and video production systems, and in particular to a distributed audio and video adaptive production system for multiple terminals. Background Art

[0002] With the rapid development of information technology, audio and video content plays an increasingly important role in people's daily life and work. From online education, video conferencing to entertainment live broadcasts and short video creation, the application scenarios of audio and video are becoming more and more extensive. At the same time, the types of terminal devices are also showing a diversified trend, including smartphones, tablets, laptops, smart TVs, etc. These devices have huge differences in hardware performance, screen resolution, network connection, etc. This puts higher requirements on audio and video production systems, requiring the system to be able to adapt to the characteristics of different terminals and provide high-quality audio and video content.

[0003] Traditional audio and video production systems are mainly based on a single-machine environment and rely on local hardware resources for audio and video processing and editing. This approach has many limitations. First, the processing power is limited by the hardware configuration of the local device. For high-definition, ultra-high-definition, and complex special effects audio and video production, performance bottlenecks often occur, resulting in slow processing speeds and low production efficiency. Second, traditional systems lack the ability to adapt to multiple terminals. The playback effects of the produced audio and video content on different terminals vary greatly, and cannot meet the viewing needs of users on various terminal devices.

[0004] With the development of cloud computing and distributed technology, some audio and video production systems based on cloud architecture have emerged. Although these systems have solved the problems of processing power and storage capacity to a certain extent, they still have some shortcomings. For example, in terms of data storage, most existing systems adopt a simple redundant backup strategy, lacking comprehensive consideration of data importance and access frequency, resulting in a waste of storage resources. In terms of task allocation, the actual processing capacity of the processing node and the correlation between tasks are not fully considered, which is prone to load imbalance and affects the overall performance of the system.

[0005] In addition, the current audio and video production system is not smart enough in terms of adaptive encoding. Although some systems can make simple bit rate adjustments based on the terminal's network bandwidth, they do not fully consider the terminal's screen resolution, processing power, and the characteristics of the audio and video content, and cannot provide users with a personalized playback experience. Moreover, the existing system also has loopholes in security protection. Audio and video data are vulnerable to attacks and leaks during transmission and storage, which poses a security risk to users.

[0006] In addition, with the development of the audio - video industry, more and more third - party developers hope to integrate audio - video production functions into their own applications. However, the existing systems lack open API interfaces and a perfect developer support system, which restricts the application scenarios of the systems and the development of the ecosystem. Therefore, it is of great practical significance to develop a distributed audio - video adaptive production system for multiple terminals. Summary of the Invention

[0007] The distributed audio - video adaptive production system for multiple terminals proposed by the present invention aims to solve the problems mentioned in the above - mentioned prior art.

[0008] To achieve the above object, the present invention adopts the following technical solutions: A distributed audio - video adaptive production system for multiple terminals includes a distributed storage cluster, audio - video processing nodes, a cloud scheduling center, a perception module, a content analysis module, and clients adapted to different terminals; The distributed storage cluster uses erasure code technology combined with a distributed file system to store audio - video materials, and uses an enhanced data redundancy rate formula to ensure data reliability, where is the number of redundant data blocks, is the number of parity check blocks, and α is an erasure code weight coefficient, which is dynamically adjusted according to data importance and storage environment; The perception module collects the hardware information, network status, and user usage habits of terminal devices in real - time, and describes the terminal characteristics through the device feature vector ; The content analysis module uses learning algorithms to perform semantic analysis, scene recognition, and emotion classification on audio - video materials to obtain content feature vectors . The audio - video processing nodes are distributed in different geographical locations and have different processing capabilities. The cloud scheduling center allocates audio - video processing tasks according to the comprehensive processing ability adaptation factor , where is the actual processing ability of the node, is the flexibility of the node to process different types of tasks, and are weighting coefficients.

[0009] Furthermore, it also includes the following modules: Real - time monitoring module: The module collects the storage utilization rate of the distributed storage cluster in real - time , where is the used storage space, is the total storage space, the task completion rate of the audio - video processing nodes , where is the number of completed tasks, is the total number of tasks, and the energy consumption E of the node; The cloud scheduling center dynamically adjusts the task allocation and storage strategy according to the data and the energy consumption - performance ratio formula .

[0010] Adaptive encoding module: Calculate adaptive encoding parameters according to the network bandwidth B, screen resolution R, processing capacity P of the terminal device, and content feature vector CFV , where is the weight coefficient.

[0011] Furthermore, the real-time monitoring module also monitors the network latency L between the client and the cloud scheduling center, and uses the network latency fluctuation coefficient to evaluate the network stability; where is the fluctuation range of the network latency, is the average network latency); when the network latency fluctuation coefficient exceeds the threshold, the cloud scheduling center adopts an adaptive buffering strategy and adjusts the buffer capacity through the buffer capacity adjustment formula where is the buffer coefficient.

[0012] Furthermore, when the cloud scheduling center adjusts the task allocation, it considers the load balancing factor of the processing node and the task correlation coefficient; where is the processing capacity already used by the i-th processing node, is the total processing capacity of the i-th processing node, and n is the number of processing nodes); the cloud scheduling center maintains a task dependency graph to record the association relationships between tasks, calculates the task correlation coefficient by analyzing the input and output data and processing flow of the tasks; for associated tasks, they are preferentially assigned to adjacent nodes or nodes with low communication costs, and the task allocation optimization formula is used to evaluate the allocation scheme.

[0013] Furthermore, the adaptive encoding module also considers the complexity C of the audio-visual content, and combines the complexity to adjust the adaptive encoding parameters to , where is the complexity adjustment coefficient.

[0014] Furthermore, the distributed storage cluster supports hierarchical storage of data, and calculates the data storage priority according to the access popularity of the data, where is the number of data accesses, is the statistical time period, and the timeliness T of the data, and stores the data on storage media with different performances, where and θ are weighting coefficients.

[0015] Furthermore, the client provides audio-visual editing functions, supporting users to perform operations such as clipping and adding subtitles to audio-visual content; during the editing process, the system calculates the operation response time prediction formula according to the processing capacity of the client device and the operation complexity OC Optimize the operation process in advance, where is the response time coefficient.

[0016] Furthermore, the system uses an encryption algorithm to encrypt audio and video data during transmission and storage, and uses an encryption strength evaluation formula to evaluate the encryption security, where is the encryption key length, is the strength coefficient; during data storage, encrypt and store audio and video materials and production projects, and dynamically adjust the encryption strategy in combination with the security level SL of the terminal device. The encryption adjustment formula is to ensure the privacy and security of audio and video data.

[0017] Furthermore, the system provides an open API interface to allow third-party developers to integrate; evaluate the API performance through the API call success rate formula where is the number of successful API calls, is the total number of API calls; if the API call success rate is low, the system will analyze the reasons, predict that the server load is too high and the interface parameters are incorrect, and optimize and adjust in a timely manner; calculate the cooperation value index according to the usage frequency UF and data interaction volume DI of the third-party application to provide differentiated services and support for different third-party applications, where and are weighting coefficients Compared with the existing technology, the beneficial effects of the present invention are: In terms of data storage, the distributed storage cluster adopts erasure code technology combined with a distributed file system, and ensures data reliability through an enhanced data redundancy rate formula. At the same time, calculate the storage priority according to the access heat and timeliness of the data, implement hierarchical storage, and store data with high access frequency and high timeliness in high-speed media, effectively reducing access latency and saving storage costs.

[0018] In terms of task allocation, the cloud scheduling center allocates tasks according to the comprehensive processing ability adaptation factor, and at the same time considers the load balancing factor and the task correlation coefficient, and realizes more reasonable task allocation through the task allocation optimization formula, avoiding uneven load on processing nodes, and improving the overall performance and processing efficiency of the system.

[0019] The adaptive coding module is a major highlight of this system. It comprehensively considers the network bandwidth, screen resolution, processing ability of the terminal device, as well as the feature vector and complexity of the audio and video content, calculates the adaptive coding parameters, and adopts hierarchical coding technology to provide personalized coding schemes for different terminals, ensuring that the audio and video can achieve the best playback effect on various terminals and improving the user experience.

[0020] The real-time monitoring module collects data from all aspects of the system in real time, such as storage utilization rate, task completion rate, energy consumption, network latency, etc., and conducts evaluation and analysis through relevant formulas. The cloud scheduling center dynamically adjusts task allocation, storage strategies, and buffer capacity based on this data to ensure the stable operation of the system, improve resource utilization efficiency, and reduce energy consumption at the same time.

[0021] The audio and video editing functions of the client are rich. The operation process is optimized in advance through the operation response time prediction formula to reduce the waiting time of users. The intelligent recommendation algorithm recommends appropriate editing templates and materials according to the user's usage habits and content feature vectors to improve the editing efficiency and personalization level.

[0022] The system performs excellently in terms of security. It uses a variety of encryption algorithms to encrypt audio and video data during transmission and storage, and dynamically adjusts the encryption strategy in combination with the terminal security level. The encryption strength evaluation formula is used to ensure the privacy and security of the data. In addition, the system provides open API interfaces, evaluates the performance through the API call success rate, and provides differentiated services for third-party applications according to the cooperation value index, expanding the application scenarios and ecosystem of the system. Brief Description of the Drawings

[0023] Figure 1 It is a schematic block diagram of the distributed audio and video adaptive production system for multi-terminals proposed by the present invention.

[0024] Figure 2 It is a schematic block diagram of the distributed audio and video adaptive production method for multi-terminals proposed by the present invention. Detailed Embodiments

[0025] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0026] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention.

[0027] In addition, the terms "first" and "second" are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present invention, "a plurality of" means two or more unless otherwise specifically defined. In addition, the terms "mounted", "connected" and "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. The present invention will be further described in detail below with reference to the accompanying drawings.

[0028] Refer to Figures 1 to 2 : A distributed audio - video adaptive production system for multiple terminals, including a distributed storage cluster, multiple audio - video processing nodes, a cloud scheduling center, an intelligent perception module, a content analysis module, and a client adapted to different terminals. The distributed storage cluster uses erasure coding technology combined with a distributed file system (such as Ceph) to store audio - video materials, and uses an enhanced data redundancy rate formula ( is the number of redundant data blocks, is the number of parity check blocks, and α is the erasure coding weight coefficient, which is dynamically adjusted according to data importance and storage environment) to ensure data reliability. The intelligent perception module collects the hardware information of the terminal device (such as the number of CPU cores, the size of GPU video memory), network status (including bandwidth, packet loss rate), and user usage habits (such as common editing functions, viewing preferences) in real - time, and comprehensively describes the terminal characteristics through the device feature vector . The content analysis module uses deep learning algorithms to perform semantic analysis, scene recognition, and emotion classification on the audio - video materials, and obtains the content feature vector for subsequent personalized production. Multiple audio - video processing nodes are distributed in different geographical locations and have different processing capabilities. The cloud scheduling center adapts according to the comprehensive processing capacity adaptation factor ( is the actual processing capacity of the node, is the flexibility of the node to process different types of tasks, and Assign audio - video processing tasks according to the weighted coefficient). The client interacts with the cloud scheduling center through the network, supporting terminal devices with different operating systems and hardware configurations. The client interface adapts its layout according to the screen size and resolution of the terminal device, and uses multi - thread technology to improve data transmission and processing efficiency. The distributed storage cluster uses erasure coding technology combined with the Ceph distributed file system to store audio - video materials. Through the enhanced data redundancy rate formula, the α value is dynamically adjusted according to the importance of the data and the storage environment to reliably store the materials. The intelligent perception module collects the hardware information, network status, and user usage habits of the terminal device in real - time, comprehensively describes the terminal characteristics with the device feature vector, and provides a basis for personalized services. The content analysis module uses deep - learning algorithms to analyze audio - video materials and obtain content feature vectors to assist in personalized production. Multiple audio - video processing nodes are distributed in different locations and have different capabilities. The cloud scheduling center reasonably allocates tasks according to the comprehensive processing ability adaptation factor. The client adapts to different terminals, the interface has an adaptive layout, and multi - thread technology improves data transmission and processing efficiency. It ensures the reliability of audio - video material storage, can provide personalized services according to different terminals and user habits, reasonably allocate tasks to improve production efficiency, and good adaptability and high - efficiency data - processing capabilities bring a good user experience In the present invention, the following modules are further included: Real - time monitoring module: This module collects the storage utilization rate of the distributed storage cluster in real - time ( is the used storage space, is the total storage space), the task completion rate of the audio - video processing node ( is the number of completed tasks, is the total number of tasks) and the energy consumption E of the node. The real - time monitoring module is equipped with high - precision sensors and data acquisition software, and these sensors and software collect data for the storage cluster and processing nodes respectively. For the storage cluster, the sensors monitor parameters such as the read - write status and temperature of the hard disks in real - time, and combine with the used and total storage space information recorded by the storage management software to accurately calculate the storage utilization rate. For the audio - video processing nodes, by monitoring the operating status of the CPU and GPU and the energy consumption data of the power module, the task completion rate and energy consumption are accurately obtained. The cloud scheduling center dynamically adjusts the task allocation and storage strategy according to these data and the energy - consumption - to - performance ratio formula When the energy - consumption - to - performance ratio of a certain processing node is too high, the cloud scheduling center will migrate some tasks to the node with a low energy - consumption - to - performance ratio. At the same time, according to the storage utilization rate, some infrequently used data will be migrated to low - cost storage media to balance the use of system resources and energy consumption. In addition, the real - time monitoring module will also visually display the collected data, facilitating the administrator to grasp the system operation status in real - time and promptly discover potential problems.

[0029] Adaptive Encoding Module: This module calculates adaptive encoding parameters based on the network bandwidth B, screen resolution R, processing capacity P of the terminal device, and the content feature vector CFV ( being the weight coefficient). The adaptive encoding module calculates parameters in a multi-model fusion manner. First, it trains historical data through a machine learning model to obtain initial values of different weight coefficients. During actual operation, it dynamically adjusts the weight coefficients according to the terminal device information and content feature vector collected in real time. The audio-video processing node encodes the audio-video according to this parameter to adapt to the playback requirements and content characteristics of different terminals. For terminals with low network bandwidth, the adaptive encoding module reduces the encoding bit rate and data volume, and selectively retains key frames and feature information according to the importance of the content; for terminals with high-resolution screens, it improves the encoding clarity and details to ensure the playback effect. In addition, the adaptive encoding module also supports the conversion of multiple encoding formats, such as H.264, H.265, etc., and selects the appropriate encoding format according to the compatibility and performance of the terminal device.

[0030] In the present invention, the real-time monitoring module also monitors the network latency L between the client and the cloud scheduling center, and uses the network latency fluctuation coefficient ( being the fluctuation range of the network latency, being the average network latency) to evaluate the network stability. The real-time monitoring module uses the ping-pong mechanism to send test data packets to the client regularly, and calculates the network latency by recording the time difference between sending and receiving. At the same time, it statistically analyzes the network latency data over a period of time to obtain the average network latency and the latency fluctuation range. When the network latency fluctuation coefficient exceeds the threshold, the cloud scheduling center adopts an adaptive buffering strategy, and adjusts the buffer capacity through the buffer capacity adjustment formula ( being the buffer coefficient) to ensure the continuity of data transmission. The cloud scheduling center dynamically adjusts the buffer sizes of the client and the server side. When the network latency fluctuates greatly, it increases the buffer capacity and caches more data in advance to cope with possible latency peaks; when the network returns to stability, it appropriately reduces the buffer capacity to reduce the data storage pressure. In addition, the cloud scheduling center also adjusts the data transmission rate and priority according to the network latency situation, and preferentially transmits key data, such as the key frames of audio-video.

[0031] In the present invention, when the cloud scheduling center adjusts the task allocation, it considers the load balancing factor ( being the processing capacity already used by the i-th processing node, is the total processing capacity of the i-th processing node, n is the number of processing nodes), and the task correlation coefficient. The cloud scheduling center maintains a task dependency graph to record the association relationships between tasks. By analyzing the input and output data and processing flow of tasks, it calculates the task correlation coefficient. For associated tasks, they are preferentially assigned to adjacent nodes or nodes with low communication costs, and the task allocation optimization formula is used to comprehensively evaluate the allocation scheme. When a new task arrives, the cloud scheduling center first calculates the load balancing factor of each processing node and the correlation coefficient between the task and the existing tasks on the node, and then selects the optimal processing node for task allocation according to the task allocation optimization formula. At the same time, the cloud scheduling center also considers the network topology and communication bandwidth between nodes to minimize the data transmission cost between tasks. If the load of a certain node is too high, the cloud scheduling center will migrate some tasks to other nodes with lower loads to achieve load balancing of the entire system. Comprehensively evaluate the allocation scheme. When a new task arrives, the cloud scheduling center first calculates the load balancing factor of each processing node and the correlation coefficient between the task and the existing tasks on the node, and then selects the optimal processing node for task allocation according to the task allocation optimization formula. At the same time, the cloud scheduling center also considers the network topology and communication bandwidth between nodes to minimize the data transmission cost between tasks. If the load of a certain node is too high, the cloud scheduling center will migrate some tasks to other nodes with lower loads to achieve load balancing of the entire system.

[0032] In the present invention, the adaptive encoding module also considers the complexity C of the audio-visual content, and adjusts the adaptive encoding parameters in combination with the complexity as ( is the complexity adjustment coefficient). The adaptive encoding module analyzes multiple dimensions such as the frame rate, resolution change, color complexity, and audio spectrum of the audio-visual content to calculate the complexity of the audio-visual content. For audio-visual content with high complexity, the encoding quality requirements are appropriately increased, and at the same time, the hierarchical encoding technology is adopted. Different quality layer data is selectively transmitted according to the terminal capabilities. During the hierarchical encoding process, the audio-visual data is divided into a base layer and an enhancement layer. The base layer contains basic audio-visual information and is suitable for low-performance terminals to play; the enhancement layer provides higher clarity, more details, and better audio effects for high-performance terminals to choose to receive. The adaptive encoding module will dynamically determine which quality layer data to transmit according to the processing capacity and network bandwidth of the terminal device. For example, for terminals with limited network bandwidth, only the base layer data is transmitted; for terminals with sufficient network bandwidth and strong processing capabilities, the base layer and some enhancement layer data are transmitted. In addition, the adaptive encoding module will also adjust the encoding strategy according to the user's viewing preferences and historical behaviors to provide a personalized playback experience for the user.

[0033] In the present invention, the distributed storage cluster supports hierarchical storage of data. According to the access popularity of the data ( is the number of data accesses, is the statistical time period) and the timeliness T of the data, calculate the data storage priority ( and θ are weighting coefficients), and store the data on storage media with different performances. The distributed storage cluster adopts a multi-level storage architecture, including a cache layer (such as SSD), a large-capacity storage layer (such as HDD), and an archival storage layer (such as a tape library). The storage management software will regularly count the access times and time of the data, calculate the access popularity and timeliness, and store the data in the appropriate storage layer according to the data storage priority. Frequently accessed data with high timeliness is stored in high-speed storage media to reduce access latency; for data with low access frequency but long-term preservation requirements, it is stored in the archival storage layer to save costs. At the same time, the storage management software will monitor the access situation of the data in real time, and automatically migrate the data to the appropriate storage layer when the access popularity or timeliness of the data changes. For example, when the access popularity of a certain data suddenly increases, it is migrated from the large-capacity storage layer to the cache layer; when the timeliness of the data decreases and the access frequency becomes low, it is migrated from the cache layer to the archival storage layer.

[0034] In the present invention, the client provides audio and video editing functions, supporting operations such as video and audio clip editing and subtitle adding for users. During the editing process, the system calculates the operation response time prediction formula according to the processing capacity of the client device and the operation complexity OC ( is the response time coefficient) to optimize the operation process in advance. The editing interface of the client adopts a modular design, and each editing function module runs independently. Through multi-threaded technology, user operations are processed in parallel to improve the editing efficiency. When the user performs an operation, the system first predicts the operation response time according to the operation complexity and the processing capacity of the client device. If the predicted response time is long, the system will perform data preloading and caching in advance, optimize the execution order of the algorithm, and reduce the user waiting time. At the same time, the client adopts an intelligent recommendation algorithm to recommend suitable editing templates and materials for the user according to the user's usage habits and content feature vectors. The intelligent recommendation algorithm is based on a deep learning model, analyzes the user's historical operation data and audio and video content features, and mines the user's interest preferences. When the user opens the editing interface, the system will recommend common editing templates and related materials according to the user's usage habits, such as subtitle styles, music materials, etc. In addition, the client also supports the user to give feedback on the recommendation results, continuously optimize the recommendation algorithm, and improve the accuracy and personalization degree of the recommendation.

[0035] In the present invention, the system uses an encryption algorithm to encrypt audio and video data during transmission and storage, and uses the encryption strength evaluation formula ( is the encryption key length, Evaluate the encryption security by the strength coefficient). The system supports multiple encryption algorithms, such as AES, RSA, etc., and selects the appropriate encryption algorithm according to the sensitivity of the data and the usage scenario. During data transmission, the SSL / TLS protocol is used to encrypt the data to ensure the security of the data during network transmission. During data storage, the audio and video materials and production projects are encrypted and stored, and only authorized users can decrypt and access them. At the same time, the encryption policy is dynamically adjusted in combination with the security level SL of the terminal device, and the encryption adjustment formula is to ensure the privacy and security of audio and video data. For terminal devices with a higher security level, a higher-strength encryption algorithm and a longer encryption key are used; for terminal devices with a lower security level, the encryption strength is appropriately reduced on the premise of ensuring a certain level of security to improve the system performance. In addition, the system will regularly update the encryption key to prevent the key from being cracked and further enhance the security of the data.

[0036] In the present invention, the system provides an open API interface to allow third-party developers to integrate. Through the API call success rate formula ( is the number of successful API calls, is the total number of API calls) to evaluate the API performance. The system provides detailed API documentation and development examples to help third-party developers integrate quickly. The API interface adopts the RESTful architecture, which has good scalability and compatibility. The system will monitor the API calls in real time, record the request parameters, response time and results of each call, and evaluate the performance of the API according to the API call success rate. If the API call success rate is low, the system will analyze the reasons, which may be high server load, incorrect interface parameters, etc., and optimize and adjust in time. At the same time, calculate the cooperation value index according to the usage frequency UF and data interaction volume DI of the third-party application ( and are weighting coefficients) to provide differentiated services and support for different third-party applications, and expand the application scenarios and ecosystem of the system. For third-party applications with a high cooperation value index, the system will provide higher service priorities, more technical support and preferential policies; for third-party applications with a low cooperation value index, the system will guide them to optimize the usage method to improve the cooperation value. The system opens the API interface for third-party developers to integrate, evaluates its performance through the API call success rate formula, and measures the operation status of the interface with intuitive data. At the same time, it provides detailed API documentation and development examples, just like preparing clear "instructions" and "operation guides" for developers to help them quickly get started and integrate. The interface adopts the RESTful architecture, just like building a solid and flexible framework, with good scalability and compatibility, and can easily handle various development needs.

[0037] The system monitors API calls in real time, detailedly records information such as request parameters. It's like having a dedicated person constantly watching the operation dynamics. Once it finds that the call success rate is low, it quickly analyzes the reasons and optimizes. In addition, it calculates the cooperation value index based on the usage frequency and data interaction volume of third-party applications, and then provides differentiated services. This measure has many benefits. It not only expands the application scenarios and ecosystem of the system, but also encourages third-party applications to optimize, achieving a win-win situation for both sides and enhancing the overall competitiveness and practicality of the system.

[0038] The above are only the preferred specific implementation manners of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes should be covered within the protection scope of the present invention.

Claims

1. A distributed audio - video adaptive production system for multiple terminals, characterized in that, It includes a distributed storage cluster, an audio and video processing node, a cloud scheduling center, a sensing module, a content analysis module, and a client that adapts to different terminals; the distributed storage cluster uses erasure code technology combined with a distributed file system to store audio and video materials, and uses an enhanced data redundancy rate formula to ensure data reliability, where is the number of redundant data blocks, is the number of parity check blocks, and α is the erasure code weight coefficient, which is dynamically adjusted according to data importance and storage environment; the sensing module collects the hardware information, network status, and user usage habits of terminal devices in real time, and describes the terminal characteristics through the device feature vector ​ The content analysis module uses learning algorithms to perform semantic analysis, scene recognition, and emotion classification on audio-visual materials, and obtains content feature vectors , the audio-visual processing nodes are distributed in different geographical locations and have different processing capabilities. The cloud scheduling center allocates audio-visual processing tasks according to the comprehensive processing capacity adaptation factor , where is the actual processing capacity of the node, is the flexibility of the node to process different types of tasks, and are weighting coefficients.

2. The distributed audio-video adaptive production system for multiple terminals according to claim 1, characterized in that, Further included are: Real-time monitoring module: The module collects the storage utilization rate of the distributed storage cluster in real time , where is the used storage space, is the total storage space, and the task completion rate of the audio and video processing node , where is the number of completed tasks, is the total number of tasks, and the energy consumption E of the node; The cloud scheduling center dynamically adjusts the task allocation and storage strategy according to the data and the energy consumption - performance ratio formula ​ 3. The multi-terminal-oriented distributed audio and video adaptive production system according to claim 1, wherein Further included are: Adaptive coding module: Calculate adaptive coding parameters according to the network bandwidth B, screen resolution R, processing capacity P of the terminal device, and content feature vector CFV , where is the weight coefficient.

4. The distributed audio - video adaptive production system for multiple terminals according to claim 2, wherein, The real-time monitoring module also monitors the network latency L between the client and the cloud scheduling center, and uses the network latency fluctuation coefficient to evaluate the network stability; where is the fluctuation range of the network latency, is the average network latency; when the network latency fluctuation coefficient exceeds the threshold, the cloud scheduling center adopts an adaptive buffering strategy and adjusts the buffer capacity through the buffer capacity adjustment formula where is the buffer coefficient.

5. The distributed audio-video adaptive production system for multiple terminals according to claim 2, wherein When the cloud scheduling center adjusts task allocation, it considers the load balancing factor of processing nodes and the task correlation coefficient; where is the processing capacity already used by the i-th processing node, is the total processing capacity of the i-th processing node, and n is the number of processing nodes; the cloud scheduling center maintains a task dependency graph to record the association relationships between tasks, and calculates the task correlation coefficient by analyzing the input and output data and processing flow of tasks; for associated tasks, they are preferentially allocated to adjacent nodes or nodes with low communication costs, and the task allocation optimization formula is used to evaluate the allocation scheme.

6. The distributed audio and video adaptive production system for multiple terminals according to claim 3, characterized in that The adaptive encoding module also considers the complexity C of the audio-visual content and adjusts the adaptive encoding parameters in combination with the complexity to be , where is the complexity adjustment coefficient.

7. The distributed audio-visual adaptive production system for multiple terminals according to any one of claims 1-6, characterized in that The distributed storage cluster supports hierarchical storage of data, according to the access heat of the data , where is the number of data accesses, is the statistical time period, and calculates the data storage priority with the timeliness T of the data By storing data on storage media with different performances, where and θ are weighting coefficients.

8. The multi-terminal-oriented distributed audio and video adaptive production system according to any one of claims 1-6, characterized in that, The client provides audio and video editing functions, supporting users to perform operations such as clipping audio and video and adding subtitles. During the editing process, the system calculates the estimated formula for the operation response time based on the processing capacity of the client device and the operation complexity OC to optimize the operation process in advance, where is the response time coefficient.

9. The distributed audio-visual adaptive production system for multiple terminals according to any one of claims 1-6, characterized in that, The system uses an encryption algorithm to encrypt audio and video data during transmission and storage, and uses an encryption strength evaluation formula to evaluate the encryption security, where is the encryption key length, is the strength coefficient; during data storage, audio and video materials and production projects are encrypted and stored, and the encryption policy is dynamically adjusted in combination with the security level SL of the terminal device. The encryption adjustment formula is .

10. The multi-terminal-oriented distributed audio and video adaptive production system according to any one of claims 1-6, characterized in that The system provides open API interfaces, allowing third-party developers to integrate; through the API call success rate formula Evaluate the API performance, where is the number of successful API calls, is the total number of API calls; If the success rate of API calls is relatively low, the system will analyze the reasons, predict that the server load is too high or the interface parameters are incorrect, and optimize and adjust in a timely manner; calculate the cooperation value index according to the usage frequency UF and data interaction volume DI of third-party applications Provide differentiated services and support for different third-party applications, where and are weighting factors.

Images