Distributed audio and video collaborative production system based on cloud architecture

Abstract

The invention discloses a distributed audio and video collaborative production system based on a cloud architecture, and relates to the technical field of audio and video processing and cloud computing, the distributed audio and video collaborative production system comprises a cloud service platform, a distributed storage module, an audio and video processing node and a client, the cloud service platform is based on a micro-service architecture, containerization and Kubernetes management are used, tasks are accurately distributed by means of a resource adaptation factor formula, and the cloud service platform is used for storing the tasks. The distributed storage module adopts a Ceph system, guarantees data reliability through a data robustness formula, carries out hierarchical storage according to data value density, distributes audio and video processing nodes in multiple places, is matched with professional hardware, processes audios and videos through a high-speed network connection platform, and has a visual interface at a client. The system has obvious advantages, the cloud platform accurately distributes tasks to improve efficiency, the storage module guarantees data reliability and hierarchical storage to reduce cost, the processing node processes audio and video with high performance, the client side is convenient to operate, the real-time cooperation and intelligent auditing module guarantees collaboration and content compliance, an interface is opened to expand an application scene, and the system competitiveness is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of audio and video processing and cloud computing, and in particular to a distributed audio and video collaborative production system based on cloud architecture. Background Art

[0002] In today's digital age, the creation and dissemination of audio and video content is showing an explosive growth trend. Whether it is film and television production companies, advertising media agencies, or self-media creators, there is an urgent need for efficient and collaborative audio and video production systems. Traditional audio and video production methods and some existing production systems have gradually exposed many limitations in the face of increasingly complex production needs and large-scale data processing.

[0003] Traditional audio and video production mainly relies on local stand-alone software, which has obvious drawbacks. First, the processing power of stand-alone software is limited by local hardware resources. For audio and video production with high definition, ultra-high definition and complex special effects, performance bottlenecks often occur, resulting in slow processing speed and extended production cycle. For example, when rendering videos with 4K or even 8K resolution, stand-alone software may take hours or even days, which seriously affects production efficiency. Secondly, team collaboration is very difficult in the stand-alone production mode. Communication and collaboration between different personnel require frequent file transfers and manual merging, which is prone to version confusion, data loss and other problems, greatly reducing the team's collaborative efficiency. Moreover, the local storage method has data security risks. Once hardware failure, natural disasters or man-made damage occurs, audio and video materials and production projects may be permanently lost, causing huge losses to the producer.

[0004] With the development of cloud computing and distributed technology, some audio and video production systems based on cloud architecture have emerged. However, the existing cloud architecture audio and video production systems also have some shortcomings. In terms of data storage, although some systems use distributed storage, they lack effective data management and optimization mechanisms. The data redundancy strategy is not scientific enough, which may lead to waste of storage resources; at the same time, there is no hierarchical storage for data of different importance and access frequency, which makes the data reading and writing efficiency low and cannot meet the production scenarios with high real-time requirements. In terms of collaborative production, although the existing system supports multi-person online collaboration, the conflict resolution mechanism is not perfect when dealing with multi-user concurrent operations, which is prone to operation conflicts and data inconsistencies. Moreover, the version control function is not powerful enough to meet the version management requirements of complex projects, which increases the difficulty of project management.

[0005] In the review process, most current systems adopt manual review or simple rule - matching review methods, which are inefficient and inaccurate. For a large amount of audio - video content, manual review requires a large amount of time and labor costs, and it is easy to miss reviews and make misjudgments; simple rule - matching review methods cannot cope with complex and changing content violation forms, and it is difficult to ensure the comprehensiveness and accuracy of reviews. In addition, existing systems lack open interfaces and are difficult to integrate with third - party applications, which limits the scalability and application scenarios of the systems. Therefore, it is of great practical significance to develop an efficient, reliable, and innovative distributed audio - video collaborative production system based on cloud architecture. Summary of the Invention

[0006] The distributed audio - video collaborative production system based on cloud architecture proposed by the present invention aims to solve the problems mentioned in the above - mentioned prior art.

[0007] To achieve the above - mentioned objectives, the present invention adopts the following technical solutions: A distributed audio - video collaborative production system based on cloud architecture includes a cloud service platform, a distributed storage module, audio - video processing nodes, and clients. The cloud service platform is built based on a microservices architecture, and containerization technology is used to deploy each microservice, and Kubernetes is used for container orchestration and management. Each microservice of the cloud service platform is independent and flexibly extensible. The cloud service platform uses the resource adaptation factor formula to allocate audio - video processing tasks, where is the resource capacity of the i - th processing node, is the current task priority of the i - th processing node, n is the number of processing nodes, and the task scheduling microservice will collect the resource capacity and task priority information of each processing node in real - time; The distributed storage module adopts the distributed file system Ceph and uses the data robustness formula to ensure data reliability, where, is the number of lost data blocks, is the total number of data blocks, is the data loss rate attenuation coefficient, T is the storage duration. The storage module will regularly check the integrity of data blocks, calculate the data robustness index, and when the data robustness is lower than a certain threshold, automatically perform data repair and redundant backup. At the same time, it supports hierarchical storage of data and stores data on storage media with different performances according to the data value density , where, is the effective information amount contained in the data, is the storage space occupied by the data; The audio - video processing nodes are distributed in different geographical locations and are connected to the cloud service platform through a high - speed network. Each processing node is equipped with professional audio - video processing hardware. The GPU acceleration card is used for compute - intensive tasks, and the FPGA chip is customized for specific audio - video processing algorithms for acceleration. The processing node runs a dedicated audio - video processing software to operate on audio - video data; The client interacts with the cloud service platform through the network, providing a visual audio - video production operation interface for users. The interface is graphically designed and supports multi - user simultaneous online collaboration. Users can complete operations such as audio - video editing, adding subtitles, and adjusting sound effects through simple operations. The client also has a real - time preview function to view the production effect at any time.

[0008] Furthermore, it also includes a real - time collaboration module. This module uses the WebSocket protocol to achieve real - time communication between clients and between the client and the cloud service platform. The WebSocket protocol supports full - duplex communication, establishing a connection between the client and the server. The real - time collaboration module supports multiple people to operate on the same audio - video project simultaneously, using the operation harmony formula to evaluate the degree of coordination of operations, where is the number of synchronous operations, is the total number of operations, is the time taken for synchronous operations, is the total operation time. The real - time collaboration module will monitor the operations of each user in real - time. When it detects that multiple users are operating on the same data, it will resolve conflicts according to preset priority rules or user - defined rules; At the same time, the real - time collaboration module also has a version control function. It uses a distributed version control system similar to Git, supporting branch management and merge operations, which is convenient for users to manage different versions of audio - video projects. The version evolution fitness formula is where is the similarity between the i - th version and the expected version, m is the number of versions. The system will optimize the version management strategy according to this formula. Users create different branches for experimental modifications and merge the branches into the main version after confirming that the modifications are correct. The system will automatically record the modification history of each version.

[0009] Furthermore, it also includes an intelligent audit module. This module uses deep - learning algorithms to audit audio - video content, including content compliance and copyright detection. The intelligent audit module uses a combined model of convolutional neural network (CNN) and recurrent neural network (RNN) to analyze the image, audio, and text information of the audio - video. CNN is good at processing image information, and RNN is suitable for processing sequence information. During the audit process, the content compliance entropy formula is used to evaluate the compliance uncertainty of audio - video content, where is the probability of the i-th compliance status, k is the number of compliance status types, the review module will pre-process the audio and video data, extract image frames, audio clips and text information, and then input them into the deep learning model for analysis. For the illegal content reviewed, the system automatically marks it and provides detailed modification suggestions.

[0010] Furthermore, when the real-time collaboration module handles concurrent operations of multiple users, it adopts a conflict resolution algorithm based on the operation dependency graph. Each operation is represented as a node in the graph, and the dependency relationship between operations is represented as an edge. The operation dependency complexity formula is used. Evaluate the complexity of the dependencies between operations, where E is the number of edges in the operation dependency graph and V is the number of operation nodes. The real-time collaboration module generates an operation dependency graph based on the user's operations. When an operation conflict occurs, the system resolves the conflict based on the operation dependency graph and the operation dependency complexity. At the same time, in order to improve the real-time nature of operation feedback, the feedback timeliness index formula is used To monitor and optimize the feedback mechanism, is the maximum feedback delay time, To average the feedback delay time, the real-time collaboration module records the feedback time of each operation and calculates the feedback timeliness index. When the feedback timeliness index exceeds a certain threshold, the system automatically adjusts the processing strategy.

[0011] Furthermore, the version control function supports users to compare and merge different versions of audio and video projects. Users can intuitively view the differences between two versions through a visual interface. When merging versions, the system will automatically analyze the differences and use the difference fusion entropy formula to calculate the difference. Assess the uncertainty in the merger process, among which, is the probability of the jth difference fusion state, l is the number of difference fusion state types, the system will compare the audio and video data of the two versions, mark the differences, and then judge the conflict based on the difference fusion entropy formula. For complex conflicts, the user will be prompted to intervene manually to make the merged version meet the user's expectations.

[0012] Furthermore, the intelligent review module supports user-defined review rules. Users can set different review indicators and weights according to their own business needs and review standards. The system will review the audio and video content according to the rules set by the user, and use the rule effectiveness ratio formula To evaluate the effectiveness of custom rules, is the number of violations correctly detected, Let \(N\) be the number of detected violations. The user sets the review rules on the client interface and assigns corresponding weights to each review metric. The intelligent review module will review the audio and video according to the rules set by the user, calculate the rule effectiveness ratio, and when the rule effectiveness ratio is lower than a certain standard, the system will prompt the user to adjust the rules.

[0013] Furthermore, the cloud service platform uses a load balancing algorithm to allocate audio and video processing tasks. The load balancing algorithm comprehensively considers the hardware resource utilization rate, network bandwidth, and current task load factors of each audio and video processing node. In addition to using the resource adaptation factor formula, it also combines the task elasticity coefficient to allocate elastic tasks, where is the time range for task adjustment, is the estimated processing time of the task. The task scheduling microservice of the cloud service platform will collect the hardware resource utilization rate, network bandwidth, and current task load information of each processing node in real time, and calculate the task elasticity coefficient according to the urgency of the task and the adjustable time range. For tasks with a high elasticity coefficient, the system will preferentially allocate them to nodes with relatively idle resources; At the same time, the cloud service platform monitors the running status of the processing nodes in real time. When a node fails or is overloaded, it will promptly migrate the task to other nodes and use the task migration smoothness formula to evaluate the migration effect, where is the actual task migration time, is the estimated task migration time. The cloud service platform will regularly check the running status of each processing node. When a node failure or overload is detected, it will immediately start the task migration mechanism. During the task migration process, the system will record the actual migration time and the estimated migration time, calculate the task migration smoothness, and if the migration smoothness is low, the system will analyze the reason and optimize the migration strategy.

[0014] Furthermore, the distributed storage module supports hierarchical storage of data. According to the access frequency and importance of the data, the data is stored on different levels of storage media. The system will automatically monitor the access pattern of the data and use the data cold-hot conversion rate formula to optimize the data migration process, where is the number of accesses from hot data to cold data, is the number of accesses from cold data to hot data, is the total number of accesses. The distributed storage module will record the access history of each data block, classify the data into hot data and cold data according to the access frequency and importance, and regularly calculate the data cold-hot conversion rate.

[0015] Furthermore, the client and the cloud service platform use an encrypted communication protocol for data transmission. When the client uploads and downloads audio and video data, the data will be encrypted. The client and the cloud service platform will perform identity authentication and key exchange when establishing a connection, and use the SSL / TLS protocol to encrypt the transmitted data. At the same time, the client also has a local data cache function. When the network is unstable, the user continues to perform some offline operations and uses the cache intelligent hit rate formula To evaluate the cache effect, is the number of requests that hit the cache, is the total number of requests, The amount of data requested for cache hits, The client caches commonly used audio and video data and operation records locally for the total requested data volume. When a user initiates a request, the local cache is checked first and the cache intelligent hit rate is calculated. When the hit rate is low, it means that the cache strategy needs to be optimized. The client adjusts the cache strategy. After the network is restored, the system automatically synchronizes local and cloud data.

[0016] Furthermore, the system provides an open API interface, allowing third-party applications to integrate with the system. Through the API interface, third-party applications can upload, download, manage projects, and query audit results of audio and video materials. The system uses the API ecological integration formula To monitor the usage of the API, is the number of third-party applications that interact with the system. is the total number of third-party applications connected. is the amount of data interacted with, For the total amount of data processed by the system, the system will provide API documentation and development examples to facilitate developers to carry out secondary development. Third-party developers can develop their own applications based on the API documentation to achieve integration with the distributed audio and video collaborative production system. The system will record the interaction of third-party applications and calculate the degree of API ecological integration.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The system has outstanding advantages in terms of resource utilization and processing efficiency. The cloud service platform accurately allocates audio and video processing tasks through the resource adaptation factor formula, fully considering the resource capacity and task priority of the processing node, so that resources are reasonably utilized and resource waste and idleness are avoided. The distributed storage module adopts the Ceph distributed file system, combined with the data robustness formula to ensure data reliability, and at the same time performs hierarchical storage based on data value density, storing frequently accessed data on high-speed storage media, improving the data reading and writing speed, greatly shortening the audio and video processing time, and improving production efficiency. Multiple audio and video processing nodes are distributed in different geographical locations and are equipped with professional audio and video processing hardware, such as GPU accelerator cards and FPGA chips, which can process large amounts of audio and video data in parallel, further improving the system's processing capabilities.

[0018] In terms of collaborative production, the real-time collaboration module uses the WebSocket protocol to achieve real-time communication and supports multiple people to operate the same audio and video project at the same time. The lock mechanism and the operation harmony formula ensure the consistency of data and the coordination of operations, effectively solving the conflict problem when multiple users operate concurrently. The version control function uses a distributed version control system similar to Git, combined with the version evolution fit formula to optimize the version management strategy, making it easier for users to manage different versions of audio and video projects, avoiding version confusion and improving the efficiency of team collaboration.

[0019] In the audit phase, the intelligent audit module uses deep learning algorithms to conduct multi-dimensional analysis of audio and video content, and combines the content compliance entropy formula to evaluate compliance uncertainty, which can quickly and accurately identify illegal content and provide detailed modification suggestions. It supports user-defined audit rules and evaluates the effectiveness of rules through the rule effectiveness ratio formula, which improves the flexibility and accuracy of audits and saves a lot of audit time and labor costs.

[0020] The system also has good scalability and openness. It provides an open API interface, allowing third-party applications to integrate with the system, and monitors API usage through the API ecological integration formula, expanding the system's application scenarios and ecosystem. The client and the cloud service platform use an encrypted communication protocol for data transmission to ensure data security and integrity. At the same time, the client's local cache function and cache intelligent hit rate formula optimize the cache strategy. Even in the case of unstable network, users can continue to perform some offline operations, and automatically synchronize data after the network is restored, improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic block diagram of a distributed audio and video collaborative production system based on cloud architecture proposed by the present invention; DETAILED DESCRIPTION

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0023] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present invention.

[0024] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality" means two or more, unless otherwise specifically defined. In addition, the terms "installation", "connection" and "connection" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations. The present invention will be further described in detail below with reference to the accompanying drawings.

[0025] Refer to Figure 1 : A distributed audio-video collaborative production system based on a cloud architecture, including a cloud service platform, a distributed storage module, multiple audio-video processing nodes and a client. The cloud service platform is built based on a microservices architecture, deploys each microservice using containerization technology (such as Docker), and performs container orchestration and management through Kubernetes. Each microservice of the cloud service platform is independent of each other and can be flexibly expanded. For example, it includes a task scheduling microservice, a resource management microservice, a user authentication microservice, etc. The cloud service platform uses the resource adaptation factor formula (where is the resource capacity of the i-th processing node, (where \(P_i\) is the current task priority of the \(i\)-th processing node and \(n\) is the number of processing nodes)to accurately allocate audio-visual processing tasks. The task scheduling microservice will collect the resource capacity and task priority information of each processing node in real time, calculate the resource adaptation factor of each processing node according to this formula, and allocate the audio-visual processing tasks to the node with the optimal adaptation factor, so that the resource allocation is adapted to the task priority.

[0026] The distributed storage module uses the distributed file system Ceph, which has the characteristics of high scalability, high performance and high reliability. The data robustness formula is used ( where \(x\) is the number of lost data blocks, \(y\) is the total number of data blocks, \(\alpha\) is the data loss rate attenuation coefficient, and \(T\) is the storage duration)to ensure data reliability. The storage module will regularly check the integrity of the data blocks, calculate the data robustness index, and automatically perform data repair and redundant backup when the data robustness is lower than a certain threshold. At the same time, it supports hierarchical storage of data. According to the data value density ( where \(I\) is the effective information volume contained in the data, \(S\) is the storage space occupied by the data)the data is stored on storage media with different performances. For example, data with high data value density and frequent access is stored on high-speed SSDs; data with low data value density and infrequent access is stored on large-capacity HDDs.

[0027] Multiple audio-visual processing nodes are dispersed in different geographical locations. This layout is not random, but based on the consideration of optimizing resource utilization and improving processing efficiency. The network conditions, power supply and hardware resource reserves in different regions are different. The distributed setting allows each node to make full use of local advantages and avoid the problem of excessive local load caused by centralized deployment. These widely distributed processing nodes are connected to the cloud service platform through a high-speed network. This high-speed network uses advanced optical fiber communication technology and efficient network transmission protocols to ensure low latency and high bandwidth for data transmission between the nodes and the cloud service platform. This enables audio-visual data to flow quickly between the nodes and the platform, ensuring the coherence of processing tasks.

[0028] Each processing node is equipped with professional audio and video processing hardware. Take the GPU acceleration card as an example. It has a large number of parallel computing cores and is designed specifically for processing large-scale data parallel operations. During audio and video encoding, the GPU acceleration card can quickly convert the original audio and video data into a format suitable for storage and transmission according to different encoding standards, such as H.264, H.265, etc., and at the same time intelligently adjust the parameters, taking into account both the file size and the video quality. During decoding, it can quickly parse the encoded data and restore high-quality audio and video signals, making the playback smooth without stuttering. In terms of special effects processing, the GPU acceleration card performs real-time rendering calculations on special effects such as lighting, particles, and 3D transitions, greatly shortening the processing time. The FPGA chip stands out with its field-programmable characteristics. R & D personnel can program it using hardware description languages, customize the internal logic circuit for specific audio and video processing algorithms, such as video format conversion, image noise reduction, etc., so as to achieve efficient and accurate task processing and reduce system resource occupancy.

[0029] The processing node runs a specially developed audio and video processing software. This software is rich and practical in functions. The encoding and decoding functions support multiple formats and parameter settings to meet the needs of different application scenarios. The editing function allows users to accurately select and splice video clips frame by frame on the timeline, easily creating smooth video content. The special effects adding function has a rich special effects library built in, covering color correction, blur, image distortion, etc. Users can easily add special effects to audio and video according to their creativity.

[0030] As an important window connecting users and the cloud service platform, the client relies on a stable and high-speed network to build a convenient interaction channel, presenting users with an intuitive and efficient visual audio and video production operation interface. This interface uses a simple and intuitive graphical design, with a regular and reasonable layout, and each function area is clearly visible. The prominent menu bar orderly houses various basic operation options, facilitating users to quickly call them; the tool panel displays common tools with clear and easy-to-understand icons, and detailed function descriptions can be viewed by hovering the mouse. It supports multiple users to collaborate online simultaneously, breaking through geographical restrictions. Users in different regions only need to log in to the same project, and the system will synchronize operations in real time to ensure smooth collaborative creation. For example, when one person edits a clip, others can immediately see the editing result, and the same is true for operations such as adding subtitles and adjusting sound effects, greatly improving the creation efficiency.

[0031] Operationally, users can complete complex production through simple drag-and-drop and clicks. For example, dragging materials from the local area into the editing area and clicking the editing button to accurately locate the editing points on the timeline; clicking the subtitle button, entering text and setting styles in the pop-up window to easily add subtitles; clicking the sound effect button and adjusting the volume and sound effect type through sliders and checkboxes. At the same time, the real-time preview function of the client is very practical. Users can click the preview at any time to view the production effect, and it also supports switching between multiple resolutions and picture quality modes, helping users evaluate the works from different angles and making the audio-visual production process smoother and more efficient.

[0032] In the present invention, there is also a real-time collaboration module, which uses the WebSocket protocol to achieve real-time communication between clients and between the client and the cloud service platform. The WebSocket protocol supports full-duplex communication and can establish a real-time and efficient connection between the client and the server. The real-time collaboration module supports multiple people to operate on the same audio-visual project simultaneously, and the system ensures data consistency through a lock mechanism. When multiple users operate on the same data, the operation harmony formula ( is the number of synchronous operations, is the total number of operations, is the time taken for synchronous operations, is the total operation time) is used to evaluate the degree of cooperation of operations. The real-time collaboration module will monitor the operations of each user in real time. When it detects that multiple users are operating on the same data, it will resolve conflicts according to preset priority rules or user-defined rules. For example, if user A and user B simultaneously attempt to modify the same audio-visual clip, the system will, according to the user-set priority, give priority to processing the operations of the high-priority user and feedback the result to other users in real time.

[0033] At the same time, the real-time collaboration module also has a version control function. It adopts a distributed version control system similar to Git, providing users with powerful and flexible version management capabilities. This distributed feature enables each collaborating user to have a complete version of the project data locally. Even in the case of unstable network, version-related operations can still be continued.

[0034] Supporting branch management and merge operations is a major highlight of this function. Users can create multiple branches according to different needs. For example, when conducting creative exploration, create an experimental branch to make various bold modifications, add new special effects, adjust the editing order, etc. Each branch independently records the modification content without interference. After confirming that the experimental modifications meet the expected effects, the branch can be merged into the main version. The version evolution fit formula ( is the similarity between the ith version and the expected version, and m is the number of versions) plays a key role in this. Based on this formula, the system will accurately analyze the similarity between each version and the expected version, thereby optimizing the version management strategy. This means that the system can intelligently filter out the version path that best meets user needs and reduce unnecessary version redundancy. The system will also automatically record the modification history of each version in detail, including modification time, modification personnel, specific modification content and other information. This recording function provides great convenience for users. If users find problems with the current version in subsequent production, or want to review the effects of a previous creative stage, they can easily roll back to the previous version at any time, making the production process of audio and video projects safer and more controllable.

[0035] The present invention also includes an intelligent audit module, which uses a deep learning algorithm to audit audio and video content, including content compliance, copyright detection and other aspects. The intelligent audit module uses a combination model of convolutional neural network (CNN) and recurrent neural network (RNN) to perform multi-dimensional analysis of the image, audio and text information of the audio and video. CNN is good at processing image information and can identify people, scenes, objects, etc. in audio and video; RNN is suitable for processing sequence information, such as voice content in audio and subtitle text in video. During the audit process, the content compliance entropy formula is used ( The audit module pre-processes the audio and video data, extracts image frames, audio clips and text information, and then inputs them into the deep learning model for analysis. For the illegal content found in the audit, the system automatically marks it and provides detailed modification suggestions, while recording the audit history for subsequent tracing and management. For example, if sensitive words or illegal images are detected in the audio or video, the system will mark the specific time point and content, and suggest that the user delete or replace them.

[0036] In the present invention, the real-time collaboration module uses a conflict resolution algorithm based on the operation dependency graph when processing multi-user concurrent operations. Each operation is represented as a node in the graph, and the dependency relationship between operations is represented as an edge. The operation dependency complexity formula is used (E is the number of edges in the operation dependency graph, and V is the number of operation nodes) to evaluate the complexity of the dependencies between operations. The real-time collaboration module generates an operation dependency graph based on the user's operations. When an operation conflict occurs, the system resolves the conflict based on the operation dependency graph and the operation dependency complexity. For example, if operation A depends on operation B, and another user tries to modify operation B at the same time, the system will determine whether to allow the modification based on the operation dependency relationship and complexity, and perform corresponding processing.

[0037] Meanwhile, to improve the real-time nature of operation feedback, the feedback timeliness index formula is used ( is the maximum feedback delay time, is the average feedback delay time) to monitor and optimize the feedback mechanism. The real-time collaboration module will record the feedback time of each operation and calculate the feedback timeliness index. When the feedback timeliness index exceeds a certain threshold, the system will automatically adjust the processing strategy, such as increasing server resources, optimizing network transmission, etc., to ensure that users can obtain operation result feedback in a timely manner.

[0038] In the present invention, the version control function supports users to compare and merge different versions of audio and video projects. Users can visually view the differences between two versions through a visual interface, including the addition and deletion of audio and video segments, the modification of special effects, the adjustment of the timeline, etc. When performing version merging, the system will automatically analyze the different content and use the difference fusion entropy formula ( is the probability of the jth difference fusion state, and l is the number of difference fusion state types) to evaluate the uncertainty during the merging process. The system will compare the audio and video data of the two versions, mark the differences, and then judge the difficulty of merging and possible conflicts according to the difference fusion entropy formula. For complex conflicts, it will prompt the user for manual intervention to ensure that the merged version meets the user's expectations. For example, if two versions perform different editing operations on the same audio and video segment, the system will prompt the user to select a suitable editing plan for merging.

[0039] In the present invention, the intelligent review module supports users to customize review rules. Users can set different review indicators and weights according to their own business requirements and review criteria. The system will accurately review the audio and video content according to the rules set by the users, and at the same time use the rule effectiveness ratio formula ( is the number of detected violations correctly, is the number of detected violations wrongly) to evaluate the effectiveness of the customized rules. Users can set review rules on the client interface, such as setting a list of sensitive words, types of violation pictures, etc., and assign corresponding weights to each review indicator. The intelligent review module will review the audio and video according to the rules set by the users and calculate the rule effectiveness ratio. When the rule effectiveness ratio is lower than a certain standard, the system will prompt the user to adjust the rules to improve the accuracy of the review.

[0040] In the present invention, the cloud service platform uses a load balancing algorithm to allocate audio and video processing tasks. The load balancing algorithm comprehensively considers factors such as the hardware resource utilization rate, network bandwidth, and current task load of each audio and video processing node. In addition to using the resource adaptation factor formula, it also combines the task elasticity coefficient ( is the flexible adjustable time range for the task, i.e., the estimated processing time of the task) to preferentially allocate flexible tasks. The task scheduling microservice of the cloud service platform will collect information such as the hardware resource utilization rate, network bandwidth, and current task load of each processing node in real time, and calculate the task elasticity coefficient according to the urgency and adjustable time range of the task. For tasks with a high elasticity coefficient, the system will preferentially allocate them to nodes with relatively idle resources to make full use of system resources.

[0041] At the same time, the cloud service platform monitors the running status of the processing nodes in real time. When a node fails or has too high a load, it can timely migrate the task to other nodes and use the task migration smoothness formula where is the actual task migration time, is the estimated task migration time) to evaluate the migration effect. The cloud service platform will regularly check the running status of each processing node. When a node failure or too high a load is detected, it will immediately start the task migration mechanism. During the task migration process, the system will record the actual migration time and the estimated migration time, and calculate the task migration smoothness. If the migration smoothness is low, the system will analyze the reasons and optimize the migration strategy to ensure the efficiency and stability of the task migration.

[0042] In the present invention, the distributed storage module supports hierarchical storage of data. According to the access frequency and importance of the data, the data is stored on different levels of storage media, such as high-speed SSDs, large-capacity HDDs, etc. The system will automatically monitor the access pattern of the data and use the data hot-cold conversion rate formula where is the number of accesses from hot data to cold data, is the number of accesses from cold data to hot data, is the total number of accesses) to optimize the data migration process. The distributed storage module will record the access history of each data block, and classify the data into hot data and cold data according to the access frequency and importance. Regularly calculate the data hot-cold conversion rate. When the data hot-cold conversion rate is high, it indicates that the access pattern of the data has changed greatly. The system will migrate the data between different storage media according to the new access pattern, migrate the frequently accessed data to the high-speed storage medium to improve the read and write speed of the data, and at the same time migrate the infrequently used data to the low-cost storage medium to reduce the storage cost.

[0043] In the present invention, encrypted communication protocols (such as SSL / TLS) are used for data transmission between the client and the cloud service platform. When the client uploads and downloads audio and video data, the data will be encrypted to ensure the security and integrity of the data during transmission. The client and the cloud service platform will perform identity authentication and key exchange when establishing a connection, and use the SSL / TLS protocol to encrypt the transmitted data. At the same time, the client also has a local data caching function, so when the network is unstable, the user can continue to perform some offline operations. Use the cache intelligent hit rate formula ( is the number of requests that hit the cache, is the total number of requests, The amount of data requested for cache hits, The client caches commonly used audio and video data and operation records locally. When a user initiates a request, the local cache is checked first. The cache intelligent hit rate is calculated. When the hit rate is low, it means that the cache strategy needs to be optimized. The client will adjust the cache strategy, such as increasing the cache capacity and updating the cache data. After the network is restored, the system automatically synchronizes the local and cloud data to ensure data consistency.

[0044] In the present invention, the system provides an open API interface, allowing third-party applications to be integrated with the system. Through the API interface, third-party applications can realize functions such as uploading, downloading, project management, and audit result query of audio and video materials. The system uses the API ecological integration degree formula ( is the number of third-party applications that interact with the system. is the total number of third-party applications connected. is the amount of data interacted with, The system monitors the usage of the API by using the API (total amount of data processed by the system). The system provides detailed API documentation and development examples to facilitate secondary development by developers. Third-party developers can develop their own applications based on the API documentation to achieve integration with the distributed audio and video collaborative production system. The system records the interaction of third-party applications and calculates the degree of API ecological integration. When the degree of API ecological integration is low, it means that the integration between the third-party application and the system is not good. The system will analyze the reasons and provide corresponding optimization suggestions, expanding the application scenarios and ecosystem of the system.

[0045] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A distributed audio - video collaborative production system based on a cloud architecture, characterized in that, It includes a cloud service platform, a distributed storage module, audio and video processing nodes, and clients. The cloud service platform is built based on a microservices architecture, deploys each microservice using containerization technology, and performs container orchestration and management through Kubernetes. Each microservice of the cloud service platform is independent and flexibly scalable. The cloud service platform uses the resource adaptation factor formula to allocate audio and video processing tasks, where is the resource capacity of the i-th processing node, is the current task priority of the i-th processing node, n is the number of processing nodes, and the task scheduling microservice will collect the resource capacity and task priority information of each processing node in real time; Distributed storage module: Adopt the distributed file system Ceph and apply the data robustness formula to ensure data reliability. Among them, is the number of lost data blocks, is the total number of data blocks, is the data loss rate attenuation coefficient, T is the storage duration. The storage module will regularly check the integrity of data blocks, calculate the data robustness index, and at the same time support hierarchical storage of data. According to the data value density store data on storage media with different performances. Among them, is the effective information amount contained in the data, is the storage space occupied by the data; Audio and video processing nodes: Distributed in different geographical locations and connected to the cloud service platform through high-speed networks. Each processing node is equipped with professional audio and video processing hardware. The GPU acceleration card is used for compute-intensive tasks, and the FPGA chip is customized for accelerating audio and video processing algorithms. The processing nodes run audio and video processing software to operate on audio and video data; Client: Interacts with the cloud service platform through the network, providing a visual operation interface for audio and video production for users. The interface is designed graphically, supports multi-user online collaboration simultaneously. Users can complete operations such as audio and video editing, adding subtitles, and adjusting sound effects through simple operations. The client also has a real-time preview function to view the production effects at any time.

2. The distributed audio and video collaborative production system based on a cloud architecture according to claim 1, wherein It also includes a real-time collaboration module. This module uses the WebSocket protocol to achieve real-time communication between clients and between the client and the cloud service platform. The WebSocket protocol supports full-duplex communication and establishes a connection between the client and the server. The real-time collaboration module supports multiple people to operate on the same audio-visual project simultaneously and uses the operation harmony formula to evaluate the degree of cooperation of the operations, where is the number of synchronous operations, is the total number of operations, is the time taken for synchronous operations, is the total operation time. The real-time collaboration module will monitor the operations of each user in real time. When it detects that multiple users are operating on the same data, it will resolve conflicts according to the preset priority rules or user-defined rules; Meanwhile, the real-time collaboration module also has a version control function. It adopts the distributed version control system of Git, supports branch management and merge operations, which facilitates users to manage different versions of audio-visual projects. The version evolution fitness formula is , where is the similarity between the i-th version and the expected version, and m is the number of versions. The system will optimize the version management strategy according to this formula. Users create different branches for experimental modifications and merge the branches into the main version after confirming that the modifications are correct. The system will automatically record the modification history of each version.

3. The distributed audio and video collaborative production system based on a cloud architecture according to claim 1, wherein, It also includes an intelligent audit module, which uses deep learning algorithms to audit audio and video content, including content compliance and copyright detection. The intelligent audit module uses a combination model of convolutional neural network CNN and recurrent neural network RNN to analyze the image, audio and text information of audio and video. CNN is good at processing image information, while RNN is suitable for processing sequence information. In the audit process, the content compliance entropy formula is used Assess compliance uncertainty of audio and video content, including: is the probability of the i-th compliance status, k is the number of compliance status types, the review module will pre-process the audio and video data, extract image frames, audio clips and text information, and then input them into the deep learning model for analysis. For the illegal content found, the system will automatically mark it and provide modification suggestions.

4. The distributed audio and video collaborative production system based on a cloud architecture according to claim 2, wherein When the real-time collaboration module processes multi-user concurrent operations, it adopts a conflict resolution algorithm based on an operation dependency graph. Each operation is represented as a node in the graph, and the dependency relationship between operations is represented as an edge. The operation dependency complexity formula is used to evaluate the dependency complexity between operations. Among them, E is the number of edges in the operation dependency graph, and V is the number of operation nodes. The real-time collaboration module will generate an operation dependency graph based on the user's operations. When an operation conflict occurs, the system will resolve the conflict according to the operation dependency graph and the operation dependency complexity; at the same time, the feedback timeliness index formula is used to monitor and optimize the feedback mechanism, where is the maximum feedback delay time, is the average feedback delay time. The real-time collaboration module will record the feedback time of each operation and calculate the feedback timeliness index.

5. The distributed audio and video collaborative production system based on a cloud architecture according to claim 2, wherein The version control function enables users to compare and merge different versions of audio and video projects. Users can visually view the differences between two versions through a graphical interface. When merging versions, the system automatically analyzes the different content and uses the differential fusion entropy formula to evaluate the uncertainty during the merging process. Here, is the probability of the j-th differential fusion state, and l is the number of differential fusion state types. The system will compare the audio and video data of the two versions, mark the differences, and then determine conflicts based on the differential fusion entropy formula. For complex conflicts, it will prompt the user for manual intervention to ensure that the merged version meets the user's expectations.

6. The distributed audio and video collaborative production system based on a cloud architecture according to claim 3, characterized in that, The intelligent review module supports users to customize review rules. Users can set different review metrics and weights according to their own business requirements and review criteria. The system will review the audio-visual content according to the rules set by the users, and at the same time use the rule effectiveness ratio formula to evaluate the effectiveness of the customized rules, is the number of detected violations, is the number of misdetected violations. Users set review rules on the client interface and assign corresponding weights to each review metric. The intelligent review module will review the audio-visual according to the rules set by the users and calculate the rule effectiveness ratio.

7. The distributed audio and video collaborative production system based on a cloud architecture according to any one of claims 1-6, characterized in that The cloud service platform uses a load balancing algorithm to allocate audio and video processing tasks. The load balancing algorithm comprehensively considers the hardware resource utilization rate, network bandwidth, and current task load factors of each audio and video processing node. In addition to using the resource adaptation factor formula, it also combines the task elasticity coefficient to allocate elastic tasks, where is the time range for task adjustment, is the estimated task processing time. The task scheduling microservice of the cloud service platform will collect the hardware resource utilization rate, network bandwidth, and current task load information of each processing node in real time, and calculate the task elasticity coefficient according to the urgency and adjustable time range of the task; Meanwhile, the cloud service platform monitors the running status of processing nodes in real time. When a certain node fails or has too high a load, it timely migrates tasks to other nodes and uses the task migration smoothness formula to evaluate the migration effect, where is the actual task migration time, is the estimated task migration time. The cloud service platform will regularly check the running status of each processing node. During the task migration process, the system will record the actual migration time and the estimated migration time and calculate the task migration smoothness.

8. The distributed audio and video collaborative production system based on a cloud architecture according to any one of claims 1-6, characterized in that The distributed storage module supports hierarchical storage of data. According to the access frequency and importance of the data, the data is stored on storage media at different levels. The system will automatically monitor the access patterns of the data and use the data hot-cold conversion rate formula to optimize the data migration process. Among them, is the number of accesses from hot data to cold data, is the number of accesses from cold data to hot data, is the total number of accesses. The distributed storage module will record the access history of each data block, classify the data into hot data and cold data according to the access frequency and importance, and regularly calculate the data hot-cold conversion rate.

9. The distributed audio and video collaborative production system based on a cloud architecture according to any one of claims 1-6, characterized in that The client and the cloud service platform use an encrypted communication protocol for data transmission. When the client uploads and downloads audio and video data, the data will be encrypted. The client and the cloud service platform will perform identity authentication and key exchange when establishing a connection, and use the SSL / TLS protocol to encrypt the transmitted data. At the same time, the client also has a local data cache function. When the network is unstable, the user continues to perform some offline operations and uses the cache intelligent hit rate formula To evaluate the cache effect, is the number of requests that hit the cache, is the total number of requests, The amount of data requested for cache hits, For the total requested data volume, the client will cache commonly used audio and video data and operation records locally. When the user initiates a request, it will first check the local cache, calculate the cache intelligent hit rate, and adjust the cache strategy as needed.

10. The distributed audio and video collaborative production system based on a cloud architecture according to any one of claims 1-6, characterized in that, The system provides open API interfaces, allowing third-party applications to integrate with the system. Through the API interfaces, third-party applications can implement functions such as uploading and downloading audio and video materials, project management, and querying audit results. The system uses the API ecological integration formula to monitor the usage of the API. Among them, is the number of third-party applications interacting with the system, is the total number of third-party applications connected, is the amount of data interacted, is the total amount of data processed by the system. The system will provide API documents and development examples to facilitate developers' secondary development. Third-party developers can develop their own applications according to the API documents to achieve integration with the distributed audio and video collaborative production system. The system will record the interaction situations of third-party applications and calculate the API ecological integration degree.

Images