Panoramic audio-visual synchronization method and synchronization system based on SLB

Abstract

The present invention relates to a panoramic audio-visual synchronization method based on SLB, and the method includes: obtaining the sound field information of each target playback device, constructing a distributed edge node transmission model and a sound field propagation model through SLB and Polar code channel control; according to the distributed edge node transmission model and the heuristic distributed transmission method, determining multiple edge nodes and realizing data synchronization of multiple edge nodes, and adaptively adjusting the resource utilization and audio and video transmission quality of each edge node; through a multi-task neural network, fusing the distributed edge node transmission model and the sound field propagation model and training the multi-task neural network; based on the trained multi-task neural network, dynamically adjusting the resource utilization and audio and video transmission quality of each edge node to realize the spatiotemporal synchronization of each edge node. The present invention can realize panoramic audio-visual synchronization through SLB, Polar code channel control mechanism and neural network.

Description

Technical Field

[0001] The present invention belongs to the technical field of panoramic audio-visual synchronization, and in particular relates to a panoramic audio-visual synchronization method and a synchronization system based on SLB. Background Art

[0002] At present, wireless communication begins to gradually replace physical connection methods (such as wires, cables, etc.) to establish communication between two devices. However, wireless communication can also introduce additional delays or lags associated with encoding, transmitting, receiving and decoding wireless packets and reconstructing original content. When different playback devices are used to present audio and video content separately, such delays may interfere with the presentation of audiovisual content and cause perceptible loss of synchronization, and wireless communication is used to transmit audio content and / or video content to appropriate playback devices. The inconsistent combination and asynchrony of audiovisual content lead to a negative audience experience. Therefore, maintaining synchronization between audio content and video content in a way that minimizes the impact on user experience has become a technical problem that needs to be solved urgently.

[0003] In the field of modern communications and multimedia processing, distributed edge computing and adaptive video streaming technologies are gaining increasing attention. With the widespread use of Internet of Things (IoT) devices and edge computing nodes, how to efficiently synchronize and transmit data between these distributed nodes has become a key challenge. Especially in the scenario of multi-target audio and video data transmission and processing, it is particularly important to ensure efficient data transmission, low latency and high reliability. Summary of the invention

[0004] In order to solve the problem of asynchrony or delay in playing audio and video on multiple devices and improve the user's playing experience, a panoramic audio-visual synchronization method based on SLB is provided in a first aspect of the present invention, comprising:

[0005] Acquire the sound field information of each target playback device, and build a distributed edge node transmission model and sound field propagation model through SLB and Polar code channel control;

[0006] According to the distributed edge node transmission model and the heuristic distributed transmission method, multiple edge nodes are determined and data synchronization of the multiple edge nodes is achieved, and resource utilization and audio and video transmission quality of each edge node are adaptively adjusted;

[0007] By means of a multi-task neural network, the distributed edge node transmission model and the sound field propagation model are integrated and the multi-task neural network is trained;

[0008] Based on the trained multi-task neural network, the resource utilization and audio and video transmission quality of each edge node are dynamically adjusted to achieve spatiotemporal synchronization of each edge node.

[0009] In some embodiments of the present invention, the sound field information of each target playback device is obtained, and a distributed edge node transmission model and a sound field propagation model are constructed through SLB and Polar code channel control, including: obtaining the position, environmental information and acoustic information of each target playback device, and constructing a sound field model according to the position, environmental information and acoustic information of each target playback device; generating identification information and routing information of each target playback device based on the access layer of SLB and Polar code channel control; and constructing a distributed edge node transmission model based on the identification information and routing information of each target playback device.

[0010] In some embodiments of the present invention, the distributed edge node transmission model is expressed as:

[0011]

[0012] Among them, i and j represent the node ordinal and routing ordinal respectively, P ij represents path selection, D ij Indicates the amount of data, T ij represents the transmission time, Q ij represents the channel quality, a, β, and γ represent the weight parameters of data volume, transmission time, and channel quality, respectively. i (t) represents the resource allocation strategy, W j (t) represents the node weight, L j (t) represents the node load.

[0013] In some embodiments of the present invention, determining multiple edge nodes and achieving data synchronization of the multiple edge nodes according to the distributed edge node transmission model and the heuristic distributed transmission method, and adaptively adjusting the resource utilization and audio and video transmission quality of each edge node include: based on the distributed edge node transmission model, determining that the multiple edge nodes exchange initial state information and form an initial state table, the initial state information including the node's bandwidth, computing power and storage status; determining the transmission route of each edge node based on the heuristic distributed transmission method, and adaptively adjusting the resource utilization and audio and video transmission quality.

[0014] Furthermore, determining the transmission route of each edge node based on the heuristic distributed transmission method and adaptively adjusting the resource utilization and the transmission quality of audio and video includes: predicting the bandwidth of each edge node based on the heuristic distributed transmission method, and dynamically adjusting the transmission rate and transmission route of each edge node according to the prediction result; adaptively adjusting the resource utilization and the transmission quality of audio and video based on the current transmission rate, transmission route and resource utilization of each edge node.

[0015] In some embodiments of the present invention, the multi-task neural network includes a first task neural network, a second task neural network and a shared layer; the first task neural network is used to extract node features from the distributed edge node transmission model; the second task neural network is used to extract sound field features from the sound field propagation model; the shared layer is used to share node features and sound field features with the first task neural network and the second task neural network.

[0016] A second aspect of the present invention provides a panoramic audio-visual synchronization system based on SLB, comprising:

[0017] The acquisition module is used to obtain the sound field information of each target playback device, and build a distributed edge node transmission model and sound field propagation model through SLB and Polar code channel control;

[0018] An adjustment module, configured to determine a plurality of edge nodes and implement data synchronization of the plurality of edge nodes according to the distributed edge node transmission model and the heuristic distributed transmission method, and adaptively adjust resource utilization and audio and video transmission quality of each edge node;

[0019] A fusion module, used to fuse the distributed edge node transmission model and the sound field propagation model through a multi-task neural network and train the multi-task neural network;

[0020] The dynamic adjustment module is used to dynamically adjust the resource utilization and audio and video transmission quality of each edge node based on the trained multi-task neural network to achieve time and space synchronization of each edge node.

[0021] According to a third aspect of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the SLB-based panoramic audio-visual synchronization method provided in the first aspect of the present invention.

[0022] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the SLB-based panoramic audio-visual synchronization method provided in the first aspect of the present invention is implemented.

[0023] The beneficial effects of the present invention are:

[0024] The present invention combines the optimization scheme of Fully Distributed Edge Synchronization, Star Flash SLB and SLE, and Polar Code channel control mechanism to improve the transmission efficiency and reliability of multicast adaptive video streaming. This scheme not only considers the basic requirements of data synchronization and transmission, but also introduces multimodal data fusion (audio and video) and adaptive weight mechanism to further optimize the transmission path and resource allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a basic flow chart of a panoramic audio-visual synchronization method based on SLB in some embodiments of the present invention;

[0026] Figure 2 It is a specific flowchart of a panoramic audio-visual synchronization method based on SLB in some embodiments of the present invention;

[0027] Figure 3 It is a structural schematic diagram of a panoramic audio-visual synchronization system based on SLB in some embodiments of the present invention;

[0028] Figure 4 It is a schematic diagram of the structure of an electronic device in some embodiments of the present invention. DETAILED DESCRIPTION

[0029] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0030] refer to Figure 1 and Figure 2 In a first aspect of the present invention, a panoramic audio-visual synchronization method based on SLB is provided, comprising:

[0031] S100. Obtain the sound field information of each target playback device, and build a distributed edge node transmission model and sound field propagation model through SLB and Polar code channel control;

[0032] S200. According to the distributed edge node transmission model and the heuristic distributed transmission method, multiple edge nodes are determined and data synchronization of the multiple edge nodes is achieved, and resource utilization and audio and video transmission quality of each edge node are adaptively adjusted;

[0033] S300. Through a multi-task neural network, the distributed edge node transmission model and the sound field propagation model are integrated and the multi-task neural network is trained;

[0034] S400. Based on the trained multi-task neural network, dynamically adjust the resource utilization and audio and video transmission quality of each edge node to achieve time and space synchronization of each edge node.

[0035] In step S100 of some embodiments of the present invention, obtaining the sound field information of each target playback device and constructing a distributed edge node transmission model and a sound field propagation model through SLB and Polar code channel control include:

[0036] S101. Obtaining the position, environmental information and acoustic information of each target playback device, and constructing a sound field model according to the position, environmental information and acoustic information of each target playback device;

[0037] Specifically, the sound field of each node is modeled, and the location, environment and sound source characteristics of the node are considered to establish a sound field propagation model.

[0038] The sound field propagation model is expressed as: S i =f(x i ,y i ,z i ,E i ),

[0039] Among them, S i represents the sound field of the ith node, x i ,y i ,z i , represents the location of the node, and Ei represents the environmental characteristics.

[0040] Then, estimate the delay D of sound propagation between different nodes. ij , adjust the transmission strategy of audio data according to channel status and environmental factors.

[0041]

[0042] Among them, D ij represents the delay of the sound from node i to node j, d ij represents the distance between two nodes, c represents the speed of sound, T env Indicates environmental delay.

[0043] S102. Based on the SLB access layer and Polar code channel control, generate identification information and routing information for each target playback device;

[0044] S103. Based on the identification information and routing information of each target playback device, a distributed edge node transmission model is constructed.

[0045] In some embodiments of the present invention, the distributed edge node transmission model is expressed as:

[0046]

[0047] Among them, i and j represent the node ordinal and routing ordinal respectively, P ij represents path selection, D ij Indicates the amount of data, T ij represents the transmission time, Q ij represents the channel quality, α, β, and γ represent the weight parameters of data volume, transmission time, and channel quality, respectively. i (t) represents the resource allocation strategy, W j (t) represents the node weight, L j (t) represents the node load.

[0048] Specifically, SLB and Polar code channel control technology are used to build a distributed edge node transmission model to ensure the efficiency and reliability of data transmission. First, node status initialization: When each edge node is initialized, its current status is exchanged, including available bandwidth, processing power, and storage status.

[0049] Si={Bi,Ci,Mi},

[0050] Among them, Si represents the state of the i-th node, Bi is the bandwidth, Ci is the computing power (computing power), and Mi is the storage state (remaining storage space utilization). Then the state is exchanged and updated: the edge nodes periodically exchange state information and update their respective state tables. The state table contains the current state of all nodes in the network.

[0051] The state table is represented as: T i (t)={S1(t),S2(t),…,S n (t)},

[0052] Among them, T i (t) represents the state table of the i-th node at time t, which contains the state information of all nodes in the network.

[0053] Next, path selection and resource allocation: select the optimal synchronization path and node according to the state table, calculate the cost of each possible path, select the path with the lowest cost for data transmission, and dynamically adjust the resource allocation strategy. Optionally, the routing selection strategy can be generated according to the Dijkstra algorithm, the minimum spanning tree algorithm, etc. After the above strategy adjustment, the distributed edge node transmission model is finally expressed as:

[0054]

[0055] Among them, P ij represents path selection, D ij is the amount of data, T ij is the transmission time, Q ijis the channel quality, α, β, γ are weight parameters, A i (t) is the resource allocation strategy, W j (t) is the node weight, L j (t) is the node load.

[0056] In step S200 of some embodiments of the present invention, the steps of determining a plurality of edge nodes and implementing data synchronization of the plurality of edge nodes according to the distributed edge node transmission model and the heuristic distributed transmission method, and adaptively adjusting resource utilization and audio and video transmission quality of each edge node include:

[0057] S201. Based on the distributed edge node transmission model, determine that the multiple edge nodes exchange initial state information and form an initial state table, wherein the initial state information includes the bandwidth, computing power and storage status of the node;

[0058] Specifically, based on historical data and real-time monitoring, the future bandwidth is predicted and the transmission rate is adjusted dynamically. The prediction model is expressed as:

[0059]

[0060] in, is the bandwidth prediction for the future k moments, f is the prediction function, and i represents the node.

[0061] S202. Determine the transmission route of each edge node based on the heuristic distributed transmission method, and adaptively adjust resource utilization and audio and video transmission quality.

[0062] Furthermore, in step S202, determining the transmission route of each edge node based on the heuristic distributed transmission method and adaptively adjusting the resource utilization and the transmission quality of audio and video include: S2021. Predicting the bandwidth of each edge node based on the heuristic distributed transmission method, and dynamically adjusting the transmission rate and transmission route of each edge node according to the prediction result; S2022. Adaptively adjusting the resource utilization and the transmission quality of audio and video based on the current transmission rate, transmission route and resource utilization of each edge node.

[0063] Specifically, according to the channel quality and bandwidth prediction results, the transmission rate of the video stream is dynamically adjusted to ensure the best transmission performance. The rate control function is expressed as:

[0064] Among them, R i (t) is the transmission rate of node i at time t, R max is the maximum transfer rate.

[0065] Then, according to the feedback information from the receiving end, the transmission strategy is adjusted in real time, including selecting an alternative path, adjusting the rate, or changing the data block size. The feedback information is expressed as: F ij (t) = {L ij (t),D ij (t),P ij (t)},

[0066] Among them, F ij (t) is the feedback information at time t, L ij (t) is the delay, D ij (t) is the amount of data, P ij (t) is the packet loss rate.

[0067] In step S300 of some embodiments of the present invention, the multi-task neural network includes a first task neural network, a second task neural network and a shared layer;

[0068] Wherein, the first task neural network is used to extract node features from the distributed edge node transmission model;

[0069] The second task neural network is used to extract sound field features from the sound field propagation model;

[0070] The sharing layer is used to share the node features and the sound field features with the first task neural network and the second task neural network.

[0071] Specifically, sound field data is collected from each edge node, including the time difference of arrival (TDOA) of the sound signal, signal strength, etc.

[0072] in,

[0073] In the formula, TDOA ij represents the arrival time difference between device i and sound source j, d ij represents the distance between the two, and c represents the speed of sound.

[0074] Furthermore, the time difference of arrival (TDOA) is calculated using the GCC-PHAT method, which is robust in noisy and reverberant environments.

[0075] The time difference calculated by the GCC-PHAT method is expressed as:

[0076]

[0077] Among them, S l and S p are the Fourier transforms of the audio sequences of the lth and pth channels respectively, τ is the delay, N is the FFT length, and K represents the number of frames of sound data.

[0078] Specifically, the structure of the multi-task neural network ensures that it can handle both the sound field propagation and node transmission tasks at the same time. The multi-task neural network contains shared layers and dedicated task branches. Input layer: The input layer is set to receive the extracted feature data. The input data includes sound field features and node state features. Task branch layer: Each task branch contains a dedicated hidden layer for processing a specific task. Output layer: Generates the prediction results of each task. The specific representation of each layer is:

[0079] Input=[S i ,T i (t)];

[0080] Shared_Layer = ReLU(W s Input+b s );

[0081] Task1_Output=Softmax(W1·Shared_Layer+b1);

[0082] Task2_Output=Softmax(W2·Shared_Layer+b).

[0083] The sound field data and transmission data are then used to train the multi-task neural network and optimize the network parameters.

[0084] The loss function is expressed as: Loss = λ1·Loss Task1 +λ2·Loss Task2 ;

[0085] Among them, Loss Task1 and Loss Task2 They represent the loss functions of the two tasks respectively, and λ1 and λ2 are weight coefficients.

[0086] In step S100 of some embodiments of the present invention, based on the trained multi-task neural network, the resource utilization rate and the transmission quality of audio and video of each edge node are dynamically adjusted to achieve spatiotemporal synchronization of each edge node.

[0087] Specifically, a multi-task neural network is deployed to each edge node to process sound field information and node status in real time to optimize data transmission and synchronization. Based on real-time feedback, the input and parameters of the multi-task neural network are dynamically adjusted to optimize the transmission quality of audio and video data and system performance.

[0088] By continuously monitoring system performance and data transmission quality, we further adjust and optimize the multi-task neural network to ensure the long-term stability and efficient operation of the system. Monitoring indicators: bandwidth utilization, computing resource utilization, and audio and video transmission quality.

[0089] It can be understood that the trained multi-task neural network can adjust the parameters involved in the distributed edge node transmission model and the sound field propagation model as a whole, thereby achieving a better synchronization solution.

[0090] refer to Figure 3 The second aspect of the present invention provides a panoramic audio-visual synchronization system 1 based on SLB, comprising:

[0091] The acquisition module 11 is used to obtain the sound field information of each target playback device, and build a distributed edge node transmission model and a sound field propagation model through SLB and Polar code channel control;

[0092] An adjustment module 12, configured to determine a plurality of edge nodes and implement data synchronization of the plurality of edge nodes according to the distributed edge node transmission model and the heuristic distributed transmission method, and adaptively adjust resource utilization and audio and video transmission quality of each edge node;

[0093] A fusion module 13 is used to fuse the distributed edge node transmission model and the sound field propagation model through a multi-task neural network and train the multi-task neural network;

[0094] The dynamic adjustment module 14 is used to dynamically adjust the resource utilization and audio and video transmission quality of each edge node based on the trained multi-task neural network to achieve time and space synchronization of each edge node.

[0095] In some specific embodiments, the acquisition module 11 includes: an acquisition unit, which is used to acquire the position, environmental information and acoustic information of each target playback device, and construct a sound field model according to the position, environmental information and acoustic information of each target playback device;

[0096] The generation unit is used to generate identification information and routing information of each target playback device based on the access layer of SLB and Polar code channel control; the construction unit is used to build a distributed edge node transmission model based on the identification information and routing information of each target playback device.

[0097] refer to Figure 4 According to a third aspect of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the panoramic audio-visual synchronization method based on SLB in the first aspect of the present invention.

[0098] The electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0099] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 4 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0100] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, the above functions defined in the method of the embodiment of the present disclosure are executed. It should be noted that the computer-readable medium described in the embodiment of the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus, or a device. In an embodiment of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus, or a device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

[0101] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more computer programs. When the one or more programs are executed by the electronic device, the electronic device:

[0102] Computer program code for performing the operations of embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, Python, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0103] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A panoramic audio-visual synchronization method based on SLB, characterized in that: include: Acquire the sound field information of each target playback device, and build a distributed edge node transmission model and sound field propagation model through SLB and Polar code channel control; According to the distributed edge node transmission model and the heuristic distributed transmission method, multiple edge nodes are determined and data synchronization of the multiple edge nodes is achieved, and resource utilization and audio and video transmission quality of each edge node are adaptively adjusted; By means of a multi-task neural network, the distributed edge node transmission model and the sound field propagation model are integrated and the multi-task neural network is trained; Based on the trained multi-task neural network, the resource utilization and audio and video transmission quality of each edge node are dynamically adjusted to achieve spatiotemporal synchronization of each edge node. The step of obtaining the sound field information of each target playback device and constructing a distributed edge node transmission model and a sound field propagation model through SLB and Polar code channel control includes: Acquiring the position, environmental information, and acoustic information of each target playback device, and constructing a sound field propagation model according to the position, environmental information, and acoustic information of each target playback device; Based on the SLB access layer and Polar code channel control, the identification information and routing information of each target playback device are generated; Based on the identification information and routing information of each target playback device, a distributed edge node transmission model is constructed; The distributed edge node transmission model is expressed as: Among them, i and j represent the node ordinal and routing ordinal respectively, P ij represents path selection, D ij Indicates the amount of data, T ij represents the transmission time, Q ij represents the channel quality, α, β, and γ represent the weight parameters of data volume, transmission time, and channel quality, respectively. i (t) represents the resource allocation strategy, W j (t) represents the node weight, L j (t) represents the node load.

2. The panoramic audio-visual synchronization method based on SLB according to claim 1, characterized in that: The method of determining a plurality of edge nodes and realizing data synchronization of the plurality of edge nodes according to the distributed edge node transmission model and the heuristic distributed transmission method, and adaptively adjusting the resource utilization and audio and video transmission quality of each edge node includes: Based on the distributed edge node transmission model, determine that the multiple edge nodes exchange initial state information and form an initial state table, wherein the initial state information includes bandwidth, computing power, and storage status of the nodes; The transmission route of each edge node is determined based on the heuristic distributed transmission method, and resource utilization and audio and video transmission quality are adaptively adjusted.

3. The panoramic audio-visual synchronization method based on SLB according to claim 2 is characterized in that: Determining the transmission route of each edge node based on the heuristic distributed transmission method and adaptively adjusting resource utilization and audio and video transmission quality includes: Predicting the bandwidth of each edge node based on the heuristic distributed transmission method, and dynamically adjusting the transmission rate and transmission route of each edge node according to the prediction result; Based on the current transmission rate, transmission route and resource utilization of each edge node, the resource utilization and audio and video transmission quality are adaptively adjusted.

4. The panoramic audio-visual synchronization method based on SLB according to claim 1, characterized in that: The multi-task neural network includes a first task neural network, a second task neural network and a shared layer; The first task neural network is used to extract node features from the distributed edge node transmission model; The second task neural network is used to extract sound field features from the sound field propagation model; The sharing layer is used to share the node features and the sound field features with the first task neural network and the second task neural network.

5. A panoramic audio-visual synchronization system based on SLB, characterized in that: include: The acquisition module is used to obtain the sound field information of each target playback device, and build a distributed edge node transmission model and sound field propagation model through SLB and Polar code channel control; An adjustment module, configured to determine a plurality of edge nodes and implement data synchronization of the plurality of edge nodes according to the distributed edge node transmission model and the heuristic distributed transmission method, and adaptively adjust resource utilization and audio and video transmission quality of each edge node; A fusion module, used to fuse the distributed edge node transmission model and the sound field propagation model through a multi-task neural network and train the multi-task neural network; A dynamic adjustment module is used to dynamically adjust the resource utilization and audio and video transmission quality of each edge node based on the trained multi-task neural network to achieve spatiotemporal synchronization of each edge node; The acquisition module comprises: An acquisition unit, used to acquire the position, environmental information and acoustic information of each target playback device, and to construct a sound field propagation model according to the position, environmental information and acoustic information of each target playback device; A generation unit, used to generate identification information and routing information of each target playback device based on the access layer of SLB and the Polar code channel control; A construction unit, used to construct a distributed edge node transmission model based on identification information and routing information of each target playback device; The distributed edge node transmission model is expressed as: Among them, i and j represent the node ordinal and routing ordinal respectively, P ij represents path selection, D ij Indicates the amount of data, T ij represents the transmission time, Q ij represents the channel quality, α, β, and γ represent the weight parameters of data volume, transmission time, and channel quality, respectively. i (t) represents the resource allocation strategy, W j (t) represents the node weight, L j (t) represents the node load.

6. An electronic device comprising: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the SLB-based panoramic audio-visual synchronization method as described in any one of claims 1 to 4.

7. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the SLB-based panoramic audio-visual synchronization method according to any one of claims 1 to 4 is implemented.

Images