Communication method and related device, communication system for multimedia stream synchronization

Abstract

The present disclosure provides a communication method and related equipment and communication system for multimedia stream synchronization, and relates to the technical field of communication. The method comprises the following steps: in response to receiving a call request initiated by a calling terminal and comprising a media stream to be transmitted, determining a corresponding computing resource address and terminal addresses of each called terminal; informing the calling terminal to perform a media stream delay test to obtain test delays between the calling terminal and each called terminal; determining a synchronization delay corresponding to the call request according to a media stream synchronization strategy of a user subscription service and the test delays; and sending the synchronization delay to each called terminal to enable each called terminal to process the media stream to be transmitted according to the synchronization delay. The present disclosure can solve the problem of poor user experience caused by the lack of support for multi-user media stream synchronization in holographic communication.

Description

Technical Field

[0001] This disclosure relates to the field of communication technology, and more specifically, to a communication method and related equipment and communication system for multimedia stream synchronization. Background Technology

[0002] With the continuous development of communication technology, holographic communication-based information interaction is becoming the mainstream interaction method in the future. Holographic display technology uses interferometry to record information such as the phase and amplitude of light waves scattered from the surface of an object, and then uses the principle of diffraction to reconstruct the three-dimensional image of the object. Holographic communication is a new type of communication that uses holographic display technology to capture images of people and surrounding objects at a remote location, transmits holographic data through a network, and projects real-time dynamic stereoscopic images in the form of holograms at the terminal, allowing for interaction. It can provide 6G (6th generation mobile networks) users with a fully immersive holographic interactive experience anytime, anywhere.

[0003] In related technologies, it is impossible to perform media stream synchronization services in multiple user access scenarios in holographic communication networks, which affects the service coverage and user experience of holographic communication.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a communication method and related equipment and system for multimedia stream synchronization, thereby solving to some extent the problem of incomplete coverage of business scenarios and poor user experience caused by the lack of support for multi-user media stream synchronization in the holographic communication process.

[0006] According to a first aspect of this disclosure, a communication method for synchronizing multimedia streams is provided, applied to a server in a holographic communication system. The method includes: responding to receiving a call request initiated by a calling terminal, comprising a media stream to be transmitted, determining the computing resource address of the media stream to be transmitted and the terminal address of at least one called terminal; notifying the calling terminal to perform a media stream latency test based on the computing resource address and the terminal address to obtain a test latency between the calling terminal and at least one called terminal; determining a synchronization latency corresponding to the call request based on a media stream synchronization strategy of a user's subscribed service and the test latency; and sending the synchronization latency to the at least one called terminal so that the at least one called terminal processes the media stream to be transmitted according to the synchronization latency.

[0007] Optionally, the media stream synchronization strategy includes a synchronization latency requirement. Determining the synchronization latency based on the media stream synchronization strategy and test latency of the user's subscribed service includes: in response to the test latency of each group of calling and called terminals respectively meeting the synchronization latency requirement, determining the synchronization latency based on the maximum value of the test latency of each group or the synchronization latency requirement; in response to the test latency of at least one group of calling and called terminals not meeting the synchronization latency requirement, notifying the communication network to perform network adjustments so that the test latency of each group of calling and called terminals meets the synchronization latency requirement after adjustment.

[0008] Optionally, the method further includes: in response to receiving a user registration request including a user identifier sent by the calling terminal, performing target service authentication on the user corresponding to the user identifier; in response to successful authentication, obtaining the media stream synchronization strategy of the user's subscribed service, and returning the registration result to the calling terminal.

[0009] Optionally, the call request includes identification information of at least one called terminal and requested service information. Determining the computing power resource address and the terminal address of at least one called terminal includes: determining the terminal address of at least one called terminal according to the identification information; allocating computing power resources to the media stream to be transmitted according to the requested service information, and determining the computing power resource address.

[0010] According to a second aspect of this disclosure, a communication method for multimedia stream synchronization is provided, applied to a calling terminal in a holographic communication system. The method includes: sending a call request including a media stream to be transmitted to a server to obtain a computing resource address allocated by the server for the media stream to be transmitted and a terminal address of at least one called terminal; performing a media stream latency test based on the computing resource address and the terminal address to determine a test latency between the calling terminal and at least one called terminal; sending the test latency to the server so that the server determines a synchronization latency corresponding to the call request based on the media stream synchronization strategy of the user's subscribed service and the test latency, and sends the synchronization latency to the at least one called terminal so that the at least one called terminal processes the media stream to be transmitted based on the synchronization latency.

[0011] Optionally, the server includes a media computing power module. The step of performing media stream latency testing based on the computing power resource address and the terminal address includes: sending a first test media stream to the media computing power module corresponding to the computing power resource address, so that the media computing power module forwards the first test media stream to at least one called terminal corresponding to the terminal address; the computing power resource address and the media computing power module correspond one-to-one; receiving a second test media stream, the second test media stream being a media stream returned by at least one called terminal via the media computing power module that corresponds to the first test media stream; and determining the test latency between the calling terminal and at least one called terminal based on the time of sending the first test media stream and the time of receiving the second test media stream.

[0012] According to a third aspect of this disclosure, a multimedia stream synchronization service device is provided, the service device comprising: a first determining module, a notification module, a second determining module, and a sending module; the first determining module being configured to, in response to receiving a call request initiated by a calling terminal including a media stream to be transmitted, determine the computing resource address of the media stream to be transmitted and the terminal address of at least one called terminal; the notification module being configured to, based on the computing resource address and the terminal address, notify the calling terminal to perform a media stream latency test to obtain a test latency between the calling terminal and at least one called terminal; the second determining module being configured to, based on the media stream synchronization strategy of the user's subscribed service and the test latency, determine the synchronization latency corresponding to the call request; and the sending module being configured to send the synchronization latency to the at least one called terminal so that the at least one called terminal processes the media stream to be transmitted according to the synchronization latency.

[0013] According to a fourth aspect of this disclosure, a terminal device for multimedia stream synchronization is provided. The terminal device includes: a first sending module, a latency testing module, and a second sending module. The first sending module is configured to send a call request including a media stream to be transmitted to a server to obtain a computing resource address allocated by the server for the media stream to be transmitted and a terminal address of at least one called terminal. The latency testing module is configured to perform a media stream latency test based on the computing resource address and the terminal address to determine a test latency between the calling terminal and at least one called terminal. The second sending module is configured to send the test latency to the server so that the server determines the synchronization latency corresponding to the call request based on the media stream synchronization strategy of the user's subscribed service and the test latency, and sends the synchronization latency to the at least one called terminal so that the at least one called terminal processes the media stream to be transmitted based on the synchronization latency.

[0014] According to a fifth aspect of this disclosure, a computer-readable storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the methods of any of the above embodiments.

[0015] According to a sixth aspect of this disclosure, a communication device for synchronizing multimedia streams is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of any of the above embodiments by executing the executable instructions.

[0016] According to a seventh aspect of this disclosure, a holographic communication system for multimedia stream synchronization is provided, characterized in that the system includes: a service device as described in the above embodiments and / or a terminal device as described in the above embodiments.

[0017] The exemplary embodiments disclosed herein may have some or all of the following beneficial effects:

[0018] In the multimedia stream synchronization communication method provided in the exemplary embodiments of this disclosure, on the one hand, by performing media stream latency tests between the calling terminal and at least one called terminal, the communication latency (test latency) between the calling terminal and each called terminal in the current call request is determined. This allows call requests to be processed based on real-time communication latency, ensuring interaction efficiency and communication quality in scenarios with multiple concurrent media streams. On the other hand, by determining the synchronization latency corresponding to the call request based on the media stream synchronization strategy of the user's subscribed service and the test latency, the latency information of different calling and called groups can be integrated. Combined with the synchronization strategy, this ensures the information synchronization effect of multiple concurrent media streams in the holographic communication system, guarantees information synchronization between multiple groups of interactive users, and improves interaction quality. Furthermore, it can expand the service scenario coverage of holographic communication and improve user experience.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0021] Figure 1 The schematic diagram illustrates a system architecture diagram of a communication method for synchronizing multimedia streams according to an embodiment of the present disclosure.

[0022] Figure 2One of the schematic flowcharts of a communication method for synchronizing multimedia streams according to an embodiment of the present disclosure is illustrated.

[0023] Figure 3 The illustration shows a schematic diagram of the user's service registration process according to an embodiment of the present disclosure.

[0024] Figure 4 The second schematic diagram illustrates a communication method for synchronizing multimedia streams according to an embodiment of the present disclosure.

[0025] Figure 5 A schematic diagram illustrating a communication interaction flowchart for multimedia stream synchronization according to an embodiment of the present disclosure is provided.

[0026] Figure 6 A schematic block diagram of a multimedia streaming synchronization service device according to an embodiment of the present disclosure is shown.

[0027] Figure 7 A schematic block diagram of a terminal device for multimedia stream synchronization according to an embodiment of the present disclosure is shown.

[0028] Figure 8 An exemplary network device block diagram according to one embodiment of the present disclosure is illustrated schematically. Detailed Implementation

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0030] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0031] Figure 1 A schematic diagram of an exemplary network architecture 100 to which a communication method for multimedia stream synchronization, according to embodiments of the present disclosure, is shown. Figure 1 As shown, the network architecture 100 may include a service control layer 110, a media computing layer 120, and a terminal side 130. The service control layer 110 includes a session control module 111 and a data warehouse 112. The session control module 111 is used for end-to-end media stream latency testing and the management of multi-party media synchronization strategies. For example, the session control module 111 can send end-to-end media stream latency test instructions to the calling terminal's concurrent stream test module and receive latency test results from the calling terminal. Based on the media stream synchronization strategy of the user's subscribed service and the media stream transmission latency test results from the calling terminal to all called terminals, the synchronization latency of the current call is determined and sent to all called terminals involved in the call. The data warehouse 112 is used for the unified storage and management of holographic user and terminal subscribed service media synchronization strategy data. The media computing layer 120 includes multiple media computing power modules 121, which are used to receive and transparently forward latency test media streams sent by the calling and called terminals.

[0032] Terminal side 130 includes multiple terminal devices, such as terminal device 1, terminal device 2, ..., terminal device N, where N is a positive integer. Each terminal device includes a media stream testing module 131, which is used for testing the end-to-end media stream latency of the holographic service. For example, as a calling terminal, it can send latency test media streams to all called terminals, receive test media streams returned by the called terminals, calculate the end-to-end transmission latency of the test media streams, and feed it back to the session control concurrent stream synchronization module. As a called terminal, it can receive latency test media streams sent by the calling terminal and send them back to the calling terminal.

[0033] Terminal devices may include, but are not limited to, smart terminals, edge computing devices (such as edge computing gateways), cloud computing devices, etc. Smart terminals may be mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, test terminals, computers, etc., any communication device that can provide computing power resources for multimedia streaming synchronization. This example does not limit this.

[0034] The business control layer 110 and the media computing layer 120 can be independent physical servers, or server clusters or distributed systems composed of multiple physical servers. Multiple servers can form a blockchain, and the server is a node on the blockchain. The server can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0035] The multimedia stream synchronization communication method provided in this embodiment can be executed in any of the service control layer 110, media computing layer 120 and multiple terminal devices 130, or in a communication system composed of service control layer 110, media computing layer 120 and multiple terminal devices 130. This example does not limit it in this way.

[0036] The technical solutions of the embodiments of this disclosure are described in detail below:

[0037] refer to Figure 2 As shown, a communication method for multimedia stream synchronization according to an example embodiment of the present disclosure may include the following steps S210 to S240.

[0038] Step S210: In response to receiving a call request initiated by a calling terminal that includes a media stream to be transmitted, determine the computing resource address of the media stream to be transmitted and the terminal address of at least one called terminal.

[0039] In this example implementation, the call request may include identification information of at least one called terminal and requested service information (such as service type, service data, etc.). The service type may include holographic data communication or at least one of audio, video, holographic, and text communication. The service data may be a media stream to be transmitted, which may include media data of multiple media types, such as at least one of audio data, video data, holographic data, and text data, or other types, which are not limited in this example. The calling terminal may be a holographic device capable of acquiring holographic data, for example, acquiring media stream data of a target scene (such as people, objects, environment, etc.) through a camera, microphone, etc. The address of each called terminal can be determined by addressing it according to its identification information.

[0040] In this example implementation, corresponding computing resource addresses can be allocated to the media streams to be transmitted between each group of calling and called terminals based on the service type and the availability of currently idle computing resources. For example, computing resources may include CPUs, GPUs, storage resources, etc.

[0041] Step S220: Based on the computing power resource address and the terminal address, notify the calling terminal to perform a media stream latency test to obtain the test latency between the calling terminal and at least one called terminal.

[0042] In this example implementation, given the computing resource address and terminal address, a test notification can be sent to the calling terminal to initiate a media stream latency test. The media stream latency test simulates the media stream communication process between the calling terminal and each called terminal to obtain the communication latency (test latency) between each group of calling and called parties during actual communication.

[0043] Step S230: Determine the synchronization delay corresponding to the call request based on the media stream synchronization strategy and test latency of the user's subscribed service.

[0044] In this example implementation, the test latency includes communication latency information between different calling and called parties. Since the communication latency between different groups of calling and called parties may vary due to differences in media stream type, data size, etc. (e.g., different types of media streams require different encoding / decoding times, and different data sizes of media streams require different parsing / processing times), it is necessary to integrate the different communication latency in the test latency to avoid the time difference problem of different called terminals acquiring media streams.

[0045] In this example implementation, the synchronization requirements and test latency in the media stream synchronization strategy of the user's subscribed service can be comprehensively considered to determine the media stream synchronization latency between different calling and called groups corresponding to this call request. For example, if the test latency meets the synchronization requirements, the larger communication latency in the test latency can be selected as the synchronization latency. If the test latency does not meet the synchronization requirements, the communication network parameters or routing corresponding to that test latency can be adjusted.

[0046] Step S240: Send a synchronization delay to at least one called terminal so that at least one called terminal processes the media stream to be transmitted according to the synchronization delay.

[0047] In this example implementation, different called terminals can process the media stream to be transmitted according to the synchronization delay to achieve synchronized display. The processing may include encoding, decoding, rendering, parsing, and other processing procedures, which are not limited in this example.

[0048] In the multimedia stream synchronization communication method provided in this disclosure, on the one hand, by performing media stream latency tests between the calling terminal and at least one called terminal, the communication latency (test latency) between the calling terminal and each called terminal in the current call request is determined. This allows call requests to be processed based on real-time communication latency, ensuring interaction efficiency and communication quality in scenarios with multiple concurrent media streams. On the other hand, by determining the synchronization latency corresponding to the call request based on the media stream synchronization strategy of the user's subscribed service and the test latency, the latency information of different calling and called groups can be integrated. Combined with the synchronization strategy, this ensures the information synchronization effect of multiple concurrent media streams in the holographic communication system, guarantees information synchronization between multiple groups of interactive users, and improves interaction quality. Furthermore, it can expand the service scenario coverage of holographic communication and improve user experience.

[0049] In some embodiments, the media stream synchronization strategy includes a synchronization latency requirement. The synchronization latency is determined based on the media stream synchronization strategy and test latency of the user's subscribed service, including: in response to the test latency of each group of calling terminals and called terminals respectively meeting the synchronization latency requirement, the synchronization latency is determined based on the maximum value of the test latency of each group or the synchronization latency requirement.

[0050] In this example implementation, if the test delays of the calling and called groups involved in the current call request both meet the synchronization delay requirements, the synchronization delay can be determined based on the test delays, or it can be determined based on the synchronization delay requirements in the media stream synchronization strategy. This example does not limit this.

[0051] For example, if the test delay of all calling and called groups is not greater than the synchronization delay requirement, then the test delay is determined to meet the synchronization delay requirement, and the maximum value of the test delay can be used as the synchronization delay.

[0052] In response to the fact that the test latency of at least one group of calling terminals and called terminals does not meet the synchronization latency requirement, the communication network is notified to make network adjustments so that the test latency of each group of calling terminals and called terminals meets the synchronization latency requirement after the adjustment.

[0053] In this example implementation, if the test latency of the calling and called groups involved in the current call request does not meet the synchronization latency requirement, a network adjustment notification can be sent to the communication network to enable network adjustments. This network adjustment notification may include information such as the difference between the test latency and the synchronization latency requirement, the current routing line, and the addresses of the calling and called terminals, so that the communication network can make adjustments based on the notification information, such as adjusting network parameters or routing lines, to ensure that the test latency meets the synchronization latency requirement.

[0054] For example, when the test delay M1 of one or more groups of calling and called parties is greater than the synchronization delay requirement M0, the absolute value of the difference between |M1-M0| can be used to notify the communication network to make network adjustments, such as adjusting network parameters (e.g., bandwidth, transmission speed limit, signal transmission power, etc.), adjusting network routing, or replacing network equipment. This example does not limit these adjustments.

[0055] In some embodiments, the method further includes: in response to receiving a user registration request including a user identifier sent by a calling terminal, performing target service authentication on the user corresponding to the user identifier. In response to successful authentication, obtaining the media stream synchronization strategy of the user's subscribed service, and returning the registration result to the calling terminal.

[0056] In this example implementation, each terminal needs to register before making a call request. Only successfully registered users can engage in normal holographic communication. The user registration request may include user identifier, registration service information (such as service name, service code, and service type), and may also include other information, which is not limited in this example. The server can authenticate the target service registered by the user based on the stored user information and the user's subscribed service information (including media stream synchronization strategy). After successful authentication, the server returns the registration result to the user terminal, which may include the registration time, the registered service, and whether the registration was successful or not.

[0057] For example, such as Figure 3 As shown, the server may include a session control module and a data warehouse (such as a user data platform), which stores user information and user-subscribed service information. The terminal registration process may include the following steps S301-S307.

[0058] Step S301: The terminal sends a user registration request to the session control module.

[0059] In step S302, the session control module forwards the user registration request to the data warehouse.

[0060] In step S303, the data warehouse performs business authentication on the user registration request. If the authentication is successful, proceed to step S304; otherwise, proceed to step S306.

[0061] Step S304: The data warehouse records the session control module accessed by the user and sends the user's user information and the user's subscribed service information to the session control module.

[0062] Step S305: The session control module records the media stream synchronization strategy of the user's subscribed service.

[0063] In step S306, the data warehouse sends an authentication failure result to the session control module.

[0064] In step S307, the session control module returns the registration result to the terminal. For example, registration successful or registration failed.

[0065] In some embodiments, the call request includes identification information of at least one called terminal and requested service information. Determining the computing resource address and the terminal address of at least one called terminal includes: determining the terminal address of at least one called terminal according to the identification information; allocating computing resources to the media stream to be transmitted according to the requested service information, and determining the computing resource address.

[0066] In this example implementation, the server (such as a data warehouse) stores user information and user subscription service information. It can search the data warehouse for the corresponding terminal address based on the called terminal's identification information. The server (such as a computing power scheduling module) can allocate computing power resources to the media stream to be transmitted and determine the corresponding computing power resource address based on the requested service information and the target network computing power resource occupancy.

[0067] This disclosure also provides a multimedia stream synchronization communication method applied to the calling terminal in a holographic communication system, referencing... Figure 4 The method includes steps S410-S430.

[0068] Step S410: Send a call request including the media stream to be transmitted to the server to obtain the computing power resource address allocated by the server for the media stream to be transmitted and the terminal address of at least one called terminal.

[0069] Step S420: Perform a media stream latency test based on the computing power resource address and the terminal address to determine the test latency between the calling terminal and at least one called terminal.

[0070] Step S430: Send a test delay to the server so that the server can determine the synchronization delay corresponding to the call request based on the media stream synchronization strategy of the user's subscribed service and the test delay, and send the synchronization delay to at least one called terminal so that at least one called terminal can process the media stream to be transmitted according to the synchronization delay.

[0071] In some embodiments, the server includes a media computing module that performs media stream latency testing based on the computing resource address and the terminal address, including:

[0072] Send a first test media stream to the media computing module corresponding to the computing resource address, so that the media computing module forwards the first test media stream to at least one called terminal corresponding to the terminal address.

[0073] In this example implementation, one computing resource address can correspond to one media computing module. The calling terminal initiates a media stream latency test and sends a first test media stream to the media computing module. The media computing module can then pass this first test media stream through to each called terminal. Alternatively, the media computing module can perform specified processing on the first test media stream before sending the processing result to each called terminal. For example, this specified processing may include rendering, encoding, decoding, transcoding, etc., and may also include other processing, which is not limited in this example.

[0074] Receive a second test media stream, which is a media stream corresponding to the first test media stream returned by at least one called terminal via the media computing module.

[0075] In this example implementation, each called terminal can send a second test media stream to the corresponding media computing module based on the received media stream, and then send it to the calling terminal after passing through or specifying the media computing module, so as to realize the simulated communication between the calling terminal and each called terminal.

[0076] The test delay between the calling terminal and at least one called terminal is determined based on the time of sending the first test media stream and the time of receiving the second test media stream.

[0077] In this example implementation, the test delay between each group of calling and called terminals can be determined based on the information time difference between terminals during simulated communication. For example, half the difference between the time the calling terminal sends the first test media stream and the time the called terminal receives the second test media stream can be used as the test delay between the calling and called terminals in that group. Alternatively, the time the calling terminal sends the first test media stream and the time the called terminal receives the first test media stream can also be used as the test delay between the calling and called terminals in that group. Furthermore, the media stream delay from the calling terminal to the called terminal can be corrected based on the media stream delay from the called terminal to the calling terminal to determine the test delay. The test delay can be determined based on actual conditions and experience; this example does not impose any limitations on this.

[0078] In some embodiments, reference Figure 5 The multimedia stream synchronization communication method is applied to a holographic communication system, which may include a calling terminal, a server and multiple called terminals. The server may include a session control module and a media computing module. The method may include the following steps S501-S512.

[0079] In step S501, the calling terminal A sends a call request to the session control module. The call request may include the media stream to be transmitted and the identification information of at least one called terminal.

[0080] In step S502, the session control module determines the computing resource address of the media stream to be transmitted and the terminal address of at least one called terminal.

[0081] In step S503, the session control module sends a latency test notification to the calling terminal A. The latency test notification may include the computing power resource address and the terminal address of at least one called terminal.

[0082] Step S504: The calling terminal A initiates a media stream latency test to each called terminal.

[0083] In step S505, the calling terminal A sends the first test media stream to the media computing module.

[0084] In this example, the first test media stream can be sent to one or more media computing modules. When sent to one media computing module, that module forwards the different media streams to each called terminal. When sent to multiple media computing modules, one calling / called group can correspond to one media computing module, or one calling / called group can correspond to multiple media computing modules; this example does not limit this.

[0085] Step S506: The media computing module forwards the first test media stream to each called terminal, such as terminal B and terminal C.

[0086] In step S507, each called terminal returns the corresponding second test media stream to the calling terminal via the original path based on the first test media stream.

[0087] In this example, each called terminal may send the second test media stream to the media computing module, which then transmits it to the calling terminal.

[0088] In step S508, the calling terminal determines the test delay based on the time difference between sending the first test media stream and receiving the second test media stream.

[0089] Step S509: The calling terminal sends a test delay to the session control module.

[0090] In step S510, the session control module determines the synchronization delay corresponding to the call request based on the media stream synchronization strategy and test latency of the user's subscribed service.

[0091] Step S511: The session control module sends a synchronization delay to each called terminal.

[0092] In step S512, each called terminal processes the calling terminal and its respective media stream to be transmitted according to the synchronization delay.

[0093] The order of the steps in the above embodiments is merely exemplary and can be adjusted as needed. Detailed descriptions of each step in the above embodiments can be found in the corresponding descriptions in the foregoing embodiments, and will not be repeated here.

[0094] Based on the above process, by extending the one-to-many scenario in parallel, a many-to-many interactive scenario can be formed. In other words, this disclosure can also be applied to the communication process between multiple calling terminals and multiple called terminals.

[0095] This disclosure addresses one-to-many or many-to-many service scenarios in holographic communication. On one hand, it supports 6G users' needs for simultaneous synchronization of multiple concurrent media streams in scenarios where multiple users access the same call, resolving the shortcomings of related technologies that do not support the synchronous transmission of single-point to multi-point or multi-point to multi-point media stream data in holographic services, thus providing an important guarantee for the implementation of holographic communication. On the other hand, the communication method of this disclosure can share the holographic communication service architecture, utilizing the holographic communication infrastructure of the operator's network to provide users with the service implementation of simultaneous synchronization of multiple user access streams, reducing service deployment costs, improving service deployment efficiency, and expanding the service deployment scope.

[0096] See Figure 6 This example embodiment also provides a multimedia stream synchronization service device 600, which includes: a first determining module 610, a notification module 620, a second determining module 630, and a sending module 640. The first determining module 610 is used to determine the computing resource address of the media stream to be transmitted and the terminal address of at least one called terminal in response to receiving a call request initiated by a calling terminal, which includes a media stream to be transmitted. The notification module 620 is used to notify the calling terminal to perform a media stream latency test based on the computing resource address and the terminal address to obtain the test latency between the calling terminal and at least one called terminal. The second determining module 630 is used to determine the synchronization latency corresponding to the call request based on the media stream synchronization strategy of the user's subscribed service and the test latency. The sending module 640 is used to send the synchronization latency to at least one called terminal so that at least one called terminal processes the media stream to be transmitted according to the synchronization latency.

[0097] In one embodiment of this disclosure, the media stream synchronization strategy includes a synchronization delay requirement. The second determining module 630 is further configured to: in response to the test delays of each group of calling terminals and called terminals respectively meeting the synchronization delay requirement, determine the synchronization delay based on the maximum value of the test delays of each group or the synchronization delay requirement; in response to the test delays of at least one group of calling terminals and called terminals not meeting the synchronization delay requirement, notify the communication network to perform network adjustments so that the test delays of each group of calling terminals and called terminals meet the synchronization delay requirement after adjustment.

[0098] In one embodiment of this disclosure, the service device 600 further includes a registration module, which is configured to: in response to receiving a user registration request including a user identifier sent by a calling terminal, authenticate the user corresponding to the user identifier for the target service; in response to successful authentication, obtain the media stream synchronization strategy of the user's subscribed service, and return the registration result to the calling terminal.

[0099] In one embodiment of this disclosure, the call request includes identification information of at least one called terminal and request service information. The first determining module 610 is further configured to: determine the terminal address of the corresponding at least one called terminal based on the identification information; allocate computing resources to the media stream to be transmitted based on the request service information, and determine the computing resource address.

[0100] See Figure 7 This example embodiment also provides a multimedia stream synchronization terminal device 700. The terminal device 700 includes: a first sending module 710, a latency testing module 720, and a second sending module 730. The first sending module 710 is used to send a call request including a media stream to be transmitted to a server to obtain the computing power resource address allocated by the server for the media stream to be transmitted and the terminal address of at least one called terminal. The latency testing module 720 is used to perform a media stream latency test based on the computing power resource address and the terminal address to determine the test latency between the calling terminal and at least one called terminal. The second sending module 730 is used to send the test latency to the server so that the server determines the synchronization latency corresponding to the call request based on the media stream synchronization strategy of the user's subscribed service and the test latency, and sends the synchronization latency to at least one called terminal so that at least one called terminal processes the media stream to be transmitted based on the synchronization latency.

[0101] In one embodiment of this disclosure, the server includes a media computing power module, and the latency testing module 720 is further configured to: send a first test media stream to the media computing power module corresponding to the computing power resource address, so that the media computing power module forwards the first test media stream to at least one called terminal corresponding to the terminal address; the computing power resource address corresponds one-to-one with the media computing power module; receive a second test media stream, the second test media stream being a media stream corresponding to the first test media stream returned by at least one called terminal through the media computing power module; and determine the test latency between the calling terminal and at least one called terminal based on the time of sending the first test media stream and the time of receiving the second test media stream.

[0102] The specific details of each module / unit involved in the service device and terminal device in the above embodiments have been described in detail in the corresponding communication methods, so they will not be repeated here.

[0103] In another aspect, this application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to implement the methods as described in the following embodiments. For example, the device may implement... Figures 2-5 The various steps shown are as follows.

[0104] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0105] Furthermore, in exemplary embodiments of this disclosure, an apparatus capable of implementing the above-described methods is also provided. Those skilled in the art will understand that various aspects of this disclosure can be implemented as systems, methods, or program products. Therefore, various aspects of this disclosure can be specifically implemented as entirely hardware embodiments, entirely software embodiments (including firmware, microcode, etc.), or embodiments combining hardware and software aspects, collectively referred to herein as "circuit," "module," or "system."

[0106] See Figure 8 , Figure 8 This is a schematic diagram of the structure of a communication device for multimedia stream synchronization provided in an embodiment of this application. Figure 8 As shown, the multimedia streaming synchronization communication device 800 includes a processor 810, a memory 820, an input / output interface 830, and a communication bus 840. The processor 810 is connected to the memory 820 and the input / output interface 830; for example, the processor 810 can be connected to the memory 820 and the input / output interface 830 via the communication bus 840. The processor 810 is configured to support the network device in performing operations. Figures 2-5 The processor 810 is a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The memory 820 is used to store program code, etc. The memory 820 may include volatile memory (VM), such as random access memory (RAM); the memory 820 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); the memory 820 may also include combinations of the above types of memory.

[0107] The input / output interface 830 is used for inputting or outputting data.

[0108] For example, processor 810 may call the above program code to perform the following operations:

[0109] In response to receiving a call request from a calling terminal including a media stream to be transmitted, the system determines the computing resource address of the media stream to be transmitted and the terminal address of at least one called terminal; based on the computing resource address and the terminal address, the system notifies the calling terminal to perform a media stream latency test to obtain the test latency between the calling terminal and at least one called terminal; based on the media stream synchronization strategy of the user's subscribed service and the test latency, the system determines the synchronization latency corresponding to the call request; and sends the synchronization latency to at least one called terminal so that at least one called terminal can process the media stream to be transmitted according to the synchronization latency.

[0110] It should be noted that the implementation of each operation can also be referenced accordingly. Figures 2-5 The corresponding description of the method embodiment shown; the processor 810 described above can also cooperate with the input / output interface 830 to perform other operations in the above method embodiment.

[0111] In some embodiments, this application also provides a holographic communication system for multimedia stream synchronization, which may include any multimedia stream synchronization service device or any multimedia stream synchronization terminal device in the above embodiments.

[0112] This invention can be implemented by adding a corresponding module to an existing holographic communication system, requiring minimal modification to the existing system and making it easy to implement.

[0113] From the description of the above embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, and includes several instructions to cause a device to execute the method according to the embodiments of this disclosure.

[0114] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0115] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps, such as omitting certain steps, combining multiple steps into one step, and / or breaking down one step into multiple steps, should all be considered part of this disclosure.

[0116] It should be understood that this disclosure, as disclosed and defined herein, extends to all alternative combinations of two or more individual features mentioned or apparent in the text and / or figures. All these different combinations constitute multiple alternative aspects of this disclosure. The embodiments described in this specification illustrate the best known mode for implementing this disclosure and will enable those skilled in the art to utilize it.

Claims

1. A multimedia stream synchronization communication method, applied to the server side of a holographic communication system, characterized in that, The method includes: In response to receiving a call request initiated by a calling terminal, which includes a media stream to be transmitted, the computing power resource address of the media stream to be transmitted and the terminal address of at least one called terminal are determined; wherein, one computing power resource address corresponds to one media computing power module; Based on the computing power resource address and the terminal address, the calling terminal is notified to perform a media stream latency test to obtain the test latency between the calling terminal and at least one called terminal; wherein, the media stream latency test is implemented in the following manner: the calling terminal sends a first test media stream to the media computing power module corresponding to the computing power resource address, so that the media computing power module forwards the first test media stream to at least one called terminal corresponding to the terminal address; The synchronization latency corresponding to the call request is determined based on the media stream synchronization strategy of the user's subscribed service and the test latency. The synchronization delay is sent to the at least one called terminal so that the at least one called terminal processes the media stream to be transmitted according to the synchronization delay.

2. The method according to claim 1, characterized in that, The media stream synchronization strategy includes synchronization latency requirements. Determining the synchronization latency based on the user's subscribed service's media stream synchronization strategy and test latency includes: In response to the fact that the test delay of each group of calling terminals and called terminals respectively meets the synchronization delay requirement, the synchronization delay is determined according to the maximum value of the test delay of each group or the synchronization delay requirement; wherein, each group of calling terminals and called terminals includes one of the calling terminals and one of the at least one called terminals; In response to the fact that the test latency of at least one group of calling terminals and called terminals does not meet the synchronization latency requirement, the communication network is notified to make network adjustments so that the test latency of each group of calling terminals and called terminals meets the synchronization latency requirement after adjustment; wherein, a group of calling terminals and called terminals includes one of the calling terminals and one of the at least one called terminals.

3. The method according to claim 1 or 2, characterized in that, The method further includes: In response to receiving a user registration request including a user identifier sent by the calling terminal, target service authentication is performed on the user corresponding to the user identifier; Upon successful authentication, the system obtains the media stream synchronization strategy for the user's subscribed services and returns the registration result to the calling terminal.

4. The method according to claim 1, characterized in that, The call request includes identification information of at least one called terminal and requested service information. Determining the computing resource address and the terminal address of at least one called terminal includes: Based on the identification information, determine the terminal address of at least one called terminal. Based on the requested service information, computing resources are allocated to the media stream to be transmitted, and the address of the computing resources is determined.

5. A multimedia stream synchronization communication method, applied to the calling terminal in a holographic communication system, characterized in that, The method includes: Send a call request to the server, including the media stream to be transmitted, to obtain the computing power resource address allocated by the server for the media stream to be transmitted and the terminal address of at least one called terminal; wherein, one computing power resource address corresponds to one media computing power module; Based on the computing power resource address and the terminal address, a media stream latency test is performed to determine the test latency between the calling terminal and at least one called terminal; wherein, the media stream latency test is implemented in the following manner: a first test media stream is sent to the media computing power module corresponding to the computing power resource address, so that the media computing power module forwards the first test media stream to at least one called terminal corresponding to the terminal address; The test delay is sent to the server so that the server determines the synchronization delay corresponding to the call request based on the media stream synchronization strategy of the user's subscribed service and the test delay, and sends the synchronization delay to the at least one called terminal so that the at least one called terminal processes the media stream to be transmitted according to the synchronization delay.

6. The method according to claim 5, characterized in that, The server includes a media computing power module, and the media stream latency test based on the computing power resource address and the terminal address includes: A first test media stream is sent to the media computing module corresponding to the computing resource address, so that the media computing module forwards the first test media stream to at least one called terminal corresponding to the terminal address; the computing resource address corresponds one-to-one with the media computing module. Receive a second test media stream, wherein the second test media stream is a media stream corresponding to the first test media stream returned by at least one called terminal through the media computing module; The test delay between the calling terminal and at least one called terminal is determined based on the time of sending the first test media stream and the time of receiving the second test media stream.

7. A multimedia stream synchronization service device, characterized in that, The service equipment includes: The first determining module is used to, in response to receiving a call request initiated by a calling terminal including a media stream to be transmitted, determine the computing power resource address of the media stream to be transmitted and the terminal address of at least one called terminal; wherein, one computing power resource address corresponds to one media computing power module; The notification module is used to notify the calling terminal to perform a media stream latency test based on the computing power resource address and the terminal address, so as to obtain the test latency between the calling terminal and at least one called terminal; wherein, the media stream latency test is implemented in the following way: the calling terminal sends a first test media stream to the media computing power module corresponding to the computing power resource address, so that the media computing power module forwards the first test media stream to at least one called terminal corresponding to the terminal address; The second determining module is used to determine the synchronization delay corresponding to the call request based on the media stream synchronization strategy of the user's subscribed service and the test delay. A sending module is configured to send the synchronization delay to the at least one called terminal, so that the at least one called terminal processes the media stream to be transmitted according to the synchronization delay.

8. A terminal device for multimedia stream synchronization, characterized in that, The terminal device includes: The first sending module is used to send a call request including the media stream to be transmitted to the server, so as to obtain the computing power resource address allocated by the server for the media stream to be transmitted and the terminal address of at least one called terminal; wherein, one computing power resource address corresponds to one media computing power module; The latency testing module is used to perform media stream latency testing based on the computing power resource address and the terminal address to determine the test latency between the terminal device and at least one called terminal; wherein, the media stream latency testing is implemented in the following way: sending a first test media stream to the media computing power module corresponding to the computing power resource address, so that the media computing power module forwards the first test media stream to at least one called terminal corresponding to the terminal address; The second sending module is used to send the test delay to the server so that the server can determine the synchronization delay corresponding to the call request based on the media stream synchronization strategy of the user's subscribed service and the test delay, and send the synchronization delay to the at least one called terminal so that the at least one called terminal can process the media stream to be transmitted according to the synchronization delay.

9. A holographic communication system for multimedia stream synchronization, characterized in that, The system includes: the service device as described in claim 7 or the terminal device as described in claim 8.

10. A network device, characterized in that, include: processor; as well as A memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of any one of claims 1-6 by executing the executable instructions.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images