Holographic communication method, device, system and computer readable storage medium

Abstract

The present disclosure provides a holographic communication method, device, system and computer readable storage medium, relating to the technical field of holographic communication, the method comprising: receiving a holographic communication call request initiated by a calling user to a called user; determining a first computing power node matched with a calling terminal of the calling user and a second computing power node matched with a called terminal of the called user; sending the holographic communication call request to the called terminal, so that the called terminal and the calling terminal establish a holographic communication connection; sending an address of the first computing power node to the calling terminal, so that the calling terminal sends a first video stream to the first computing power node for holographic processing after the holographic communication connection is established, and then sends the first video stream to the called terminal; and sending an address of the second computing power node to the called terminal, so that the called terminal sends a second video stream to the second computing power node for holographic processing after the holographic communication connection is established, and then sends the second video stream to the calling terminal.

Description

Technical Field

[0001] This disclosure relates to the field of holographic communication technology, and in particular to a holographic communication method, apparatus, system and computer-readable storage medium. Background Technology

[0002] Holographic technology is a technology that can reproduce any object in three dimensions, including the human body, equipment, buildings, natural landscapes, etc.

[0003] Holographic communication is a communication method that organically combines the transmission capabilities and information interaction capabilities of a communication network with holographic technology to provide users with a three-dimensional, immersive, holographic interactive experience for video communication.

[0004] In related technologies, the two parties typically conduct video communication with the support of current 5G service networks. Summary of the Invention

[0005] The inventors noted that, in the relevant technologies, the current 5G service network cannot support the holographic processing and transmission of video streams, thus making it impossible to achieve holographic communication.

[0006] To address the aforementioned problems, the present disclosure proposes the following solutions.

[0007] According to one aspect of the embodiments of this disclosure, a holographic communication method is provided, comprising: receiving a holographic communication call request initiated by a calling user to a called user; determining a first computing power node matching the calling terminal of the calling user and a second computing power node matching the called terminal of the called user; sending the holographic communication call request to the called terminal so that the called terminal and the calling terminal establish a holographic communication connection; sending the address of the first computing power node to the calling terminal so that the calling terminal, after the holographic communication connection is established, sends a first video stream to the first computing power node for holographic processing and then sends it to the called terminal; and sending the address of the second computing power node to the called terminal so that the called terminal, after the holographic communication connection is established, sends a second video stream to the second computing power node for holographic processing and then sends it to the calling terminal.

[0008] In some embodiments, the holographic communication call request carries a first identifier of the calling user and a second identifier of the called user; determining a first computing power node matching the calling terminal and a second computing power node matching the called terminal includes: determining a first computing power requirement of the calling terminal based on the first terminal capability information of the calling terminal corresponding to the first identifier; determining a computing power node that meets the first computing power requirement as a first computing power node matching the calling terminal; determining a second computing power requirement of the called terminal based on the second terminal capability information of the called terminal corresponding to the second identifier; and determining a computing power node that meets the second computing power requirement as a second computing power node matching the called terminal.

[0009] In some embodiments, the method further includes: receiving a first registration request sent by the calling terminal, the first registration request carrying the first identifier and the first terminal capability information; storing the first identifier and the first terminal capability information correspondingly if it is confirmed that the calling user has subscribed to holographic communication service; receiving a second registration request sent by the called terminal, the second registration request carrying the second identifier and the second terminal capability information; storing the second identifier and the second terminal capability information correspondingly if it is confirmed that the called user has subscribed to holographic communication service.

[0010] In some embodiments, the method further includes: receiving a first encryption request sent by a first terminal, wherein the first terminal is one of the calling terminal and the called terminal; generating a first encryption key and a first decryption key according to the first encryption request; sending the first encryption key to the first terminal so that the first terminal uses the first encryption key to encrypt an initial video stream to obtain one of the first video stream and the second video stream; and sending the first decryption key to a second terminal so that the second terminal uses the first decryption key to decrypt one of the first video stream and the second video stream, wherein the second terminal is the other of the calling terminal and the called terminal.

[0011] In some embodiments, the first encryption request carries a first type of the first video data to be encrypted; generating a first encryption key and a first decryption key according to the first encryption request includes: generating a first encryption subkey and a first decryption subkey corresponding to the first type, wherein the first encryption key includes the first encryption subkey and the first decryption key includes the first decryption subkey.

[0012] In some embodiments, the initial video stream includes second video data to be encrypted, and the first encryption request carries a second type of the second video data to be encrypted; generating a first encryption key and a first decryption key according to the first encryption request further includes: generating a second encryption subkey and a second decryption subkey corresponding to the second type, wherein the first encryption key further includes the second encryption subkey, the first decryption key further includes the second decryption subkey, the second encryption subkey is different from the first encryption subkey, and the second decryption subkey is different from the first decryption key.

[0013] In some embodiments, the first encryption subkey and the first decryption subkey are generated using a first algorithm; the second encryption subkey and the second decryption subkey are generated using a second algorithm, wherein the second algorithm is different from the first algorithm.

[0014] In some embodiments, the method further includes: sending a first QoS indicator parameter of the holographic communication service subscribed by the calling user to the network side, so that the network side controls the transmission of the first video stream according to the first QoS indicator parameter; and / or sending a second QoS indicator parameter of the holographic communication service subscribed by the called user to the network side, so that the network side controls the transmission of the second video stream according to the second QoS indicator parameter.

[0015] In some embodiments, the method further includes: determining a third terminal that is within a first preset range from the calling terminal and has registered for holographic communication services, based on the current location of the calling terminal; sending the address of the first computing power node to the third terminal, so that the third terminal sends the collected third video stream to the first computing power node for holographic processing before sending it to the called terminal.

[0016] In some embodiments, the method further includes: determining a fourth terminal that is within a first preset range from the called terminal and has registered for holographic communication services, based on the current location of the called terminal; sending the address of the second computing node to the fourth terminal so that the fourth terminal can send the collected fourth video stream to the second computing node for holographic processing before sending it to the called terminal.

[0017] In some embodiments, the method further includes: the first session management module accessed by the calling terminal receiving the holographic communication call request; sending the holographic communication call request to the called terminal includes: the first session management module sending the holographic communication call request to a second session management module accessed by the called terminal; the second session management module sending the holographic communication call request to the called terminal.

[0018] According to another aspect of the embodiments of this disclosure, a holographic communication device is provided, comprising: a receiving module configured to receive a holographic communication call request initiated by a calling user to a called user; a determining module configured to determine a first computing power node matching the calling terminal of the calling user and a second computing power node matching the called terminal of the called user; a first sending module configured to send the holographic communication call request to the called terminal so that the called terminal and the calling terminal establish a holographic communication connection; a second sending module configured to send the address of the first computing power node to the calling terminal so that the calling terminal sends a first video stream to the first computing power node for holographic processing after the holographic communication connection is established and then sends it to the called terminal; and a third sending module configured to send the address of the second computing power node to the called terminal so that the called terminal sends a second video stream to the second computing power node for holographic processing after the holographic communication connection is established and then sends it to the calling terminal.

[0019] According to another aspect of the present disclosure, a holographic communication device is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute the method described in any of the above embodiments based on instructions stored in the memory.

[0020] According to another aspect of the present disclosure, a holographic communication system is provided, comprising: the holographic communication device described in any of the above embodiments; a first computing node configured to perform holographic processing on the first video stream before sending it to the called terminal; and a second computing node configured to perform holographic processing on the second video stream before sending it to the calling terminal.

[0021] In some embodiments, the system further includes: the calling terminal, configured to receive the address of the first computing power node and send the first video stream to the first computing power node after the holographic communication connection is established; and the called terminal, configured to receive the address of the second computing power node and send the second video stream to the second computing power node after the holographic communication connection is established.

[0022] According to another aspect of the present disclosure, a computer-readable storage medium is provided, including computer program instructions, wherein the computer program instructions, when executed by a processor, implement the method described in any of the above embodiments.

[0023] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program, when executed by a processor, implements the method described in any of the above embodiments.

[0024] In this embodiment, by determining a first computing power node matching the calling terminal of the calling user and sending the address of the first computing power node to the calling terminal, the calling terminal can send the first video stream to the first computing power node for holographic processing after the holographic communication connection is established before sending it to the called terminal. Similarly, by determining a second computing power node matching the called terminal of the called user and sending the address of the second computing power node to the called terminal, the called terminal can send the second video stream to the second computing power node for holographic processing after the holographic communication connection is established before sending it to the calling terminal. This ensures that the first computing power node can perform holographic processing on the first video stream sent by the calling terminal, and also ensures that the second computing power node can perform holographic processing on the second video stream sent by the called terminal, thereby achieving reliable holographic communication.

[0025] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating a holographic communication method according to some embodiments of the present disclosure;

[0028] Figure 2 This is a flowchart illustrating a holographic communication method according to other embodiments of the present disclosure;

[0029] Figure 3 This is a schematic diagram of the structure of a holographic communication system according to some embodiments of the present disclosure;

[0030] Figure 4 This is a flowchart illustrating a holographic communication method according to some embodiments of the present disclosure;

[0031] Figure 5 This is a schematic diagram of the structure of a holographic communication device according to some embodiments of the present disclosure;

[0032] Figure 6 This is a schematic diagram of the structure of a holographic communication device according to other embodiments of the present disclosure;

[0033] Figure 7 This is a schematic diagram of the structure of a holographic communication system according to other embodiments of the present disclosure. Detailed Implementation

[0034] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0035] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0036] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0037] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0038] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0039] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0040] Figure 1 This is a flowchart illustrating a holographic communication method according to some embodiments of the present disclosure.

[0041] In step 102, a holographic communication call request initiated by the calling user to the called user is received.

[0042] In some embodiments, a holographic communication call request initiated by the calling user to the called user can be received through a first session management module accessed by the calling user's calling terminal.

[0043] In some embodiments, a holographic communication call request may carry a first identifier of the calling user and a second identifier of the called user. For example, the first identifier may be the username used by the calling user when registering for the holographic communication service, and the second identifier may be the username used by the called user when registering for the holographic communication service.

[0044] In step 104, a first computing power node matching the calling terminal of the calling user and a second computing power node matching the called terminal of the called user are determined.

[0045] Here, matching the first computing node with the calling terminal can be understood as the first computing node being able to perform holographic processing on the video stream sent by the calling terminal. Similarly, matching the second computing node with the called terminal can be understood as the second computing node being able to perform holographic processing on the video stream sent by the called terminal.

[0046] In step 106, a holographic communication call request is sent to the called terminal so that the called terminal and the calling terminal can establish a holographic communication connection.

[0047] In some embodiments, the first session management module accessed by the calling terminal can send a query request to the data warehouse module regarding the session management module accessed by the called terminal. Upon receiving the query request, the data warehouse module queries which session management module the called terminal is accessing and sends the corresponding query result (e.g., the called terminal is accessing the second session management module) to the first session management module. Furthermore, the first session management module can send the received holographic communication call request to the second session management module accessed by the called terminal, which then forwards the holographic communication call request to the called terminal.

[0048] It should be noted that the data warehouse module can store information such as subscriber information, subscription service information, various terminal information, and registration information for holographic communication services. Among them, the registration information can include the session management module information accessed by various terminals when registering for holographic communication services.

[0049] In step 108, the address of the first computing node is sent to the calling terminal so that after the holographic communication connection is established, the calling terminal can send the first video stream to the first computing node for holographic processing and then send it to the called terminal.

[0050] In some embodiments, the address of the first computing node can be sent to the calling terminal by the first session management module accessed by the calling terminal.

[0051] It should be understood that the first video stream is the video stream captured by the calling terminal during holographic communication between the calling and called users. The first video stream can reflect the environment in which the calling terminal is located.

[0052] It should be noted that holographic processing can be the processing of a video stream using holographic technology, so that the holographically processed video stream can reproduce the content in the video stream in three dimensions.

[0053] In step 110, the address of the second computing node is sent to the called terminal so that the called terminal can send the second video stream to the second computing node for holographic processing after the holographic communication connection is established, and then send it to the calling terminal.

[0054] In some embodiments, the address of the second computing node can be sent to the called terminal by the second session management module accessed by the called terminal.

[0055] It should be understood that the second video stream is the video stream captured by the called terminal during holographic communication between the called and calling users. The second video stream can reflect the environment in which the called terminal is located.

[0056] It should be noted that the execution order of steps 110 and 108 is not limited.

[0057] In the above embodiments, by determining a first computing power node matching the calling terminal of the calling user and sending the address of the first computing power node to the calling terminal, the calling terminal can send the first video stream to the first computing power node for holographic processing after the holographic communication connection is established before sending it to the called terminal. Similarly, by determining a second computing power node matching the called terminal of the called user and sending the address of the second computing power node to the called terminal, the called terminal can send the second video stream to the second computing power node for holographic processing after the holographic communication connection is established before sending it to the calling terminal. In this way, it can be ensured that the first computing power node can perform holographic processing on the first video stream sent by the calling terminal, and that the second computing power node can perform holographic processing on the second video stream sent by the called terminal, thereby achieving reliable holographic communication.

[0058] In some embodiments, the first computing power requirement of the calling terminal can be determined based on the first terminal capability information of the calling terminal corresponding to the first identifier of the calling user. For example, the first terminal capability information may include information such as the video codec types supported by the calling terminal. Accordingly, the first computing power requirement may include the computing power required to encode and decode the video stream sent by the calling terminal, as determined by the video encoding types supported by the calling terminal.

[0059] In some embodiments, a computing power node that meets the first computing power requirement can be determined as the first computing power node matched with the calling terminal. For example, the first computing power node that meets the first computing power requirement can be matched with the calling terminal based on the currently schedulable computing power nodes and the computing power that the computing power nodes can provide.

[0060] In some embodiments, the second computing power requirement of the called terminal can be determined based on the second terminal capability information of the called terminal corresponding to the second identifier of the called user. For example, the second terminal capability information may be information such as the video codec types supported by the called terminal. The second computing power requirement may be the computing power required to encode and decode the video stream sent by the called terminal, determined based on the video encoding types supported by the called terminal.

[0061] In some embodiments, a computing power node that meets the second computing power requirement can be determined as the second computing power node matched with the called terminal. For example, a second computing power node that meets the second computing power requirement can be matched with the called terminal based on the currently schedulable computing power nodes and the computing power that the computing power nodes can provide.

[0062] In the above embodiments, during the determination of the first computing power node, the first computing power requirement determined based on the first terminal capability information of the calling terminal is considered, which can further ensure that the first computing power node can perform holographic processing on the first video stream sent by the calling terminal. Similarly, during the determination of the second computing power node, the second computing power requirement determined based on the second terminal capability information of the called terminal is considered, which can further ensure that the second computing power node can perform holographic processing on the second video stream sent by the called terminal, thereby achieving more reliable holographic communication.

[0063] In some embodiments, the calling terminal and the called terminal may register for holographic communication services separately before engaging in holographic communication.

[0064] Therefore, prior to step 104, the first session management module, which is accessed by the calling terminal, can receive the first registration request sent by the calling terminal. The first registration request may carry the calling user's first identifier and the calling terminal's first terminal capability information. If it is confirmed that the calling user has subscribed to the holographic communication service, the first session management module can store the first identifier and the first terminal capability information accordingly and notify the calling terminal that registration is successful.

[0065] In some embodiments, the first session management module can forward the first registration request to the data warehouse module. The data warehouse module can query the holographic communication service subscription information of the calling user based on the first identifier of the calling user carried in the first registration request, so as to confirm whether the calling user has subscribed to the holographic communication service.

[0066] In some embodiments, if it is confirmed that the calling user has signed up for holographic communication service, the data warehouse module can also store the calling user's first identifier and the information of the first session management module to which the calling user has accessed.

[0067] In some embodiments, if it is confirmed that the calling user has signed up for holographic communication service, the data warehouse module can also send the calling user's holographic communication service signing information to the first session management module, so that the first session management module can store the first identifier, the first terminal capability information and the calling user's holographic communication service signing information accordingly.

[0068] It should be understood that after receiving a holographic communication call request from a calling user to a called user, the first session management module can query the calling user's holographic communication service subscription information based on the calling user's first identifier carried in the holographic communication call request, in order to confirm the calling user's holographic communication service permissions. Only after confirming that the calling user has holographic communication service permissions will the first session management module process the holographic communication call request.

[0069] Similarly, prior to step 104, the called terminal can also register for holographic communication services through the second session management module. For details, please refer to the relevant embodiments described above. Further elaboration will not be repeated here.

[0070] In the above embodiments, the first session management module can store the first identifier of the calling user and the first terminal capability information of the calling terminal in a corresponding manner by processing the first registration request sent by the calling terminal, so that the corresponding information can be directly called when processing the holographic communication call request initiated by the calling user, thereby improving the communication efficiency of holographic communication.

[0071] In some embodiments, the first Quality of Service (QoS) index parameter of the holographic communication service subscribed by the calling user can be sent to the network side so that the network side can control the transmission of the first video stream sent by the calling terminal according to the first QoS index parameter.

[0072] In some embodiments, the second QoS indicator parameters of the holographic communication service subscribed by the called user can be sent to the network side so that the network side can control the transmission of the second video stream sent by the called terminal according to the second QoS indicator parameters.

[0073] It should be noted that QoS parameters can be parameters such as latency, bandwidth, and jitter that the network side must meet during the transmission of the video stream between the transmission start point, intermediate nodes, and transmission end point in holographic communication. For example, the first QoS parameter can be the latency, bandwidth, and jitter parameters that the network side must meet during the transmission of the first video stream from the calling terminal (transmission start point) to the first computing node (intermediate node) and from the first computing node (intermediate node) to the called terminal (transmission end point).

[0074] In the above embodiments, by sending the first QoS indicator parameter (second QoS indicator parameter) of the holographic communication service subscribed by the calling user (called user) to the network side, the network side can control the transmission of the first video stream (second video stream) according to the corresponding parameters, thereby improving the transmission quality of the video stream during transmission and thus improving the communication quality of holographic communication.

[0075] Figure 2This is a flowchart illustrating a holographic communication method according to other embodiments of the present disclosure.

[0076] and Figure 1 Compared to the embodiments shown, Figure 2 The illustrated embodiment also includes steps 202 to 208.

[0077] In step 202, a first encryption request sent by the first terminal is received.

[0078] Here, the first terminal can be either the calling terminal or the called terminal.

[0079] For example, the first terminal can identify data in the unencrypted initial video stream to determine whether there is data to be encrypted in the initial video stream. If the initial video stream contains data such as fingerprints, facial features, or gait data that needs to be encrypted, the first terminal will send a first encryption request.

[0080] It should be understood that the initial video stream is an unencrypted video stream, such as the original video stream captured by the first terminal, or a video stream after other processing of the original video stream.

[0081] In step 204, a first encryption key and a first decryption key are generated based on the first encryption request.

[0082] In some embodiments, the first encryption key may include at least one encryption subkey, and the first decryption key may include at least one decryption subkey.

[0083] In some embodiments, the initial video stream may include first video data to be encrypted. The first encryption request may carry a first type of the first video data to be encrypted. For example, the first video data to be encrypted may be facial data, and its type is facial type.

[0084] In some embodiments, a first encryption subkey and a first decryption subkey corresponding to a first type can be generated according to a first encryption request.

[0085] In step 206, the first encryption key is sent to the first terminal so that the first terminal can use the first encryption key to encrypt the initial video stream to obtain one of the first video stream and the second video stream.

[0086] In some embodiments, if the first terminal is the calling terminal, the first session management module can send the first encryption key to the calling terminal.

[0087] In other embodiments, if the first terminal is the called terminal, the second session management module can send the second encryption key to the called terminal.

[0088] In step 208, the first decryption key is sent to the second terminal so that the second terminal can use the first decryption key to decrypt one of the first video stream and the second video stream.

[0089] Here, the second terminal is the other one of the calling terminal and the called terminal.

[0090] In some embodiments, if the first terminal is the calling terminal, then the second terminal is the called terminal. For example, the first decryption key can be sent to the called terminal by the second session management module.

[0091] In other embodiments, if the first terminal is the called terminal, then the second terminal is the calling terminal. For example, the first session management module can send the first decryption key to the calling terminal.

[0092] In the above embodiments, corresponding encryption and decryption keys can be generated according to the type of video data to be encrypted in the initial video stream, thereby improving the security of the video stream during transmission and thus improving the security of holographic communication.

[0093] In some embodiments, the initial video stream may include first video data to be encrypted and second video data to be encrypted. The first encryption request may carry a first type of the first video data to be encrypted and a second type of the second video data to be encrypted.

[0094] In some embodiments, a first encryption subkey and a first decryption subkey corresponding to a first type can be generated according to the first encryption request, as well as a second encryption subkey and a second decryption subkey corresponding to a second type, wherein the first encryption subkey and the second encryption subkey are different, and the first decryption subkey and the second decryption subkey are different.

[0095] In the above embodiments, different encryption and decryption keys can be generated according to different types of video data to be encrypted in the initial video stream, thereby further improving the security of the video stream during transmission and thus further improving the security of holographic communication.

[0096] In some embodiments, a first encryption subkey and a first decryption subkey can be generated using a first algorithm, and a second encryption subkey and a second decryption subkey can be generated using a second algorithm, wherein the first algorithm and the second algorithm are different, the first encryption subkey and the second encryption subkey are different, and the first decryption subkey and the second decryption subkey are different.

[0097] In the above embodiments, different encryption subkeys and different decryption subkeys can be generated through different algorithms, thereby further improving the security of video streams during transmission and thus further improving the security of holographic communication.

[0098] In some embodiments, a third terminal that is within a first preset range from the calling terminal and is registered for holographic communication services can be determined based on the current location of the calling terminal. For example, the third terminal may be a pre-deployed terminal registered for holographic communication services, and its relevant information (such as deployment location, terminal capability information, etc.) may be stored in a data warehouse module.

[0099] In some embodiments, the first preset range may be a range pre-set in the first session management module accessed by the calling terminal. For example, the area containing a circle with the current location of the calling terminal as the center and a radius of 5 meters may be pre-set as the first preset range.

[0100] In some embodiments, the first session management module accessed through the calling terminal can send the current location of the calling terminal to the data warehouse module, so that the data warehouse module can determine a third terminal that is within a first preset range of distance from the calling terminal and has registered for holographic communication services, and then wake up the third terminal to collect the video stream collected when the calling user and the called user are conducting holographic communication.

[0101] It should be understood that the third video stream can also reflect the environment in which the calling terminal is located. The third video stream, together with the first video stream, can provide a more comprehensive picture of the environment in which the calling terminal is located.

[0102] In some embodiments, the address of the first computing node can be sent to the third terminal so that the third terminal can send the acquired third video stream to the first computing node for holographic processing before sending it to the called terminal. For example, the address of the first computing node can be sent to the third terminal by the first session management module.

[0103] In the above embodiments, a third video stream reflecting the environment of the calling terminal can also be collected by a third terminal, and the third video stream can be sent to the second computing node for holographic processing before being sent to the called terminal. In this way, the called user can obtain a picture presented by the first video stream and the third video stream, thereby further improving the communication quality of holographic communication.

[0104] In some embodiments, a fourth terminal that is within a second preset range from the called terminal and is registered for holographic communication services can be determined based on the current location of the called terminal. For example, the fourth terminal may be a pre-deployed terminal registered for holographic communication services, and its relevant information (such as deployment location, terminal capability information, etc.) may be stored in a data warehouse module.

[0105] In some embodiments, the second preset range may be a range preset in the second session management module accessed by the called terminal. For example, the area containing a circle with the current location of the called terminal as the center and a radius of 3 meters may be preset as the second preset range.

[0106] In some embodiments, the second session management module accessed by the called terminal can send the current location of the called terminal to the data warehouse module, so that the data warehouse module can determine the fourth terminal that is within a second preset range of distance from the called terminal and has registered holographic communication services, and then wake up the fourth terminal to collect the video stream collected when the called user and the calling user are conducting holographic communication.

[0107] It should be understood that the fourth video stream can also reflect the environment in which the called terminal is located. The fourth video stream, together with the second video stream, can provide a more comprehensive picture of the environment in which the called terminal is located.

[0108] In some embodiments, the address of the second computing node can be sent to the fourth terminal so that the fourth terminal can send the acquired fourth video stream to the second computing node for holographic processing before sending it to the calling terminal. For example, the address of the second computing node can be sent to the fourth terminal by the second session management module.

[0109] In the above embodiments, a fourth video stream reflecting the environment of the called terminal can also be collected by the fourth terminal, and the fourth video stream can be sent to the second computing node for holographic processing before being sent to the calling terminal. In this way, the calling user can obtain the picture presented by the second video stream and the fourth video stream, thereby further improving the communication quality of holographic communication.

[0110] Figure 3 This is a schematic diagram of the structure of a holographic communication system according to some embodiments of the present disclosure.

[0111] like Figure 3 As shown, the holographic communication system 300 may include a service control layer and a mixed-media distributed access and computing layer. The service control layer includes a session management module, a data warehouse module, a sensitive media security management module, a mixed-media computing resource intelligent scheduling module, and a QoS rule engine module. The mixed-media distributed access and computing layer includes multiple mixed-media computing modules, such as mixed-media computing module 1 to mixed-media computing module n.

[0112] The business control layer is configured to provide business process control services for holographic communication services.

[0113] The session management module is configured to provide services such as registration, authentication, business data management, and call session control for various terminals of holographic communication service subscribers.

[0114] The data warehouse module is configured to provide a unified storage and management service for user information, business information, and registration information of various terminals of holographic communication service subscribers.

[0115] The sensitive media security management module is configured to provide encrypted management services for information that needs to be encrypted during holographic communication (such as fingerprints, images, gait information, etc.).

[0116] The intelligent scheduling module for mixed media computing resources is configured to provide intelligent scheduling services for various video streams that require holographic processing during holographic communication, and to provide mixed media computing power modules deployed in the network.

[0117] The intelligent scheduling module for hybrid computing resources is configured to store information about schedulable computing nodes (such as the computing power and address of schedulable computing nodes) so that when a terminal's computing power requirement is received, a computing node that meets that requirement can be allocated to the terminal.

[0118] The QoS rules engine module is configured to provide a unified storage and management service for QoS indicator parameters of various user-subscribed holographic communication services. QoS indicator parameters can be parameters such as latency, bandwidth, and jitter that the network side must meet during the transmission of the video stream between the transmission start point, intermediate nodes, and transmission end point in holographic communication.

[0119] The mixed-media distributed access and computing layer is configured to provide services for holographic processing of video streams transmitted during holographic communication.

[0120] Each mixed media computing module is configured to perform holographic processing of various video streams under the scheduling of the mixed media computing resource intelligent scheduling module.

[0121] It should be noted that, Figure 3 The hybrid computing module in the illustrated embodiment can be Figure 1 The computing node in the illustrated embodiment.

[0122] Figure 4 This is a flowchart illustrating a holographic communication method according to some embodiments of the present disclosure.

[0123] Figure 4 For a detailed description of each module involved in the illustrated embodiment, please refer to [link / reference needed]. Figure 3 The illustrated embodiment.

[0124] In step 402, the first session management module receives the holographic communication call request sent by the calling terminal.

[0125] Here, the first session management module is the session management module accessed by the calling terminal. The holographic communication call request is initiated by the calling user to the called user, and the holographic communication call request carries the first identifier of the calling user and the second identifier of the called user.

[0126] In some embodiments, the first identifier may be the username used by the calling user when registering for the holographic communication service, and the second identifier may be the username used by the called user when registering for the holographic communication service.

[0127] In step 404, the first session management module confirms whether the calling user has the service permission for holographic communication services.

[0128] It should be noted that the first session management module can store the calling user's first identifier and holographic communication service subscription information. This allows the module to query the calling user's holographic communication service subscription information based on the calling user's first identifier carried in the holographic communication call request, thereby confirming whether the calling user has holographic communication service permissions. If it is confirmed that the calling user has holographic communication service permissions, the first session management module will execute the subsequent step 406.

[0129] In step 406, the first session management module sends a query request to the data warehouse module for the session management module accessed by the called user.

[0130] Here, the query request may include a second identifier of the called user.

[0131] For example, the data warehouse module can query the session management module accessed by the called user based on the called user's second identifier, which is the second session management module.

[0132] In step 408, the data warehouse module sends the query result of the called user accessing the second session management module to the first session management module.

[0133] In step 410, the first session management module forwards the holographic communication call request to the second session management module.

[0134] In step 412, the first session management module sends the first computing power request of the calling terminal to the mixed media computing power resource intelligent scheduling module.

[0135] In some embodiments, the first session management module can determine the first computing power requirement of the calling terminal based on the first terminal capability information of the calling terminal corresponding to the first identifier of the calling user.

[0136] In step 414, the intelligent scheduling module for mixed media computing resources determines the first mixed media computing module that matches the calling terminal.

[0137] In some embodiments, the intelligent scheduling module for mixed-media computing resources can determine the mixed-media computing power module that meets the first computing power requirement as the first mixed-media computing power module to be matched with the calling terminal. For example, the intelligent scheduling module for mixed-media computing power resources can match the calling terminal with the first mixed-media computing power module that meets the first computing power requirement based on the currently schedulable mixed-media computing power modules and the computing power that the mixed-media computing power modules can provide.

[0138] In step 416, the intelligent scheduling module for mixed media computing resources sends the address of the first mixed media computing module to the first session management module.

[0139] In step 418, the first session management module receives the first encryption request sent by the calling terminal.

[0140] In some embodiments, the calling terminal can identify data in the unencrypted initial video stream to determine whether there is data to be encrypted in the initial video stream. If the initial video stream contains data such as fingerprints, facial features, or gait data that needs to be encrypted, the calling terminal will send a first encryption request to the first session management module.

[0141] In step 420, the first session management module forwards the first encryption request to the sensitive media security control module.

[0142] In step 422, the sensitive media security control module generates a first encryption key and a first decryption key based on the first encryption request.

[0143] After receiving the first encryption request, the sensitive media security control module can generate a first encryption key and a first decryption key according to the specific implementation method in step 204. For detailed explanation, please refer to the relevant embodiments in step 204, which will not be repeated here.

[0144] In step 424, the sensitive media security control module sends the first encryption key to the first session management module.

[0145] In step 426, the sensitive media security control module sends the first decryption key to the second session management module.

[0146] In step 428, the first session management module sends the first QoS request to the QoS rules engine module.

[0147] In some embodiments, the first QoS request may carry a first identifier of the calling user, so that the QoS rule engine module can determine the first QoS indicator parameters of the holographic communication service subscribed by the calling user based on the first identifier.

[0148] In step 430, the QoS rule engine module sends the first QoS indicator parameter to the network side so that the network side can control the transmission of the first video stream according to the first QoS indicator parameter.

[0149] It should be noted that QoS parameters can be parameters such as latency, bandwidth, and jitter that the network side must meet during the transmission of the video stream between the transmission start point, intermediate nodes, and transmission end point in holographic communication. For example, the first QoS parameter can be the latency, bandwidth, and jitter parameters that the network side must meet during the transmission of the first video stream from the calling terminal (transmission start point) to the first computing node (intermediate node) and from the first computing node (intermediate node) to the called terminal (transmission end point).

[0150] In step 432, the first session management module sends the first encryption key to the calling terminal so that the calling terminal can use the first encryption key to encrypt the initial video stream to obtain the first video stream.

[0151] In step 434, the first session management module sends the address of the first mixed media computing power module to the calling terminal, so that after the holographic communication connection is established, the calling terminal sends the first video stream to the first mixed media computing power module for holographic processing and then sends it to the called terminal.

[0152] It should be understood that the first video stream is the video stream captured by the calling terminal during holographic communication between the calling and called users. The first video stream can reflect the environment in which the calling terminal is located.

[0153] In step 436, the second session management module sends the first decryption key to the called terminal so that the called terminal can decrypt the received first video stream.

[0154] In step 438, the second session management module sends a holographic communication call request to the called terminal so that the called terminal and the calling terminal can establish a holographic communication connection.

[0155] In step 440, the calling terminal sends the first video stream to the first mixed media computing module.

[0156] In step 442, the first mixed media computing module performs holographic processing on the first video stream.

[0157] It should be noted that holographic processing can be the processing of a video stream using holographic technology, so that the holographically processed video stream can reproduce the content in the video stream in three dimensions.

[0158] In step 444, the first mixed media computing module sends the holographically processed first video stream to the called terminal.

[0159] It should be noted that after receiving the holographic communication call request from the first session management module, the second session management module can combine the intelligent scheduling module for mixed media computing resources, the sensitive media security control module, and the QoS rule engine module to execute the relevant steps of the called terminal during the holographic communication process. The specific implementation method is similar to the implementation methods shown in steps 412 to 436 above. After the holographic communication connection is established, the called terminal can send the second video stream to the corresponding mixed media computing power module for holographic processing before sending it to the calling terminal. The specific implementation method is similar to the implementation methods in steps 440 to 444 above. Further details will not be provided here.

[0160] It should be noted that, Figure 4 The execution order of the steps in the illustrated embodiments is not limited.

[0161] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus embodiments, since they largely correspond to the method embodiments, the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0162] Figure 5 This is a schematic diagram of the structure of a holographic communication device according to some embodiments of the present disclosure.

[0163] like Figure 5 As shown, the holographic communication device 500 includes a receiving module 501 configured to receive a holographic communication call request initiated by a calling user to a called user; a determining module 502 configured to determine a first computing power node matching the calling user's calling terminal and a second computing power node matching the called user's called terminal; a first sending module 503 configured to send the holographic communication call request to the called terminal so that the called terminal and the calling terminal can establish a holographic communication connection; a second sending module 504 configured to send the address of the first computing power node to the calling terminal so that the calling terminal can send a first video stream to the first computing power node for holographic processing after the holographic communication connection is established before sending it to the called terminal; and a third sending module 505 configured to send the address of the second computing power node to the called terminal so that the called terminal can send a second video stream to the second computing power node for holographic processing after the holographic communication connection is established before sending it to the calling terminal.

[0164] Figure 6 This is a schematic diagram of the structure of a holographic communication device according to other embodiments of the present disclosure.

[0165] like Figure 6As shown, the holographic communication device 600 includes a memory 601 and a processor 602 coupled to the memory 601. The processor 602 is configured to execute the method of any of the foregoing embodiments based on instructions stored in the memory 601.

[0166] The memory 601 may include, for example, system memory, fixed non-volatile storage media, etc. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs.

[0167] The holographic communication device 600 may also include an input / output interface 603, a network interface 604, and a storage interface 607. These interfaces 603, 604, and 607, as well as the memory 601 and processor 602, can be connected, for example, via a bus 606. The input / output interface 603 provides a connection interface for input / output devices such as displays, mice, keyboards, and touchscreens. The network interface 604 provides a connection interface for various networked devices. The storage interface 607 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0168] Figure 7 This is a schematic diagram of the structure of a holographic communication system according to other embodiments of the present disclosure.

[0169] like Figure 7 As shown, the holographic communication system 700 may include the holographic communication device 500 / 600 of any of the above embodiments, a first computing node 701, and a second computing node 702. The first computing node 701 is configured to perform holographic processing on the first video stream before sending it to the called terminal; the second computing node 702 is configured to perform holographic processing on the second video stream before sending it to the calling terminal.

[0170] In some embodiments, the holographic communication system 700 may further include a calling terminal 703 and a called terminal 704. The calling terminal 703 may be configured to receive the address of the first computing node 701 and send a first video stream to the first computing node 701 after the holographic communication connection is established; the called terminal 704 may be configured to receive the address of the second computing node 702 and send a second video stream to the second computing node 702 after the holographic communication connection is established.

[0171] This disclosure also provides a computer-readable storage medium including computer program instructions that, when executed by a processor, implement the method of any of the above embodiments.

[0172] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the method of any of the above embodiments.

[0173] The embodiments of this disclosure have now been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein based on the above description.

[0174] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0175] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that the functions specified in one or more flowchart illustrations and / or one or more blocks in a block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate functions for implementing the functions in the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0176] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0177] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0178] While specific embodiments of this disclosure have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments or equivalent substitutions can be made to some technical features without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A holographic communication method, comprising: Receive holographic communication call requests initiated by the calling user to the called user; Determine a first computing power node that matches the calling terminal of the calling user and a second computing power node that matches the called terminal of the called user; The holographic communication call request is sent to the called terminal so that the called terminal and the calling terminal can establish a holographic communication connection. The address of the first computing power node is sent to the calling terminal so that the calling terminal can send the first video stream to the first computing power node for holographic processing after the holographic communication connection is established, and then send it to the called terminal. The address of the second computing node is sent to the called terminal, so that after the holographic communication connection is established, the called terminal sends the second video stream to the second computing node for holographic processing and then sends it to the calling terminal.

2. The method according to claim 1, wherein, The holographic communication call request carries the first identifier of the calling user and the second identifier of the called user; Determining the first computing power node matched with the calling terminal and the second computing power node matched with the called terminal includes: The first computing power requirement of the calling terminal is determined based on the first terminal capability information of the calling terminal corresponding to the first identifier; The computing power node that meets the first computing power requirement is determined as the first computing power node that matches the calling terminal. The second computing power requirement of the called terminal is determined based on the second terminal capability information of the called terminal corresponding to the second identifier; The computing power node that meets the second computing power requirement is determined as the second computing power node that matches the called terminal.

3. The method according to claim 2, further comprising: Receive a first registration request sent by the calling terminal, wherein the first registration request carries the first identifier and the first terminal capability information; If it is confirmed that the calling user has signed up for holographic communication service, the first identifier and the first terminal capability information are stored accordingly. Receive a second registration request sent by the called terminal, the second registration request carrying the second identifier and the second terminal capability information; If it is confirmed that the called user has signed up for holographic communication service, the second identifier and the second terminal capability information are stored accordingly.

4. The method according to claim 1, further comprising: Receive a first encryption request sent by a first terminal, wherein the first terminal is one of the calling terminal and the called terminal; Generate a first encryption key and a first decryption key according to the first encryption request; The first encryption key is sent to the first terminal so that the first terminal can use the first encryption key to encrypt the initial video stream to obtain one of the first video stream and the second video stream; The first decryption key is sent to the second terminal so that the second terminal can use the first decryption key to decrypt one of the first video stream and the second video stream. The second terminal is the other of the calling terminal and the called terminal.

5. The method according to claim 4, wherein: The initial video stream includes first video data to be encrypted, and the first encryption request carries a first type of the first video data to be encrypted. Generating a first encryption key and a first decryption key according to the first encryption request includes: Generate a first encryption subkey and a first decryption subkey corresponding to the first type, wherein the first encryption key includes the first encryption subkey and the first decryption key includes the first decryption subkey.

6. The method according to claim 5, wherein: The initial video stream also includes second video data to be encrypted, and the first encryption request carries a second type of the second video data to be encrypted. Generating the first encryption key and the first decryption key according to the first encryption request further includes: Generate a second encryption subkey and a second decryption subkey corresponding to the second type. The first encryption key further includes the second encryption subkey, and the first decryption key further includes the second decryption subkey. The second encryption subkey is different from the first encryption subkey, and the second decryption subkey is different from the first decryption subkey.

7. The method according to claim 6, wherein: The first encryption subkey and the first decryption subkey are generated using the first algorithm; The second encryption subkey and the second decryption subkey are generated using a second algorithm, which is different from the first algorithm.

8. The method according to any one of claims 1-7, further comprising: The first Quality of Service (QoS) index parameter of the holographic communication service subscribed by the calling user is sent to the network side so that the network side can control the transmission of the first video stream according to the first QoS index parameter. and / or The second QoS indicator parameter of the holographic communication service subscribed by the called user is sent to the network side so that the network side can control the transmission of the second video stream according to the second QoS indicator parameter.

9. The method according to any one of claims 1-7, further comprising: Based on the current location of the calling terminal, a third terminal that is within a first preset range from the calling terminal and has registered for holographic communication services is identified. The address of the first computing node is sent to the third terminal, so that the third terminal can send the collected third video stream to the first computing node for holographic processing before sending it to the called terminal.

10. The method according to any one of claims 1-7, further comprising: Based on the current location of the called terminal, a fourth terminal that is within a second preset range and has registered for holographic communication services is determined; The address of the second computing node is sent to the fourth terminal, so that the fourth terminal can send the collected fourth video stream to the second computing node for holographic processing before sending it to the called terminal.

11. The method according to any one of claims 1-7, wherein, The first session management module accessed by the calling terminal receives the holographic communication call request; Sending the holographic communication call request to the called terminal includes: The first session management module sends the holographic communication call request to the second session management module accessed by the called terminal; The second session management module sends the holographic communication call request to the called terminal.

12. A holographic communication device, comprising: The receiving module is configured to receive holographic communication call requests initiated by the calling user to the called user; The determination module is configured to determine a first computing power node that matches the calling terminal of the calling user and a second computing power node that matches the called terminal of the called user. The first sending module is configured to send the holographic communication call request to the called terminal so that the called terminal and the calling terminal can establish a holographic communication connection. The second sending module is configured to send the address of the first computing power node to the calling terminal, so that the calling terminal can send the first video stream to the first computing power node for holographic processing after the holographic communication connection is established, and then send it to the called terminal. The third sending module is configured to send the address of the second computing power node to the called terminal, so that the called terminal can send the second video stream to the second computing power node for holographic processing after the holographic communication connection is established, and then send it to the calling terminal.

13. A holographic communication device, comprising: Memory; as well as A processor coupled to the memory is configured to execute the method of any one of claims 1-11 based on instructions stored in the memory.

14. A holographic communication system, comprising: The holographic communication device according to claim 12 or 13; The first computing node is configured to perform holographic processing on the first video stream before sending it to the called terminal; The second computing node is configured to perform holographic processing on the second video stream before sending it to the calling terminal.

15. The system of claim 14, further comprising: The calling terminal is configured to receive the address of the first computing power node and send the first video stream to the first computing power node after the holographic communication connection is established. The called terminal is configured to receive the address of the second computing node and send the second video stream to the second computing node after the holographic communication connection is established.

16. A computer-readable storage medium comprising computer program instructions, wherein, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1-11.

17. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the method described in any one of claims 1-11.

Images