Information processing methods, information transmission method, terminal, system, device and medium

Abstract

The present application provides information processing methods, an information transmission method, a user terminal, a communication system, a network device and a computer-readable storage medium. An information processing method comprises: performing feature extraction on collected first multi-dimensional image data, so as to obtain feature information of the first multi-dimensional image data; converting the feature information into first behavior instruction data, the first behavior instruction data being used for indicating behavior information of a first user to which a first user terminal belongs; and transmitting the first behavior instruction data to a second user terminal via a dedicated channel, the dedicated channel being a data transmission channel established in advance on the basis of a real-time communication technology, and the second user terminal being configured to perform information reconstruction according to the first behavior instruction data, so as to obtain reconstructed first multi-dimensional image data.

Description

Information processing method, information transmission method, terminal, system, device and medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202311216083.3 filed on September 19, 2023, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present application relates to the field of communication technology, and in particular to an information processing method, an information transmission method, a user terminal, a communication system, a network device, and a computer-readable storage medium. Background Art

[0004] With the development of the fifth generation mobile communication technology (5G) and the sixth generation wireless systems (6G) networks, various new video communication methods have emerged, including virtual reality (VR), augmented reality (AR), and holographic projection.

[0005] While 5G and 6G wireless networks boast increasingly faster uplink and downlink speeds, and their bandwidths are increasing, the data volumes transmitted by various applications are also growing. For example, for holographic video communication, for a life-size display or a 70-inch TV screen, the dynamic transmission volume of a single data stream is approximately 1.9 Tbps, with a required latency of 1 to 5 ms. If applied to multi-person conferencing scenarios, the data volume will increase exponentially, making it difficult for existing wireless bandwidth to meet these requirements, resulting in low bandwidth utilization.

[0006] Public content

[0007] The present application provides an information processing method, which is applied to a first user terminal, including: performing feature extraction on collected first multidimensional image data to obtain feature information of the first multidimensional image data; converting the feature information into first behavior instruction data, wherein the first behavior instruction data is used to indicate behavior information of a first user to which the first user terminal belongs; transmitting the first behavior instruction data to a second user terminal via a dedicated channel, wherein the dedicated channel is a data transmission channel established in advance based on real-time communication technology, and the second user terminal is configured to reconstruct information based on the first behavior instruction data to obtain reconstructed first multidimensional image data.

[0008] The present application provides an information processing method, which is applied to a second user terminal, comprising: receiving first behavior instruction data of a first user terminal through a dedicated channel; the dedicated channel is a data transmission channel established in advance based on real-time communication technology, the first behavior instruction data is the behavior instruction data obtained by the first user terminal performing feature extraction on collected first multidimensional image data and converting the feature information of the extracted first multidimensional image data, and the first behavior instruction data is used to indicate behavior information of a first user to whom the first user terminal belongs; and information is reconstructed based on the first behavior instruction data to obtain reconstructed first multidimensional image data.

[0009] The present application provides an information transmission method, which is applied to a communication system, comprising: receiving first behavior instruction data from a first user terminal, where the first behavior instruction data is behavior instruction data obtained by the first user terminal performing feature extraction on collected first multidimensional image data and converting feature information of the extracted first multidimensional image data, and the first behavior instruction data is used to indicate behavior information of a first user to which the first user terminal belongs; transmitting the first behavior instruction data to a second user terminal via a dedicated channel, where the dedicated channel is a data transmission channel established in advance based on real-time communication technology, and the second user terminal is configured to reconstruct information based on the first behavior instruction data to obtain reconstructed first multidimensional image data.

[0010] The present application provides a user terminal, comprising: an extraction module, configured to perform feature extraction on collected first multidimensional image data to obtain feature information of the first multidimensional image data; a conversion module, configured to convert the feature information into first behavior instruction data, wherein the first behavior instruction data is used to indicate behavior information of a first user to which the first user terminal belongs; a transmission module, configured to transmit the first behavior instruction data to a second user terminal via a dedicated channel, wherein the dedicated channel is a data transmission channel pre-established based on real-time communication technology, and the second user terminal is configured to reconstruct information based on the first behavior instruction data to obtain reconstructed first multidimensional image data.

[0011] The present application provides a user terminal, comprising: a receiving module, configured to receive first behavior instruction data of a first user terminal through a dedicated channel; the dedicated channel is a data transmission channel established in advance based on real-time communication technology, the first behavior instruction data is behavior instruction data obtained by the first user terminal performing feature extraction on collected first multidimensional image data and converting the feature information of the extracted first multidimensional image data, and the first behavior instruction data is used to indicate behavior information of a first user to whom the first user terminal belongs; a reconstruction module, configured to perform information reconstruction based on the first behavior instruction data to obtain reconstructed first multidimensional image data.

[0012] The present application provides a communication system, including: a receiving module, configured to receive first behavior instruction data of a first user terminal, where the first behavior instruction data is behavior instruction data obtained by the first user terminal performing feature extraction on collected first multidimensional image data and converting feature information of the extracted first multidimensional image data, and the first behavior instruction data is used to indicate behavior information of a first user to which the first user terminal belongs; a transmission module, configured to transmit the first behavior instruction data to a second user terminal via a dedicated channel, where the dedicated channel is a data transmission channel established in advance based on real-time communication technology, and the second user terminal is configured to reconstruct information based on the first behavior instruction data to obtain reconstructed first multidimensional image data.

[0013] The present application provides a network device, comprising: at least one processor; a memory on which at least one computer program is stored, and the at least one computer program is executed by the at least one processor, so that the at least one processor implements the above-mentioned information processing method of the present application.

[0014] The present application provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor, so that the computer program implements the above-mentioned information processing method of the present application.

[0015] With respect to the above embodiments and other aspects of the present application and their implementation, further description is provided in the accompanying drawings, detailed description and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] FIG1 shows a flowchart of an information processing method provided in the present application and applied to a first user terminal.

[0017] FIG2 is a schematic diagram showing the principles of holographic image acquisition and generation provided by the present application.

[0018] FIG3 is a schematic diagram showing the principle of holographic image reconstruction provided by the present application.

[0019] FIG4 illustrates an information processing method provided by the present application and applied to a second user terminal.

[0020] FIG5 shows an information processing method provided by the present application and applied to a communication system.

[0021] FIG6 shows a schematic diagram of an exemplary architecture of a communication system provided in this application.

[0022] FIG7 shows a schematic diagram of an exemplary detailed architecture of the communication system provided in this application.

[0023] FIG8 shows an exemplary flowchart of an information processing method provided in the present application and applied to a first user terminal.

[0024] FIG9 shows an exemplary flowchart of the information processing method provided in the present application and applied to the second user terminal.

[0025] FIG10 shows a schematic structural diagram of a first user terminal provided in this application.

[0026] FIG11 shows a schematic structural diagram of a second user terminal provided in this application.

[0027] FIG12 shows a schematic structural diagram of the communication system provided in this application.

[0028] FIG13 shows a block diagram of an exemplary hardware architecture of a computing device capable of implementing the methods and apparatuses according to the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of this application more clear, the following will describe the embodiments of this application in detail with reference to the accompanying drawings. It should be noted that, unless there is a conflict, the various embodiments and features in the embodiments of this application can be combined with each other in any way.

[0030] With the development of 5G and 6G networks, various new video communication methods have emerged, including virtual reality (VR), augmented reality (AR), and holographic projection. VR is a computer simulation system that allows users to create and experience virtual worlds. It uses computers to generate a simulated environment and exploits human binocular parallax to achieve a three-dimensional dynamic visual experience and physical behavior system simulation by fusing multi-source information, immersing users in the virtual environment. AR is a technology that calculates the position and angle of camera images in real time and adds corresponding images. It seamlessly integrates real-world and virtual-world information, aiming to overlay virtual images on the real world on a screen. Holography uses the principle of interference to record and reconstruct three-dimensional images of objects using diffraction. Holography records and reconstructs all information about the amplitude and phase of an object's light waves, resulting in a three-dimensional image that is identical to the original. This three-dimensional image can be used to observe the reconstructed virtual image of the hologram from different angles, allowing different sides of the object to be seen, creating a visual effect and a sense of depth.

[0031] Holography differs from conventional photography. In conventional photography, light emitted by an object, or reflected and refracted by its surface, is imaged through an objective lens, the light intensity recorded on a photographic plate, and the flat image of the object is then displayed on photographic paper. Holography, on the other hand, is a lensless imaging method that uses the principle of light interference to record all information about the object, namely its amplitude and phase, on a holographic plate. Hence the name holography. When a holographic image is reproduced, the object appears three-dimensional and lifelike.

[0032] Digital holography uses a charge-coupled device (CCD) to record the interference fringes produced by the coherent superposition of light waves reflected from an object and a reference light wave (a real-time recording process). This process is then used to reconstruct the object's appearance in a computer. This process is known as digital reconstruction. Because this technology records not only the amplitude but also the phase information of the object, it can more realistically represent the original object.

[0033] While 5G and 6G wireless networks are experiencing increasingly rapid uplink and downlink speeds, and bandwidth is increasing, the data volumes required for various applications are also increasing. For example, for holographic video communication, for a life-size display or a 70-inch TV screen, the dynamic transmission volume of a single data stream is approximately 1.9 Tbps, with a required latency of 1 to 5 ms. If this is applied to multi-person conferencing scenarios, the data volume will increase exponentially, making it difficult for existing wireless bandwidth to meet these requirements.

[0034] In related technologies, video content can be compressed to reduce its data size. However, this method is generally only applicable to two-dimensional images and is not suitable for the compression of three-dimensional images such as VR, AR, and holograms. In related technologies, no technology has yet been found to achieve real-time holographic communication. Furthermore, due to the differences in the terminals used by the communicating parties, the use of video compression has shortcomings when adapting to different types of terminals. For example, when a mobile phone communicates with a dedicated large-screen device, the images captured on the mobile phone will become blurry when displayed on the dedicated large-screen device.

[0035] The present application provides an information processing method, an information transmission method, a user terminal, a communication system, a network device and a computer-readable storage medium, which can reduce bandwidth occupancy and improve bandwidth utilization.

[0036] FIG1 is a flowchart of an information processing method provided in the present application and applied to a first user terminal.

[0037] In a first aspect, referring to FIG. 1 , the present application provides an information processing method, which can be applied to a first user terminal. The method includes steps S110 to S130 .

[0038] S110 , performing feature extraction on the collected first multi-dimensional image data to obtain feature information of the first multi-dimensional image data.

[0039] In this step, the multidimensional image data includes at least one of a multidimensional image and corresponding audio data. The multidimensional image has a dimension greater than or equal to 2. For example, the multidimensional image can be a two-dimensional image, a three-dimensional image, or an image of higher dimensions. When the multidimensional image is a three-dimensional image, it can be at least one of VR image data, VR video data, AR image data, AR video data, holographic image data, and holographic video data.

[0040] In this step, exemplarily, when the first multidimensional image data includes a multidimensional image, the corresponding feature information includes but is not limited to: at least one of features such as color, shape, texture, depth, etc.; when the first multidimensional image data includes corresponding audio data, the corresponding feature information includes but is not limited to: at least one of tone and intonation.

[0041] The following uses multi-dimensional image data as holographic image data as an example to illustrate the principles of collecting and generating holographic image data. Figure 2 is a schematic diagram of the principles of collecting and generating holographic images provided by this application.

[0042] As shown in Figure 2, the holographic image data acquisition device is a laser. The laser emits a laser beam, which is split into two by a beam splitter. One beam, called the object beam, shines on the object being photographed; the other beam, called the reference beam, shines directly on the photosensitive film, or holographic plate. When the beam is reflected by the object, its reflected beam also shines on the film, completing the holographic recording process.

[0043] As can be seen from Figure 2, when collecting digital holographic images, the object light wave emitted from the object surface interferes with the reference light wave on the CCD target surface, and the CCD records the light intensity distribution and sends it to the computer for storage, forming a digital matrix, namely a digital hologram.

[0044] In this step, the first user terminal is a terminal belonging to the first user, and the first user's terminal performs a data acquisition operation, such as a digital holographic image and related sensor data.

[0045] S120: Convert the characteristic information into first behavior instruction data, where the first behavior instruction data is used to indicate behavior information of a first user belonging to the first user terminal.

[0046] For example, the first behavioral instruction data may include: rotating the body 15 degrees clockwise, tilting the head up 45 degrees, etc. In this step, the feature information of the first multi-dimensional image data may be converted into behavioral instruction data corresponding to the user, so as to perform behavioral recognition on the first user. Behavior recognition is the process of extracting, representing, and recognizing motion features from a video sequence (e.g., multiple frames of images) containing user motion behaviors, and falls within the scope of semantic understanding of multi-dimensional image data.

[0047] S130, transmitting the first behavior instruction data to the second user terminal through a dedicated channel, where the dedicated channel is a data transmission channel pre-established based on real-time communication technology, and the second user terminal is configured to reconstruct information according to the first behavior instruction data to obtain reconstructed first multi-dimensional image data, and the reconstructed first multi-dimensional image data is used to display the behavior information of the first user on the second user terminal.

[0048] In this step, the dedicated channel can be a data transmission channel pre-established based on Web Real-Time Communication (WebRTC). This dedicated channel is different from a conventional audio and video channel. For example, the dedicated channel can be a channel established based on a streaming media protocol. Streaming media protocols are generally divided into protocols for transmitting control messages and protocols for transmitting media data. The media data transmission protocol is generally used to transmit audio and video media data, such as the Real-time Transport Protocol (RTP), the Real-time Transport Control Protocol (RTCP), and the Secure Real-time Transport Protocol (SRTP).

[0049] According to the information processing method provided by the present application, the first user terminal can perform feature extraction on the collected first multidimensional image data, convert the feature information of the extracted first multidimensional image data into first behavioral instruction data, and transmit the first behavioral instruction data to the second user terminal through a dedicated channel. The dedicated channel is a data transmission channel established in advance based on real-time communication technology. The second user terminal can reconstruct information based on the first behavioral instruction data to obtain the reconstructed first multidimensional image data. According to this method, converting the multidimensional image data into behavioral instruction data before transmitting it is conducive to reducing the traffic volume of data transmission, thereby reducing the bandwidth occupation and improving the bandwidth utilization; and the data transmission channel established based on real-time communication technology, that is, the dedicated channel, allows the user terminal to establish a point-to-point connection with the communication network without the help of an intermediate medium. Therefore, transmitting the first behavioral instruction data to the second user terminal through the dedicated channel can achieve high-efficiency data transmission.

[0050] In this step, after receiving the first behavior instruction data, the second user terminal may perform an information reconstruction operation (ie, information reconstruction) according to the behavior instruction data, and present the reconstructed multi-dimensional image data.

[0051] The following takes multi-dimensional image data as holographic image data as an example to illustrate the reconstruction principle of the holographic image.

[0052] FIG3 is a schematic diagram showing the principle of holographic image reconstruction provided by this application.

[0053] In practical applications, because holographic film (negative) records the phase and amplitude information of light, the diffracted light waves of a linearly recorded sinusoidal hologram under coherent laser illumination generally produce two images: the original image (also called the initial image) and the conjugate image. The reproduced image has a strong sense of three-dimensionality and a realistic visual effect. Each part of the hologram records the light information of each point on the object, so in principle, each part can reproduce the entire image of the original object. Through multiple exposures, multiple different images can be recorded on the same negative, and they can be displayed separately without interfering with each other.

[0054] As shown in Figure 3, the projection device can project the holographic image at different angles onto a holographic projection film (transparent film), use the diffraction principle to reproduce the object's light wave information, and display the image to form a holographic projection imaging, thereby realizing the three-dimensional image reproduction of the object.

[0055] According to the information processing method provided by the present application, the first user terminal can perform feature extraction on the collected first multidimensional image data, convert the feature information of the extracted first multidimensional image data into first behavioral instruction data, and transmit the first behavioral instruction data to the second user terminal through a dedicated channel. The dedicated channel is a data transmission channel pre-established based on real-time communication technology. The second user terminal can reconstruct information based on the first behavioral instruction data to obtain the reconstructed first multidimensional image data. According to this method, converting the multidimensional image data into behavioral instruction data before transmitting it is conducive to reducing the traffic volume of data transmission, thereby reducing bandwidth occupancy and improving bandwidth utilization. Moreover, the data transmission channel established based on real-time communication technology, i.e., the dedicated channel, allows the user terminal to establish a point-to-point connection with the communication network without the aid of an intermediary medium. Therefore, transmitting the first behavioral instruction data to the second user terminal through the dedicated channel can achieve high-efficiency data transmission.

[0056] According to this information processing method, the problems of excessive bandwidth occupancy and low bandwidth utilization in current VR, AR, and holographic communications can be addressed by abstracting and describing the information that the user terminal needs to transmit (feature extraction and transformation), obtaining corresponding behavior instruction data, and transmitting the transformed behavior instruction data to the user terminal of the communication counterpart through a dedicated channel in the network. The user terminal of the communication counterpart then reconstructs the information based on the behavior instruction data, and converts the received behavior instruction into multi-dimensional image data through information reconstruction. That is, in this method, when the first user terminal communicates with the second user terminal, it no longer transmits the collected original data itself, but transmits the behavior instruction data that has been abstracted (i.e., processed by feature extraction and transformation), greatly reducing the amount of data transmitted. Taking holographic communication as an example, in actual scenarios, using the information processing method provided by this application, due to the significant reduction in data transmission volume, bandwidth occupancy is reduced by more than 80%, greatly improving bandwidth utilization.

[0057] In some embodiments, the first multidimensional image data includes multidimensional image data, and the feature information is a multidimensional image data feature; the above-mentioned step S110 may specifically include: preprocessing the multidimensional image data collected by the image sensor, the preprocessing including at least one of the following: image denoising, image alignment, posture correction and motion compensation; performing feature extraction on the preprocessed multidimensional image data to obtain multidimensional image data features.

[0058] In this embodiment, the first user terminal performs data preprocessing, such as image denoising, image alignment, posture correction, motion compensation, etc., to improve the quality and usability of the data.

[0059] In some embodiments, the step of converting the feature information into the first behavior instruction data in step S120 may specifically include: matching the feature information with the semantic knowledge base to obtain semantic information corresponding to the feature information, the semantic information including descriptive information of the entity behavior; converting the descriptive information of the entity behavior into the corresponding first behavior instruction data.

[0060] For example, a semantic knowledge base can include knowledge from various fields. If the multidimensional image data is holographic image data, the semantic knowledge base includes indications in the holographic field, such as at least one of entities, relationships, and events. An event is a piece of knowledge that can be triggered by an action or a change in state.

[0061] Exemplarily, the semantic knowledge base may include semantic information of multiple multidimensional image data. The similarity between the feature information of the first multidimensional image data and the feature information of all multidimensional image data in the semantic knowledge base is calculated to obtain the image data with the highest similarity, and the semantic information of the multidimensional image data with the highest similarity is used as the corresponding semantic information matched to the feature information of the first multidimensional image data.

[0062] In this embodiment, the first user terminal matches and analyzes the extracted feature information with the semantic knowledge base to obtain semantic information, and then semantically represents the semantic information to obtain first behavioral instruction data. After identifying the multidimensional image data as corresponding behavioral instruction data, the data transmission traffic can be greatly reduced.

[0063] In some embodiments, the dedicated channel includes: a first dedicated channel between the first user terminal and the first communication network, and a second dedicated channel between the second user terminal and the second communication network.

[0064] In this embodiment, before the step of transmitting the first behavior instruction data to the second user terminal through the dedicated channel in step S130, the information processing method further includes the following steps: S11 to S13.

[0065] S11. Send a first session request to a first communication network corresponding to a first user terminal. The first session request includes at least: first dedicated channel media information and service type indication information of the first user terminal. The first communication network is configured to forward the first dedicated channel media information and service type indication information to the second user terminal through a second communication network corresponding to the second user terminal.

[0066] S12, receiving, through the first communication network, a session response from the second user terminal forwarded by the second communication network; the session response is a session response generated and sent to the second communication network by the second user terminal after receiving the first dedicated channel media information and the service type indication information, and the session response includes the second dedicated channel media information of the second user terminal; the first communication network is further configured to forward the session response to the first user terminal; the second communication network is further configured to establish a second dedicated channel with the second user terminal based on the received second dedicated channel media information.

[0067] S13: In response to the session response, establish a first dedicated channel with the first communication network according to the first dedicated channel media information.

[0068] In the above steps S11 to S13, the dedicated channel consists of a first dedicated channel, a first communication network, a second communication network and a second dedicated channel. When establishing the first dedicated channel and the second dedicated channel, it is necessary to indicate one's own media information to the participating parties, such as the user terminal of the communication counterpart, and to indicate a specific type of call service through service type indication information. For example, when the service type indication information is a holographic communication media mark, it indicates that the session request is configured to implement a holographic communication service.

[0069] In some embodiments, transmitting the first behavior instruction data to the second user terminal through a dedicated channel in step S130 may specifically include: sending the first behavior instruction data to the first communication network through the first dedicated channel; the first communication network is configured to send the first behavior instruction data to the second communication network, and the second communication network is configured to send the first behavior instruction data to the second user terminal through the second dedicated channel.

[0070] In this embodiment, the first behavior instruction data is transmitted through a pre-established dedicated channel (including a first dedicated channel and a second dedicated channel). The dedicated channel is different from the audio and video channel, and can realize a point-to-point connection between the user terminal and the communication network. The first behavior instruction data is transmitted to the second user terminal through the dedicated channel, which can realize efficient data transmission.

[0071] In some implementations, the first session request further includes: first reestablishment capability information, where the first reestablishment capability information is used to indicate the information reestablishment capability of the first user terminal.

[0072] In this embodiment, the first communication network is further configured to: send a first session request to the first application server; the first application server is configured to select corresponding first model data for the first user terminal based on the reconstruction capability information of the first user terminal, and instruct the first communication network to send a second session request to the second communication network, wherein the second session request includes: first dedicated channel media information and service type indication information; and, before sending the session response to the first user terminal, add an identifier of the first model data in the session response.

[0073] The second communication network is further configured to: send a second session request to a second application server corresponding to the second user terminal, the second application server being configured to select corresponding second model data for the second user terminal based on stored reconstruction capability information of the second user terminal, and instruct the second communication network to send a third session request to the second user terminal, where the third session request includes: first dedicated channel media information, service type indication information, and an identifier of the second model data.

[0074] The second user terminal is also configured to: after downloading the second model data through the second dedicated channel, perform feature extraction on the collected second multidimensional image data through the second model data and convert the feature information of the extracted second multidimensional image data into second behavior instruction data, and send the second behavior instruction data to the second communication network through the second dedicated channel.

[0075] In this embodiment, after the above-mentioned step S13, the information processing method further includes: downloading first model data through a first dedicated channel; receiving second behavior instruction data from the first communication network through the first dedicated channel, the second behavior instruction data being data transmitted from the second communication network to the second communication network; reconstructing the second behavior instruction data according to the first model data to obtain reconstructed second multidimensional image data.

[0076] In this embodiment, the first application server can select a corresponding model package for the user terminal based on the information reconstruction capability of the user terminal, so that the user terminal can download the model package through a dedicated channel, and use the downloaded model package to reconstruct the received behavioral instruction data to obtain reconstructed multi-dimensional image data.

[0077] Through the information processing method provided by the present application, the first user terminal can extract features from the collected first multidimensional image data, convert the extracted feature information of the first multidimensional image data into first behavioral instruction data, and transmit the first behavioral instruction data to the second user terminal through a dedicated channel. The dedicated channel is a data transmission channel pre-established based on real-time communication technology. The second user terminal can reconstruct information based on the first behavioral instruction data to obtain the reconstructed first multidimensional image data. According to this method, converting the multidimensional image data into behavioral instruction data before transmitting it is conducive to reducing the traffic volume of data transmission, thereby reducing bandwidth occupancy and improving bandwidth utilization. In addition, the data transmission channel established based on real-time communication technology, i.e., the dedicated channel, allows the user terminal to establish a point-to-point connection with the communication network without the aid of an intermediary medium. Therefore, transmitting the first behavioral instruction data to the second user terminal through the dedicated channel can achieve high-efficiency data transmission.

[0078] FIG4 is an information processing method provided by the present application and applied to a second user terminal.

[0079] In a second aspect, referring to FIG. 4 , the present application provides an information processing method, which can be applied to a second user terminal. The method includes steps S410 and S420 .

[0080] S410, receiving first behavior instruction data of the first user terminal through a dedicated channel; the dedicated channel is a data transmission channel established in advance based on real-time communication technology, and the first behavior instruction data is behavior instruction data obtained by the first user terminal extracting features from the collected first multidimensional image data and converting the feature information of the extracted first multidimensional image data, and the first behavior instruction data is used to indicate behavior information of the first user to whom the first user terminal belongs.

[0081] S420 , reconstructing information according to the first line of instruction data to obtain reconstructed first multi-dimensional image data.

[0082] According to the information processing method provided in the present application, the second user terminal can receive the first behavior instruction data of the first user terminal through a dedicated channel. The first behavior instruction data is the behavior instruction data of the first user terminal performing feature extraction on the collected first multidimensional image data, converting the feature information of the extracted first multidimensional image data into the first behavior instruction data, and transmitting the first behavior instruction data to the second user terminal through the dedicated channel. In addition, the dedicated channel is a data transmission channel established in advance based on real-time communication technology. The second user terminal can reconstruct information based on the first behavior instruction data to obtain the reconstructed first multidimensional image data.

[0083] According to the information processing method, multi-dimensional image data can be converted into behavioral instruction data before being transmitted, which is beneficial to reducing the business volume of data transmission, thereby reducing bandwidth occupancy and improving bandwidth utilization; and, the data transmission channel established based on real-time communication technology, that is, the dedicated channel, allows the user terminal to establish a point-to-point connection with the communication network without the aid of an intermediate medium. Therefore, the first behavioral instruction data is transmitted to the second user terminal through the dedicated channel, which can achieve efficient data transmission.

[0084] In some embodiments, the dedicated channel includes: a first dedicated channel between the first user terminal and the first communication network, and a second dedicated channel between the second user terminal and the second communication network.

[0085] The first communication network is configured to: send the received first session request of the first user terminal to the first application server, and the first application server is configured to select corresponding first model data for the first user terminal based on the reconstruction capability information of the first user terminal, and instruct the first communication network to send a second session request to the second communication network, where the second session request includes: first dedicated channel media information and service type indication information.

[0086] The second communication network is configured to: send a second session request to a second application server corresponding to the second user terminal, the second application server is configured to select corresponding second model data for the second user terminal based on the stored reconstruction capability information of the second user terminal, and instruct the second communication network to send a third session request to the second user terminal, where the third session request includes: first dedicated channel media information, service type indication information, and an identifier of the second model data.

[0087] Before step S410, the information processing method also includes: receiving a third session request sent by the second communication network; sending a session response to the second communication network in response to the third session request, wherein the session response includes second dedicated channel media information of the second user terminal; and establishing a second dedicated channel with the second communication network through the second dedicated channel media information.

[0088] In this embodiment, the second user terminal can obtain the first dedicated channel media information, service type indication information and the identifier of the second model data carried in the third session request through the third session request, and determine the service type of this communication based on the information carried in the third session request, and download the corresponding model package through the identifier of the second model data; it can also carry its own dedicated channel media information, i.e., the second dedicated channel media information, through the session response, for communication negotiation with the first user terminal through the session response.

[0089] In some embodiments, step S420 may specifically include: downloading second model data through a second dedicated channel; and reconstructing the first behavior instruction data using the second model data to obtain reconstructed first multi-dimensional image data.

[0090] In this embodiment, the second model data is downloaded via a second dedicated channel, and the first behavior instruction data is processed using the second model data to obtain reconstructed first reconstructed data. Both the first dedicated channel and the second dedicated channel are different from the audio and video channels. The dedicated channels can establish a point-to-point connection between the corresponding user terminal and the communication network. Transmitting the first behavior instruction data to the second user terminal via the dedicated channel can achieve efficient data transfer.

[0091] In some embodiments, the first user terminal is further configured to: in response to a session response, establish a first dedicated channel with the first communication network according to the first dedicated channel media information, and download the first model data through the first dedicated channel; after downloading the second model data through the second dedicated channel in the above steps, the information processing method further includes: performing feature extraction on the collected second multidimensional image data, and converting the feature information of the extracted second multidimensional image data into second behavior instruction data; sending the second behavior instruction data to the second communication network through the second dedicated channel; the second communication network is configured to transmit the second behavior instruction data to the first communication network, so that the first communication network transmits the second behavior instruction data to the first user terminal through the first dedicated channel; the first user terminal is configured to: perform information reconstruction on the second behavior instruction data according to the first model data to obtain reconstructed second multidimensional image data.

[0092] In this embodiment, the second user terminal can also perform feature extraction and conversion on the collected second multidimensional image data to obtain second behavior instruction data, and transmit the second behavior instruction data to the first user terminal through a dedicated channel, so that the first user terminal can reconstruct the second behavior instruction data according to the model package corresponding to its own reconstruction capability to obtain reconstructed second multidimensional image data.

[0093] According to the information processing method provided in the present application, the second user terminal can receive the first behavior instruction data of the first user terminal through a dedicated channel. The first behavior instruction data is the behavior instruction data of the first user terminal performing feature extraction on the collected first multidimensional image data, converting the feature information of the extracted first multidimensional image data into the first behavior instruction data, and transmitting the first behavior instruction data to the second user terminal through the dedicated channel. In addition, the dedicated channel is a data transmission channel established in advance based on real-time communication technology. The second user terminal can reconstruct information based on the first behavior instruction data to obtain the reconstructed first multidimensional image data.

[0094] According to the information processing method, multi-dimensional image data can be converted into behavioral instruction data before being transmitted, which is beneficial to reducing the business volume of data transmission, thereby reducing bandwidth occupancy and improving bandwidth utilization; and, the data transmission channel established based on real-time communication technology, that is, the dedicated channel, allows the user terminal to establish a point-to-point connection with the communication network without the aid of an intermediate medium. Therefore, the first behavioral instruction data is transmitted to the second user terminal through the dedicated channel, which can achieve efficient data transmission.

[0095] FIG5 is an information processing method applied to a communication system provided by the present application.

[0096] In a third aspect, referring to FIG. 5 , the present application provides an information transmission method, which can be applied to a communication system. The method includes steps S510 and S520 .

[0097] S510, receiving first behavior instruction data of the first user terminal, the first behavior instruction data is behavior instruction data obtained by the first user terminal extracting features from the collected first multidimensional image data and converting the feature information of the extracted first multidimensional image data, and the first behavior instruction data is used to indicate behavior information of the first user to whom the first user terminal belongs.

[0098] S520, transmitting the first behavior instruction data to the second user terminal through a dedicated channel, where the dedicated channel is a data transmission channel pre-established based on real-time communication technology, and the second user terminal is configured to reconstruct information according to the first behavior instruction data to obtain reconstructed first multi-dimensional image data.

[0099] Through the information transmission method provided by the present application, the communication system can receive first behavior instruction data from a first user terminal and transmit the first behavior instruction data to a second user terminal through a dedicated channel. The first behavior instruction data is the behavior instruction data of the first user terminal performing feature extraction on the collected first multidimensional image data, converting the feature information of the extracted first multidimensional image data into the first behavior instruction data, and transmitting the first behavior instruction data to the second user terminal through a dedicated channel. In addition, the dedicated channel is a data transmission channel established in advance based on real-time communication technology. The second user terminal can reconstruct information according to the first behavior instruction data to obtain the reconstructed first multidimensional image data.

[0100] In some embodiments, the dedicated channel includes a first dedicated channel and a second dedicated channel; the communication system includes at least a first communication network and a second communication network; before receiving the first behavior instruction data of the first user terminal, the information processing method further includes the following steps: S21, receiving a first session request from the first user terminal through the first communication network, the first session request including at least: first dedicated channel media information and service type indication information of the first user terminal; S22, sending the first dedicated channel media information and service type indication information to the second communication network through the first communication network, the second communication network being a communication network corresponding to an identifier of the second user terminal; S23, sending the first dedicated channel media information and service type indication information to the second user terminal through the second communication network; S24, receiving and forwarding a session response from the second user terminal to the first communication network through the second communication network, the session response including second dedicated channel media information of the second user terminal; S25, sending second dedicated channel media information to the first user terminal through the first communication network, and establishing a first dedicated channel with the first user terminal based on the first dedicated channel media information; S26, establishing a second dedicated channel with the second user terminal through the second communication network using the second dedicated channel media information.

[0101] In this embodiment, a first dedicated channel is established between the first user terminal and the first communication network, and a second dedicated channel is established between the second user terminal and the second communication network. The first dedicated channel and the second dedicated channel are different from the audio and video channels, and a point-to-point connection can be achieved between the user terminal and the communication network. The first behavior instruction data is transmitted to the second user terminal through the dedicated channel, which can achieve efficient data transmission.

[0102] In some embodiments, the communication system further includes: a first application server and a second application server; the above-mentioned step S22 may specifically include: sending a first session request to the first application server through the first communication network, wherein the first session request also includes reconstruction capability information of the first user terminal; the first application server selects corresponding first model data for the first user terminal based on the reconstruction capability information of the first user terminal, and instructs the first communication network to send a second session request to the second communication network, wherein the second session request includes: first dedicated channel media information and service type indication information.

[0103] In this embodiment, the above-mentioned step S23 may specifically include: sending a second session request to the second application server through the second communication network; the second application server selects corresponding second model data for the second user terminal based on the stored reconstruction capability information of the second user terminal, and instructs the second communication network to send a third session request to the second user terminal, where the third session request includes: first dedicated channel media information, service type indication information and an identifier of the second model data.

[0104] In this embodiment, the above step S25 may specifically include: adding an identifier of the first model data to the session response via the first communication network; and sending the session response carrying the identifier of the first model data and the second dedicated channel media information to the first user terminal.

[0105] In this embodiment, the first application server is configured to select corresponding model data for the first user terminal based on the reconstruction capability information of the first user terminal, and the second application server is configured to select corresponding model data for the second user terminal based on the reconstruction capability information of the second user terminal, for subsequent information reconstruction of the behavioral instruction data received from the communication counterpart.

[0106] In some embodiments, the step of transmitting the first behavior instruction data to the second user terminal through a dedicated channel in step S520 may specifically include: sending the first behavior instruction data received through the first dedicated channel to the second communication network; and in the second communication network, sending the first behavior instruction data to the second user terminal through the second dedicated channel.

[0107] In this embodiment, the dedicated channel is composed of a first dedicated channel, a first communication network, a second communication network and a second dedicated channel, and efficient transmission of behavior instruction data is achieved based on the dedicated channel.

[0108] In some embodiments, after the above-mentioned step S26, the information processing method further includes: receiving second behavior instruction data from a second user terminal through a second dedicated channel in a second communication network, where the second behavior instruction data is behavior instruction data obtained by the second user terminal through second model data, performing feature extraction on the collected second multidimensional image data and converting the feature information of the extracted second multidimensional image data; transmitting the second behavior instruction data to the first communication network through the second communication network; and sending the second behavior instruction data to the first user terminal through the first dedicated channel in the first communication network, where the first user terminal is configured to reconstruct information of the second behavior instruction data according to the first model data to obtain reconstructed second multidimensional image data.

[0109] In this embodiment, the second user terminal can extract and convert features of the collected second multidimensional image data through a dedicated channel to obtain second behavioral instruction data, and transmit the second behavioral instruction data to the first user terminal through the dedicated channel, so that the first user terminal can reconstruct information on the second behavioral instruction data according to a model package corresponding to its own reconstruction capability to obtain reconstructed second multidimensional image data.

[0110] Figure 6 is a schematic diagram of an exemplary architecture of a communication system provided in this application. Taking the communication system corresponding to a communication-side terminal as an example, as shown in Figure 6, the communication system may include: a communication network 610 and an application service 620. The application scenario of the communication system also includes: information reconstruction terminal 1 (first user terminal) and information reconstruction terminal 2 (second user terminal).

[0111] The communication network 610 is configured to support the negotiation of newly added dedicated channels (different from the audio and video channels), while also providing terminal access authentication, addressing between terminals and between terminals and application servers, and the transmission of behavioral instruction data. The application service 620 is configured to add model management functions, including the storage, addition, update, and deletion of information models and reconstruction models. During the communication establishment process, it supports the negotiation of newly added dedicated channels, the selection of corresponding model packages for different terminal types, and the forwarding of one-to-many behavioral instructions.

[0112] In FIG6 , the first user terminal and the second user terminal may use the same communication network. In actual application scenarios, the first user terminal and the second user terminal may also use different communication networks.

[0113] The following describes the specific process of the information processing method and information transmission method provided by this application in conjunction with Figure 6.

[0114] In some embodiments, the information processing method and the information transmission method may include the following steps: S701 to S723.

[0115] S701: A first user terminal sends a first session request to a communication network of the first user.

[0116] The first session request includes the dedicated channel media information of the first user terminal, a specific type of call service indication, and capability characteristics supporting information reconstruction, etc. The specific type of call service indication is service type indication information, such as holographic communication service, VR communication service, AR communication service, etc.

[0117] S702: The first communication network receives a first session request and reports the first session request to a first application server according to a specific type of call service indication in the first session request.

[0118] The first application server is a server corresponding to the application service of the first user, and the type of the application service corresponds to the business type.

[0119] S703: The first application server selects corresponding model data for the first user terminal by analyzing the information reconstruction capability of the first user terminal in the first session request.

[0120] Exemplarily, the information reconstruction capability includes at least one of the following: imaging capability, GPU capability, screen size, etc. The model data is used to subsequently instruct the first user terminal to use the model data for information generation and information reconstruction.

[0121] S704: The first application server instructs the first communication network to continue the call, where the instruction carries dedicated channel media information and a specific type of call service instruction.

[0122] In this step, the first application server may instruct the first communication network to continue the call through a second session request, where the second session request carries dedicated channel media information and a specific type of call service indication, i.e., a capability feature of information reconstruction without the first user terminal.

[0123] S705: The first communication network sends a session request to the second communication network according to the identification information of the second user terminal.

[0124] In this step, the identification information of the second user terminal may be a user number, for example, including but not limited to at least one of the first user's International Mobile Subscriber Identity (IMSI) and Mobile Station International Subscriber Directory Number (MSISDN). The first communication network may connect to the network corresponding to the identification information using the identification information of the second user terminal.

[0125] S706: The second communication network receives the session request and reports session information to the second application server according to the specific type of call service indication in the session request.

[0126] S707 , the second application server selects corresponding model data for the second user terminal according to the information reconstruction capability of the second user terminal when it is registered, and instructs the second user terminal to use the model data for information generation and information reconstruction.

[0127] S708: The second application server instructs the second communication network to continue the call. The instruction carries dedicated channel media information, a specific type of call service indication, and an identifier of the second model data.

[0128] In this step, the second application server may instruct the second communication network to continue the call through a third session request, where the third session request carries a specific type of call service indication and a model data identifier required for information generation and information reconstruction.

[0129] S709: The second communication network sends a session request to the second user terminal according to the identification information of the second user terminal.

[0130] In this step, the identification information of the second user terminal may be a user number, for example including but not limited to at least one of the IMSI and MSISDN of the second user.

[0131] S710, the second user terminal determines whether to use the information generation and reconstruction function this time and performs initialization of terminal-related capabilities based on the information in the session request, such as dedicated channel media information, a specific type of call service indication, and a model data identifier required for information generation and information reconstruction.

[0132] In this step, the initialization may include, for example, the initialization of the information generation function and the initialization of the information reconstruction function.

[0133] S711: The second user terminal returns a session response to the second communication network. The session response carries local dedicated channel media information (ie, dedicated channel media information of the second user terminal).

[0134] S712: The second communication network forwards the session response to the first communication network.

[0135] S713: The first communication network inserts a model data identifier required for information generation and information reconstruction used by the first user terminal into the session response, and forwards the session response to the first user terminal.

[0136] S714: After receiving the session response, the first user terminal establishes a dedicated channel with the first communication network and downloads the specified model data through the dedicated channel.

[0137] S715 , after confirming that the first user terminal has received the session response, the second user terminal establishes a dedicated channel with the second communication network and downloads the specified model data through the dedicated channel.

[0138] S716: The second user terminal uses the terminal hardware to collect information, generates information using the downloaded model data, and sends the second user's behavior instruction data to the second communication network through a dedicated channel.

[0139] S717: The second communication network forwards the second user's behavior instruction data to the first communication network through a dedicated channel.

[0140] S718: The first communication network forwards the second user's behavior instruction data to the first user terminal through a dedicated channel.

[0141] S719: The first user terminal reconstructs the behavior instruction data of the second user using the downloaded model data, and presents the reconstructed data on the first user terminal.

[0142] S720: The first user collects information using terminal hardware, generates information using the downloaded model data, and sends the first user's behavior instruction data to the first communication network through a dedicated channel.

[0143] S721: The first communication network forwards the behavior instruction data of the first user to the second communication network through a dedicated channel.

[0144] S722: The second communication network forwards the first user's behavior instruction data to the second user terminal through a dedicated channel.

[0145] S723: The second user terminal reconstructs the behavior instruction data of the first user using the downloaded model data, and presents the reconstructed data to the second user.

[0146] Through the above steps S701 to S723, real-time communication of three-dimensional information can be completed based on the information reconstruction method.

[0147] Figure 7 is a schematic diagram of an exemplary detailed architecture of the communication system provided in this application. Structures identical or equivalent to those in Figure 6 are numbered the same. As shown in Figure 7 , in the first communication system, a first communication network 610 includes a Session Border Controller (SBC) 611, a Call Session Control Function (CSCF) node 612, a multimedia system application server 613, and a service interaction platform 614.

[0148] The session edge controller 611 can serve as an access module in the Internet Multimedia System (IMS), responsible for providing access services to information reconstruction terminals such as the first user terminal and the second user terminal, and can support the establishment of dedicated channels and the forwarding of behavioral instruction data.

[0149] The call session control function node 612 can serve as a session control and routing module in the IMS, providing an interrogation call session control function (I-CSCF) and a serving call session control function (S-CSCF). The main functions of the I-CSCF include at least one of the following: allocating a serving call session control function (S-CSCF) to the user when the user registers, and being responsible for sending the session to the S-CSCF where the user is registered or a designated S-CSCF in the business process; the main functions of the S-CSCF include at least one of the following: providing registration and authentication for mobile phones and AR devices; providing session initialization protocol (SIP) session management, including session establishment, session refresh, session release, session maintenance, emergency call and message management, etc.; providing service triggering function to trigger the call to the corresponding multimedia system application server. The multimedia system application server can be, for example, a Voice over Long-Term Evolution Application Server (VoLTE AS).

[0150] The multimedia system application server 613 can be, for example, a VoLTE AS. As an application service in the IMS, the multimedia system application can provide basic supplementary services, call continuity and other functions. In some implementations, the VoLTE AS reports the call event to the service interaction platform 614, receives the control instructions issued by the service interaction platform 614, and implements the management of the dedicated data channel, such as at least one of creation, modification, and deletion.

[0151] The service interaction platform 614 provides a unified service portal for applications and terminals. Specific functions include at least one of the following: providing subscriptions to user registration events or call events; providing automatic addressing for different services; providing in-call signaling control capabilities, including but not limited to connecting calls, rejecting calls, and playing audio to terminals; providing in-call media control capabilities, including but not limited to anchoring terminal audio and video media to the media plane, anchoring the terminal's dedicated data channel to the media plane, and managing audio and video media channels and dedicated data channels, such as at least one of creation, modification, and deletion; and providing forwarding control capabilities for behavioral instruction data.

[0152] As shown in Figure 7, the communication system may also include: a Home Subscriber Server (HSS) 615. The HSS may be responsible for storing user authentication information, specific information of subscribers, dynamic information of subscribers, etc., and cooperate with the CSCF to complete the registration and authentication of the information reconstruction terminal; provide storage service transparent data storage for the VoLTE AS, etc.

[0153] In some embodiments, the first communication network includes: a first session edge controller, a first call session control function node, a first multimedia system application server, and a first service interaction platform connected in sequence, the first session edge controller is connected to the first user terminal, and the first service interaction platform is connected to the first application server; the second communication network includes: a second session edge controller, a second call session control function node, a second multimedia system application server, and a second service interaction platform connected in sequence, the second session edge controller is connected to the second user terminal, and the second service interaction platform is connected to the second application server; and the first call session control function node is connected to the second multimedia system application server for communication transmission from the first communication network to the second communication network; and the second call session control function node is connected to the first multimedia system application server for communication transmission from the second communication network to the second communication network.

[0154] In this embodiment, in the communication network, the first communication network implements the functions of the communication network through the first session edge controller, the first call session control function node, the first multimedia system application server, and the first service interaction platform.

[0155] FIG8 is an exemplary flowchart of the information processing method provided in the present application and applied to the first user terminal.

[0156] As shown in FIG8 , the information processing method may include the following steps S801 to S806 .

[0157] S801, data acquisition.

[0158] In this step, the first user terminal obtains data transmitted by holographic communication through hardware facilities, such as holographic images, related sensor data, etc.

[0159] S802, data preprocessing.

[0160] In this step, the first user terminal preprocesses the acquired data, including image denoising, image alignment, posture correction, motion compensation and other processing, to improve the quality and usability of the data.

[0161] S803, feature extraction.

[0162] In this step, the first user terminal extracts feature information in the holographic image from the preprocessed data, such as color, shape, texture, depth and other features.

[0163] S804, semantic analysis.

[0164] In this step, the first user terminal matches and analyzes the extracted feature information with the semantic knowledge base to obtain semantic information. The semantic knowledge base may include knowledge in the holographic field, such as entities, relationships, events, etc.

[0165] S805, semantic representation.

[0166] In this step, the first user terminal represents and stores the semantic information obtained by parsing, and obtains corresponding behavioral instruction data, for example, blink the left eye three times, each blink lasting 50 milliseconds, etc.

[0167] S806, data transmission.

[0168] In this step, the first user terminal establishes a session with the communication network and transmits the information data with completed semantic representation, that is, the behavioral instruction data obtained by the semantic representation, to a remote device, such as the second user terminal.

[0169] Through the above steps S801 to S806, any user terminal can convert the acquired data for holographic communication into corresponding behavior instruction data, and transmit the behavior instruction data to the second user terminal to realize information reconstruction in the second user terminal.

[0170] FIG9 is an exemplary flowchart of the information processing method provided in the present application and applied to the second user terminal.

[0171] As shown in FIG. 9 , the information processing method may include the following steps S901 to S903 .

[0172] S901, information presentation.

[0173] In this step, the second user terminal may establish a session with the communication network to receive the behavior instruction data transmitted in the communication network, that is, the behavior instruction data represented by semantics.

[0174] S902, semantic analysis.

[0175] In this step, the second user terminal matches and analyzes the received information data with the semantic knowledge base to obtain semantic information corresponding to the behavioral instruction to be presented, for example, blink the left eye three times, each blink lasting 50 milliseconds.

[0176] S903, data transmission.

[0177] In this step, the second user terminal loads the corresponding data model according to the semantic information, applies the semantic information to the existing scene, and uses the terminal's own hardware capabilities to present information such as sound and image on the second user terminal.

[0178] Through the above steps S901 to S903, the second user terminal can receive the first behavior instruction data from the first user terminal through the dedicated channel, and reconstruct information according to the first behavior instruction data to obtain reconstructed first multi-dimensional image data.

[0179] The following describes the process of two user terminals of the same type supporting holographic communication capabilities using the information processing method and information transmission method provided in this application to interact with each other using the example of implementing holographic communication in an IMS network, where the first user terminal and the second user terminal can use the same IMS network. It should be understood that the first user terminal and the second user terminal can use the same IMS network or different IMS networks, and the processing flow corresponding to using the same IMS network or different IMS networks is the same. In some embodiments, the information processing method and the information transmission method include the following steps: S1001 to S1031.

[0180] S1001: A first user terminal sends an INVITE request to an SBC. The INVITE request carries dedicated channel media information of the first user terminal, a service indication of holographic communication, and terminal capability information.

[0181] Exemplarily, the holographic communication service indication may be implemented by the following method:

[0182] Add a holographic communication media label to the media line (m=line) of the multimedia session's application layer control protocol (Session Description Protocol, SDP). m=application 10000 UDP / DTLS / SCTP webrtc-datachannel a=dcmap:1002label=”holo”

[0183] In WebRTC, point-to-point message communication is achieved through the Data Channel using the User Datagram Protocol (UDP), Datagram Transport Layer Security (DTLS), or Stream Control Transmission Protocol (SCTP).

[0184] Each media level section begins with an "m=" line and continues to the next media level section or the end of the entire session.

[0185] Exemplarily, the terminal capabilities include at least one of: graphics processing unit (GPU) capabilities, three-dimensional (3D) rendering capabilities, screen size, resolution, multi-dimensional acquisition capabilities, and the like.

[0186] S1002: The SBC forwards the INVITE request to the S-CSCF.

[0187] S1003: The S-CSCF forwards the INVITE request to the VoLTE AS.

[0188] S1004, the VoLTE AS reports the call event to the service interaction platform, and carries the dedicated channel media information of the first user terminal, the holographic communication service indication, and the terminal capability information.

[0189] S1005: The service interaction platform reports the call event to the information reconstruction application server corresponding to the first user terminal according to the holographic communication service instruction.

[0190] S1006: After receiving the request, the information reconstruction application server corresponding to the first user terminal selects model package 1 for the first user terminal according to the terminal capability information of the first user terminal.

[0191] Table 1 below shows an example of the specific content of the model package 1.

[0192] Table 1 Specific contents of the model package

[0193] Exemplarily, a model package, such as Model Package 1, includes: a data analysis model, a data presentation model, a semantic model, a behavior data model, and the like, and records the corresponding information in the system. For example, the data analysis model is configured to parse the corresponding behavior instruction data to obtain descriptive information about the entity's behavior; the semantic model is configured to determine the semantic information corresponding to the descriptive information about the entity's behavior; the behavior data model can be configured to convert the corresponding semantic information into feature information about the corresponding behavior; and the data presentation model is configured to present the corresponding behavior using this feature information.

[0194] S1007: The application server corresponding to the first user terminal instructs the service interaction platform to continue the call and carries the dedicated channel media information and holographic communication service indication of the first user terminal.

[0195] S1008: The service interaction platform instructs the VoLTE AS to continue the call.

[0196] S1009: The VoLTE AS forwards the INVITE request to the I / S-CSCF.

[0197] S1010: The I / S-CSCF forwards the INVITE session request to the VoLTE AS of the second user terminal according to the identification information of the second user terminal.

[0198] S1011, the VoLTE AS corresponding to the second user terminal reports a call event to the service interaction platform, and carries a holographic communication service indication and terminal capability information.

[0199] S1012: The service interaction platform corresponding to the second user terminal reports the call event to the information reconstruction application server corresponding to the second user terminal according to the holographic communication service indication.

[0200] S1013: After receiving the request, the information reconstruction application server corresponding to the second user terminal selects model package 1 for the second user terminal according to the terminal capability information of the second user terminal during registration.

[0201] Table 2 below shows an example of the specific content of the model package 1.

[0202] It can be seen from Table 1 and Table 2 that the first user terminal and the second user terminal have the same terminal capability information, and therefore the corresponding model packages are also the same.

[0203] In Table 2, model package 1 includes data analysis model, data presentation model, semantic model, behavioral data model, etc., and the corresponding information is recorded in the system.

[0204] S1014: The application server corresponding to the second user terminal instructs the service interaction platform to continue the call.

[0205] S1015: The service interaction platform instructs the VoLTE AS to continue the call.

[0206] S1016, the VoLTE AS forwards the INVITE request to the I / S-CSCF. The INVITE request carries the holographic communication service indication, media description, and model package identification information.

[0207] For example, the holo tag in SDP indicates a holographic communication service, and the package indicates the model package identifier. m = application 10000 UDP / DTLS / SCTP webrtc-datachannel a = dcmap:1002 label = "holo" a = package:1

[0208] S1017: The I / S-CSCF sends an INVITE request to the SBC where the second user terminal is registered.

[0209] S1018: The SBC registered with the second user terminal forwards the INVITE request to the second user terminal.

[0210] S1019, when the second user terminal receives the INVITE call request, it performs preparations for holographic communication according to the holographic communication service indication in the call request, including but not limited to initializing the presentation device, graphics processing unit (GPU) function, etc.

[0211] S1020: The second user terminal returns a 183 response to the SBC corresponding to the second user terminal.

[0212] S1021: The SBC corresponding to the second user terminal forwards the 183 response to the S-CSCF.

[0213] S1022: The S-CSCF forwards the response to the VoLTE AS. The VoLTE AS interacts with the media function (MRF) to create a dedicated channel.

[0214] S1023: The second user terminal downloads the data of the model package 1 through a dedicated channel.

[0215] S1024: The VoLTE AS corresponding to the second user terminal sends a 183 response to the I / S-CSCF. The 183 response carries the media and network information (IP address, port number, etc.) of the second user terminal.

[0216] S1025 , the I / S-CSCF forwards the 183 response to the VoLTE AS corresponding to the first user terminal.

[0217] S1026: The VoLTE AS corresponding to the first user terminal inserts the model identification information corresponding to the first user into the 183 response and forwards it to the I / S-CSCF. For example, a new a=package attribute is added to the m line in the SDP to indicate the model package information. m=application 10000 UDP / DTLS / SCTP webrtc-datachannel a=dcmap:1002 lable="holo" a=package:1

[0218] S1027 : The I / S-CSCF forwards the 18x response to the SBC where the first user terminal is registered.

[0219] S1028: The SBC forwards the 18x response to the first user terminal.

[0220] S1029: The first user terminal uses the media information in the 18x response to establish a connection with the first user application server and downloads the data of the model package 1 through a dedicated channel.

[0221] S1030: The first user terminal and the second user terminal complete a subsequent IMS precondition process.

[0222] The Precondition process includes reserving network resources before a session is established.

[0223] S1031: After the second user terminal answers the call, the second user terminal returns a 200 OK response to the first user terminal via the IMS network, and the first user terminal establishes a media path with the second user terminal.

[0224] In some embodiments, user data may or may not be transferred through an application server. After receiving the user data from the second user terminal, the first user terminal uses model package 1 to generate holographic presentation content; after receiving the user data from the second user, the second user terminal uses model package 1 to generate holographic presentation content.

[0225] Through the above steps, the first user terminal and the second user terminal complete the holographic communication process.

[0226] The following takes the implementation of holographic communication in an IMS network, where the first user terminal and the second user terminal can use the same IMS network as an example, to describe the process of two different types of user terminals supporting holographic communication capabilities interacting using the information processing method and information transmission method provided in this application. It should be understood that the first user terminal and the second user terminal can use the same IMS network or different IMS networks, and the processing flow corresponding to using the same IMS network or different IMS networks is the same. In some embodiments, the information processing method and the information transmission method include the following steps: S1101 to S1131.

[0227] S1101: A first user terminal sends an INVITE request to an SBC. The INVITE request carries dedicated channel media information of the first user terminal, a service indication of holographic communication, and terminal capability information.

[0228] In this step, the terminal capability information may reflect the characteristics of the corresponding user terminal.

[0229] For example, the holographic communication service indication can be implemented by the following method: adding a holographic communication media label in the media line (m=line) of the SDP: m=application 10000 UDP / DTLS / SCTP webrtc-datachannel a=dcmap:1002 label=”holo”

[0230] The specific content of the holographic communication media mark can be customized according to actual needs and is not specifically limited in this application.

[0231] S1102: The SBC forwards the INVITE request to the S-CSCF.

[0232] S1103: The S-CSCF forwards the INVITE request to the VoLTE AS.

[0233] S1104, the VoLTE AS reports the call event to the service interaction platform, and carries the holographic communication service indication and terminal characteristics.

[0234] S1105: The service interaction platform reports the call event to the information reconstruction application server corresponding to the first user terminal according to the holographic communication service instruction.

[0235] S1106: After receiving the request, the information reconstruction application server corresponding to the first user terminal selects model package 1 for the first user terminal according to the terminal capability information of the first user terminal.

[0236] The specific content of model package 1 can be found in Table 1 above and will not be described in detail here.

[0237] S1107: The application server corresponding to the first user terminal instructs the service interaction platform to continue the call.

[0238] S1108: The service interaction platform instructs the VoLTE AS to continue the call.

[0239] S1109: The VoLTE AS forwards the INVITE request to the I / S-CSCF.

[0240] S1110 , the I / S-CSCF forwards the INVITE session request to the VoLTE AS corresponding to the second user terminal according to the identification information of the second user terminal.

[0241] S1111, the VoLTE AS corresponding to the second user terminal reports the call event to the service interaction platform, and carries the holographic communication service indication and terminal characteristics.

[0242] S1112, the service interaction platform corresponding to the second user terminal reports the call event to the information reconstruction application server corresponding to the second user terminal according to the holographic communication service indication.

[0243] S1113, after receiving the request, the information reconstruction application server corresponding to the second user terminal selects model package 2 for the second user terminal according to the terminal capability information of the second user during registration.

[0244] Table 3 below shows an example of the specific content of model package 3.

[0245] Table 3 Specific contents of the model package

[0246] Model package 2 includes data analysis model, data presentation model, semantic model, behavioral data model, etc., and records the corresponding information in the system.

[0247] S1114. The application server corresponding to the second user terminal instructs the service interaction platform to continue the call and carries the dedicated channel media information and holographic communication service indication of the first user terminal.

[0248] S1115. The service interaction platform instructs the VoLTE AS to continue the call.

[0249] S1116. The VoLTE AS forwards the INVITE request to the I / S-CSCF. The INVITE request carries the holographic communication service indication, media description, and model package identification information.

[0250] For example, the holo tag in SDP indicates a holographic communication service, and the package indicates the model package identifier. m = application 10000 UDP / DTLS / SCTP webrtc-datachannel a = dcmap:1002 label = "holo" a = package:2

[0251] package:2 indicates that the model package is model package 2.

[0252] S1117: The I / S-CSCF sends an INVITE request to the SBC where the second user terminal is registered.

[0253] S1118: The SBC registered with the second user terminal forwards the INVITE request to the second user terminal.

[0254] S1119, when the second user terminal receives the INVITE call request, it performs preparations for holographic communication according to the holographic communication service indication in the call request, including but not limited to initializing the presentation device, graphics processing unit (GPU) function, etc.

[0255] S1120: The second user terminal returns a 183 response to the SBC corresponding to the second user terminal.

[0256] S1121: The SBC corresponding to the second user terminal forwards the 183 response to the S-CSCF.

[0257] S1122: The S-CSCF forwards the response to the VoLTE AS. The VoLTE AS interacts with the media function (MRF) to create a dedicated channel with the second user terminal.

[0258] S1123: The second user terminal downloads the data of model package 2 through a dedicated channel.

[0259] S1124, the second user terminal sends a 183 response to the I / S-CSCF corresponding to the VoLTE AS. The 183 response carries the media and network information (IP address, port number, etc.) of the second user terminal.

[0260] S1125 , the I / S-CSCF forwards the 183 response to the VoLTE AS corresponding to the first user terminal.

[0261] S1126: The VoLTE AS corresponding to the first user terminal inserts the model identification information corresponding to the first user into the 183 response and forwards it to the I / S-CSCF. For example, a new a=package attribute is added to the m line in the SDP to indicate the model package information. m=application 10000 UDP / DTLS / SCTP webrtc-datachannel a=dcmap:1002 lable="holo" a=package:1

[0262] S1127 , the I / S-CSCF forwards the 18x response to the SBC where the first user terminal is registered.

[0263] S1128 , the SBC forwards the 18x response to the first user terminal.

[0264] S1129: The first user terminal uses the media information in the 18x response to establish a connection with the first user application server and downloads the data of the model package 1 through a dedicated channel.

[0265] S1130: The first user terminal and the second user terminal complete a subsequent IMS precondition process.

[0266] S1131: After the second user answers the call, the second user terminal returns a 200 OK response to the first user terminal via the IMS network, and the first user terminal establishes a media path with the second user terminal.

[0267] In some embodiments, user data may or may not be transferred through an application server. After receiving the user data from the second user terminal, the first user terminal generates holographic presentation content using model package 1; after receiving the user data from the second user, the second user terminal generates holographic presentation content using model package 2.

[0268] Through the above steps, the first user terminal and the second user terminal complete the holographic communication process.

[0269] The following describes the process of interaction between a user terminal capable of holographic communication and a user terminal without holographic communication capabilities, using the example of implementing holographic communication in an IMS network, where a first user terminal and a second user terminal can use the same IMS network. It should be understood that the first user terminal and the second user terminal can use the same IMS network or different IMS networks, and the corresponding processing flow is the same regardless of whether the same IMS network or different IMS networks is used. In some embodiments, the method includes the following steps: S1201 to S1228.

[0270] S1201: A first user terminal sends an INVITE request to an SBC. The INVITE request includes the dedicated channel media information of the first user terminal, a service indication of holographic communication, and terminal capability information.

[0271] Exemplarily, the holographic communication service indication may be implemented by the following method:

[0272] Add the holographic communication media label to the media line (m=line) of the SDP. m=application 10000 UDP / DTLS / SCTP webrtc-datachannel a=dcmap:1002label="holo"

[0273] Exemplarily, the terminal capabilities include at least one of: graphics processing unit (GPU) capabilities, three-dimensional (3D) rendering capabilities, screen size, resolution, multi-dimensional acquisition capabilities, and the like.

[0274] S1202: The SBC forwards the INVITE request to the S-CSCF.

[0275] S1203: The S-CSCF forwards the INVITE request to the VoLTE AS.

[0276] S1204, the VoLTE AS reports the call event to the service interaction platform, and carries the dedicated channel media information of the first user terminal, the holographic communication service indication, and the terminal capability information.

[0277] S1205: The service interaction platform reports the call event to the information reconstruction application server corresponding to the first user terminal according to the holographic communication service instruction;

[0278] S1206: After receiving the request, the information reconstruction application server corresponding to the first user terminal selects model package 1 for the first user terminal according to the terminal capability information of the first user.

[0279] S1207: The application server corresponding to the first user terminal instructs the service interaction platform to continue the call.

[0280] S1208: The service interaction platform instructs the VoLTE AS to continue the call.

[0281] S1209: The VoLTE AS forwards the INVITE request to the I / S-CSCF.

[0282] S1210: The I / S-CSCF forwards the INVITE session request to the VoLTE AS corresponding to the second user terminal according to the identification information of the second user terminal.

[0283] S1211, the VoLTE AS corresponding to the second user terminal reports the call event to the service interaction platform, and carries the holographic communication service indication and terminal capability information.

[0284] S1212: The service interaction platform corresponding to the second user terminal reports the call event to the information reconstruction application server corresponding to the second user terminal according to the holographic communication service indication.

[0285] S1213: After receiving the request, the information reconstruction application server corresponding to the second user terminal determines that the terminal characteristics of the second user do not support holographic communication, and instructs the service interaction platform to delete the dedicated channel and continue the call.

[0286] S1214: The service interaction platform instructs the VoLTE AS to delete the dedicated channel and continue the call.

[0287] S1215: The VoLTE AS forwards the INVITE request to the I / S-CSCF. The INVITE request does not carry the holographic communication service indication, media description, and model package identification information. For example, the holographic communication related information is deleted in the SDP.

[0288] The deleted holographic communication corresponding information includes the following content: “m=application 10000 UDP / DTLS / SCTP webrtc-datachannel a=dcmap:1002 label=”holo” a=package:2”.

[0289] S1216: The I / S-CSCF sends an INVITE request to the SBC where the second user terminal is registered.

[0290] S1217: The SBC registered with the second user terminal forwards the INVITE request to the second user terminal.

[0291] S1218: When the second user terminal receives the INVITE call request, it prepares resources according to the basic call of non-holographic communication.

[0292] S1219: The second user terminal returns a 183 response to the SBC corresponding to the second user terminal.

[0293] S1220: The SBC corresponding to the second user terminal forwards the 183 response to the S-CSCF.

[0294] S1221: The S-CSCF forwards the response to the VoLTE AS.

[0295] S1222: The VoLTE AS corresponding to the second user terminal sends a 183 response to the I / S-CSCF, setting the holographic media port number to 0, indicating that it is unavailable. m = application 0 UDP / DTLS / SCTP webrtc-datachannel

[0296] S1223: The I / S-CSCF forwards the 183 response to the VoLTE AS corresponding to the first user terminal.

[0297] S1224, the VoLTE AS corresponding to the first user terminal determines that the second user does not support holographic communication, and does not insert the model identification information corresponding to the first user in the 183 response and forwards it to the I / S-CSCF.

[0298] S1225 , the I / S-CSCF forwards the 18x response to the SBC where the first user terminal is registered.

[0299] S1226: The SBC forwards the 18x response to the first user terminal.

[0300] S1227: The first user terminal and the second user terminal complete the subsequent IMS precondition process.

[0301] S1228: After the second user answers the call, the second user terminal returns a 200 OK response to the first user terminal via the IMS network, and the first user terminal establishes a common audio and video media path with the second user terminal.

[0302] Through the above steps, the ordinary audio and video call process between the user terminal with holographic communication capability and the user terminal without holographic communication capability is completed.

[0303] FIG10 is a schematic structural diagram of the first user terminal provided in this application.

[0304] In the fourth aspect, referring to Figure 10, the present application provides a first user terminal 1000, including: an extraction module 1010, configured to perform feature extraction on the collected first multidimensional image data to obtain feature information of the first multidimensional image data; a conversion module 1020, configured to convert the feature information into first behavior instruction data, and the first behavior instruction data is used to indicate the behavior information of the first user to whom the first user terminal belongs; a transmission module 1030, configured to transmit the first behavior instruction data to the second user terminal through a dedicated channel, and the dedicated channel is a data transmission channel established in advance based on real-time communication technology. The second user terminal is configured to reconstruct information according to the first behavior instruction data to obtain reconstructed first multidimensional image data.

[0305] In some embodiments, the first multidimensional image data includes multidimensional image data, and the feature information is a multidimensional image data feature; the extraction module 1010 is specifically configured to: preprocess the multidimensional image data collected by the image sensor, and the preprocessing includes at least one of the following: image denoising, image alignment, posture correction and motion compensation; perform feature extraction on the preprocessed multidimensional image data to obtain multidimensional image data features.

[0306] In some embodiments, the conversion module 1020 is specifically configured to: match the feature information with the semantic knowledge base to obtain semantic information corresponding to the feature information, the semantic information including descriptive information of the entity behavior; and convert the descriptive information of the entity behavior into corresponding first behavior instruction data.

[0307] In some embodiments, the dedicated channel includes: a first dedicated channel between the first user terminal and the first communication network, and a second dedicated channel between the second user terminal and the second communication network; the first user terminal 1000 also includes: a sending module, configured to send a first session request to the first communication network corresponding to the first user terminal before transmitting the first behavior instruction data to the second user terminal through the dedicated channel, the first session request including at least: first dedicated channel media information and service type indication information of the first user terminal, the first communication network being configured to forward the first dedicated channel media information and service type indication information to the second user terminal through the second communication network corresponding to the second user terminal; receiving A module is configured to receive, through the first communication network, a session response from a second user terminal forwarded by a second communication network; the session response is a session response generated and sent to the second communication network by the second user terminal after receiving the first dedicated channel media information and service type indication information, and the session response includes the second dedicated channel media information of the second user terminal; the first communication network is further configured to forward the session response to the first user terminal; the second communication network is further configured to establish a second dedicated channel with the second user terminal based on the received second dedicated channel media information; an establishment module is configured to establish a first dedicated channel with the first communication network based on the first dedicated channel media information in response to the session response.

[0308] In some embodiments, the transmission module 1030 is specifically configured to: send the first behavior instruction data to the first communication network through a first dedicated channel; the first communication network is configured to send the first behavior instruction data to the second communication network, and the second communication network is configured to send the first behavior instruction data to the second user terminal through a second dedicated channel.

[0309] In some embodiments, the first session request also includes: first reconstruction capability information, the first reconstruction capability information is used to indicate the information reconstruction capability of the first user terminal; the first communication network is further configured to: send the first session request to the first application server; the first application server is configured to select corresponding first model data for the first user terminal based on the reconstruction capability information of the first user terminal, and instruct the first communication network to send a second session request to the second communication network, the second session request includes: first dedicated channel media information and service type indication information; and before sending the session response to the first user terminal, add an identifier of the first model data to the session response; the second communication network is further configured to: send the second session request to the second communication network The second application server corresponding to the second user terminal is configured to select corresponding second model data for the second user terminal based on the stored reconstruction capability information of the second user terminal, and instruct the second communication network to send a third session request to the second user terminal, wherein the third session request includes: first dedicated channel media information, service type indication information, and an identifier of the second model data; the second user terminal is further configured to: after downloading the second model data through the second dedicated channel, perform feature extraction on the collected second multidimensional image data through the second model data and convert the feature information of the extracted second multidimensional image data into second behavior instruction data, and send the second behavior instruction data to the second communication network through the second dedicated channel.

[0310] The first user terminal 1000 also includes: a downloading module, configured to download the first model data through the first dedicated channel after establishing the first dedicated channel with the first communication network according to the first dedicated channel media information; a receiving module, further configured to receive second behavior instruction data from the first communication network through the first dedicated channel, the second behavior instruction data being data transmitted from the second communication network to the second communication network; an information reconstruction module, configured to reconstruct the second behavior instruction data according to the first model data to obtain reconstructed second multidimensional image data.

[0311] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of convenience and brevity, detailed descriptions of known methods are omitted here. The specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the aforementioned method implementations and will not be repeated here.

[0312] FIG11 is a schematic structural diagram of a second user terminal provided in this application.

[0313] In the fifth aspect, referring to Figure 11, the present application provides a second user terminal 1100, including: a receiving module 1110, configured to receive first behavior instruction data of a first user terminal through a dedicated channel; the dedicated channel is a data transmission channel established in advance based on real-time communication technology, and the first behavior instruction data is the behavior instruction data obtained by the first user terminal extracting features from the collected first multidimensional image data and converting the feature information of the extracted first multidimensional image data, and the first behavior instruction data is used to indicate the behavior information of the first user to whom the first user terminal belongs; a reconstruction module 1120, configured to reconstruct information according to the first behavior instruction data to obtain reconstructed first multidimensional image data.

[0314] In some embodiments, the dedicated channel includes: a first dedicated channel between a first user terminal and a first communication network, and a second dedicated channel between a second user terminal and a second communication network; the first communication network is configured to: send a first session request received from the first user terminal to a first application server, the first application server is configured to select corresponding first model data for the first user terminal based on the reconstruction capability information of the first user terminal, and instruct the first communication network to send a second session request to the second communication network, the second session request includes: first dedicated channel media information and service type indication information; the second communication network is configured to: send a second session request to a second application server corresponding to the second user terminal, the second application server is configured to select corresponding second model data for the second user terminal based on the stored reconstruction capability information of the second user terminal, and instruct the second communication network to send a third session request to the second user terminal, the third session request includes: first dedicated channel media information, service type indication information and an identifier of the second model data.

[0315] In some embodiments, the receiving module 1110 is further configured to receive a third session request sent by the second communication network before receiving the first behavior instruction data of the first user terminal through the dedicated channel; the second user terminal 1100 also includes: a sending module, configured to send a session response to the second communication network in response to the third session request, and the session response includes the second dedicated channel media information of the second user terminal; and an establishing module, configured to establish a second dedicated channel with the second communication network through the second dedicated channel media information.

[0316] In some embodiments, the reconstruction module 1120 is specifically configured to: download the second model data through the second dedicated channel; and reconstruct the first behavior instruction data using the second model data to obtain reconstructed first multi-dimensional image data.

[0317] In some embodiments, the first user terminal is further configured to: in response to a session response, establish a first dedicated channel with the first communication network according to the first dedicated channel media information, and download the first model data through the first dedicated channel; the second user terminal 1100 also includes: an extraction module, configured to perform feature extraction on the collected second multidimensional image data after downloading the second model data through the second dedicated channel, and convert the feature information of the extracted second multidimensional image data into second behavior instruction data; a sending module, configured to send the second behavior instruction data to the second communication network through the second dedicated channel; the second communication network is configured to transmit the second behavior instruction data to the first communication network, so that the first communication network transmits the second behavior instruction data to the first user terminal through the first dedicated channel; the first user terminal is configured to: reconstruct the second behavior instruction data according to the first model data to obtain reconstructed second multidimensional image data.

[0318] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of convenience and brevity, detailed descriptions of known methods are omitted here. The specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the aforementioned method implementations and will not be repeated here.

[0319] FIG12 is a schematic diagram of the structure of the communication system provided in this application.

[0320] In the sixth aspect, referring to Figure 12, the present application provides a communication system 1200, including: a receiving module 1210, configured to receive first behavior instruction data of a first user terminal, the first behavior instruction data is the behavior instruction data obtained by the first user terminal extracting features from the collected first multidimensional image data and converting the feature information of the extracted first multidimensional image data, and the first behavior instruction data is used to indicate the behavior information of the first user to whom the first user terminal belongs; a transmission module 1220, configured to transmit the first behavior instruction data to the second user terminal through a dedicated channel, the dedicated channel is a data transmission channel established in advance based on real-time communication technology, and the second user terminal is configured to reconstruct information according to the first behavior instruction data to obtain reconstructed first multidimensional image data.

[0321] In some embodiments, the dedicated channel includes a first dedicated channel and a second dedicated channel; the communication system includes at least a first communication network and a second communication network; the transmission module 1220 is further configured to: before receiving the first behavior instruction data of the first user terminal, receive a first session request from the first user terminal through the first communication network, the first session request including at least: first dedicated channel media information and service type indication information of the first user terminal; send the first dedicated channel media information and service type indication information to the second communication network through the first communication network, the second communication network being a communication network corresponding to the identifier of the second user terminal; send the first dedicated channel media information and service type indication information to the second user terminal through the second communication network; receive and forward the session response of the second user terminal to the first communication network through the second communication network, the session response including the second dedicated channel media information of the second user terminal; send the second dedicated channel media information to the first user terminal through the first communication network, and establish a first dedicated channel with the first user terminal based on the first dedicated channel media information; establish a second dedicated channel with the second user terminal using the second dedicated channel media information through the second communication network.

[0322] In some embodiments, the communication system further includes: a first application server and a second application server; when the transmission module 1220 is configured to send the first dedicated channel media information and service type indication information to the second communication network through the first communication network, it is specifically configured as follows: sending a first session request to the first application server through the first communication network, the first session request also including the reconstruction capability information of the first user terminal; the first application server selects the corresponding first model data for the first user terminal based on the reconstruction capability information of the first user terminal, and instructs the first communication network to send a second session request to the second communication network, the second session request including: the first dedicated channel media information and service type indication information.

[0323] When the transmission module 1220 is configured to send the first dedicated channel media information and service type indication information to the second user terminal through the second communication network, it is specifically configured as follows: sending a second session request to the second application server through the second communication network; the second application server selects the corresponding second model data for the second user terminal based on the stored reconstruction capability information of the second user terminal, and instructs the second communication network to send a third session request to the second user terminal, where the third session request includes: the first dedicated channel media information, the service type indication information and the identifier of the second model data.

[0324] When the transmission module 1220 is configured to send the second dedicated channel media information to the first user terminal through the first communication network, it is specifically configured as follows: adding the identifier of the first model data in the session response through the first communication network; and sending the session response carrying the identifier of the first model data and the second dedicated channel media information to the first user terminal.

[0325] In some embodiments, when the transmission module 1220 is configured to transmit the first behavior instruction data to the second user terminal through a dedicated channel, it is specifically configured as follows: sending the first behavior instruction data received through the first dedicated channel to the second communication network; in the second communication network, sending the first behavior instruction data to the second user terminal through the second dedicated channel.

[0326] In some embodiments, after being configured to establish a second dedicated channel with the second user terminal via the second communication network using the second dedicated channel media information, the transmission module 1220 is further configured to: receive second behavior instruction data from the second user terminal via the second dedicated channel in the second communication network, where the second behavior instruction data is behavior instruction data obtained by the second user terminal extracting features from the collected second multidimensional image data through the second model data and converting the feature information of the extracted second multidimensional image data; transmit the second behavior instruction data to the first communication network via the second communication network; and send the second behavior instruction data to the first user terminal via the first dedicated channel in the first communication network, where the first user terminal is configured to reconstruct the second behavior instruction data based on the first model data to obtain reconstructed second multidimensional image data.

[0327] In some embodiments, the first communication network includes: a first session edge controller, a first call session control function node, a first multimedia system application server, and a first service interaction platform connected in sequence, the first session edge controller is connected to the first user terminal, and the first service interaction platform is connected to the first application server; the second communication network includes: a second session edge controller, a second call session control function node, a second multimedia system application server, and a second service interaction platform connected in sequence, the second session edge controller is connected to the second user terminal, and the second service interaction platform is connected to the second application server; and the first call session control function node is connected to the second multimedia system application server for communication transmission from the first communication network to the second communication network; and the second call session control function node is connected to the first multimedia system application server for communication transmission from the second communication network to the second communication network.

[0328] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of convenience and brevity, detailed descriptions of known methods are omitted here. The specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the aforementioned method implementations and will not be repeated here.

[0329] In a seventh aspect, the present application also provides a network device.

[0330] 13 , the electronic device includes: at least one processor 1301; at least one memory 1302; the memory 1302 stores at least one computer program that can be executed by the at least one processor 1301, and the at least one computer program is executed by the at least one processor 1301 so that the at least one processor 1301 can execute the above-mentioned information processing method and information transmission method.

[0331] In some embodiments, the electronic device further includes at least one I / O interface 1303 connected between the processor 1301 and the memory 1302 .

[0332] A processor is a device with data processing capabilities, including but not limited to a central processing unit (CPU); a memory is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); an I / O interface (read-write interface) is connected between the processor and the memory, enabling information exchange between the memory and the processor, including but not limited to a data bus (Bus).

[0333] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor / processing core, the processor / processing core implements the above-mentioned information processing method and information transmission method. The computer-readable storage medium can be volatile or non-volatile computer-readable storage medium.

[0334] Those skilled in the art will appreciate that all or some of the steps, systems, and functional modules / units in the apparatus disclosed above may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0335] The method provided in this application can be widely used in assisted manufacturing, assisted education, AR / VR video conferencing, digital twins and other fields, and has broad application prospects.

[0336] In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation.

[0337] Some or all of the physical components may be implemented as software executed by a processor (such as a central processing unit (CPU), a digital signal processor, or a microprocessor), or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or temporary media). As is well known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH) or other disk storage; compact disc (CD-ROM), digital versatile disc (DVD) or other optical disc storage; magnetic cassettes, tapes, disk storage or other magnetic storage; any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0338] This application has disclosed example embodiments, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly indicated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the present application as set forth in the appended claims.

Images