An immersive multimedia service control system, method, electronic device, and storage medium

Abstract

The present application discloses an immersive multimedia service control system, method, electronic device, and storage medium, which relate to the fields of wireless communication and terminals. The system includes a service control layer, a media processing layer, and a database. The service control layer is responsible for the service control of immersive multimedia services, including access authentication, intelligent distribution, computing power scheduling, and quality of service scheduling. The media processing layer is used for the processing of multimedia data of immersive multimedia services, including media extraction, compression encoding, online modeling, media rendering, media encapsulation, and multi-stream synchronization. The database is used to store user data and model data. The present application also discloses an immersive multimedia service control method and device. The present application enables the existing 5G service network architecture to support not only traditional audio streams, video streams, and text streams, but also new types of multimedia data streams such as multi-channel and multi-concurrent XR media streams or holographic media streams, achieving capabilities such as the distribution of multimedia data streams, multi-stream synchronization, and specific media information processing.

Description

Background Art

[0002] XR (Extended Reality) refers to an environment that combines the real and virtual and allows human-computer interaction generated by computer technology and wearable devices, including various forms such as AR (Augmented Reality), VR (Virtual Reality), and MR (Mixed Reality). The integration of the three visual interaction technologies realizes an immersive experience of seamless conversion between the virtual world and the real world, opening up a new mobile intelligent experience mode. As the fifth-generation mobile communication network, the peak theoretical transmission speed of 5G can reach dozens of GB per second, which injects new vitality into the evolution and scenario applications of XR technology. However, the existing 5G service network architecture does not support the intelligent distribution, multi-stream synchronization, and specific multimedia data processing capabilities of multimedia data.

[0003] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0004] To solve the above problems, the purpose of the present disclosure is to provide an immersive multimedia service control system, method, electronic device, and storage medium, which can realize end-to-end business control of multimedia data processing, computing power scheduling, intelligent distribution, QoS scheduling, etc., and solve the defect that the existing 5G service network architecture cannot support such services. In addition, the present disclosure supports the access of various types of terminals, and based on service policies and user privacy protection, it can realize the processing and encapsulation of specific multimedia data.

[0005] Other features and advantages of the present disclosure will become apparent through the following detailed description, or be learned in part through the practice of the present disclosure.

[0006] According to one aspect of the present disclosure, there is provided an immersive multimedia service control system, characterized in that it includes a service control layer, a media processing layer, and a database;

[0007] The service control layer is used for the service control of immersive multimedia services, including computing power scheduling, QoS scheduling, intelligent distribution, and access authentication;

[0008] The media processing layer is used for the processing of multimedia data of immersive multimedia services. The multimedia data includes holographic streams, XR streams, video streams, audio streams, text streams, and signaling streams. The processing of the multimedia data includes media extraction, media rendering, media encapsulation, online modeling, compression coding, and multi-stream synchronization;

[0009] The database is used to store user data and model data.

[0010] In one embodiment, the service control layer includes:

[0011] It includes an access authentication module, a computing power scheduling module, a service quality scheduling module, and an intelligent distribution module: Among them,

[0012] The access authentication module is used to receive a service request initiated by a user, query the user's ID from the user data, and after passing the authentication, respond to the service access request and return the address of the intelligent distribution module to the user;

[0013] The intelligent distribution module is used to receive multimedia data from the terminal and distribute different types of multimedia data to the media processing layer for further processing;

[0014] The computing power scheduling module is used to receive a computing power resource request, query the status of network computing power resources and allocate computing power resources, and return the computing power node address;

[0015] The service quality scheduling module is used to receive the synchronized media stream, query the terminal capabilities and network performance, and match the corresponding QoS policies.

[0016] In one embodiment, the media processing layer includes:

[0017] The media extraction module is used to receive the distributed multimedia data and extract predetermined media data from the multimedia data;

[0018] The compression and encoding module is used to perform compression and encoding processing on the predetermined multimedia data to obtain the compressed and numbered multimedia data;

[0019] The online modeling module is used to receive the compressed and numbered multimedia data, query the user's 3D / holographic model data in the database, initiate a computing power resource request to the computing power scheduling module, receive the returned computing power node address, use the computing power node resources to perform modeling on the compressed and numbered multimedia data, generate user model data, and write the user model data into the database;

[0020] The media rendering module is used to apply for computing power resources from the computing power scheduling module to complete the rendering processing of the model data;

[0021] The media encapsulation module is used to perform corresponding encapsulation processing on the rendered data based on business policies and user privacy protection;

[0022] The multi-stream synchronization module is used to synchronize the encapsulated multimedia data stream with other data streams processed in parallel.

[0023] In one embodiment, an immersive multimedia service control system further includes a terminal.

[0024] In one embodiment, the terminal is a holographic terminal, an HMD terminal, a common terminal, or a combination of the above.

[0025] According to another aspect of the present disclosure, an immersive multimedia service control method is provided, and the specific steps of the method include:

[0026] The terminal initiates a service request, receives the service request initiated by the user through the access authentication module, queries the user's ID from the user data, and after the authentication is passed, responds to the service access request and returns the address of the intelligent distribution module to the user;

[0027] The terminal sends the multimedia data to the intelligent distribution module;

[0028] Receiving multimedia data from the terminal through the intelligent distribution module, distributing different types of multimedia data to the media processing layer for further processing;

[0029] Receiving distributed multimedia data through the media extraction module, and extracting predetermined media data from the multimedia data;

[0030] The compression and encoding module compresses and encodes the predetermined multimedia data to obtain compressed and numbered multimedia data;

[0031] The compressed and numbered multimedia data is received through the online modeling module, the 3D / holographic model data of the user in the database is queried, a computing resource request is initiated to the computing resource scheduling module, the computing node address is received, the compressed and numbered multimedia data is modeled using the computing node resources, the user model data is generated, and the user model data is written into the database;

[0032] Apply for computing resources from the computing scheduling module through the media rendering module to complete the rendering process of the model data;

[0033] The media encapsulation module performs corresponding encapsulation processing on the rendered data based on business policies and user privacy protection;

[0034] Synchronizing the encapsulated multimedia data stream with other data streams processed in parallel through the multi-stream synchronization module;

[0035] Through the service quality scheduling module, receiving the synchronized multimedia data stream, querying the terminal capability and network performance, and matching the corresponding QoS strategy;

[0036] Forward the processed multimedia data stream to the address of the next-hop routing node through the intelligent distribution module.

[0037] According to another aspect of the present disclosure, there is provided an immersive multimedia service control device, including:

[0038] At least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute an immersive multimedia service control method as described in the second aspect above.

[0039] According to yet another aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program, which when executed by a processor implements an immersive multimedia service control method as described in the second aspect above.

[0040] According to yet another aspect of the present disclosure, there is provided a computer program product, the computer program product including a computer program stored on a computer-readable storage medium, the computer program including program instructions, which when executed by a processor cause the processor to execute an immersive multimedia service control method as described in the second aspect above.

[0041] An immersive multimedia service control system, method, electronic device and storage medium provided by an embodiment of the present disclosure include a service control layer, a media processing layer and a database. The technical solution of the present disclosure can implement end-to-end media processing, computing power scheduling, intelligent distribution, QoS scheduling and other service controls, enabling the existing 5G service network architecture to support not only traditional audio streams, video streams and text streams, but also new multimedia data streams such as multi-channel and multi-concurrent XR media streams or holographic media streams, and realizing the distribution of multimedia data streams, multi-stream synchronization, processing of specific media information and other capabilities. At the same time, the system architecture supports access of various types of terminals, and based on service policies and user privacy protection, it can implement the processing and encapsulation of specific multimedia data streams.

[0042] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. Brief Description of the Drawings

[0043] The drawings here are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0044] Figure 1 It is an architecture diagram of an immersive multimedia service control system provided by an exemplary embodiment of the present application;

[0045] Figure 2 It is a schematic structural diagram of the service control layer of an immersive multimedia service control system provided by an exemplary embodiment of the present application;

[0046] Figure 3 It is a schematic structural diagram of the media processing layer of an immersive multimedia service control system provided by an exemplary embodiment of the present application;

[0047] Figure 4 It is a schematic structural diagram of the terminal of an immersive multimedia service control system provided by an exemplary embodiment of the present application;

[0048] Figure 5 It is a flowchart of service access and intelligent distribution of multimedia data for an immersive multimedia service control method provided by an exemplary embodiment of the present application;

[0049] Figure 6 It is a flowchart of multimedia data processing and multimedia data modeling for an immersive multimedia service control method provided by an exemplary embodiment of the present application;

[0050] Figure 7 It is a flowchart of multimedia data rendering, encapsulation, synchronization, and QoS scheduling for an immersive multimedia service control method provided by an exemplary embodiment of the present application;

[0051] Figure 8 It is an immersive multimedia service control device provided by an exemplary embodiment of the present application. Detailed Implementation Manner

[0052] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art. The features, structures, or characteristics described can be combined in any suitable manner in one or more embodiments.

[0053] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0054] The solution provided by this application can achieve service controls such as end-to-end multimedia data processing, computing power scheduling, intelligent distribution, and QoS scheduling under the existing 5G service network architecture. It enables an immersive experience for users, breaks through the limit of flat visual experience, enters the three-dimensional space experience stage, promotes the integration of the real world and the digital world, and thus changes the ways of users' work, life, study, and entertainment. For ease of understanding, several terms involved in this application will be explained first below.

[0055]

Artificial Intelligence-related Terms

[0056] Artificial Intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines to enable the machines to have the functions of perception, reasoning, and decision-making.

[0057] Computer Vision Technology (CV) Computer vision is a science that studies how to enable machines to "see". More specifically, it refers to machine vision that uses cameras and computers to replace human eyes for target recognition, tracking, and measurement, and further performs graphic processing to make the computer-processed images more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish artificial intelligence systems that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D (3-Dimension) technology, virtual reality, augmented reality, simultaneous localization and mapping, etc. technologies, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0058] The solution provided by the embodiments of this application relates to technologies such as immersive multimedia service control in wireless communication, and is specifically described through the following embodiments:

[0059] Figure 1 It is an architecture diagram of an immersive multimedia service control system provided by an exemplary embodiment of this application. The system includes: a service control layer 110, a media processing layer 120, and a database 130.

[0060] Figure 2 It is a schematic structural diagram of the service control layer of an immersive multimedia service control system provided by an exemplary embodiment of this application. The service control layer includes an access authentication module 111, an intelligent distribution module 112, a computing power scheduling module 113, and a quality of service scheduling module 114.

[0061] The access authentication module 111 is used to receive a service request initiated by a user, query the user's ID from user data, and after successful authentication, respond to the service access request and return the address of the intelligent distribution module to the user;

[0062] The intelligent distribution module 112 is used to receive multimedia data from a terminal and distribute different types of multimedia data to the media processing layer for further processing;

[0063] The computing power scheduling module 113 is used to receive a computing power resource request, query the status of network computing power resources and allocate computing power resources, and return the address of a computing power node;

[0064] The quality of service scheduling module 114 is used to receive the synchronized media stream, query the terminal capabilities and network performance, and match the corresponding QoS policy.

[0065] Figure 3 It is a schematic structural diagram of the media processing layer of an immersive multimedia service control system provided by an exemplary embodiment of the present application. The media processing layer includes: a media extraction module 121, a compression and encoding module 122, an online modeling module 123, a media rendering module 124, a media encapsulation module 125, and a multi-stream synchronization module 126.

[0066] The media extraction module 121 is used to receive the distributed multimedia data and extract predetermined media data from the multimedia data;

[0067] The compression and encoding module 122 is used to perform compression and encoding processing on the predetermined multimedia data to obtain the compressed and numbered multimedia data;

[0068] The online modeling module 123 is used to receive the compressed and numbered multimedia data, query the user's 3D / holographic model data in the database, initiate a computing power resource request to the computing power scheduling module, receive the returned computing power node address, use the computing power node resources to model the compressed and numbered multimedia data, generate user model data, and write the user model data into the database;

[0069] The media rendering module 124 is used to apply for computing power resources from the computing power scheduling module to complete the rendering processing of the model data;

[0070] The media encapsulation module 125 is used to perform corresponding encapsulation processing on the rendered data based on service policies and user privacy protection;

[0071] The multi-stream synchronization module 126 is used to synchronize the encapsulated multimedia data stream with other data streams processed in parallel.

[0072] Figure 4 It is a schematic structural diagram of a terminal of an immersive multimedia service control system provided by an exemplary embodiment of the present application. The terminal 140 is one or several of a holographic terminal, an HMD terminal, and an ordinary terminal.

[0073] Figure 5 It is a flowchart of service access and intelligent distribution of multimedia data of an immersive multimedia service control method provided by an exemplary embodiment of the present application, including the following steps:

[0074] Step a: The terminal initiates a service request. After the access authentication module queries the user ID from the user data and is correct, it responds to the service access request and returns the address of the intelligent distribution module, which is specifically divided into:

[0075] Step a1: Service request;

[0076] Step a2: User ID query request;

[0077] Step a3, query result is returned, responding to the service access request;

[0078] Step a4, access permission, intelligent distribution module address.

[0079] Step b: The terminal sends the media data to the intelligent distribution module.

[0080] Step c: the intelligent distribution module distributes different types of media data to various media processing function modules for further processing according to the type identifier of the media stream.

[0081] Figure 6 This is a flowchart of multimedia data processing and multimedia data modeling of an immersive multimedia service control method provided by an exemplary embodiment of the present application, comprising the following steps:

[0082] In step d, the media extraction module extracts specific media data (such as head, face, and facial features) from the distributed media data.

[0083] In step e, the extracted media data is compressed and encoded, and then modeled by the online modeling module.

[0084] Step f, the online modeling module queries the model data to see if there is a 3D / holographic model of the current user (f1), and the model data returns the query result (f2).

[0085] Step g, the online modeling module initiates a model call request (g1), and the model data returns the existing user model or the preset 3D / holographic model (g2) according to the query result.

[0086] In step h, the online modeling module initiates a computing resource request to the computing resource scheduling module (h1). The computing resource scheduling module queries the network computing resource status and returns the computing node address (h2).

[0087] In step i, the online modeling module uses computing node resources to complete the modeling operation and writes the generated user model into the database.

[0088] In step j, the model data is sent to the next processing module.

[0089] Figure 7 This is a flowchart of multimedia data rendering encapsulation synchronization and QoS scheduling of an immersive multimedia service control method provided by an exemplary embodiment of the present application, including the following steps:

[0090] Step k: The multimedia data rendering module applies for computing resources from the computing power scheduling module to complete the rendering operation of the model data.

[0091] Step l: The rendered data is sent to the media encapsulation module, where corresponding encapsulation and processing operations are performed based on business policies and user privacy protection.

[0092] Step m: The encapsulated data stream, together with other data streams processed in parallel, undergoes media stream synchronization under the processing of the multi-stream synchronization module.

[0093] Step n: The synchronized media stream is sent to the QoS scheduling module, where corresponding QoS policies are matched based on the current terminal capabilities and network performance.

[0094] Step o: The processed multimedia service stream is forwarded by the intelligent distribution module by querying the address of the next-hop routing node.

[0095] Figure 8 An immersive multimedia service control device provided by an exemplary embodiment of the present application. The immersive multimedia service control device 800 can be any type of terminal, such as a mobile phone, a game console, a tablet computer, an e-book reader, smart glasses, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a smart home device, an AR (Augmented Reality) device, a VR (Virtual Reality) device, etc. mobile terminals. Alternatively, the immersive multimedia service control device 800 can also be a personal computer (PC), such as a laptop computer and a desktop computer, etc., such as mobile phones, tablet computers, personal computers, etc.

[0096] Among them, the immersive multimedia service control device 800 can be installed with an application program for providing an immersive multimedia service control system.

[0097] Preferably, the immersive multimedia service control device 800 includes: one or more processors 810 and a memory 820. Figure 8 Here, one processor 810 is taken as an example.

[0098] The processor 810 and the memory 820 can be connected through a bus or other means. Figure 8 Taking the connection through the bus as an example.

[0099] The memory 820, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and units, such as program instructions / units corresponding to an immersive multimedia service control device 800 in an embodiment of the present invention. For example, a service control unit, a media processing unit, and a database unit. The processor 810 executes various functional applications and data processing of an immersive multimedia service control device by running the non-transitory software programs, instructions, and units stored in the memory 820, that is, implements an immersive multimedia service control method in the above method embodiment.

[0100] The memory 820 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of an immersive multimedia service control device, etc. In addition, the memory 820 may include a high-speed random access memory and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 820 may optionally include a memory remotely set relative to the processor 810, and these remote memories can be connected to the immersive multimedia service control device through a network.

[0101] Optionally, the above network connection can be a wireless network or a wired network using standard communication technologies and / or protocols. The network is usually the Internet, but can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network, or a virtual private network). In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged through the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec), etc. can be used to encrypt all or some of the links. In other embodiments, custom and / or proprietary data communication technologies can also be used to replace or supplement the above data communication technologies.

[0102] The one or more units are stored in the memory 820 and, when executed by the one or more processors 810, perform an immersive multimedia service control method in any of the above method embodiments. For example, execute method steps a to c described above Figure 5 in, Figure 6 method steps d to j in, and implement Figure 7 method steps k to o in.

[0103] An embodiment of the present invention also provides a computer-readable storage medium storing computer-executable instructions that, when executed by one or more processors, for example, by Figure 8 one of the processors 810 in, enable the one or more processors 810 to perform an immersive multimedia service control method in the above method embodiments. For example, execute method steps a to c described above Figure 5 in, Figure 6 method steps d to j in, and implement Figure 7 method steps k to o in.

[0104] An embodiment of the present invention also provides a computer program product. The computer program product includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions that, when executed by a computer, for example, by Figure 8 one of the processors 810 in, cause the computer to execute the method steps as described above. For example, execute method steps a to c described above Figure 5 in, Figure 6 method steps d to j in, and implement Figure 7 method steps k to o in.

[0105] A program product for implementing the above method according to an embodiment of the present invention is described. It may be in the form of a portable compact disc read-only memory (CD-ROM) and include program code and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0106] The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0107] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0108] The program code contained on the readable medium may be transmitted by any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above.

[0109] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, executed as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by using an Internet service provider to connect through the Internet).

[0110] Optionally, the clients of the application programs installed in different devices 800 are the same, or the clients of the application programs installed on two devices 800 are the clients of the same type of application program on different control system platforms. Based on the differences in the terminal platforms, the specific forms of the clients of the application program can also be different. For example, the client of the application program can be a mobile phone client, a PC client, or a World Wide Web (Web) client, etc.

[0111] Those skilled in the art can understand that the number of the above-mentioned devices can be more or less. For example, the above-mentioned terminal can be only one, or there can be dozens or hundreds of the above-mentioned terminals, or even more. The embodiments of the present application do not limit the number and types of the terminals.

[0112] It should be noted that the above-mentioned drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present invention, rather than for limiting purposes. It is easy to understand that the processes shown in the above-mentioned drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easy to understand that these processes can be executed synchronously or asynchronously, for example, in multiple modules.

[0113] Those skilled in the art to which the present invention pertains can understand that various aspects of the present invention can be implemented as a system, a method, or a program product. Therefore, various aspects of the present invention can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to herein as "circuitry", "module", or "system".

[0114] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented through software, or can be implemented by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0115] It should be noted that although several modules or units of devices for action execution are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more of the above-mentioned modules or units can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0116] In addition, although the steps of the methods in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in that specific order, or that all of the steps shown must be performed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0117] From the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the methods according to the embodiments of the present disclosure.

[0118] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed herein. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the appended claims.

Claims

1. An immersive multimedia service control system, characterized in that, Including: Business control layer, including: an access authentication module, which is used to receive a service request initiated by a user, query the user's ID from user data, and after successful authentication, return the address of the intelligent distribution module in response to the service access request to the user; an intelligent distribution module, which is used to receive multimedia data from a terminal and distribute different types of multimedia data to the media processing layer for further processing; a computing power scheduling module, which is used to receive a computing power resource request, query the status of network computing power resources and allocate computing power resources, and return the computing power node address; a service quality scheduling module, which is used to receive the synchronized media stream, query the terminal capabilities and network performance, and match the corresponding QoS policy; Media processing layer, including: a media extraction module, which is used to receive the distributed multimedia data and extract predetermined media data from the multimedia data; a compression and encoding module, which is used to perform compression and encoding processing on the predetermined multimedia data to obtain compressed and encoded multimedia data; an online modeling module, which is used to receive the compressed and encoded multimedia data, query the user's 3D / holographic model data in the database, initiate a computing power resource request to the computing power scheduling module, receive the returned computing power node address, use the computing power node resources to perform modeling on the compressed and encoded multimedia data, generate user model data, and write the user model data into the database; a media rendering module, which is used to apply for computing power resources from the computing power scheduling module to complete the rendering processing of the model data; a media encapsulation module, which is used to perform corresponding encapsulation processing on the rendered data based on business policies and user privacy protection; a multi-stream synchronization module, which is used to synchronize the encapsulated multimedia data stream with other data streams processed in parallel; Database, which is used to store user data and model data.

2. The immersive multimedia service control system according to claim 1, wherein It also includes a terminal.

3. An immersive multimedia service control system according to claim 2, characterized in that, The terminal is one or several of a holographic terminal, an HMD terminal, and an ordinary terminal.

4. An immersive multimedia service control method, characterized in that, The specific steps of the method include: The terminal initiates a service request. The access authentication module receives the service request initiated by the user, queries the user's ID from user data, and after successful authentication, returns the address of the intelligent distribution module in response to the service access request to the user; The terminal sends the multimedia data to the intelligent distribution module; Receive the multimedia data from the terminal through the intelligent distribution module, and distribute different types of multimedia data to the media processing layer for further processing; Receive the distributed multimedia data through the media extraction module and extract the predetermined media data from the multimedia data; Perform compression and encoding processing on the predetermined multimedia data through the compression and encoding module to obtain compressed and encoded multimedia data; Receive the compressed and encoded multimedia data through the online modeling module, query the user's 3D / holographic model data in the database, initiate a computing power resource request to the computing power scheduling module, receive the returned computing power node address, use the computing power node resources to perform modeling on the compressed and encoded multimedia data, generate user model data, and write the user model data into the database; Apply for computing power resources from the computing power scheduling module through the media rendering module to complete the rendering processing of the model data; Through the media encapsulation module, corresponding encapsulation processing is performed on the rendered data based on business policies and user privacy protection; Through the multi-stream synchronization module, synchronize the encapsulated multimedia data stream with other data streams processed in parallel; Through the quality of service scheduling module, receive the synchronized multimedia data stream, query the terminal capabilities and network performance, and match the corresponding QoS policies; Through the intelligent distribution module, forward the processed multimedia data stream to the address of the next-hop routing node.

5. An immersive multimedia service control electronic device, characterized in that, Comprising: At least one processor and a memory for communicatively connecting with the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the immersive multimedia service control method described in claim 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the immersive multimedia service control method described in claim 4.

Images