Non-traditional video live broadcast system and method based on Web technology

Abstract

The invention relates to the technical field of network live broadcast, in particular to a non-traditional video live broadcast system and method based on Web technology, the system comprises a server side, a user side and an anchor side, the server side comprises the following modules: a resource pre-loading module, a video pre-loading module, a video pre-loading module, a video pre-loading module, a video pre-loading module, a video pre-loading module, a video pre-loading module and a video pre-loading module when the user side opens a live broadcast room corresponding to a certain anchor side; the core resources of the live broadcast room are preloaded to a user side through a Web technology; the core resources comprise a virtual role model, a scene background and a special effect; the information stream transmission module is used for obtaining a control instruction of the virtual role transmitted by the anchor side after the user side opens the live broadcast room and establishes connection with the live broadcast room, and forwarding the control instruction to the user side in real time, so that the user side renders the animation, audio and special effect of the virtual role according to the control instruction; the control instruction comprises an action instruction and a voice instruction. According to the invention, efficient and stable resource loading can be realized so as to achieve low-delay and smooth live broadcast experience.

Description

Technical Field

[0001] The present invention relates to the technical field of webcasting, and particularly to a non - traditional video live - streaming system and method based on Web technology. Background Art

[0002] In recent years, live - streaming technology has been widely applied, covering many fields from entertainment to education, and from commerce to social interaction. However, traditional live - streaming technology focuses on audio - video stream transmission, and its underlying architecture has extremely high requirements for network bandwidth, device performance, and server decoding capabilities. Especially in large - scale concurrency or complex scenarios, it is often difficult to meet the requirements of low latency and high interactivity. At the same time, with the rapid rise of virtual live streamers, users' expectations for the personalization and interactivity of live - streaming content continue to increase. Virtual live - streaming technology provides a more immersive experience for viewers through means such as 3D modeling, real - time motion capture, and voice synthesis, but traditional live - streaming technology is difficult to support the highly customized needs of virtual live streamers for dynamic scenes.

[0003] Currently, the main bottleneck of traditional live - streaming technology lies in its video - stream - centered transmission method. This method not only requires high video decoding capabilities from terminal devices but also extremely high bandwidth to support real - time video content updates. This results in many users of low - performance devices being unable to enjoy high - quality live - streaming content. At the same time, when the network conditions are slightly poor, the live - streaming delay and stuttering are significant, seriously affecting the interactive experience. For innovative live - streaming scenarios such as virtual live streamers, this limitation is even more prominent because virtual live - streaming rooms usually contain complex dynamic special effects and highly interactive functions, requiring higher real - time performance from the system. Therefore, it is necessary to explore a new live - streaming technology to achieve a low - latency and smooth live - streaming experience. Summary of the Invention

[0004] One of the purposes of the present invention is to provide a non - traditional video live - streaming system based on Web technology, which can achieve efficient and stable resource loading to achieve a low - latency and smooth live - streaming experience.

[0005] To achieve the above - mentioned purpose, a non - traditional video live - streaming system based on Web technology is provided, which includes a server - side, a user - side, and a host - side. The server - side includes the following modules:

[0006] A resource pre - loading module: used to pre - load the core resources of the live - streaming room corresponding to a certain host - side to the user - side through Web technology when the user - side opens the live - streaming room; the core resources include virtual character models, scene backgrounds, and special effects;

[0007] Information flow transmission module: After opening the live broadcast room on the user side and establishing a connection with it, it is used to obtain the control instructions of the virtual character transmitted by the host side and forward the control instructions to the user side in real time, so that the user side can render the animation, audio and special effects of the virtual character according to the control instructions; the control instructions include action instructions and voice instructions.

[0008] Furthermore, the resource preloading module includes the following sub-modules:

[0009] First access analysis sub-module: When the user side accesses the live broadcast room, it is used to initialize the connection, establish a stable connection with the server through Web technology to support real-time two-way data transmission, analyze and judge whether the user side is accessing the live broadcast room for the first time. If so, start the resource chunk preloading mechanism; if not, execute the local cache check sub-module;

[0010] Local cache check sub-module: It is used to enable the user side to check whether the local cache has stored relevant core resources; if the core resources exist and the versions are consistent, the user side directly reads and loads from the local cache; if the core resources do not exist or the versions are inconsistent, start the resource chunk preloading mechanism, send the latest core resources to the user side, and then the user side reads and loads;

[0011] Resource version verification module: It is used to enable the user side to verify the integrity and consistency of the core resources through the hash verification mechanism to ensure that the data during the transmission process is not damaged; if the verification fails, request the server to send the latest core resources and overwrite.

[0012] Furthermore, the resource chunk preloading mechanism is: loading the core resources required by the live broadcast room in a chunked manner; the chunking method is specifically: dividing the core resources into multiple task chunks {R1, R2,..., RN}, calculating the priority Pi of each task chunk, and then sorting the resource queue Q = {R1, R2,..., RN} from high to low according to the priority sorting algorithm, and then using the parallel loading technology to load the task chunks simultaneously through multi-threading or asynchronous requests; the priority algorithm formula is:

[0013] P i = ω1·S i + ω2·D i + ω3·U i

[0014] Among them, S i represents the importance of the core resources, D i represents the dependency of the core resources, U i represents the adaptability of the user side device performance and network conditions, and ω1, ω2, and ω3 are weight coefficients respectively.

[0015] Furthermore, the server also includes:

[0016] Dynamic resource update module: used to load new core resources to the user side through a lightweight dynamic update mechanism during the live broadcast; the dynamic update mechanism is: obtaining a dynamic request sent by the user side to the server according to the live broadcast scenario requirements, and the dynamic request is the new core resource that needs to be loaded.

[0017] Resource loading optimization module: used to dynamically adjust the quality of core resources according to the network bandwidth and computing power of the user side device. If the bandwidth is low or the device performance is weak, the low-quality version of the core resource is preferentially loaded, and the higher-quality core resource is loaded after the conditions improve; at the same time, after the core resource is loaded, the client immediately performs progressive rendering, preferentially displays the basic content, and then gradually renders the picture according to the priority of the core resource and the current hardware state; and continues to load other low-priority core resources in the client background.

[0018] Further, after the user first enters the live broadcast room for connection initialization, the server further includes the following modules:

[0019] Identity authentication module: used to obtain the identity authentication request sent by the user side and confirm the user information and permissions.

[0020] Timestamp synchronization module: used to record the exact time at this moment after the identity authentication is passed and the core resource loading is completed, generate a basic timestamp, and use it as the starting reference time for subsequent time synchronization; all subsequent control instructions and dynamic information are synchronized based on this timestamp. During the synchronization process, a time synchronization request is sent to the host side to obtain the current timestamp returned by the host side, so that the client and the host side align the reference time and start real-time data transmission; during the transmission process, a persistent connection is maintained, and the control instructions of the host side and the interactive information of the user are processed into small data packets through compression and optimization algorithms and transmitted between the user side and the server through WebSocket.

[0021] Host side: generates a timestamp based on its local clock and attaches it to each control instruction; the action instructions of the control instruction include action type, action parameters, and timestamp, and the action parameters include position, speed, and direction.

[0022] Further, it further includes the following modules:

[0023] Packet Loss Analysis and Processing Module: It is used to transmit data using redundant data encoding. When part of the data is lost, the lost information is recovered through redundant data. It is also used to enable the client to send an acknowledgement message ACK to the server regularly to ensure that the data has been received. If the acknowledgement signal is not received within the timeout period, a retransmission request is triggered, and the data is retransmitted through the TCP retransmission mechanism. And in the case of severe packet loss, the client is enabled to smooth out the data loss through a packet loss compensation strategy. The packet loss compensation strategy is as follows: Before the lost data packet is retransmitted, a delayed or blank rendering frame is inserted, and the time delay between the client and the host is adjusted in real time according to the network conditions to receive subsequent data packets in advance.

[0024] Automatic Reconnection Module: It is used to issue a prompt of connection interruption or recovery to the client when a connection disconnection is detected, ensuring that the user knows that the live content is paused and will resume as soon as possible, and enabling the client to automatically attempt to reconnect to the server. After the connection is restored, the client is enabled to restore the timestamp before the disconnection, and catch up with the latest timestamp and status through acceleration to ensure a smooth transition of the live broadcast. The exponential backoff algorithm is adopted to control the reconnection frequency during the disconnection process. If the disconnection time exceeds the time threshold, all necessary states are synchronized after reconnection, and the states include, for example, timestamps, actions, and voices. And the scene rendering is restored based on the timestamp at the time of reconnection, so that the content of the client live broadcast is consistent with the host at the time of recovery.

[0025] Further, after the client receives a control instruction with a timestamp, the content and timestamp corresponding to each control instruction are extracted, and the content includes actions, voices, and special effects. The timestamp of the control instruction is compared with the local clock for timestamp offset correction, and the delay between the timestamp and the current time of the local clock is calculated for adjusting the rendering timing.

[0026] If the delay exceeds the set duration, wait for the set time before rendering the control instruction.

[0027] If the delay does not exceed the set duration, render immediately; and reduce the error through a timestamp correction algorithm.

[0028] The timestamp correction algorithm is as follows: The average value of the timestamp differences is used for smooth adjustment; and the delay is calculated in real time during the process: Assume that the current timestamp of the client is current_timestamp, and the timestamp of the control instruction received from the host is control_timestamp, then the delay can be calculated by the following formula:

[0029] Delay = control_timestamp - current_timestamp

[0030] If the delay is negative, it means that the control instruction has expired and a new request is needed; if the delay is positive, it means that the control instruction is still valid and rendering should be performed on time.

[0031] Further, during the timestamp synchronization process, the client uses an adaptive algorithm to correct the drift and adopts the NTP time synchronization protocol to ensure the stability, unity, and accuracy of timestamp synchronization.

[0032] Further, after the client determines the rendering timing based on the delay, it then renders the animation, audio, and special effects; the animation is the skeletal animation of the virtual character. When rendering the skeletal animation of the virtual character, the client calls the resources in the resource library in real time, calculates, and updates the skeletal positions of the virtual character to complete the animation rendering; the rendering of the audio is for the audio of the virtual character, and the rendering of the special effects is for the real-time special effects rendering of the barrage and gifts sent by other clients.

[0033] The second object of the present invention is to provide a non-traditional video live broadcast method based on Web technology, and the method utilizes the non-traditional video live broadcast system based on Web technology as described above.

[0034] Principle and advantages:

[0035] 1. In this solution, by leveraging Web technology and adopting the chunk loading and local caching technology, the resources of the live broadcast room are pre-loaded into the user terminal, realizing the gradual loading and dynamic update of resources, reducing the network pressure of real-time loading, and significantly reducing the consumption of bandwidth and computing resources, providing a more smooth and efficient live broadcast experience for users.

[0036] 2. By changing to the real-time information flow transmission mode to replace the traditional audio and video stream transmission, the host side only needs to transmit a small amount of action, voice, and interaction information, and the client renders the scene immediately according to the received information. In this way, lower latency, higher customization, and interactivity can be achieved, thus significantly reducing the network bandwidth requirements. It brings revolutionary technical support to the application scenarios of virtual hosts.

[0037] 3. The client dynamically renders only the virtual character and the scene by receiving the real-time transmitted action and voice control instructions. It realizes a virtual live broadcast room environment with low latency, high interactivity, and high customization. At the same time, it integrates the timestamp correction algorithm and synchronization mechanism to ensure the real-time synchronization of voice, action, and scene, providing support for a low-latency and smooth live broadcast experience and providing a highly immersive viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flowchart of a non-traditional video live broadcast method based on Web technology according to an embodiment of the present invention;

[0039] Figure 2 is the core resource loading flowchart;

[0040] Figure 3 is the live broadcast information transmission flowchart;

[0041] Figure 4 is the dynamic rendering flowchart;

[0042] Figure 5 is the overall system architecture diagram of the system. Detailed implementation mode

[0043] The following is a further detailed description through specific implementation modes:

[0044] Embodiment

[0045] A non-traditional video live broadcast system based on Web technology is basically as Figure 5 shown, including a server, a user terminal and a host terminal. The server includes the following modules:

[0046] Resource preloading module: When the user terminal opens the live broadcast room corresponding to a certain host terminal, preload the core resources of the live broadcast room to the user terminal through Web technology; the core resources include virtual character models, scene backgrounds and special effects;

[0047] The resource preloading module includes the following sub-modules:

[0048] First access analysis sub-module: When the user terminal accesses the live broadcast room, perform connection initialization, establish a stable connection with the server through Web technology (WebSocket technology) to support real-time two-way data transmission, analyze and judge whether the user terminal is accessing the live broadcast room for the first time. If so, start the resource chunk preloading mechanism; if not, execute the local cache check sub-module;

[0049] The resource chunk preloading mechanism is as follows: Load the core resources required for the live broadcast room in a chunked manner; the chunking method is specifically: divide the core resources into multiple task chunks {R1, R2,..., RN}, calculate the priority Pi of each task chunk, and then sort the resource queue Q = {R1, R2,..., RN} from high to low according to the priority sorting algorithm, and then use parallel loading technology to load the task chunks simultaneously through multi-threading or asynchronous requests; the priority algorithm formula is:

[0050] P i = ω1·S i + ω2·D i + ω3·U i

[0051] where S iIndicates the importance of the core resources, D i Indicates the degree of dependence of the core resources, U i Indicates the adaptability between the performance of the client device and the network conditions, where ω1, ω2, and ω3 are weight coefficients respectively.

[0052] The resource task allocation formula for loading task blocks of resources simultaneously by multi-threading or asynchronous requests is:

[0053]

[0054] Among them, N is the total number of resources of the core resources, and T is the number of available threads.

[0055] Local cache check sub-module: Used to enable the client to check whether the local cache has stored relevant core resources; if the core resources exist and the versions are consistent, enable the client to directly read and load from the local cache to avoid repeated downloads; if the core resources do not exist or the versions are inconsistent, start the resource chunk preloading mechanism, send the latest core resources to the client, and then enable the client to read and load.

[0056] Resource version verification module: Used to enable the client to verify the integrity and consistency of the core resources through a hash verification mechanism to ensure that the data during the transmission process is not damaged; if the verification fails, request the server to send the latest core resources and overwrite them.

[0057] Dynamic resource update module: Used to load new core resources (such as special scenes, temporary special effects, etc.) to the client through a lightweight dynamic update mechanism during the live broadcast process after the live broadcast room is initialized; the dynamic update mechanism is: obtain the dynamic request sent by the client to the server according to the live broadcast scene requirements, and the dynamic request is the new core resources that need to be loaded; instead of loading all possible core resources at one time. This method effectively avoids network bandwidth congestion and excessive consumption of the client's computing power, ensuring the smoothness and real-time nature of the live broadcast.

[0058] Resource loading optimization module: Used to dynamically adjust the quality of the core resources according to the network bandwidth and computing power of the client device. If the bandwidth is low or the device performance is weak, preferentially load the low-quality version of the core resources, and then load the higher-quality core resources after the conditions improve; at the same time, after the core resources are loaded, enable the client to immediately perform progressive rendering, preferentially display the basic content, and then gradually render the picture according to the priority of the core resources and the current hardware state; and continue to load other low-priority core resources in the background of the client.

[0059] Loading Completion Notification Module: It is used to send a loading completion notification to the client after the core resources are loaded. And start the timestamp synchronization with the host side. All subsequent control instructions and dynamic information will be synchronized based on this timestamp to ensure the consistency of scenes, actions, voices, etc. between the host side and the client side, and avoid visual differences caused by delays or out-of-sync.

[0060] After the user first enters the live broadcast room for connection initialization, the identity authentication module is executed.

[0061] Identity Authentication Module: It is used to obtain the identity authentication request sent by the client side and confirm the user information and permissions;

[0062] Timestamp Synchronization Module: It is used to record the exact time at this moment after the identity authentication is passed and the core resources are loaded, generate a basic timestamp, and use it as the starting reference time for subsequent time synchronization. By adopting the method of timestamp verification and synchronization, it ensures the consistency of scenes, actions, voices, etc. between the host side and the client side, and avoids visual differences caused by delays or out-of-sync. All subsequent control instructions and dynamic information are synchronized based on this timestamp. During the synchronization process, a time synchronization request is sent to the host side to obtain the current timestamp returned by the host side, so that the client side and the host side align the reference time and start real-time data transmission; during the transmission process, a persistent connection will be maintained, and the control instructions of the host side (such as host actions, voices, scene interactions, etc.) and the interaction information of the user (such as sending bullet screens, voting, etc.) will be processed into small data packets through compression and optimization algorithms and efficiently transmitted between the user side and the server side through WebSocket technology;

[0063] Host Side: Generate a timestamp based on its local clock and attach it to each control instruction; the action instructions of the control instructions include action type, action parameters, and timestamp, and the action parameters include position, speed, and direction.

[0064] Packet Loss Analysis and Processing Module: It is used to transmit data using redundant data encoding (such as FEC, error correction code). When part of the data is lost, the lost information is recovered through redundant data; it is also used to enable the client to send an acknowledgment message ACK to the server regularly to ensure that the data has been received; if no acknowledgment signal is received within the timeout, a retransmission request is triggered, and data retransmission is performed through the TCP retransmission mechanism; and in the case of relatively serious packet loss, the client is enabled to smooth out data loss through a packet loss compensation strategy; the packet loss compensation strategy is: before the lost data packet is retransmitted, insert a delayed or blank rendering frame to avoid video freeze or desynchronization, and adjust the time delay between the client and the host in real time according to the network conditions, receive subsequent data packets in advance, and reduce the user's perception during packet loss; this solution designs this module to address common problems during the connection process, such as network packet loss and disconnection, to ensure the stability and smoothness of the live broadcast.

[0065] Automatic Reconnection Module: It is used to issue a prompt of connection interruption or recovery to the client when detecting a connection disconnection, ensuring that the user knows that the live content is paused and will resume as soon as possible, and enabling the client to automatically attempt to reconnect to the server; after the connection is restored, the client is enabled to restore the timestamp before the disconnection, and accelerate to catch up with the latest timestamp and status to ensure a smooth transition of the live broadcast; during the disconnection process, the exponential backoff algorithm is adopted to control the reconnection frequency to avoid network congestion caused by overly frequent retries; if the disconnection time exceeds the time threshold, all necessary statuses are synchronized after reconnection, and the statuses include, for example, timestamps, actions, and voices; and the scene rendering is restored based on the timestamp at the time of reconnection, enabling the content of the client's live broadcast to be consistent with the host at the time of recovery.

[0066] The host generates timestamps based on its local clock and attaches them to each control instruction; the action instructions of the control instructions include action types, action parameters, and timestamps, and the action parameters include position, speed, and direction.

[0067] After the client receives the control instruction with a timestamp, it extracts the content and timestamp corresponding to each control instruction to implement the parsing operation of the control instruction, and the content includes actions, voices, and special effects; the timestamp of the control instruction is compared with the local clock for timestamp offset correction, and the delay between the timestamp and the current time of the local clock is calculated for adjusting the rendering timing.

[0068] If the delay exceeds the set duration, wait for the set time and then render the control instruction.

[0069] If the delay does not exceed the set duration, render immediately; and reduce the error through the timestamp correction algorithm.

[0070] The timestamp correction algorithm is as follows: Use the average value of the timestamp differences for smooth adjustment; and calculate the latency in real time during the process: Assume that the current timestamp of the client is current_timestamp, and the control instruction timestamp received from the host is control_timestamp, then the latency can be calculated by the following formula:

[0071] Delay = control_timestamp - current_timestamp

[0072] If delay is negative, it means the control instruction has expired and needs to be requested again; if delay is positive, it means the control instruction is still valid and is rendered on time. During the timestamp synchronization process, the client uses an adaptive algorithm to correct the drift and adopts the NTP time synchronization protocol to ensure the stability, unity, and accuracy of the timestamp synchronization.

[0073] Information flow transmission module: After the client opens the live broadcast room and establishes a connection with it, it obtains the control instructions of the virtual character transmitted by the host and forwards the control instructions to the client in real time, enabling the client to render the animation, audio, and special effects of the virtual character according to the control instructions; the control instructions include action instructions and voice instructions.

[0074] After the client determines the rendering timing based on the latency, it then renders the animation, audio, and special effects, and finally displays the rendering result; the animation is the skeletal animation of the virtual character. When rendering the skeletal animation of the virtual character, the client calls the resources in the resource library in real time according to the received control instructions, calculates and updates the skeletal position of the virtual character, and completes the animation rendering; the rendering of the audio is for the audio of the virtual character, and the rendering of the special effects is for the real-time special effects rendering of the barrage and gifts sent by other clients.

[0075] A non-traditional video live broadcast method based on Web technology, as Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 shown, includes the following steps:

[0076] S1. When the client opens the live broadcast room corresponding to a certain host, preload the core resources of the live broadcast room to the client through Web technology; the core resources include virtual character models, scene backgrounds, and special effects;

[0077] As Figure 2 shown, includes the following sub-modules:

[0078] S101. First access analysis sub-step: When the user terminal accesses the live broadcast room, perform connection initialization, establish a stable connection with the server through Web technology (WebSocket technology) to support real-time two-way data transmission, analyze and determine whether the user terminal is accessing the live broadcast room for the first time. If so, start the resource chunk preloading mechanism; if not, execute the local cache check sub-step;

[0079] The resource chunk preloading mechanism is as follows: Load the core resources required for the live broadcast room in a chunked manner; the chunking method is specifically: divide the core resources into multiple task chunks {R1, R2,..., RN}, calculate the priority Pi of each task chunk, and then sort the resource queue Q = {R1, R2,..., RN} from high to low according to the priority sorting algorithm. Then, adopt parallel loading technology to load the task chunks simultaneously through multi-threading or asynchronous requests; the priority algorithm formula is:

[0080] P i = ω1·S i + ω2·D i + ω3·U i

[0081] where S i represents the importance of the core resources, D i represents the dependency degree of the core resources, U i represents the adaptability of the user terminal device performance and network conditions, and ω1, ω2, and ω3 are weight coefficients respectively.

[0082] The resource task allocation formula for loading task chunks simultaneously through multi-threading or asynchronous requests is:

[0083]

[0084] where N is the total number of core resources, and T is the number of available threads.

[0085] S102. Local cache check sub-step: Enable the user terminal to check whether the local cache has stored the relevant core resources; if the core resources exist and the versions are the same, enable the user terminal to directly read and load from the local cache to avoid repeated downloads; if the core resources do not exist or the versions are inconsistent, start the resource chunk preloading mechanism, send the latest core resources to the user terminal, and then enable the user terminal to read and load.

[0086] S103. Resource version verification step: Enable the user terminal to verify the integrity and consistency of the core resources through a hash verification mechanism to ensure that the data during the transmission process is not damaged; if the verification fails, request the server to send the latest core resources and overwrite them.

[0087] S104. Dynamic resource update step: During the live broadcast after the initialization of the live broadcast room is completed, load new core resources (such as special scenes, temporary special effects, etc.) for the user side through a lightweight dynamic update mechanism; the dynamic update mechanism is: obtain the dynamic request sent by the user side to the server according to the live broadcast scene requirements, and the dynamic request is the new core resource that needs to be loaded; instead of loading all possible core resources at one time. This method effectively avoids network bandwidth congestion and excessive consumption of the client's computing power, ensuring the smoothness and real-time nature of the live broadcast.

[0088] S105. Resource loading optimization step: Dynamically adjust the quality of the core resources according to the network bandwidth and computing power of the user side device. If the bandwidth is low or the device performance is weak, preferentially load the low-quality version of the core resources, and then load the higher-quality core resources after the conditions improve; at the same time, after the core resources are loaded, make the client immediately perform progressive rendering, first display the basic content, and then gradually render the picture according to the priority of the core resources and the current hardware status; and continue to load other low-priority core resources in the client background.

[0089] S106. Loading completion notification step: After the core resources are loaded, send a loading completion notification to the user side. And start the timestamp synchronization with the anchor side. All subsequent control instructions and dynamic information will be synchronized based on this timestamp to ensure the consistency of the scene, actions, voices, etc. between the anchor side and the user side, and avoid visual differences caused by delays or out-of-sync.

[0090] S2. Information flow transmission step: After the user side opens the live broadcast room and establishes a connection with it, obtain the control instructions of the virtual character transmitted by the anchor side, and forward the control instructions to the user side in real time, so that the user side renders the animation, audio, and special effects of the virtual character according to the control instructions; the control instructions include action instructions and voice instructions. The S201 includes the following steps:

[0091] S201. Enable the user side to establish a connection with the live broadcast room and transmit information;

[0092] S202. The user side renders the animation, audio, and special effects in real time according to the control instructions.

[0093] As Figure 3 shown, the step S201 specifically includes the following steps:

[0094] S2011. Connection request step: The user side sends a connection request to the server, and the server makes a connection response.

[0095] S2012. When the user performs connection initialization, execute the identity authentication step.

[0096] Identity authentication steps: The server obtains the identity authentication request sent by the client and verifies the user information and permissions;

[0097] S2013. Timestamp synchronization step: After the identity authentication is passed and the core resources are loaded, record the exact time at this moment to generate a basic timestamp as the starting reference time for subsequent time synchronization. Adopt the method of timestamp verification and synchronization to ensure the consistency of scenarios, actions, voices, etc. between the host side and the client side, and avoid visual differences caused by delays or desynchronization. All subsequent control instructions and dynamic information are synchronized based on this timestamp. During the synchronization process, send a time synchronization request to the host side to obtain the current timestamp returned by the host side, align the reference time of the client side and the host side, and start real-time data transmission; during the transmission process, maintain a persistent connection, and process the control instructions of the host side (such as host actions, voices, scene interactions, etc.) and the interaction information of users (such as sending bullet screens, voting, etc.) into small data packets through compression and optimization algorithms, and efficiently transmit them between the user side and the server through WebSocket technology;

[0098] Host side: Generate a timestamp based on its local clock and attach it to each control instruction; the action instructions of the control instruction include action type, action parameters, and timestamp, and the action parameters include position, speed, and direction.

[0099] S2014. The server obtains the control instructions of the virtual character transmitted by the host side and forwards the control instructions to the client side in real time to achieve information interaction.

[0100] S2015. Transmission packet loss analysis and processing step: Use redundant data encoding (such as FEC, error correction code) to transmit data. When part of the data is lost, recover the lost information through redundant data; it is also used to make the client send an acknowledgment message ACK to the server regularly to ensure that the data has been received; if the acknowledgment signal is not received within the timeout, trigger a retransmission request and perform data retransmission through the TCP retransmission mechanism; and in the case of relatively serious packet loss, make the client smooth out the data loss through the packet loss compensation strategy; the packet loss compensation strategy is: insert a delayed or blank rendering frame before the lost data packet is retransmitted to avoid frame freezes or desynchronization, and adjust the time delay between the user side and the host side in real time according to the network conditions, receive subsequent data packets in advance, and reduce the user perception during packet loss; this solution designs this module to address common problems during the connection process, such as network packet loss, disconnection, etc., to ensure the stability and smoothness of the live broadcast.

[0101] S2016, Automatic Reconnection Steps: When a connection disconnection is detected, a prompt of connection interruption or recovery will be sent to the client side to ensure that the user knows that the live content is paused and will resume as soon as possible, and the client side will automatically attempt to reconnect to the server; after the connection is restored, the client side will resume the timestamp before the disconnection, and catch up with the latest timestamp and status through acceleration to ensure a smooth transition of the live broadcast; during the disconnection process, the exponential backoff algorithm is used to control the reconnection frequency to avoid network congestion caused by overly frequent retries; if the disconnection time exceeds the time threshold, all necessary statuses will be synchronized after reconnection, and the statuses include timestamps, actions, and voices; and based on the timestamp at the time of reconnection, the scene rendering will be restored to ensure that the content of the client side live broadcast is consistent with the host side when it is restored.

[0102] The host side generates a timestamp based on its local clock and attaches it to each control instruction; the action instruction of the control instruction includes an action type, action parameters, and a timestamp, and the action parameters include position, speed, and direction.

[0103] Timestamp Analysis and Delay Calculation Steps: After the client side receives a control instruction with a timestamp, extract the content and timestamp corresponding to each control instruction to implement the parsing operation of the control instruction, and the content includes actions, voices, and special effects; compare the timestamp of the control instruction with the local clock to perform timestamp offset correction, and calculate the delay between the timestamp and the current time of the local clock for adjusting the rendering timing; as Figure 4 shown.

[0104] Rendering Time Analysis Steps: Analyze the rendering time according to the delay;

[0105] If the delay exceeds the set duration, wait for the set time and then render the control instruction;

[0106] If the delay does not exceed the set duration, render immediately; and reduce the error through the timestamp correction algorithm;

[0107] The timestamp correction algorithm is as follows: Use the average value of the timestamp difference for smooth adjustment; and calculate the delay in real time during the process: Assume that the current timestamp of the client side is current_timestamp, and the control instruction timestamp received from the host side is control_timestamp, then the delay can be calculated by the following formula:

[0108] Delay = control_timestamp - current_timestamp

[0109] If the delay is negative, it indicates that the control instruction has expired and a new request is needed; if the delay is positive, it indicates that the control instruction is still valid and rendering should be performed on time. During the timestamp synchronization process, the client uses an adaptive algorithm to correct the drift and adopts the NTP time synchronization protocol to ensure the stability, unity, and accuracy of the timestamp synchronization.

[0110] The specific process of the client rendering the animation, audio, and special effects in real time according to the control instruction in step S202 is as follows:

[0111] After the client determines the rendering timing based on the delay, it then renders the animation, audio, and special effects, and finally displays the rendering result; the animation is the skeletal animation of the virtual character. When rendering the skeletal animation of the virtual character, the client, according to the received control instruction, calls the resources in the resource library in real time, calculates and updates the skeletal position of the virtual character, and completes the animation rendering; the rendering of the audio is for the audio of the virtual character, and the rendering of the special effects is for the real-time special effects of the barrage and gifts sent by other clients.

[0112] As Figure 5 shown, the overall system architecture of the new live broadcast method based on Web technology is as follows:

[0113] Host side: Responsible for the production of the core content of the virtual live room and the generation of instructions, mainly including the following modules:

[0114] Virtual character control module: Used to collect the action, expression, and voice data of the host and convert this data into control instructions.

[0115] Resource management module: Organizes and optimizes the virtual character models, scene resources, and dynamic special effects, and is responsible for uploading the resources to the server.

[0116] Data transmission module: Packages the control instructions and necessary resource information and sends them to the client in real time through the network.

[0117] Server side: Mainly used for data transfer, processing, and synchronization to ensure real-time performance and consistency in a multi-user environment. The server side also includes a communication module and a security module;

[0118] The communication module runs through the host side, server side, and client side, and is responsible for high-efficiency and low-latency data transmission. The specific functions include:

[0119] Instruction transmission: Transmits action, voice, and interaction data through a lightweight protocol.

[0120] Scene resource transmission: Adopts chunking and compression technologies to transmit high-quality scene resources and optimize the use of network bandwidth.

[0121] Status feedback: Monitor the network status in real time, and provide packet loss compensation and latency adjustment mechanisms to ensure the stability of the live streaming experience.

[0122] The security module is used to protect user privacy and content copyright, specifically including:

[0123] Data encryption module: Encrypt the data transmitted in real time to prevent interception or tampering in the middle.

[0124] Permission control module: Verify the access to the resources and functions in the live broadcast room to ensure data security.

[0125] User terminal: Used to receive the data transmitted by the server and render the content of the virtual live broadcast room in real time, mainly including the following modules:

[0126] Resource loading module: Pre-load the core resources and dynamically update the local cache to ensure smooth switching of scenes during the live broadcast.

[0127] Rendering engine module: Dynamically render virtual characters, scene special effects and interactive elements according to control instructions, supporting high frame rate and high-quality rendering.

[0128] Interaction module: Allow users to interact with virtual anchors in real time, such as sending bullet screens, giving gifts or participating in interactive tasks in virtual scenes.

[0129] The above are only embodiments of the present invention. Specific structures and common knowledge such as characteristics well known in the art are not described in detail here. Those of ordinary skill in the art know all the common general technical knowledge in the technical field to which the invention belongs before the application date or priority date, can know all the existing technologies in this field, and have the ability to apply the conventional experimental means before this date. Those of ordinary skill in the art can, under the inspiration given in this application, combine their own abilities to improve and implement this solution. Some typical well-known structures or well-known methods should not become obstacles for those of ordinary skill in the art to implement this application. It should be noted that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can still be made, and these should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent. The protection scope required by this application should be based on the content of its claims, and the specific implementation manners and the like recorded in the specification can be used to interpret the content of the claims.

Claims

1. A non - traditional video live - streaming system based on Web technology, characterized in that, It includes a server, a user terminal, and a host terminal. The server includes the following modules: Resource preloading module: When the user terminal opens the live room corresponding to a certain host terminal, it preloads the core resources of the live room to the user terminal through web technology; the core resources include virtual character models, scene backgrounds, and special effects; Information flow transmission module: After the user terminal opens the live room and establishes a connection with it, it obtains the control instructions of the virtual character transmitted by the host terminal and forwards the control instructions to the user terminal in real time, enabling the user terminal to render the animation, audio, and special effects of the virtual character according to the control instructions; the control instructions include action instructions and voice instructions.

2. The non - traditional video live - streaming system based on Web technology according to claim 1, characterized in that: The resource preloading module includes the following sub-modules: First access analysis sub-module: When the user terminal accesses the live room, it initializes the connection, establishes a stable connection with the server through web technology to support real-time two-way data transmission, analyzes and judges whether the user terminal is accessing the live room for the first time. If so, it starts the resource chunk preloading mechanism; if not, it executes the local cache check sub-module; Local cache check sub-module: Enables the user terminal to check whether the local cache stores the relevant core resources; If the core resources exist and the versions are the same, the user terminal directly reads and loads from the local cache; if the core resources do not exist or the versions are inconsistent, it starts the resource chunk preloading mechanism, sends the latest core resources to the user terminal, and then the user terminal reads and loads them; Resource version verification module: Enables the user terminal to verify the integrity and consistency of the core resources through a hash verification mechanism to ensure that the data during transmission is not damaged; If the verification fails, it requests the server to send the latest core resources and overwrite them.

3. An unconventional video live streaming system based on Web technology according to claim 2, characterized in that: The resource chunk preloading mechanism is: loading the core resources required by the live room in a chunked manner; the chunking method is specifically: dividing the core resources into multiple task chunks {R1, R2,..., RN}, calculating the priority Pi of each task chunk, and then sorting the resource queue Q = {R1, R2,..., RN} from high to low according to the priority sorting algorithm. Then, using parallel loading technology, it simultaneously loads the task chunks through multi-threading or asynchronous requests; the priority algorithm formula is: P i = ω1·S i + ω2·D i + ω3·U i Among them, S i represents the importance of the core resource, D i represents the degree of dependence of the core resource, U i represents the adaptability of the user device performance and network conditions, and ω1, ω2, and ω3 are weight coefficients respectively.

4. An unconventional video live streaming system based on Web technology according to claim 3, characterized in that: The server also includes: Dynamic resource update module: During the live broadcast, it loads new core resources to the user terminal through a lightweight dynamic update mechanism; the dynamic update mechanism is: obtaining the dynamic request sent by the user terminal according to the live broadcast scene requirements, and the dynamic request is the new core resources that need to be loaded; Resource loading optimization module: Dynamically adjusts the quality of the core resources according to the network bandwidth and computing power of the user terminal device. If the bandwidth is low or the device performance is weak, it preferentially loads the low-quality version of the core resources, and then loads the higher-quality core resources after the conditions improve; at the same time, after the core resources are loaded, the client immediately performs progressive rendering, preferentially displays the basic content, and then gradually renders the picture according to the priority of the core resources and the current hardware state; and continues to load other low-priority core resources in the client background.

5. The non - traditional video live - streaming system based on Web technology according to claim 4, wherein: After the user first enters the live broadcast room for connection initialization, the server further includes the following modules: Identity authentication module: used to obtain the identity authentication request sent by the client and confirm the user information and permissions; Timestamp synchronization module: used to record the exact time at this moment and generate a basic timestamp as the starting reference time for subsequent time synchronization after the identity authentication is passed and the core resources are loaded; all subsequent control instructions and dynamic information are synchronized based on this timestamp. During the synchronization process, a time synchronization request is sent to the host side to obtain the current timestamp returned by the host side, so that the client side and the host side align the reference time and start real-time data transmission; During the transmission process, a persistent connection is maintained, and the control instructions of the host side and the interactive information of the user are processed into small data packets through compression and optimization algorithms and transmitted between the user side and the server through WebSocket; Host side: generates a timestamp based on its local clock and attaches it to each control instruction; the action instructions of the control instruction include action type, action parameters, and timestamp, and the action parameters include position, speed, and direction.

6. The non-traditional video live broadcast system based on Web technology according to claim 5, characterized in that: It further includes the following modules: Transmission packet loss analysis and processing module: used to transmit data using redundant data encoding. When part of the data is lost, the lost information is recovered through redundant data; it is also used to make the client regularly send an acknowledgment message ACK to the server to ensure that the data has been received; if the acknowledgment signal is not received within the timeout, a retransmission request is triggered and the data is retransmitted through the TCP retransmission mechanism; and in the case of relatively serious packet loss, the client is made to smooth out the data loss through a packet loss compensation strategy; the packet loss compensation strategy is: before the lost data packet is retransmitted, insert a delayed or blank rendering frame, and adjust the time delay between the client and the host side in real time according to the network conditions to receive subsequent data packets in advance; Automatic reconnection module: used to send a prompt of connection interruption or recovery to the client when it detects that the connection is disconnected, ensuring that the user knows that the live content is paused and will resume as soon as possible, and making the client automatically attempt to reconnect to the server; after the connection is restored, the client resumes the timestamp before the disconnection and catches up with the latest timestamp and status through acceleration to ensure a smooth transition of the live broadcast; An exponential backoff algorithm is used to control the reconnection frequency during the disconnection process; If the disconnection time exceeds the time threshold, all necessary states are synchronized after reconnection, and the states include, for example, timestamp, action, and voice; and the scene rendering is restored based on the timestamp at the time of reconnection, so that the content of the client's live broadcast is consistent with the host side when it is restored.

7. An unconventional video live streaming system based on Web technology according to claim 6, characterized in that: After the client receives the control instruction with a timestamp, it extracts the content and timestamp corresponding to each control instruction, and the content includes action, voice, and special effects; the timestamp of the control instruction is compared with the local clock for timestamp offset correction, and the delay between the timestamp and the current time of the local clock is calculated for adjusting the rendering timing; If the delay exceeds the set duration, wait for the set time before rendering the control instruction; If the delay does not exceed the set duration, render immediately; and reduce the error through a timestamp correction algorithm; The timestamp correction algorithm is as follows: use the average value of the timestamp difference for smooth adjustment; and calculate the delay in real time during the process: assume that the current timestamp of the client is current_timestamp, and the control instruction timestamp received from the host is control_timestamp, then the delay can be calculated by the following formula: Delay = control_timestamp - current_timestamp If delay is negative, it means the control instruction has expired and needs to be requested again; if delay is positive, it means the control instruction is still valid and is rendered on time.

8. An unconventional video live streaming system based on Web technology according to claim 7, characterized in that: During the process of timestamp synchronization, the client uses an adaptive algorithm to correct the drift and adopts the NTP time synchronization protocol to ensure the stability, unity, and accuracy of timestamp synchronization.

9. The non - traditional video live - streaming system based on Web technology according to claim 7, characterized in that: After the client determines the rendering timing based on the delay, it then performs the rendering of animations, audio, and special effects; the animation is the skeletal animation of the virtual character. When rendering the skeletal animation of the virtual character, the client calls the resources in the resource library in real time according to the received control instruction, calculates and updates the skeletal position of the virtual character, and completes the animation rendering; the rendering of the audio is for the audio of the virtual character, and the rendering of the special effects is for the real-time special effects rendering of the barrage and gifts sent by other clients.

10. A non - traditional video live - streaming method based on Web technology, characterized in that: The non-traditional video live broadcast system based on Web technology as described in any one of claims 1-9 above is applied.

Images