Message processing method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a message processing method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: obtaining a first interaction message, the first interaction message comprises a message issued by a server in response to an interaction request of a client, and the first interaction message comprises a response timestamp, the response timestamp indicates the time when the server receives the interaction request; according to the response timestamp and delay time, target execution time of an interaction task corresponding to the first interaction message is determined, and the delay time comprises multimedia data stream delay time corresponding to the interaction task; and in response to the target execution time and the local timestamp meeting a preset condition, executing the interaction task. And the target execution time of the interactive task is combined with the delay time of the multimedia data stream, so that the execution of the interactive task and the time synchronization of the multimedia data stream are realized.

Description

Technical Field

[0001] This disclosure relates to data processing technology, and more particularly to a message processing method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] Live streaming is a method of transmitting video and audio content in real time via the internet. Hosts can interact with guests via video link, providing live content to viewers.

[0003] In live streaming, interactive content is usually sent to the members in the live stream in the form of messages. After receiving the messages, the members render the images to achieve the effect of multi-person live streaming interaction.

[0004] However, there is a desynchronization issue between the execution of interactive messages and the multimedia stream. Summary of the Invention

[0005] This disclosure provides a message processing method, apparatus, electronic device, storage medium, and program product to realize the execution of interactive messages and the synchronization of multimedia streams.

[0006] In a first aspect, embodiments of this disclosure provide a message processing method, including:

[0007] Obtain a first interactive message, the first interactive message including a message sent by the server in response to the client's interactive request, the first interactive message including a response timestamp, the response timestamp indicating the time when the server received the interactive request;

[0008] Based on the response timestamp and the delay time, the target execution time of the interactive task corresponding to the first interactive message is determined, wherein the delay time includes the delay time of the multimedia data stream corresponding to the interactive task;

[0009] The interactive task is executed in response to the target execution time and local timestamp meeting preset conditions.

[0010] Secondly, embodiments of this disclosure also provide a message processing apparatus, including:

[0011] The acquisition module is used to acquire a first interactive message, which includes a message sent by the server in response to the client's interactive request. The first interactive message includes a response timestamp, which indicates the time when the server received the interactive request.

[0012] The determining module is used to determine the target execution time of the interactive task corresponding to the first interactive message based on the response timestamp and the delay time, wherein the delay time includes the delay time of the multimedia data stream corresponding to the interactive task;

[0013] The execution module is used to execute the interactive task in response to the target execution time and local timestamp meeting preset conditions.

[0014] Thirdly, embodiments of this disclosure also provide an electronic device, characterized in that the electronic device comprises:

[0015] One or more processing devices;

[0016] Storage device for storing one or more programs.

[0017] When the one or more programs are executed by the one or more processing devices, the one or more processing devices implement a message processing method as provided in the disclosed embodiments.

[0018] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a message processing method as provided in embodiments of this disclosure.

[0019] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the message processing method provided in embodiments of this disclosure.

[0020] In this embodiment, after obtaining the first interactive message, the target execution time of the interactive task corresponding to the first interactive message is determined based on the response timestamp and multimedia stream delay time included in the first interactive message. Then, based on whether the comparison result between the target execution time and the local timestamp meets a preset condition, it is determined whether to execute the interactive task. During the execution of the interactive task, the time for the server to obtain the interactive request and the multimedia data stream delay time are considered, so that the target execution time of the interactive task is combined with the multimedia data stream delay time, achieving time synchronization between the execution of the interactive task and the multimedia data stream. Attached Figure Description

[0021] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0022] Figure 1 This is a flowchart illustrating a message processing method provided in an embodiment of this disclosure;

[0023] Figure 2 This is a flowchart illustrating another message processing method provided in this embodiment of the present disclosure;

[0024] Figure 3 This is a flowchart illustrating another message processing method provided in an embodiment of this disclosure;

[0025] Figure 4 This is a schematic diagram of the structure of a message processing device provided in an embodiment of this disclosure;

[0026] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0027] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0028] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0029] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0030] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0031] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0032] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0033] In current live streaming, many functions and logics require multi-person interaction to trigger, necessitating the synchronization of information to all viewers to achieve the final effect. In multi-person live streaming, interaction is typically achieved through message delivery or push notifications of streaming fields (i.e., the `sei` field that follows the data stream), with interactive messages written in the message body. Upon receiving the message, certain logic is used to complete the final interaction or render the visuals, creating a smooth multi-person live streaming interaction effect. However, currently, interactive messages and multimedia data streams, such as audio and video streams, suffer from asynchrony, resulting in a poor user experience in multi-person interactive scenarios and issues like audio-visual desynchronization. For example, when a live stream host sends flowers to guests, the host's client triggers an API call to send messages to the guests' and viewers' clients, ensuring they receive the flower effect. However, due to end-to-end latency, the time the receiver receives the message and the time it receives the multimedia data stream are not synchronized, thus failing to guarantee that the message and multimedia data stream are consumed simultaneously. For example, if a broadcaster describes "I'm sending you a flower" at 8:08 AM and performs the action of sending it, the broadcaster's client calls the server interface to send a message to other clients. However, due to delays, the received multimedia data stream is the one from 8:06 AM, meaning the flower image hasn't been rendered yet. Specifically, the broadcaster describes "I'm sending you a flower" at 8:08 AM and performs the action of sending it. The broadcaster's client calls the server interface to send a message to other clients. However, due to delays, the other clients receive the audio / video stream from 8:06 AM, which lacks the audio and video content. Since the message is rendered at 8:06 AM, the message and audio / video stream are out of sync. Audio / video streams refer to the continuous transmission and processing of audio and video data.

[0034] To address the aforementioned technical problems, this disclosure provides a message processing method. Figure 1This is a flowchart illustrating a message processing method provided in an embodiment of this disclosure. This embodiment is applicable to situations where interactive messages are processed, such as on a client side, to ensure the execution of interactive messages and the synchronization of multimedia data streams. This method can be executed by a message processing device, which can be implemented in software and / or hardware, optionally through an electronic device such as a mobile terminal or PC. Interactive messages can be considered as messages that enable interaction during live streaming, such as messages from a server indicating interaction between objects connected by eye contact. Interactive messages include a first interactive message and a second interactive message. The first interactive message can be a real-time interactive message. The second interactive message can be a historical interactive message. In this embodiment, the client can be an endpoint that receives the first interactive message and renders the interactive effect corresponding to the first interactive message.

[0035] like Figure 1 As shown, the method includes:

[0036] S110, Obtain the first interactive message.

[0037] The first interactive message includes a message sent by the server in response to the client's interactive request. The first interactive message includes a response timestamp, which indicates the time when the server received the interactive request.

[0038] In this context, the client, also known as the live interaction object, can be considered as the object participating in the live connection process, such as the host's client and the guest's client. The first interactive message can be a message sent by the server to achieve the interactive effect during the live interaction by calling an interface. An interaction request can trigger the server to send the first interactive message to realize the interaction request. The server determines the time of obtaining the client's interaction request as the response timestamp. The response timestamp can represent the time when the interactive message was triggered. In the subsequent synchronization process, the response timestamp is synchronized with the audio and video streams at the same time.

[0039] In this embodiment, the terminal side corresponding to the electronic device can be a client other than the client that triggered the first interactive message. This client can obtain the first interactive message from the server to render the interactive effect. The first interactive message additionally carries a response timestamp to facilitate synchronization between the first interactive message and the corresponding multimedia data. Multimedia data can refer to a data set containing multiple different types of media formats, which are integrated to convey information. Multimedia data can include audio, images, and / or video.

[0040] S120. Determine the target execution time of the interactive task corresponding to the first interactive message based on the response timestamp and the delay time.

[0041] The delay time includes the delay time of the multimedia data stream. A multimedia data stream can refer to a continuous, dynamic data sequence composed of multimedia data. An interactive task can be considered a task executed by the client to present the interactive effect corresponding to the interactive message. For example, an interactive task might be to receive flowers, where receiving flowers can be considered the presented interactive effect. The target execution time can be considered the time when the interactive task should be executed on the client. In this embodiment, the interactive task corresponding to the first interactive message is not executed immediately after receiving the first interactive message, nor is it executed at the time corresponding to the response timestamp, but rather after the target execution time arrives. For example, if the response timestamp is 8:08, the client's local time at 8:08 does not necessarily mean the interactive task will be executed; the delay time must also be considered to determine the target execution time of the interactive task.

[0042] In this embodiment, the delay time can be considered as the time delay between the multimedia data stream acquired by the client and the local time. For example, due to end-to-end delay, if the multimedia data stream acquired by the client at 8:08 is the multimedia data of 8:06, the interactive task corresponding to 8:08 cannot be executed directly at this time. The interactive task must be executed after the multimedia data of 8:08 is acquired, so as to represent the interactive effect of the interactive task of 8:08 being rendered on the multimedia data of 8:08.

[0043] This operation can perform mathematical calculations based on the response timestamp and delay time to determine the target execution time of the interactive task corresponding to the first interactive message. The calculation method is not limited here; for example, the target execution time can be determined by summing the response timestamp and delay time. Alternatively, the target execution time can be determined by a value greater than or less than the sum of the response timestamp and delay time, and the difference between the target execution time and the sum can be less than a set threshold.

[0044] S130. In response to the target execution time and local timestamp meeting preset conditions, the interactive task is executed.

[0045] The local timestamp can be considered as the timestamp representing the current time on the client's local machine. The comparison result can be considered as the result obtained by comparing the target execution time with the local timestamp. The preset conditions can be considered as pre-set conditions for determining whether to execute the interactive task. The preset conditions can be used to constrain the comparison results. In response to the comparison result meeting the preset conditions, the interactive task can be executed. The preset conditions are not limited here; for example, they can represent that the target execution time is not less than the local timestamp.

[0046] In one embodiment, in response to the comparison result indicating that the local timestamp is greater than or equal to the target execution time, it can be considered that the preset conditions are met. In this embodiment, an interactive task can be triggered to render the human-computer interaction interface on the client and realize the presentation of interactive effects.

[0047] The technical solution of this embodiment, after obtaining the first interactive message, determines the target execution time of the interactive task corresponding to the first interactive message based on the response timestamp and multimedia stream delay time included in the first interactive message. Then, based on whether the comparison result of the target execution time and the local timestamp meets a preset condition, it is determined whether to execute the interactive task. During the execution of the interactive task, the time for the server to obtain the interactive request and the multimedia data stream delay time are considered, so that the target execution time of the interactive task is combined with the multimedia data stream delay time, achieving time synchronization between the execution of the interactive task and the multimedia data stream.

[0048] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0049] In one embodiment, the step of executing the interactive task in response to the target execution time and local timestamp satisfying a preset condition includes:

[0050] The target execution time and the local timestamp are compared to obtain the comparison result;

[0051] In response to the comparison result indicating that the target execution time is later than or equal to the local timestamp, the interactive task is executed based on the task type of the interactive task.

[0052] Task type can be considered as information representing the type of interactive task. Task types can be classified according to different criteria. Here, we do not limit the classification method of task types. For example, there can be tasks with additional preset conditions, as well as tasks that are executed on a timer, and so on.

[0053] In this embodiment, the preset condition can be that the target execution time is later than or equal to the local timestamp. After comparing the target execution time with the local timestamp, if the preset condition is met, the task type of the interactive task can be further considered to determine whether to execute the interactive task. In response to the task type indicating that the interactive task is a timed task, a countdown can be started, and the interactive task can be executed after the countdown expires. When the task type indicates that the interactive task is a task with additional preset conditions, the interactive task is executed after the additional preset conditions are met. Here, the additional preset conditions are not limited; they can be any conditions added after the target execution time has expired. For example, the additional preset conditions may limit the execution priority of multiple tasks, such as interactive task A needing to be executed only after interactive task B has been completed.

[0054] In one embodiment, executing the interactive task according to the task type of the interactive task includes:

[0055] In response to the task type indication that the interactive task is a delayed task, the decision to execute the interactive task is made through a delayed processing task timer.

[0056] Delayed execution tasks can be considered tasks that need to be executed later, such as an interactive task scheduled for 8:08. The task type indicates that the interactive task is a delayed execution task, meaning it will be delayed until 8:18. Therefore, the interactive task will execute at 8:18. The required delay time for a delayed task can be directly determined by the server, either by specifying the exact delay time (e.g., ten minutes) or by specifying the delayed time (e.g., 8:18).

[0057] In this embodiment, the delayed processing task timer can be considered a tool for counting down the time for delayed processing tasks. In response to the determination that the interaction needs to be delayed based on the task type, a countdown can be performed using the delayed processing task timer. Once the countdown is complete, the execution of the interaction task is determined, thus enabling the use of the delayed processing interaction task timer to determine whether to execute the interaction task.

[0058] Figure 2 This is a flowchart illustrating another message processing method provided in this disclosure embodiment. This embodiment also includes a method for processing messages when a second interactive message is not received, such as... Figure 2 As shown, this disclosure includes the following steps:

[0059] S210. In response to not obtaining the second interaction message, obtain the second interaction message.

[0060] A scenario where the second interactive message is not received can be considered an abnormal scenario, which can be defined as a scenario that affects the normal presentation of interactive effects during the live stream. In one embodiment, the abnormal scenario includes a scenario where the second interactive message is missing within a set time prior to the current time. Abnormal scenarios can include interrupted broadcasts, resuming broadcasts, or unexpectedly joining a live stream. An interrupted broadcast refers to the phenomenon where the live stream signal is suddenly interrupted due to various reasons, and the transmission of live video and audio stops. Resuming broadcasts refers to the process of resuming the transmission of the live stream signal and restoring live video and audio after the interruption problem is resolved. Unexpectedly joining a live stream refers to entering the live stream at a certain point in time after the live stream has started, without prior planning.

[0061] The second interactive message includes interactive messages received within a set time period prior to the current time. The current time can be considered the time at which the abnormal scenario is determined. The set time can be considered a pre-defined message used to limit a historical time period. The second interactive message can be a message received by the client within the set time period prior to the current time. The task type indication of the second interactive message indicates that the interactive task is a delayed task. If the second interactive message is lost, it will cause the interactive task corresponding to the second interactive message to fail to execute after its delayed execution time has elapsed.

[0062] This embodiment can obtain and process a second interactive message in abnormal scenarios, so that the interactive task corresponding to the second interactive message can be executed correctly.

[0063] S220. Determine the target execution time corresponding to the second interactive message, and execute the interactive task corresponding to the second interactive message based on the target execution time.

[0064] In this embodiment, after obtaining the second interactive message, the target execution time of the second interactive message is determined, and the interactive task corresponding to the second interactive message is executed according to the target execution time. For example, the target execution time of the interactive task corresponding to the second interactive message is determined based on the delay time corresponding to the second interactive message and the response timestamp in the second interactive message. In response to the time represented by the local timestamp reaching the target execution time corresponding to the second interactive message, the interactive task of the second interactive message is executed. For example, the second interactive message can be processed in the same way as the first interactive message. In this embodiment, the first and second interactive messages can be considered as interactive messages obtained at different times, and the processing method for the interactive messages can be the same. For example, the response timestamp within them is extracted and combined with the delay time to determine the target execution time of the task. In response to the target execution time being greater than or equal to the local timestamp, the interactive task is executed based on the task type, such as delayed execution.

[0065] In this embodiment, by acquiring the second interactive message, the execution of the interactive task corresponding to the second interactive message is restored, thus avoiding the problem of the interactive task corresponding to the second interactive message being missing due to the absence of the second interactive message, which would lead to a mismatch between the interactive message and the multimedia data stream.

[0066] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0067] In one embodiment, obtaining the first interactive message includes:

[0068] The first interactive message transmitted by the server is obtained through a dynamic container;

[0069] The interactive task corresponding to the first interactive message pair is created by the task processor of the dynamic container;

[0070] The interactive task is associated with a delay processor, which is used to control the execution of the interactive task.

[0071] A dynamic container can be considered as a container that dynamically updates the client with the ability to process interactive messages and executes the message processing. A deferred processor can be considered as a processor used by the dynamic container to delay the processing of interactive tasks. The deferred processor can be further controlled by a task timer.

[0072] The message processing method disclosed herein can be executed by a dynamic container on the client side, which obtains the first interactive message transmitted by the server. Then, the dynamic container creates an interactive task corresponding to the first interactive task through a task processor, adds the created interactive task to a queue, and associates it with a delay processor for controlling the execution of the interactive task.

[0073] In one example, when the task processor of the dynamic container receives the first interactive message, it creates a corresponding interactive task and associates the interactive task with the task timer. The interactive task can be added to the task list associated with the task timer. The task timer calls the client's method to get the delay time, such as the delay time retrieval method, to get the delay time.

[0074] The task timer can determine when the following preset conditions are met, such as notifying the client to render the human-computer interaction interface:

[0075] The sum of the response timestamp and the delay time is greater than the local timestamp. During the execution of interactive tasks, corresponding tasks can be executed according to the task type, such as countdowns.

[0076] The task timer will stop when all interactive tasks associated with it are removed from the task list. When an interactive task is added to an empty task list, the task timer will start, scheduling the interactive tasks in the list and determining whether their target execution time has arrived.

[0077] In one embodiment, determining the target execution time of the interactive task corresponding to the first interactive message based on the response timestamp and the delay time includes:

[0078] The delay time is obtained by calling the delay time acquisition method through the delay processor;

[0079] The sum of the response timestamp and the delay time is determined as the target execution time of the interactive task corresponding to the first interactive message.

[0080] Different methods are used to obtain latency in different scenarios. In real-time communication scenarios, such as when multiple parties are connected and can be considered to be in a Real-Time Communication (RTC) room, the latency method can be considered as obtaining the latency of the RTC room. In non-real-time communication scenarios during live streaming, the latency can be obtained from fields accompanying the multimedia data stream.

[0081] In this embodiment, the sum of the response timestamp and the delay time is determined as the target execution time of the interactive task corresponding to the first interactive message.

[0082] Figure 3 This is a flowchart illustrating another message processing method provided in this disclosure embodiment. See also... Figure 3 The host triggers the server to send the first interactive message. The server adds a response timestamp to the first interactive message. Then, the first interactive message is transmitted as a random message to all members of the live stream, namely guests, host, and viewers.

[0083] The client corresponding to the guest processes the task associated with the first interactive message, i.e., the interactive task, and then starts a scheduled task. The timing of starting the scheduled task is not limited; it can be started after the target execution time has arrived, provided the task type is determined to be a scheduled task (i.e., a delayed execution task). The start time of the scheduled task countdown is not limited; it can begin counting down from when it is confirmed to be a scheduled task, or it can start counting down from the response timestamp. The scheduled task can be processed synchronously after it is started. Rendering operations can be performed on the client after the scheduled time has elapsed.

[0084] After the broadcaster's client receives the first interactive message, it can determine whether to process it. It retrieves the delay time from the RTC room, and the dynamic container creates a task queue. Once the target execution time for the interactive task arrives, the dynamic container executes the task based on its type; for example, it might schedule the task to execute at a set time (i.e., delay the execution). After the scheduled time arrives, rendering operations can be performed on the broadcaster's client. The scheduled time can be determined by the server.

[0085] The client corresponding to the viewer receives the first interactive message and can determine whether to process it. This is done by following fields in the multimedia data stream, such as `sei` (a field in the structure of the audio / video stream), to obtain the delay time. A dynamic container creates a task queue. After the target execution time of the interactive task arrives, the dynamic container executes the interactive task based on its type. For example, if the dynamic container executes the task on a timed basis, it will delay the execution of the interactive task. Once the timed period arrives, rendering operations can be performed on the client corresponding to the broadcaster.

[0086] The message processing method disclosed herein can be considered a dynamic multi-task streaming processing method, ensuring that the multimedia data stream obtained by the members in the live broadcast and the execution time of the interactive task are consistent. The multimedia data stream can be obtained from the Content Delivery Network (CDN) server, and the interactive message corresponding to the interactive task can be a message from the live broadcast dimension.

[0087] This disclosure addresses the problem that existing message processing mechanisms cannot handle concurrent tasks according to their actual execution time and order, instead processing related logic as soon as a message is received. For example, in a live stream with nine guests, each guest triggers a task, but some tasks require scheduled execution or have a specific execution order. Therefore, this disclosure rationally handles the response order of each initial interaction message, ensuring the correct execution order and actual performance of tasks. For instance, this disclosure provides a timer for delayed task processing, with a set duration, such as 0.1 seconds, to poll the task list and calculate whether a preset condition is met, such as response timestamp + delay time > current local timestamp, to ensure consistent execution time for each task across different devices and viewpoints. During execution, the decision to execute a task can be made based on the response timestamp corresponding to each guest.

[0088] This disclosure addresses the inability to perform richer multi-user interactions using scheduled execution capabilities. For example, if the first interactive message is at 8:08 (e.g., the response timestamp is 8:08), the corresponding interactive task is executed at 8:16. This disclosure can determine whether to execute the interactive task based on a delayed processing task timer.

[0089] This disclosure addresses the anomaly that can occur when handling scheduled interactive tasks in abnormal scenarios, such as a first interactive message at 8:08 AM requiring the corresponding interactive task to be executed at 8:16 AM, but the viewer enters the live stream at 8:10 AM. In this scenario, the client can obtain a second interactive message through a dynamic container and restore the actual effect of the current live stream based on the second interactive message.

[0090] The dynamic container provides the ability to add new message processing capabilities, dynamically updating the application on the client side to respond to the processing of the first interactive message. The dynamic container employs dynamic update capabilities, deploying basic response capabilities on the client side. Through these dynamic update capabilities, it processes new logic and invokes client capabilities to display new interactive effects.

[0091] This disclosure implements a mechanism for processing follow-up messages, ensuring that the execution of interactive messages and audio / video streams are synchronized, achieving audio-visual synchronization. For example, a response timestamp is added to the first interactive message, and the dynamic container processes the interactive task corresponding to the first interactive message based on the response timestamp, the local timestamp, and the delay time.

[0092] Figure 4 This is a schematic diagram of the structure of a message processing device provided in an embodiment of this disclosure, as shown below. Figure 4 As shown, the device includes:

[0093] The acquisition module 410 is used to acquire a first interactive message, the first interactive message including a message sent by the server in response to the client's interactive request, the first interactive message including a response timestamp, the response timestamp indicating the time when the server received the interactive request;

[0094] The determining module 420 is used to determine the target execution time of the interactive task corresponding to the first interactive message based on the response timestamp and the delay time, wherein the delay time includes the delay time of the multimedia data stream corresponding to the interactive task;

[0095] The execution module 430 is used to execute the interactive task in response to the target execution time and local timestamp meeting preset conditions.

[0096] The technical solution provided in this disclosure involves an acquisition module obtaining a first interactive message, and a determination module determining the target execution time of the interactive task corresponding to the first interactive message based on the response timestamp and multimedia stream delay time included in the first interactive message. Then, the execution module determines whether to execute the interactive task based on whether the comparison result between the target execution time and the local timestamp meets preset conditions. During the execution of the interactive task, the time for the server to obtain the interactive request and the multimedia data stream delay time are considered, enabling the target execution time of the interactive task to be synchronized with the multimedia data stream time by combining the multimedia data stream delay time.

[0097] In one embodiment, the execution module 430 includes:

[0098] The comparison unit is used to compare the target execution time with the local timestamp to obtain the comparison result;

[0099] An execution unit is configured to execute the interactive task based on the task type of the interactive task in response to the comparison result indicating that the target execution time is later than or equal to the local timestamp.

[0100] In one embodiment, the execution unit is specifically used for:

[0101] In response to the task type indication that the interactive task is a delayed task, the decision to execute the interactive task is made through a delayed processing task timer.

[0102] In one embodiment, the message processing apparatus further includes an exception module, configured to:

[0103] In response to the current abnormal scenario, a second interactive message received within a set time period before the current time is obtained, the second interactive message including interactive messages received within the set time period before the current time;

[0104] The second interactive message is used as the first interactive message, and the operation of determining the target execution time corresponding to the first interactive message and executing the interactive task corresponding to the first interactive message is performed.

[0105] In one embodiment, the abnormal scenario includes a scenario where a second interactive message is missing within a set time period prior to the current time.

[0106] In one embodiment, the acquisition module 410 is specifically used for

[0107] The first interactive message transmitted by the server is obtained through a dynamic container;

[0108] The interactive task corresponding to the first interactive message pair is created by the task processor of the dynamic container;

[0109] The interactive task is associated with a delay processor, which is used to control the execution of the interactive task.

[0110] In one embodiment, the determining module 420 is specifically used for:

[0111] The delay time is obtained by calling the delay time acquisition method through the delay processor;

[0112] The sum of the response timestamp and the delay time is determined as the target execution time of the interactive task corresponding to the first interactive message.

[0113] The message processing apparatus provided in this disclosure can execute the message processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0114] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0115] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Refer to the following... Figure 5 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 5 A structural diagram of the terminal device or server in the 500.

[0116] Electronic devices 500 include:

[0117] One or more processing devices 501;

[0118] Storage device 508, for storing one or more programs,

[0119] When the one or more programs are executed by the one or more processing devices 501, the one or more processing devices 501 implement the methods provided in the embodiments of this disclosure.

[0120] The terminal devices in this disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0121] like Figure 5 As shown, electronic device 500 may include a processing unit (e.g., central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An edit / output (I / O) interface 505 is also connected to bus 504.

[0122] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0123] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0124] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0125] The electronic device provided in this embodiment and the message processing method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0126] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the message processing method provided in the above embodiments.

[0127] It should be noted that the computer-readable medium described above in this disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof.

[0128] The computer storage medium may be a storage medium for computer-executable instructions, which, when executed by a computer processor, are used to perform the methods provided in this disclosure.

[0129] Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0130] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0131] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0132] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire a first interactive message, the first interactive message including a message sent by the server in response to an interactive request from the client, the first interactive message including a response timestamp indicating the time when the server received the interactive request;

[0133] Based on the response timestamp and the delay time, the target execution time of the interactive task corresponding to the first interactive message is determined, wherein the delay time includes the delay time of the multimedia data stream corresponding to the interactive task;

[0134] The interactive task is executed in response to the target execution time and local timestamp meeting preset conditions.

[0135] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0137] The modules or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of modules or units do not necessarily limit the specific unit; for example, an acquisition module can also be described as a "first interactive message acquisition module".

[0138] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0139] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0140] This disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the message processing method provided in this disclosure.

[0141] According to one or more embodiments of this disclosure, [Example 1] provides a message processing method, including:

[0142] Obtain a first interactive message, the first interactive message including a message sent by the server in response to the client's interactive request, the first interactive message including a response timestamp, the response timestamp indicating the time when the server received the interactive request;

[0143] Based on the response timestamp and the delay time, the target execution time of the interactive task corresponding to the first interactive message is determined, wherein the delay time includes the delay time of the multimedia data stream corresponding to the interactive task;

[0144] The interactive task is executed in response to the target execution time and local timestamp meeting preset conditions.

[0145] According to one or more embodiments of this disclosure, [Example 2] provides the method of Example 1, wherein the interactive task is executed in response to the target execution time and local timestamp satisfying a preset condition, including:

[0146] The target execution time and the local timestamp are compared to obtain the comparison result;

[0147] In response to the comparison result indicating that the target execution time is later than or equal to the local timestamp, the interactive task is executed based on the task type of the interactive task.

[0148] According to one or more embodiments of this disclosure, [Example 3] provides the method of Example 2, wherein executing the interactive task according to the task type of the interactive task includes:

[0149] In response to the task type indication that the interactive task is a delayed task, the decision to execute the interactive task is made through a delayed processing task timer.

[0150] According to one or more embodiments of this disclosure, Example 4 provides the method described in Example 1, further comprising:

[0151] In response to the failure to obtain the second interactive message, the second interactive message is obtained, the second interactive message including interactive messages received within a set time prior to the current time;

[0152] Determine the target execution time corresponding to the second interactive message and execute the interactive task corresponding to the second interactive message.

[0153] According to one or more embodiments of this disclosure, [Example 5] provides the method of Example 1, wherein obtaining the first interactive message includes:

[0154] The first interactive message transmitted by the server is obtained through a dynamic container;

[0155] The interactive task corresponding to the first interactive message pair is created by the task processor of the dynamic container;

[0156] The interactive task is associated with a delay processor, which is used to control the execution of the interactive task.

[0157] According to one or more embodiments of this disclosure, [Example 6] provides the method described in Example 5, wherein determining the target execution time of the interactive task corresponding to the first interactive message based on the response timestamp and the delay time includes:

[0158] The delay time is obtained by calling the delay time acquisition method through the delay processor;

[0159] The sum of the response timestamp and the delay time is determined as the target execution time of the interactive task corresponding to the first interactive message.

[0160] According to one or more embodiments of this disclosure, [Example 7] provides a message processing apparatus, including:

[0161] The acquisition module is used to acquire a first interactive message, which includes a message sent by the server in response to the client's interactive request. The first interactive message includes a response timestamp, which indicates the time when the server received the interactive request.

[0162] The determining module is used to determine the target execution time of the interactive task corresponding to the first interactive message based on the response timestamp and the delay time, wherein the delay time includes the delay time of the multimedia data stream corresponding to the interactive task;

[0163] The execution module is used to execute the interactive task in response to the target execution time and local timestamp meeting preset conditions.

[0164] According to one or more embodiments of this disclosure, [Example 8] provides an electronic device, the electronic device comprising:

[0165] One or more processing devices;

[0166] Storage device for storing one or more programs.

[0167] When the one or more programs are executed by the one or more processing devices, the one or more processing devices implement the message processing method as described in any of Examples 1-7.

[0168] According to one or more embodiments of this disclosure, [Example 9] provides a storage medium containing computer-executable instructions that, when executed by a computer processor, are used to perform a message processing method as described in any of Examples 1-7.

[0169] According to one or more embodiments of this disclosure, [Example 10] provides a computer program product including a computer program that, when executed by a processor, implements the message processing method according to any one of Examples 1-6.

[0170] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0171] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0172] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A message processing method, characterized in that, include: Obtain a first interactive message, the first interactive message including a message sent by the server in response to the client's interactive request, the first interactive message including a response timestamp, the response timestamp indicating the time when the server received the interactive request; Based on the response timestamp and the delay time, the target execution time of the interactive task corresponding to the first interactive message is determined, wherein the delay time includes the delay time of the multimedia data stream corresponding to the interactive task; The interactive task is executed in response to the target execution time and local timestamp meeting preset conditions.

2. The method according to claim 1, characterized in that, The step of executing the interactive task in response to the target execution time and local timestamp meeting a preset condition includes: The target execution time and the local timestamp are compared to obtain the comparison result; In response to the comparison result indicating that the target execution time is later than or equal to the local timestamp, the interactive task is executed based on the task type of the interactive task.

3. The method according to claim 2, characterized in that, Executing the interactive task according to the task type of the interactive task includes: In response to the task type indication that the interactive task is a delayed task, the decision to execute the interactive task is made through a delayed processing task timer.

4. The method according to claim 1, characterized in that, Also includes: In response to the failure to obtain the second interactive message, the second interactive message is obtained, the second interactive message including interactive messages received within a set time prior to the current time; Determine the target execution time corresponding to the second interactive message, and execute the interactive task corresponding to the second interactive message based on the target execution time.

5. The method according to claim 1, characterized in that, The acquisition of the first interactive message includes: The first interactive message transmitted by the server is obtained through a dynamic container; The interactive task corresponding to the first interactive message pair is created by the task processor of the dynamic container; The interactive task is associated with a delay processor, which is used to control the execution of the interactive task.

6. The method according to claim 5, characterized in that, Determining the target execution time of the interactive task corresponding to the first interactive message based on the response timestamp and the delay time includes: The delay time is obtained by calling the delay time acquisition method through the delay processor; The sum of the response timestamp and the delay time is determined as the target execution time of the interactive task corresponding to the first interactive message.

7. A message processing device, characterized in that, include: The acquisition module is used to acquire a first interactive message, which includes a message sent by the server in response to the client's interactive request. The first interactive message includes a response timestamp, which indicates the time when the server received the interactive request. The determining module is used to determine the target execution time of the interactive task corresponding to the first interactive message based on the response timestamp and the delay time, wherein the delay time includes the delay time of the multimedia data stream corresponding to the interactive task; The execution module is used to execute the interactive task in response to the target execution time and local timestamp meeting preset conditions.

8. An electronic device, characterized in that, The electronic device includes: One or more processing devices; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processing devices, the one or more processing devices implement the message processing method as described in any one of claims 1-6.

9. A storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the message processing method as described in any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the message processing method according to any one of claims 1-6.

Images