Content generation method and device, and recorded content data playing method and device

Abstract

The embodiment of the invention provides a content generation method and device and a recorded content data playing method and device.The method comprises the steps that in the process of receiving a multimedia data stream, if it is determined that display content of a first video of the multimedia data stream changes, first data is obtained, the first data is associated with first display content obtained after the display content is changed; and the first audio of the multimedia data stream, the first data and the corresponding content change time information are stored in an associated manner, so that recorded content data of the multimedia data stream can be generated.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a content generation method and device, and a method and device for playing recorded content data. Background Art

[0002] Currently, for recording of online multimedia, video and audio can usually be recorded separately, and the video and audio are stored in association to generate recording data of the online multimedia.

[0003] However, in some scenarios (such as video conferencing scenarios), the speaker explains the shared content most of the time, and the video frame in the video stream will change only when the shared content is switched (for example, turning pages).

[0004] For the above-mentioned video conferencing scenario, there will be many video frames with the same content in the video stream of the recorded data, and there will be a lot of redundant data in the recorded data. The amount of information contained in the video content is not high relative to the storage space occupied by the recorded data. Summary of the invention

[0005] One or more embodiments of the present disclosure provide a content generation method and device, and a method and device for playing recorded content data, so as to reduce redundant data in recorded data, reduce the data volume of recorded data, and increase the proportion of information in recorded data.

[0006] According to a first aspect, there is provided a content generation method, comprising:

[0007] In the process of receiving the multimedia data stream, if it is determined that the display content of the first video of the multimedia data stream has changed, obtaining first data, wherein the first data is associated with the first display content obtained after the display content has changed;

[0008] The first audio of the multimedia data stream, the first data and their corresponding content change time information are stored in association to generate recorded content data of the multimedia data stream.

[0009] According to a second aspect, a method for playing recorded content data is provided, comprising:

[0010] In response to a content playback instruction for recorded content data of a multimedia data stream, obtaining first audio, first data, and playback time information corresponding to the first data in the recorded content data, wherein the first data is data obtained when it is determined that the display content of a first video of the multimedia data stream changes during the process of receiving the multimedia data stream, the first data is associated with first display content obtained after the display content changes, and the playback time information corresponds to the time when the display content changes;

[0011] In the playback interface, the first audio and the first data are played in association with each other based on the playback time information corresponding to the first data.

[0012] According to a third aspect, there is provided a content generation device, comprising:

[0013] A first acquisition module is configured to acquire first data if it is determined that the display content of the first video of the multimedia data stream has changed during the process of receiving the multimedia data stream, wherein the first data is associated with the first display content obtained after the display content has changed;

[0014] The storage module is configured to associate and store the first audio of the multimedia data stream, the first data and their corresponding content change time information to generate recorded content data of the multimedia data stream.

[0015] According to a fourth aspect, there is provided a content generation device, comprising:

[0016] a second acquisition module, configured to, in response to a content playback instruction for recorded content data of a multimedia data stream, acquire a first audio, first data, and playback time information corresponding to the first data in the recorded content data, wherein the first data is data acquired when it is determined that the display content of a first video of the multimedia data stream changes during the process of receiving the multimedia data stream, the first data is associated with a first display content obtained after the display content changes, and the playback time information corresponds to the time when the display content changes;

[0017] The first playing module is configured to play the first audio and the first data in an associated manner in a playing interface based on the playing time information corresponding to the first data.

[0018] According to a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method described in the first aspect or the second aspect.

[0019] According to a sixth aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in the first aspect or the second aspect is implemented.

[0020] According to the method and device provided by the embodiments of the present disclosure, in the process of receiving a multimedia data stream, if it is determined that the display content of the first video of the multimedia data stream has changed, the first data associated with the first display content obtained after the display content has changed is obtained, and then the first audio, the first data of the multimedia data stream and their corresponding content change time information are associated and stored to generate recorded content data of the multimedia data stream. In this way, the generated recorded content data includes the audio of the multimedia data stream and the data associated with the first display content obtained after the display content has changed. This can reduce redundant data in the recorded data to a certain extent, increase the proportion of information in the recorded data, and reduce the data volume of the recorded data, saving data storage occupancy. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram of an implementation framework of an embodiment disclosed in the present disclosure;

[0023] Figure 2 A schematic diagram of a flow chart of a content generation method provided in an embodiment;

[0024] Figures 3A-3F A schematic diagram of the relevant interfaces of the content generation process provided in the embodiment;

[0025] Figure 4 Another flowchart of the content generation method provided in the embodiment;

[0026] Figure 5 Another flowchart of the content generation method provided in the embodiment;

[0027] Figure 6 Another flowchart of the content generation method provided in the embodiment;

[0028] Figure 7 A schematic diagram of a flow chart of a method for playing recorded content data provided in an embodiment;

[0029] Figure 8 A schematic block diagram of a content generation device provided in an embodiment;

[0030] Fig. 9 A schematic block diagram of a device for playing recorded content data provided in an embodiment;

[0031] Fig.10 A schematic block diagram of an electronic device provided in an embodiment. DETAILED DESCRIPTION

[0032] The technical solution of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0033] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0034] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0035] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0036] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0037] The present disclosure discloses a content generation method and device, and a method and device for playing recorded content data. Figure 1 A schematic diagram of an implementation scenario of an embodiment of the content generation method disclosed in the present disclosure is shown, which is a schematic diagram of a scenario for recording online activities, such as online meetings and online teaching, etc. The following is an example of an online meeting scenario. In this implementation scenario, several clients participating in the online meeting are shown, such as Figure 1 The several clients shown may include client A, client B and client C, as well as a server that provides online conference services for the several clients.

[0038] During the online conference, any client participating in the online conference can act as a speaker, and its corresponding member participating in the online conference can act as a spokesperson. Any client participating in the online conference can share data with other clients participating in the online conference. Among them, when the client that is the speaker changes or the spokesperson of the client that is the speaker changes, the display content of the first video of the multimedia data stream generated by the online conference (for example, the display content of the video interface of the first video) will change. When any client participating in the online conference shares data with other clients participating in the online conference, the display content of the first video includes at least part of the shared data shared in the first video, and the display content of the first video will change during the demonstration of the shared data.

[0039] Specifically, during an online meeting, the server can receive multimedia data streams generated by each client participating in the online meeting, and synchronize the obtained multimedia data streams among the clients participating in the online meeting (for example, data included in the multimedia data stream obtained from client A, such as audio and / or video, is synchronized to clients B and C; data included in the multimedia data stream obtained from client B is synchronized to clients A and C; data included in the multimedia data stream obtained from client C is synchronized to clients A and B).

[0040] During an online meeting, members participating in the online meeting can trigger a recording instruction based on their client (such as client A). Correspondingly, after receiving the recording instruction, the server records the multimedia data stream generated during the meeting.

[0041] During the recording process, during the online conference, if a client (for example, client A) in the online conference shares data, that is, client A responds to the data sharing operation of the corresponding member, and shares the data that needs to be shared (called shared data 1) with other clients (for example, client B and client C) in the online conference, at least part of the shared data 1 will be displayed in the video interface of the online conference. At this time, the multimedia data stream received by the server may include audio (called first audio) and video (the video of the video interface is called first video). The server responds to the instruction to record the conference process. In the process of receiving the multimedia data stream, if it is determined that the display content of the first video of the multimedia data stream (that is, the video interface of the online conference) has changed, the first data is obtained, and the first data is associated with the first display content obtained after the display content changes. The display content may include at least part of the shared data shared in the first video, and then the first audio, the first data and the corresponding content change time information of the multimedia data stream are associated and stored to generate the recorded content data of the multimedia data stream.

[0042] In the above process, in the process of receiving the multimedia data stream, if it is determined that the display content of the first video of the multimedia data stream has changed, the first data associated with the first display content obtained after the display content has changed is obtained, and then the first audio, the first data and the corresponding content change time information of the multimedia data stream are associated and stored to generate the recorded content data of the multimedia data stream. In this way, the generated recorded content data includes the audio of the multimedia data stream and the data associated with the first display content obtained after the display content has changed. This can reduce the redundant data in the recorded data to a certain extent, increase the proportion of information in the recorded data, and reduce the data volume of the recorded data, saving data storage occupancy.

[0043] The content generation method and device, and the method and device for playing recorded content data provided by the present disclosure are described in detail below in conjunction with specific embodiments.

[0044] Figure 2 The flowchart of the content generation method in one embodiment of the present disclosure is shown. The method can be implemented by a target program, which can exist in the form of a server (or a plug-in of the server) or a client (or a plug-in of a client). The target program can be installed in an electronic device, which can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. Figure 2 As shown, the method includes the following steps S210-S220:

[0045] In step S210, during the process of receiving the multimedia data stream, if it is determined that the display content of the first video of the multimedia data stream has changed, first data is obtained, and the first data is associated with the first display content obtained after the display content has changed.

[0046] In one implementation, when several clients are carrying out online activities, if there is a need to record the multimedia data stream generated by the online activities, the clients participating in the online activities can send a recording instruction to the target program to instruct the target program to record the multimedia data stream generated by the online activities. The online activities can be, for example, but not limited to: online conferences (including audio-only conferences and video conferences), online teaching (including audio-only teaching and video teaching), etc.

[0047] In one case, considering that there may be various requirements in online activity scenarios, for example, in some online activity scenarios, only the audio in the online activity needs to be recorded, and in some online activity scenarios, the audio and video in the online activity need to be recorded (it may also be necessary to record audio in part and audio and video in part). And considering that in some online activity scenarios that need to record audio and video, such as online video conferencing scenarios and online video teaching scenarios, there may be a large amount of static content in the display content of the video (for example, in an online video conferencing scenario, a part of a shared data is shared for a long time in the video to explain the part of the content).

[0048] In view of this, in order to provide users with better services and meet the different needs of different users, such as Figure 3A As shown, the content-related display interface of the client may display a "recording" control, and triggering the "recording" control may display several controls corresponding to different recording modes, such as Figure 3A As shown, it may include: controls for a first recording mode (capable of recording multimedia data streams containing audio and video generated by online activities), controls for a second recording mode (capable of recording multimedia data streams containing audio generated by online activities), and controls for a third recording mode (capable of recording multimedia data streams containing audio generated during an online activity before video is generated, for example, before data is shared during an online meeting, and recording the audio in the multimedia data stream generated during the meeting and data associated with the display content obtained after the display content of the video therein changes after data sharing).

[0049] Understandable, such as Figure 3AThe interface shown is only an illustrative example. In one implementation, after the target program detects the recording instruction generated by triggering the "recording" control, it can directly record the audio in the multimedia data stream generated by the online activity, as well as the data associated with the display content obtained after the display content of the video therein changes.

[0050] In one implementation, the content-related display interface may also display relevant information (such as nicknames and / or avatars of members) corresponding to at least some of the clients participating in the online activity. Figure 3A As shown, the nicknames of the members corresponding to client 1 are AA, the nicknames of the members corresponding to client 2 are BB, and the nicknames of the members corresponding to client 3 are CC.

[0051] In one implementation, any client participating in an online activity can generate a recording instruction in response to the operation of its user, and send the recording instruction to the target program. Accordingly, the target program responds to the recording instruction and records the multimedia data stream of the activity process of the online activity. Specifically, in the process of receiving the multimedia data stream, it is monitored in real time whether the display content of the first video of the multimedia data stream changes. If it is determined that the display content of the first video of the multimedia data stream changes, the first data associated with the first display content obtained after the display content changes is obtained.

[0052] In one implementation, the display content may include imaging content of the speaker in the online activity (e.g., the speaker's avatar, frames in the speaker's video stream). In another implementation, the display content may include at least part of the shared data shared in the first video. The shared data shared in the first video may be data in any type of format. In one embodiment, the shared data may be data in any of the following forms: data in a document form, data in a slide form, and data in an online table form. In one implementation, the data shared in the first video (i.e., shared data) may be one or more.

[0053] In one implementation, the display content of the first video of the multimedia data stream may be changed because the first member triggers a first preset operation (or a second preset operation) through its first client during an online activity. The first preset operation may cause the display content of the first video in the multimedia data stream to change. The first preset operation may include, for example, browsing and editing operations of the first member through its first client on the shared data shared during the online activity, wherein the browsing operation may include, but is not limited to, opening, scrolling, page turning, and / or zooming operations. The first client may be a client that shares the shared data. In one case, the first member may be a member that shares data in an online activity.

[0054] In another implementation, the display content of the first video of the multimedia data stream may change because the first member triggers certain specific controls corresponding to the online activity through his first client, such as triggering a control for inviting members to participate in online activities, so as to invite new members to participate in online activities such as online meetings; or triggering a control for switching the speaker of an online activity, so as to switch client 1 as the speaker to client 2, and the imaging content of the speaker of the corresponding online activity is changed.

[0055] In view of the above situation, the target program determines whether the display content of the first video of the multimedia data stream has changed in a manner that may include: the target program detects whether the first member has triggered a first preset operation during the online activity, or the target program may extract video frames from the video segment of the first video of the received multimedia data stream at a regular interval, and determines whether the display content of the first video of the multimedia data stream has changed by judging the similarity between the latest extracted video frame and the historical video frame extracted from the previous video segment of the first video. Next, if the target program determines that the display content of the first video of the multimedia data stream has changed, the target program obtains first data, which is associated with the first display content obtained after the display content has changed.

[0056] Afterwards, in step S220, the first audio, the first data and their corresponding content change time information of the multimedia data stream are stored in association to generate recorded content data of the multimedia data stream.

[0057] After the target program obtains the first data associated with the first display content obtained after the display content changes, it can also obtain the content change time information corresponding to the first data. It can be understood that the first audio can include a plurality of audio frames, each audio frame has a corresponding time stamp, and accordingly, the content change time information corresponds to an audio frame in the first audio, and each data in the first data corresponds to an audio frame corresponding to each content change time information in the first audio.

[0058] In one implementation, after responding to a recording instruction, the target program may record the corresponding data identifier of the multimedia data stream in a preset storage space (for example, in an online meeting scenario, the data identifier may be a meeting ID, and in an online teaching scenario, the data identifier may be a teaching video ID). Thereafter, the target program may store the first audio, the first data and the corresponding content change time information of the multimedia data stream in the preset storage space corresponding to the corresponding data identifier of the multimedia data stream to generate recorded content data of the multimedia data stream.

[0059] In one embodiment, if the target program detects a recording end instruction for a multimedia data stream, the target program can obtain the recorded content data of the multimedia data stream based on the first audio, first data and corresponding content change time information of the associated stored multimedia data stream; and then feed back a notification message related to the recorded content data to the client where the specified member is located.

[0060] In one implementation, the target program can feed back notification messages related to the recorded content data to the clients of the members who initiate or participate in online activities (such as online meetings or online teaching). Optionally or additionally, these members can send notification message sharing instructions to the target program through their clients, and accordingly, the target program responds to the notification message sharing instructions and shares the notification message to the clients of other users designated by the members who initiate the online activities.

[0061] In one embodiment, the notification message related to the recorded content data may include an access address of the recorded content data, and any designated member may view the recorded content data by accessing the access address. In one implementation, any designated member may directly click on the access address, and the client of the member generates an access instruction in response to the click operation on the access address and sends it to the target program, and the target program responds to the access instruction and provides at least part of the audio of the first audio in the recorded content data and data corresponding to at least part of the audio in the first data to the corresponding playback client, so that the playback client plays at least part of the audio of the first audio and data corresponding to at least part of the audio in the first data in an associated manner.

[0062] The playback client may be any playback client that can play audio and video and is authorized by the member who clicks on the access address of the recorded content data and is installed in the device where the member's client is located.

[0063] In another implementation, any member of the designated members can copy the access address and paste it into any playback client installed on the device where their client is located to trigger an access instruction to the access address. Accordingly, the target program responds to the access instruction and provides at least part of the audio of the first audio in the recorded content data and data corresponding to at least part of the audio in the first data to the corresponding playback client, so that the playback client associates the playback of at least part of the audio of the first audio and data corresponding to at least part of the audio in the first data.

[0064] Through the above method, the recorded content data can be accessed and played, providing users with viewing services for the recorded content data.

[0065] In this embodiment, during the process of receiving a multimedia data stream, if it is determined that the display content of the first video of the multimedia data stream has changed, first data associated with the first display content obtained after the display content has changed is obtained, and then the first audio, first data and corresponding content change time information of the multimedia data stream are associated and stored to generate recorded content data of the multimedia data stream. In this way, the generated recorded content data includes the audio of the multimedia data stream and data associated with the first display content obtained after the display content has changed. This can reduce redundant data in the recorded data to a certain extent, increase the proportion of information in the recorded data, reduce the amount of recorded data, and save data storage space.

[0066] In order to better provide users with high-quality services, the target program can provide users with multiple recording methods for multimedia data streams. For example, when the first member (i.e., the user) selects the aforementioned third recording method to record the multimedia data stream, the target program can record the multimedia data stream in any of the following multiple recording methods. In one implementation, the first member (or other members related to the multimedia data stream) can set a recording method for the multimedia data stream, and the target program can record the multimedia data stream based on the recording method set by the first member.

[0067] In one embodiment, Figure 4 As shown, a recording method is provided. Accordingly, the method may include the following steps S410-S430: In step S410, during the process of receiving a multimedia data stream, a first video segment of a first video of the multimedia data stream is obtained.

[0068] In one implementation, the first video clip may be at least a portion of the video stream corresponding to the first video of the multimedia data stream. In another implementation, the first video clip may be a video file generated by the target program based on at least a portion of the video stream corresponding to the first video during the process of receiving the multimedia data stream. For example, the target program may generate a corresponding video file based on at least a portion of the video stream corresponding to the first video in the multimedia data stream received in real time at preset time intervals. During an online activity, the target program may receive in real time the multimedia data stream generated by the activity. The multimedia data stream may include multimedia data streams generated by each client participating in the online activity.

[0069] After the target program obtains the first video clip, in step S420, in response to detecting a first instruction generated by the first member triggering a first preset operation during an online activity, a first video frame corresponding to the first instruction is extracted from the first video clip and included in the first data, wherein the online activity process is a process of generating a multimedia data stream.

[0070] In this step, when the target program receives the multimedia data stream generated by the online activity, after detecting the first instruction generated by the first member triggering the first preset operation during the online activity, it can be determined that the display content of the first video has changed. Accordingly, the target program responds to the first instruction and extracts the first video frame corresponding to the first instruction from the first video clip, and classifies it into the first data. The first instruction may include the time corresponding to the first display content obtained after the display content changes, which can be considered as the time when the first member triggers the first preset operation for the first video, and the time is the content change time information corresponding to the first data associated with the first display content obtained after the display content changes. The first data may include each video frame obtained after each display content change.

[0071] In one implementation, the first preset operation may include, but is not limited to, one or more of the following operations: browsing operations and editing operations on the shared data shared in the first video (i.e., the shared data shared during the online activity). In one case, the browsing operation may include, but is not limited to: opening operations, scrolling operations, zooming operations, page turning operations, and closing operations on the shared data. For example, in the case where the first preset operation is a scrolling operation on the shared data, the time to trigger the first preset operation may be the time when the scrolling operation on the shared data ends, and the first video frame is the first display content obtained after the scrolling operation on the shared data causes the display content to change. In one case, the editing operation may include, but is not limited to, an annotation operation on the shared data shared in the first video.

[0072] In one implementation, after the target program extracts the first video frame corresponding to the first instruction from the first video clip, it can also configure a video frame identifier for the first video frame. The video frame identifier can include a number representing the extraction order of the corresponding first video frame. The number i can also represent that the corresponding first video frame is the i-th extracted video frame during this recording process, and can also represent the i-th change in the display content of the first video during this recording process. For example, the video frame identifier corresponding to the first video frame 1 is 3, indicating that the first video frame 1 is the third video frame extracted during this recording process, and is the video frame obtained after the display content of the first video changes for the third time during this recording process; for another example, the video frame identifier corresponding to the first video frame 2 is 10, indicating that the first video frame 2 is the 10th video frame extracted during this recording process, and is the video frame obtained after the display content of the first video changes for the 10th time during this recording process. The video frame identifier corresponding to the first video frame can then be included in the first data.

[0073] In another implementation, after the target program extracts the first video frame corresponding to the first instruction from the first video clip, it can also feed back the first video frame (and its corresponding video frame identifier) ​​to at least the first client, so that the first client displays the first video frame (and its corresponding video frame identifier) ​​in a preset display area of ​​the video interface of the first video, prompting the first member that the corresponding video frame has been extracted for the first preset operation triggered by it, such as Figure 3B When data is shared in an online activity, the first video frame may display the shared data, for example, displaying a directory of the shared data and the content of the shared data.

[0074] Then, in step S430, the first audio, the first data and the corresponding content change time information of the multimedia data stream are stored in association to generate the recorded content data of the multimedia data stream. Figure 2 The step S220 shown is not repeated here.

[0075] In one embodiment, taking into account that users may have the need to annotate the content in the extracted video frame (such as the first video frame), in order to provide users with better services and improve user experience, after step S420, the method may also include the following step 11: In step 11, a labeling instruction for the first video frame is received, wherein the labeling instruction carries the labeling content of the first member for the first video frame; correspondingly, in step S430, the first audio, the first data and their corresponding content change time information and the labeling content are associated and stored.

[0076] In one implementation, the annotation instruction may be an instruction generated by the first member triggering a preset annotation operation on the first video frame displayed by the aforementioned first client. For example, the first member may click on the first video frame displayed by the first client, and accordingly, the first client displays an annotation control indicating that annotation is to be made to the first video frame; then the first client enters the annotation environment for the first video frame in response to the first member's triggering operation on the annotation control, and the first member may annotate the first video frame in the annotation environment, and after the annotation is completed, trigger a control indicating that the annotation is completed; the first client generates an annotation instruction carrying the annotation content in response to the first member's triggering operation on the control indicating that the annotation is completed, and sends it to the target program, and the target program responds to the annotation instruction and associates and stores the first audio, the first data and their corresponding content change time information and the annotation content carried by the annotation instruction.

[0077] In one case, the annotation instruction for the first video frame may also carry the annotation position of the annotation content in the first video frame. Accordingly, the target program stores the first audio, the first data and its corresponding content change time information, the annotation content carried by the annotation instruction and the annotation position. Subsequently, in the process of viewing the recorded content data, the corresponding annotation content may be displayed at the corresponding annotation position of the first video frame, so as to provide a better viewing experience for the user.

[0078] In another implementation, when the first member sets the permission for other members participating in the online activity to annotate the video frames, the target program can also accept annotation instructions from other members for the extracted video frames (e.g., the first video frame), and then associate and store the first audio, the first data and its corresponding content change time information, and the annotation content carried by the annotation instructions.

[0079] exist Figure 4 In the process shown, each time the first member triggers the first preset operation during the online activity, the target program can generate a first instruction in response to the detection of the first preset operation triggered by the first member, extract a video frame corresponding to the first instruction from the video clip of the first video of the acquired multimedia data stream, and classify it into the first data to associate and store the first audio, first data and corresponding content change time information of the multimedia data stream. Correspondingly, in this implementation, after the recording of the multimedia data stream is completed, the generated recorded content data may include: the first audio of the recorded multimedia data stream, the first data including several video frames and their corresponding content change time information, wherein each video frame is associated with the first display content obtained after the display content of the first video changes, that is, the first display content obtained after the display content of the first video changes. In another case, the first data may also include the annotation content and annotation position annotated for any video frame, and the video frame identifiers of several video frames. This reduces the redundant data in the recorded data to a certain extent, increases the proportion of information in the recorded data, reduces the data volume of the recorded data, and saves the data storage space.

[0080] In one case, when the first preset operation is a closing operation for shared data, the target program can end the sharing process of the shared data in response to the first instruction generated for the first preset operation. Subsequently, before the target program detects that the members participating in the online activity share data again (and before the recording of the multimedia data stream is ended), only the audio of the multimedia data stream can be recorded.

[0081] In one embodiment, the target program detects the recording end instruction for the first video, ends the recording of the multimedia data stream, and obtains the recorded content data of the multimedia data stream. Then, a notification message related to the recorded content data can be generated, and the notification message related to the recorded content data is fed back to the client where the designated member is located. In one implementation, the notification message includes the access address of the recorded content data. The designated member can play and view the recorded content data by accessing the access address. For a clear layout, the playback process of the recorded content data is described in detail later.

[0082] In another embodiment, the target program also provides other recording methods for multimedia data streams, such as Figure 5 As shown, the method may include the following steps S510-S550:

[0083] In step S510, in the process of receiving the multimedia data stream, a second video segment of the first video of the multimedia data stream is obtained. It can be understood that the process of the target program obtaining the second video segment can be referred to in Figure 4 The process of obtaining the first video clip in is not repeated here.

[0084] Then, in step S520, if the arrival of the moment corresponding to the first frame extraction cycle is detected, the second video frame corresponding to the moment corresponding to the first frame extraction cycle is extracted from the second video segment. In this step, the target program may preset a frame extraction cycle for the first video, referred to as the first frame extraction cycle, and each time the arrival of the moment corresponding to the first frame extraction cycle is detected, a video frame corresponding to the moment corresponding to the first frame extraction cycle is extracted from the acquired second video segment, referred to as the second video frame. For example, the first frame extraction cycle is 5 moments, and in the case where moment 1 is the moment corresponding to the first frame extraction cycle, the moment corresponding to the first frame extraction cycle may also include moment 1+5n; in the case where moment 2 is the moment corresponding to the first frame extraction cycle, the moment corresponding to the first frame extraction cycle may also include moment 2+5n, where n is a positive integer.

[0085] Then, in step S530, the similarity value between the second video frame and the third video frame is calculated, and the third video frame is a video frame obtained after the latest display content change in the first data. The third video frame is a video frame extracted from the second video segment or a historical video segment before it, and the historical video segment is also a video segment corresponding to the first video. In this step, after the target program extracts the second video frame, it can use a preset image similarity algorithm to calculate the similarity value between the second video frame and the third video frame. The preset image similarity algorithm can be any algorithm in the relevant technology that can calculate the similarity between images.

[0086] Then, in step S540, if the similarity value is lower than the preset similarity threshold, the second video frame is included in the first data, and the first data includes each video frame obtained after each display content change. In this step, if the target program determines that the similarity value between the second video frame and the third video frame is lower than the preset similarity threshold, it can be determined that the display content of the first video has changed, and accordingly, the second video frame of the first display content obtained after the display content has changed is included in the first data. Then, the timestamp corresponding to the second video frame can also be determined as the corresponding content change time information.

[0087] In one implementation, after the target program classifies the second video frame into the first data, it can also configure a video frame identifier for the second video frame, which can be a number representing the number of times the display content of the first video changes during the recording process; thereafter, the video frame identifier corresponding to the second video frame can also be included in the first data.

[0088] In another implementation, after the target program classifies the second video frame into the first data, it can also feed back the second video frame (and its corresponding video frame identifier) ​​to the first client, so that the first client displays the second video frame (and its corresponding video frame identifier), prompting the first member to extract the corresponding video frame in response to the changes in the display content of the first video.

[0089] Next, in step S550, the first audio, the first data and the corresponding content change time information of the multimedia data stream are stored in association to generate the recorded content data of the multimedia data stream. Figure 2 The step S220 shown is not repeated here.

[0090] In one implementation, taking into account that users may have the need to annotate the content in the extracted video frame (such as the second video frame), in order to provide users with better services and improve user experience, after step S540, the method may also include the following steps: receiving a labeling instruction for the second video frame, wherein the labeling instruction carries the labeling content of the first member for the second video frame; correspondingly, in step S550, the first audio, the first data and their corresponding content change time information and the labeling content of the multimedia data stream are associated and stored.

[0091] In yet another embodiment, Figure 5As shown, the method may further include step S560: in step S560, if the similarity value is not lower than a preset similarity threshold, the second video frame is discarded. In this step, if the target program determines that the similarity value between the second video frame and the third video frame is not lower than the preset similarity threshold, it can be determined that the display content of the first video has not changed. In order to reduce redundant data in the recorded data and reduce the data volume of the recorded content data, the target program discards the second video frame.

[0092] In this implementation, after the recording of the multimedia data stream is completed, the generated recorded content data may include: the first audio of the multimedia data stream, a number of first data and their corresponding content change time information. Among them, the first data include: video frames whose similarity values ​​between each two adjacent frames extracted at a fixed time are lower than a preset similarity threshold, and may also include the annotation content and annotation position annotated for the video frame, and the video frame identifier. This can reduce the redundant data in the recorded data to a certain extent, increase the proportion of information in the recorded data, reduce the amount of recorded data, and save data storage space.

[0093] In one implementation, in order to provide users with better services while avoiding waste of computing resources in the content generation process, the first frame extraction period, that is, the frame extraction frequency can be dynamically adjusted in the embodiment of the present disclosure. Accordingly, the method can also include the following steps 31-32: in step 31, the frame loss frequency of discarded video frames within the most recent preset time period during the process of receiving the multimedia data stream is counted; and in step 32, based on the frame loss frequency, the first frame extraction period is adjusted.

[0094] In this implementation, the target program can count the frame loss frequency of discarded video frames within the most recent preset time period during the process of receiving the multimedia data stream. The frame loss frequency can reflect the frequency of changes in the display content of the first video (for example, the frequency of the first member performing the first preset operation on the shared data); in order to avoid wasting computing resources during the recording process, the target program can adjust the length of the first frame extraction period based on the frame loss frequency, wherein the higher the frame loss frequency, the longer the first frame extraction period can be.

[0095] In one embodiment, the target program detects a recording end instruction for a multimedia data stream, ends recording of the multimedia data stream, obtains recorded content data corresponding to the multimedia data stream, and then generates a notification message related to the recorded content data. Afterwards, the target program can send the notification message related to the recorded content data to the client of the designated member so that the designated member can view the recorded content data.

[0096] In another embodiment, the target program also provides other recording methods for multimedia data streams, such as Figure 6As shown, the method may include the following steps S610-S630:

[0097] In step S610, shared data is obtained during the process of receiving the multimedia data stream. In this step, the shared data is shared data associated with the multimedia data stream, that is, when the shared data is data in an online activity (such as an online meeting), the target program can obtain the shared data during the process of receiving the multimedia data stream.

[0098] In one implementation, the shared data can be stored locally on the device where the client (e.g., the second client) that shares the shared data is located. When the target program detects that the second client starts to share the shared data, it can apply to the member corresponding to the second client (the second member) for permission to obtain the shared data. After the second member authorizes, the shared data can be obtained locally from the device where the second client is located. In another implementation, the shared data is online data. Accordingly, the target program can obtain the storage location information of the shared data with the authorization of the second member to obtain the shared data.

[0099] Then, in step S620, in response to detecting that the second member generates a second instruction to trigger a second preset operation on the shared data, the operation track data corresponding to the second preset operation is obtained and included in the first data. In this step, after detecting that the second member generates a second instruction to trigger a second preset operation on the shared data, the target program obtains the operation track data corresponding to the second preset operation carried by the second instruction in response to the second instruction, and includes the operation track data in the first data.

[0100] The second preset operation may be an operation that causes the display content of the first video to change. In one implementation, the second preset operation may include but is not limited to browsing operations and editing operations on the shared data, wherein the browsing operation may include but is not limited to: opening operations, scrolling operations, page turning operations, zooming operations, and closing operations on the shared data.

[0101] In another implementation, the operation track data may include the second preset operation triggered by the second member for the shared data and / or the corresponding operation data. Specifically, when the second preset operation is an opening operation for the shared data, the operation track data at least includes: an opening operation. Wherein, when the second preset operation is an opening operation for the shared data, the operation track data at least includes an opening operation, and each time the shared data is opened, the shared data can be displayed from a specified position of the shared data, wherein the specified position can be, for example, the home page position of the shared data. In another case, the operation track data may also include an opening display position corresponding to the opening operation, and accordingly, in this case, when the shared data is opened, the shared data may be displayed from the opening display position corresponding to the opening operation of the shared data. Wherein, the opening display position corresponding to the opening operation can be determined based on the position displayed when the shared data was closed the last time.

[0102] When the second preset operation is a scrolling operation on the shared data, the operation track data includes: the scrolling operation and data indicating a scrolling end position of the scrolling operation in the shared data.

[0103] In one case, when the second preset operation is a scrolling operation for shared data, the operation trajectory data may also include: the scrolling start position corresponding to the scrolling operation. In one implementation, the data indicating the scrolling end position of the scrolling operation in the shared data may include: the scrolling start position corresponding to the scrolling operation, and the scrolling distance (for example, the operation trajectory data may include: scrolling yyPX downward (or upward) from the scrolling start position, i.e., position xx). In another implementation, the data indicating the scrolling end position of the scrolling operation in the shared data may directly include: the scrolling end position (for example: scrolling down to position zz; for another example, scrolling yyPX downward from the scrolling start position, i.e., position xx, to position zz). Wherein, PX is pixel, which means pixel, which is the smallest unit displayed on the display screen. The above-mentioned scrolling down yyPX may refer to scrolling down yy pixels in the vertical direction.

[0104] In another case, when the second preset operation is a scrolling operation on the shared data, the operation trajectory data may include: the scrolling operation and data indicating the scrolling end position of the scrolling operation in the shared data, the scrolling start position corresponding to the scrolling operation, and information such as the scrolling duration and scrolling speed of the scrolling operation, so as to better reflect the operation trajectory of the shared data in the recorded content data; for example, the operation trajectory data may include: at a speed of t1, after x seconds, scrolling down yyPX from the xx position of the shared data to the zz position of the shared data.

[0105] When the second preset operation is a zoom operation on the shared data, the operation track data may include: the zoom operation and data indicating the zoom multiple of the shared data by the zoom operation. The target program may determine the display position of the shared data after the zoom operation based on the display position of the shared data before the zoom operation and the data indicating the zoom multiple of the shared data by the zoom operation.

[0106] When the second preset operation is a page turning operation for shared data, the operation track data may include: a page turning operation and data indicating the end position of the page turning operation in the shared data. In one case, when the second preset operation is a page turning operation for shared data, the operation track data may also include: the page turning start position corresponding to the page turning operation. In one implementation, the data indicating the end position of the page turning operation in the shared data may include: the page turning start position corresponding to the page turning operation, and the number of pages turned (for example, the operation track data may include: from the page turning start position, i.e., page aa, x pages have been turned). In another implementation, the data indicating the end position of the page turning operation in the shared data may directly include: the end position of the page turning (for example: turning to page MM; for another example, from the page turning start position, i.e., page aa, x pages have been turned to page MM). In another case, when the second preset operation is a page turning operation for shared data, the operation trajectory data may include: the page turning operation and data indicating the end position of the page turning operation in the shared data, the page turning start position corresponding to the page turning operation, and information such as the page turning duration and the page turning speed of the page turning operation, so as to better reflect the operation trajectory of the shared data in the recorded content data; for example, the operation trajectory data may include: at a speed of t2, after y seconds, turning from page aa of the shared data, page x, to page MM of the shared data.

[0107] When the second preset operation is a closing operation for shared data, the operation track data may at least include: a closing operation. In one case, when the second preset operation is a closing operation for shared data, the operation track data may also include the display position of the shared data when closing. In another case, when the operation track data may at least include: a closing operation, the operation end position corresponding to the second preset operation before the closing operation may be determined as the display position of the shared data when closing. In one implementation, the closing operation for shared data may also be manifested as an operation of the second member triggering the end of data sharing.

[0108] After the target program obtains the first data, in step S630, the shared data, the first audio, the first data and their corresponding content change time information are stored in association. In one case, when the target program obtains the shared data itself, the shared data, the first audio, the first data and their corresponding content change time information can be stored in association. When the shared data is in the form of online data and the storage location information of the shared data is obtained, the storage location information of the shared data, the first audio, the first data and their corresponding content change time information can be stored in association to achieve the storage of the shared data, the first audio, the first data and their corresponding content change time information.

[0109] In this implementation, after recording of the multimedia data stream is completed, the generated recorded content data may include: the first audio of the multimedia data stream, the shared data associated with the multimedia data stream, a number of first data and their corresponding content change time information, wherein the several first data include the operation track data corresponding to each second preset operation triggered by the second member on the shared data. This can reduce the redundant data in the recorded data to a certain extent, increase the proportion of information in the recorded data, reduce the data volume of the recorded data, and save the data storage space.

[0110] In one embodiment, the target program detects a recording end instruction for a multimedia data stream, ends the recording of the multimedia data stream, and obtains the recorded content data corresponding to the multimedia data stream. Then, a notification message related to the recorded content data can be generated, and the notification message can be sent to the client of the specified member so that the specified member can view the recorded content data.

[0111] In one embodiment, during an online activity (such as an online meeting), members participating in the online activity can set a variety of mixed recording methods for the activity process of the online activity. The aforementioned three recording methods can be arbitrarily combined to record the multimedia data stream of the activity process of the online activity. For example, during the time from xx to yy of the online activity, for example, during the process of sharing data A, the multimedia data stream generated by the online activity can be recorded by a recording method. For example, the recording method is to extract the video frame corresponding to the first instruction from the video segment of the first video of the multimedia data stream after detecting the first instruction generated by triggering the first preset operation for the first video of the multimedia data stream, and record the multimedia data stream to obtain the audio from xx to yy of the multimedia data stream generated by the online activity, and the recorded content data of several video frames as the first display content in the video from xx to yy of the multimedia data stream and the time information of their content changes.

[0112] During the time from yy+ time to zz time of the online activity, for example, during the process of sharing data B, the multimedia data stream generated by the online activity is recorded by another recording method. For example, the other recording method is that when the target program detects the second instruction generated by triggering the second preset operation for the shared data B, the operation trajectory data corresponding to the second preset operation is obtained and incorporated into the first data, and the multimedia data stream is recorded to obtain the audio from yy+ time to zz time of the multimedia data stream generated by the online activity, the shared data B, the operation trajectory data of the shared data and the recorded content data of the content change time information.

[0113] In another embodiment, the target program may also provide the following recording method: during the process of receiving the multimedia data stream, if the target program determines that the display content of the first video of the multimedia data stream has changed, the target program obtains the first data associated with the first display content obtained after the display content has changed, wherein the first data may include: a set of video frames corresponding to the process of the display content of the first video changing and a video frame as the first display content after the change. For example: in response to detecting the first instruction generated by the first member triggering the first preset operation on the first video, a set of video frames of the operation process (scrolling process) of the first preset operation (for example, a scrolling operation) and a video frame as the first display content after the change at the end of the scrolling are obtained from the video clip of the first video currently obtained, and the first audio, the first data and the corresponding content change time information of the multimedia data stream are then associated and stored to generate the recorded content data of the multimedia data stream, wherein the content change time information may be a time range, the start time of which is the start time of the first preset operation and the end time of which is the end time of the first preset operation.

[0114] By means of the above method, it is possible to record the multimedia data stream and obtain the recorded content data of the multimedia data stream with a relatively low data volume. The following is an introduction to the playing process of the above recorded content data.

[0115] like Figure 7 As shown, a flowchart of a method for playing recorded content data is shown. The method can be implemented by a first program. The first program can exist in the form of a server (or a plug-in of the server) or a client (or a plug-in of a client). The first program can be installed in an electronic device, and the electronic device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. Figure 7 As shown, the method includes the following steps S710-S720:

[0116] In step S710, in response to a content playback instruction for recorded content data of a multimedia data stream, first audio in the recorded content data, first data, and playback time information corresponding to the first data are obtained, the first data including data obtained when it is determined that the display content of a first video of the multimedia data stream changes during the process of receiving the multimedia data stream, the first data is associated with a first display content obtained after the display content changes, and the playback time information corresponds to the time when the display content changes.

[0117] The recorded content data of the multimedia data stream is data generated based on the content generation process in the aforementioned embodiment.

[0118] Any user (e.g., target user) who obtains the recorded content data and / or the access address of the recorded content data can access and view the recorded content data. Specifically, upon receiving a content playback instruction for the recorded content data of the multimedia data stream, the first program obtains the first audio, the first data, and the playback time information corresponding to the first data in the recorded content data in response to the content playback instruction. The user who obtains the recorded content data and / or the access address of the recorded content data can include, for example, the aforementioned designated member or other user who is authorized to access the recorded content data.

[0119] Then, in step S720, in the playback interface, the first audio and the first data are played in association based on the playback time information corresponding to the first data. In one implementation, when the first program exists in the form of a server, after obtaining the first audio, the first data and the playback time information corresponding to the first data in the recorded content data, the first audio, the first data and the playback time information corresponding to the first data in the recorded content data can be sent to a related playback client, and the playback client plays the first audio and the first data in association based on the playback time information corresponding to the first data in its playback interface.

[0120] In another implementation, when the first program exists in the form of a client (for example, called a playback client), the playback client responds to a content playback instruction for the recorded content data of the multimedia data stream, obtains the first audio, the first data and the playback time information corresponding to the first data in the recorded content data, and in the playback interface, associates and plays the first audio and the first data based on the playback time information corresponding to the first data.

[0121] In one embodiment, the playback time information corresponding to the first data includes first playback time information. In step S720, the steps may include: when playing the audio frame corresponding to the first playback time information in the first audio, in the playback interface, the data corresponding to the first playback time information in the first data is associated and displayed.

[0122] It can be understood that the first data may include data obtained when it is determined that the display content of the first video of the multimedia data stream changes during the process of receiving the multimedia data stream, the first data is associated with the first display content obtained after the display content changes, and the corresponding first data includes data associated with the first display content obtained after each change in the display content of the first video, each data corresponds to a playback time information corresponding to the time when the display content changes, and each audio frame in the first audio corresponds to a timestamp. When the playback time information corresponding to the first data includes the first playback time information, when the first program plays the audio frame corresponding to the first playback time information in the first audio, the data corresponding to the first playback time information in the first data is associated and displayed on the playback interface to ensure the synchronous playback of the first audio and the first data, thereby improving the user's viewing experience.

[0123] In one embodiment, the first data includes a plurality of target video frames as the first display content, and the play time information corresponding to the first data includes the play time information corresponding to each target video frame.

[0124] Accordingly, in one embodiment, the method may further include the following steps 41-42:

[0125] In step 41, in response to a first play instruction for recorded content data, second play time information corresponding to the target video frame to be played carried by the first play instruction is determined based on the play time information corresponding to each target video frame.

[0126] In step 42, the target video frame to be played is displayed on the play interface, and the audio frame corresponding to the second play time information in the first audio is played in association.

[0127] In order to provide users with better services, the first program can provide users with multiple interactive modes for playing recorded content data. In one implementation, the playback interface can display a control indicating the next video frame to be played (which can be called a next frame playback control) and / or a control indicating the previous video frame to be played (which can be called a previous frame playback control), such as Figure 3C The target user can control which target video frame and its corresponding audio frame in the first audio to be played by triggering the next frame play control or the previous frame play control.

[0128] In yet another implementation, Figure 3C As shown, a control indicating display of a video frame display sequence (which may be referred to as a sequence display control) may also be displayed on the playback interface. After the target user triggers the sequence display control, the first program may display a video frame display sequence including thumbnails of several target video frames on the playback interface, such as Figure 3DAs shown, the target user can select which target video frame and its corresponding audio frame in the first audio to play from the video frame display sequence.

[0129] Specifically, the first playback instruction may be an instruction generated when the first program detects that the target user triggers a third preset operation in the playback interface, wherein the third preset operation may include but is not limited to: triggering the operation of a control in the playback interface that indicates playing the next frame of the video (next frame playback control), or triggering the operation of a control displayed in the playback interface that indicates playing the previous frame of the video (previous frame playback control), or selecting the thumbnail of any target video frame in the video frame display sequence displayed in the playback interface.

[0130] Accordingly, after the first program detects the first play instruction, in response to the first play instruction, based on the play time information corresponding to each target video frame, the first play instruction determines the second play time information corresponding to the target video frame to be played carried by the first play instruction. It can be understood that in one implementation, the first play instruction can carry the video frame identifier of the target video frame to be played, that is, the target video frame to be played. Based on the video frame identifier of the target video frame to be played carried by the first play instruction, and the correspondence between each target video frame (video frame identifier) ​​and the play time, the second play time information corresponding to the target video frame to be played carried by the first play instruction can be determined. Then, the target video frame to be played can be displayed on the play interface, and the audio frame corresponding to the second play time information in the first audio can be associated and played.

[0131] In yet another implementation, Figure 3C As shown, the recorded content data may also include audio transcription text content generated based on the first audio, wherein the audio transcription text content may include the identification of each speaker (e.g., at least some of the members participating in the online activity mentioned above) and their speech content. Correspondingly, the playback interface may also include an area for displaying the audio transcription text content, such as Figures 3C-3F As shown, the display area of ​​the first data and the display area of ​​the audio transcription text content can be distributed left and right.

[0132] In one case, the first program displays the target video frame to be played in the playback interface, and while playing the audio frame corresponding to the second playback time information in the first audio, it can synchronously display the text content corresponding to the audio frame corresponding to the second playback time information in the audio transcription text content in a preset display mode. The preset display mode can be a highlight display or a color change display, etc., which can make the target better pay attention to the display mode of the text content corresponding to the currently displayed target video frame to be played.

[0133] In yet another embodiment, the method may further include the following steps 51-52:

[0134] In step 51, in response to a second play instruction for recorded content data, based on the play time information corresponding to each target video frame, a target video frame corresponding to the third play time information carried by the second play instruction is determined.

[0135] In step 52, on the play interface, the target video frame corresponding to the third play time information is displayed, and the audio frame corresponding to the third play time information in the first audio is played in association.

[0136] In order to provide users with better services, in one implementation, Figure 3C As shown, the playback interface may also display a first playback control for controlling whether to play the first audio and several target video frames. Accordingly, the target user may control whether to play the first audio and several target video frames by triggering the first playback control.

[0137] In yet another implementation, Figure 3C As shown, the playback interface may also display a playback progress bar generated based on the total playback time of the first audio. Accordingly, the target user may control the progress of playing the recorded content data by dragging (and / or clicking) the playback progress bar.

[0138] Specifically, the second playback instruction may be an instruction generated when the first program detects that the target user triggers a fourth preset operation in the playback interface, wherein the fourth preset operation may include but is not limited to: triggering the operation of the aforementioned first playback control in the playback interface, triggering the operation of the aforementioned playback progress bar, and triggering the operation of any transcribed text in the audio transcribed text content displayed on the playback interface.

[0139] Correspondingly, after the first program detects the second playback instruction, it determines the time information carried by the second playback instruction in response to the second playback instruction as the third playback time information, and based on the correspondence between each target video frame and the playback time information, that is, the playback time information corresponding to each target video frame, determines the target video frame corresponding to the third playback time information carried by the second playback instruction; then, on the playback interface, the target video frame corresponding to the third playback time information is displayed, and the audio frame corresponding to the third playback time information in the first audio is played in an associated manner.

[0140] It can be understood that each target video frame corresponds to playback time information, for example, target video frame 1 corresponds to playback time information 1, and target video frame 2 corresponds to playback time information 2, wherein target video frame 2 is the next video frame of target video frame 1. When the third playback time information carried by the second playback instruction is playback time information 1, or the time information between playback time information 1 and playback time information 2, based on the playback time information corresponding to each target video frame, it can be determined that the target video frame corresponding to the third playback time information carried by the second playback instruction is target video frame 1. For another example, the next video frame of target video frame 2 is target video frame 3, wherein target video frame 3 corresponds to playback time information 3. When the third playback time information carried by the second playback instruction is playback time information 2, or the time information between playback time information 2 and playback time information 3, based on the playback time information corresponding to each target video frame, it can be determined that the target video frame corresponding to the third playback time information carried by the second playback instruction is target video frame 2.

[0141] Through the above-mentioned multiple playback interaction methods, whether to play the first audio and several target video frames in the recorded content data, and controlling the playback progress of the first audio and several target video frames in the recorded content data can be controlled to improve the user experience.

[0142] In yet another embodiment, the first data includes operation track data generated when the second preset operation is triggered by the shared data shared in the first video, and the playback time information corresponding to the first data includes playback time information corresponding to the operation track data.

[0143] Correspondingly, in one embodiment, before step S720, the step also includes obtaining shared data in the recorded content data, where the shared data is data associated with the multimedia data stream, that is, data shared in the first video. Then, in step S720, the following steps may be included: when playing an audio frame corresponding to the playback time information in the first audio, in the playback interface, the shared data is displayed based on the operation trajectory data associated with the playback time information.

[0144] Each audio frame in the first audio corresponds to a timestamp (i.e., time information), and each operation track data corresponds to playback time information. Accordingly, each audio frame in the first audio corresponds to each operation track data. When the first program plays the audio frame corresponding to the playback time information in the first audio, the shared data is displayed in association with the operation track data corresponding to the playback time information in the playback interface. For example, the operation track data corresponding to the playback time information 4 includes: a scrolling operation and data indicating the scrolling end position of the scrolling operation in the shared data, such as scrolling up rrPX from the xx position of the shared data. Accordingly, in step S720, when playing the audio frame corresponding to the playback time information 4 in the first audio, the scrolling process of scrolling up rrPX from the xx position of the shared data is displayed in association, and the content of the rrPX position of the shared data is displayed, so as to bring a real online activity experience to the target user.

[0145] In one implementation, the playback interface may display a playback progress bar generated based on the first audio, and the target user may control the playback progress of the first audio and shared data by dragging (and / or clicking) the playback progress bar. In another implementation, the playback interface may display a playback control (i.e., the aforementioned first playback control), and the first playback control may be triggered to control whether to play the first audio and shared data. In another implementation, the playback interface may also display audio transcription text content generated based on the first audio, and the target user may control the playback progress of the corresponding first audio and shared data by selecting any transcribed text in the audio transcription text content.

[0146] In another embodiment, in order to better improve the user experience, the first program also provides a function of freely browsing shared data in the recorded content data. When this function is activated, the target user can actively select the location of the shared data he wants to view, and then play the corresponding audio. In one implementation, when the target user needs to freely browse the shared data, he can trigger the mode control displayed on the playback interface, such as Figure 3E As shown, the mode control is a control for instructing free browsing of shared data. The first program responds to the mode conversion instruction generated by the trigger operation of the mode control and sets the browsing mode of the shared data to the free browsing mode so that the target user can freely browse the shared data.

[0147] In one implementation, the operation track data is used to indicate the operation end position in the shared data. The method may further include the following steps 61-64:

[0148] In step 61, shared data is acquired. And in step 62, in the process of freely browsing the shared data displayed on the playback interface, in response to the audio playback instruction for the recorded content data, based on the current browsing position of the shared data and the operation end position in the shared data indicated by the operation track data, the operation track data matching the current browsing position is determined.

[0149] In the process of freely browsing the shared data, if the first program detects an audio playback instruction (i.e., an instruction to play audio) from the target user for the recorded content data, the first program determines the current browsing position of the shared data in response to the audio playback instruction, and determines the operation track data matching the current browsing position from the operation end position in the shared data indicated by the operation track data. The operation track data matching the current browsing position is, for example, the operation track data whose indicated operation end position is the same as the current browsing position, or the display content corresponding to the indicated operation end position includes the current browsing position, etc.

[0150] In one implementation, the audio playback instruction may be an instruction generated when the target user triggers a designated control in the playback interface during free browsing of shared data displayed on the playback interface, wherein the designated control may be the aforementioned first playback control.

[0151] After the operation track data matching the current browsing position is determined, in step 63, the fourth playback time information corresponding to the current browsing position is determined based on the operation track data matching the current browsing position and the playback time information corresponding to the operation track data. In this step, each operation track data corresponds to playback time information, and based on the playback time information corresponding to each operation track data, the fourth playback time information corresponding to the operation track data matching the current browsing position is determined, that is, the fourth playback time information corresponding to the current browsing position.

[0152] Next, in step 64, the current browsing position of the shared data is displayed on the play interface, and the audio frame corresponding to the fourth play time information in the first audio is played in association.

[0153] In this implementation, during the free browsing of shared data, the corresponding audio frame can be played according to the current browsing position of the shared data to improve the user experience.

[0154] Considering that when demonstrating shared data, the shared data can be demonstrated from front to back, or the shared data can be rolled back. For example: the operation track data includes operation track data 1, operation track data 2 and operation track data 3, wherein the operation track data 1 indicates that at the 5th second (of the first audio), the shared data is scrolled down by 100PX to the mm position of the shared data; the operation track data 2 indicates that at the 200th second (of the first audio) (from the mm position) the shared data is scrolled down by 50PX to the nn position of the shared data; the operation track data 3 indicates that at the 500th second (of the first audio) (from the nn position) the shared data is scrolled up by 50PX to the mm position of the shared data. Correspondingly, the mm position of the shared data can correspond to two playback time information, namely the 5th second and the 500th second. When the current browsing position is the mm position, it can be determined that the mm position corresponds to multiple playback time information. For this case, the multiple playback time information corresponding to the current browsing position can be displayed to the target user, so that the target user can select the audio frame corresponding to which playback time information he needs to play.

[0155] In view of this, in yet another embodiment, step 64 may specifically include the following steps 641-642:

[0156] In step 641, if it is determined that there are multiple fourth play time information corresponding to the current browsing position, the multiple fourth play time information are displayed at a designated position of the play interface.

[0157] In step 642, in response to a selection instruction for target playback time information among multiple fourth playback time information, shared data is displayed in the playback interface based on the current browsing position, and the audio frame corresponding to the target playback time information in the first audio is played in an associated manner.

[0158] In this implementation, if the first program determines that there are multiple fourth playback time information corresponding to the current browsing position, multiple fourth playback time information are displayed at the designated position of the playback interface. Figure 3F As shown, at the specified position of the playback interface (such as Figure 3F ) displays a plurality of fourth play time information corresponding to the current browsing position (such as Figure 3F As shown, the fourth playback time information 1 and the fourth playback time information 2) are provided for the target user to select the audio position that he needs to play. Then, the first program responds to the selection instruction for the target playback time information in the plurality of fourth playback time information, determines the audio frame corresponding to the target playback time information from the first audio, displays the shared data based on the current browsing position in the playback interface, and plays the audio frame corresponding to the target playback time information in the first audio in association.

[0159] In one implementation, during the free browsing of shared data, such as Figure 3F As shown, the playback interface may also display a control "Exit" indicating exiting the process of freely browsing shared data. When the first program detects that the target user triggers the control "Exit" and generates an exit instruction, it may exit the process of freely browsing shared data.

[0160] In another implementation, if it is determined that the fourth playback time information corresponding to the current browsing position is one, the first program directly displays the shared data based on the current browsing position in the playback interface, and associates the audio frame corresponding to the fourth playback time information in the first audio for playback.

[0161] In another implementation, the target user can comment on and / or annotate the shared data while viewing the recorded content data, that is, viewing the shared data and the first audio. Accordingly, the target program can store the comment content and / or annotation content added by the target user to the shared data corresponding to the user identifier of the target user. Subsequently, when the target user views the recorded content data again, the comment content and / or annotation content added by the target user to the shared data can be simultaneously viewed.

[0162] In one implementation, the target user can also set the comment content and / or annotation content added to the shared data to be public, and accordingly, other users who can view the recorded content data can view the comment content and / or annotation content added to the shared data by the target user when viewing the recorded content data. In addition, if the comment content and / or annotation content added to the shared data by the target user is public, other users who can view the shared data can also view the comment content and / or annotation content added to the shared data by the target user.

[0163] The foregoing describes specific embodiments of the present disclosure, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims may be performed in an order different from that in the embodiments, and the desired results may still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible, or may be advantageous.

[0164] Corresponding to the above method embodiment, the present disclosure provides a content generation device 800, whose schematic block diagram is as follows: Figure 8 As shown, including:

[0165] A first acquisition module 810 is configured to acquire first data if it is determined that the display content of the first video of the multimedia data stream has changed during the process of receiving the multimedia data stream, wherein the first data is associated with the first display content obtained after the display content has changed;

[0166] The storage module 820 is configured to store the first audio of the multimedia data stream, the first data and their corresponding content change time information in an associated manner to generate recorded content data of the multimedia data stream.

[0167] In an optional embodiment, the first acquisition module 810 is specifically configured to acquire a first video segment of a first video of the multimedia data stream; in response to detecting a first instruction generated by a first member triggering a first preset operation during an online activity, extract a first video frame corresponding to the first instruction from the first video segment and include it in the first data, wherein the online activity process is the process of generating the multimedia data stream.

[0168] In an optional implementation, the first preset operation includes one or more of the following operations: a browsing operation and an editing operation for the shared data shared in the first video.

[0169] In an optional implementation, it also includes:

[0170] A first receiving module (not shown in the figure), configured to receive a marking instruction for the first video frame, wherein the marking instruction carries the marking content of the first member for the first video frame;

[0171] The storage module 820 is specifically configured to store the first audio, the first data and its corresponding content change time information and the annotated content in an associated manner.

[0172] In an optional implementation, the first data includes each video frame acquired after each display content change;

[0173] The first acquisition module 810 is specifically configured to acquire a second video segment of the first video of the multimedia data stream;

[0174] If it is detected that the time corresponding to the first frame extraction period arrives, extracting the second video frame corresponding to the first frame extraction period from the second video segment;

[0175] Calculating a similarity value between the second video frame and a third video frame, where the third video frame is a video frame obtained after the latest display content change in the first data;

[0176] If the similarity value is lower than a preset similarity threshold, the second video frame is included in the first data.

[0177] In an optional embodiment, the device further comprises:

[0178] The discarding module (not shown in the figure) is configured to discard the second video frame if the similarity value is not lower than a preset similarity threshold.

[0179] In an optional implementation, it also includes:

[0180] A statistics module (not shown in the figure), configured to count the frame loss frequency of discarded video frames within a recent preset time period during the process of receiving the multimedia data stream;

[0181] An updating module (not shown in the figure) is configured to update the first frame extraction period based on the frame loss frequency.

[0182] In an optional implementation, the display content includes at least part of the shared data shared in the first video.

[0183] In an optional implementation, the first acquisition module 810 is specifically configured to acquire the shared data;

[0184] In response to detecting a second instruction generated by a second member triggering a second preset operation on the shared data, obtaining operation track data corresponding to the second preset operation and incorporating the data into the first data;

[0185] The storage module 820 is specifically configured to store the shared data, the first audio, the first data and their corresponding content change time information in an associated manner.

[0186] In an optional implementation, the operation track data includes a second preset operation triggered by the second member with respect to the shared data and / or corresponding operation data generated.

[0187] In an optional implementation, when the second preset operation is an opening operation for the shared data, the operation track data at least includes: an opening operation;

[0188] When the second preset operation is a scrolling operation on the shared data, the operation track data includes: the scrolling operation and data indicating a scrolling end position of the scrolling operation in the shared data;

[0189] When the second preset operation is a scaling operation on the shared data, the operation track data includes: the scaling operation and data indicating a scaling multiple of the shared data by the scaling operation;

[0190] When the second preset operation is a page turning operation for the shared data, the operation track data includes: the page turning operation and data indicating a page turning end position of the page turning operation in the shared data;

[0191] When the second preset operation is a closing operation for the shared data, the operation track data at least includes: the closing operation.

[0192] In an optional implementation manner, the shared data is data in any of the following forms: data in a document form, data in a slide form, and data in an online table form.

[0193] In an optional implementation, it also includes:

[0194] A generating module (not shown in the figure), configured to generate the recorded content data if a recording end instruction for the multimedia data stream is detected;

[0195] The feedback module (not shown in the figure) is configured to feed back the notification message related to the recorded content data to the client of the designated member.

[0196] In an optional implementation manner, the notification message includes an access address of the recorded content data; and the device further includes:

[0197] A module (not shown in the figure) is provided, configured to provide at least part of the audio of the first audio and data corresponding to at least part of the audio in the first data to a corresponding playback client in response to an access instruction for the access address, so that the playback client plays at least part of the audio of the first audio and data corresponding to at least part of the audio in the first data in an associated manner.

[0198] Corresponding to the above method embodiment, the present disclosure provides a playback device 900 for recorded content data, the schematic block diagram of which is shown in FIG. Fig. 9 As shown, including:

[0199] The second acquisition module 910 is configured to, in response to a content playback instruction for recorded content data of a multimedia data stream, acquire a first audio, first data, and playback time information corresponding to the first data in the recorded content data, wherein the first data includes data acquired when it is determined that the display content of a first video of the multimedia data stream changes during the process of receiving the multimedia data stream, the first data is associated with a first display content obtained after the display content changes, and the playback time information corresponds to the time when the display content changes;

[0200] The first playing module 920 is configured to play the first audio and the first data in an associated manner in a playing interface based on the playing time information corresponding to the first data.

[0201] In an optional embodiment, the playback time information corresponding to the first data includes first playback time information, and the first playback module 910 is specifically configured to associate and display the data corresponding to the first playback time information in the first data on the playback interface when playing the audio frame corresponding to the first playback time information in the first audio.

[0202] In an optional implementation, the first data includes several target video frames as the first display content, and the playback time information corresponding to the first data includes playback time information corresponding to each target video frame.

[0203] In an optional implementation, it also includes:

[0204] A first determination module (not shown in the figure) is configured to determine, in response to a first playback instruction for the recorded content data, second playback time information corresponding to the target video frame to be played carried by the first playback instruction based on the playback time information corresponding to each target video frame;

[0205] The second playback module (not shown in the figure) is configured to display the target video frame to be played on the playback interface, and to play the audio frame corresponding to the second playback time information in the first audio in an associated manner.

[0206] In an optional implementation, it also includes:

[0207] A second determination module (not shown in the figure) is configured to determine, in response to a second playback instruction for the recorded content data, a target video frame corresponding to third playback time information carried by the second playback instruction based on the playback time information corresponding to each target video frame;

[0208] The third playback module (not shown in the figure) is configured to display the target video frame corresponding to the third playback time information on the playback interface, and to play the audio frame corresponding to the third playback time information in the first audio in an associated manner.

[0209] In an optional implementation, the first data includes operation track data generated by triggering a second preset operation for the shared data shared in the first video, and the playback time information corresponding to the first data includes playback time information corresponding to the operation track data.

[0210] In an optional implementation, the first playback module is specifically configured to display the shared data in a playback interface based on the operation track data corresponding to the playback time information when playing the audio frame corresponding to the playback time information in the first audio.

[0211] In an optional implementation, the operation track data is used to indicate the operation end position in the shared data: the device further includes:

[0212] A third acquisition module (not shown in the figure), configured to acquire the shared data;

[0213] A third determination module (not shown in the figure) is configured to, in a process of freely browsing the shared data displayed on the playback interface, respond to an audio playback instruction for the recorded content data, determine the operation track data matching the current browsing position based on the current browsing position of the shared data and the operation end position in the shared data indicated by the operation track data;

[0214] a fourth determining module (not shown in the figure), configured to determine fourth playback time information corresponding to the current browsing position based on the operation track data matching the current browsing position and the playback time information corresponding to the operation track data;

[0215] The fourth playback module (not shown in the figure) is configured to display the current browsing position of the shared data on the playback interface, and associate and play the audio frame corresponding to the fourth playback time information in the first audio.

[0216] In an optional implementation, the fourth playback module is specifically configured to display the plurality of fourth playback time information at a designated position of the playback interface if it is determined that the fourth playback time information corresponding to the current browsing position is multiple;

[0217] In response to a selection instruction for the target playback time information among the multiple fourth playback time information, the current browsing position of the shared data is displayed on the playback interface, and the audio frame corresponding to the target playback time information in the first audio is played in an associated manner.

[0218] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0219] Reference below Fig.10 , which shows a schematic structural diagram of an electronic device suitable for implementing an embodiment of the present disclosure. Fig.10The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0220] like Fig.10 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0221] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Fig.10 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Fig.10 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0222] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1009, or installed from a storage device 1008, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0223] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer executes the computer provided in the present disclosure. Figure 2 The content generation method shown, or the execution of the content generation method provided by the present disclosure Figure 7 The method for playing the recorded content data shown.

[0224] It should be noted that the computer-readable medium described in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0225] The computer-readable medium may be included in the electronic device; or it may exist independently and not be installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in the process of receiving the multimedia data stream, if it is determined that the display content of the first video of the multimedia data stream changes, obtain the first data, the first data is associated with the first display content obtained after the display content changes; the first audio of the multimedia data stream, the first data and the corresponding content change time information are stored in association to generate the recorded content data of the multimedia data stream. Or, the electronic device: in response to the content playback instruction for the recorded content data of the multimedia data stream, obtain the first audio, the first data and the playback time information corresponding to the first data in the recorded content data, the first data includes the data obtained when it is determined that the display content of the first video of the multimedia data stream changes during the process of receiving the multimedia data stream, the first data is associated with the first display content obtained after the display content changes, and the playback time information corresponds to the time when the display content changes; in the playback interface, based on the playback time information corresponding to the first data, the first audio and the first data are played in association.

[0226] Computer program code for performing the operations of the disclosed embodiments may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0227] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium and computing device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0228] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0229] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present invention in detail. It should be understood that the above description is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A content generation method, comprising: In the process of receiving the multimedia data stream, if it is determined that the display content of the first video of the multimedia data stream has changed, obtaining first data, wherein the first data is associated with the first display content obtained after the display content has changed; The first audio of the multimedia data stream, the first data and their corresponding content change time information are stored in association to generate recorded content data of the multimedia data stream.

2. The method of claim 1, wherein: If it is determined that the display content of the first video of the multimedia data stream has changed, obtaining the first data includes: Acquire a first video segment of a first video of the multimedia data stream; In response to detecting a first instruction generated by a first member triggering a first preset operation during an online activity, a first video frame corresponding to the first instruction is extracted from the first video clip and included in the first data, wherein the online activity process is a process of generating the multimedia data stream.

3. The method of claim 2, wherein: The first preset operation includes one or more of the following operations: a browsing operation and an editing operation for the shared data shared in the first video.

4. The method of claim 2, further comprising: receiving a marking instruction for the first video frame, wherein the marking instruction carries marking content of the first member for the first video frame; The associated storage of the first audio of the multimedia data stream, the first data and their corresponding content change time information includes: The first audio, the first data, and their corresponding content change time information and the annotated content are stored in association.

5. The method of claim 1, wherein: The first data includes each video frame acquired after each display content changes; If it is determined that the display content of the first video of the multimedia data stream has changed, obtaining the first data includes: Acquire a second video segment of the first video of the multimedia data stream; If it is detected that the time corresponding to the first frame extraction period arrives, extracting from the second video segment a second video frame corresponding to the time corresponding to the first frame extraction period; Calculating a similarity value between the second video frame and a third video frame, where the third video frame is a video frame obtained after the latest display content change in the first data; If the similarity value is lower than a preset similarity threshold, the second video frame is included in the first data.

6. The method of claim 5, further comprising: If the similarity value is not lower than a preset similarity threshold, the second video frame is discarded.

7. The method of claim 6, further comprising: Counting the frame loss frequency of discarded video frames within a recent preset time period during the process of receiving the multimedia data stream; Based on the frame loss frequency, the first frame extraction period is updated.

8. The method of claim 1, wherein: The display content includes at least part of the shared data shared in the first video.

9. The method according to claim 8, wherein if it is determined that the display content of the first video of the multimedia data stream has changed, obtaining the first data comprises: Acquiring the shared data; In response to detecting a second instruction generated by a second member triggering a second preset operation on the shared data, obtaining operation track data corresponding to the second preset operation and incorporating the data into the first data; The associated storage of the first audio of the multimedia data stream, the first data and their corresponding content change time information includes: The shared data, the first audio, the first data and their corresponding content change time information are stored in association.

10. The method of claim 9, wherein: The operation track data includes a second preset operation triggered by the second member with respect to the shared data and / or corresponding operation data generated.

11. The method of claim 10, wherein: When the second preset operation is an opening operation for the shared data, the operation track data at least includes: an opening operation; When the second preset operation is a scrolling operation on the shared data, the operation track data includes: the scrolling operation and data indicating a scrolling end position of the scrolling operation in the shared data; When the second preset operation is a scaling operation on the shared data, the operation track data includes: the scaling operation and data indicating a scaling multiple of the shared data by the scaling operation; When the second preset operation is a page turning operation for the shared data, the operation track data includes: the page turning operation and data indicating a page turning end position of the page turning operation in the shared data; When the second preset operation is a closing operation for the shared data, the operation track data at least includes: the closing operation.

12. The method of claim 2, wherein: The shared data is data in any of the following forms: data in a document form, data in a slide form, and data in an online table form.

13. The method according to any one of claims 1 to 12, further comprising: If a recording end instruction for the multimedia data stream is detected, generating the recording content data; Feedback the notification message related to the recorded content data to the client of the designated member.

14. The method according to claim 13, wherein the notification message includes an access address of the recorded content data; the method further comprises: In response to an access instruction for the access address, at least part of the audio of the first audio and data corresponding to at least part of the audio in the first data are provided to a corresponding playback client, so that the playback client plays at least part of the audio of the first audio and data corresponding to at least part of the audio in the first data in an associated manner.

15. A method for playing recorded content data, comprising: In response to a content playback instruction for recorded content data of a multimedia data stream, obtaining first audio, first data, and playback time information corresponding to the first data in the recorded content data, wherein the first data includes data obtained when it is determined that the display content of a first video of the multimedia data stream changes during the process of receiving the multimedia data stream, the first data is associated with first display content obtained after the display content changes, and the playback time information corresponds to the time when the display content changes; In the playback interface, the first audio and the first data are played in association with each other based on the playback time information corresponding to the first data.

16. The method of claim 15, wherein: The playing time information corresponding to the first data includes first playing time information, and in the playing interface, based on the playing time information corresponding to the first data, associating and playing the first audio and the first data, includes: When playing the audio frame corresponding to the first playback time information in the first audio, on the playback interface, the data corresponding to the first playback time information in the first data is displayed in an associated manner.

17. The method as claimed in claim 15, wherein the first data includes a plurality of target video frames as the first display content, and the play time information corresponding to the first data includes the play time information corresponding to each target video frame.

18. The method of claim 17, further comprising: In response to a first playback instruction for the recorded content data, based on the playback time information corresponding to each target video frame, determine second playback time information corresponding to the target video frame to be played carried by the first playback instruction; On the playback interface, the target video frame to be played is displayed, and the audio frame corresponding to the second playback time information in the first audio is played in an associated manner.

19. The method of claim 17, further comprising: In response to a second playback instruction for the recorded content data, determining a target video frame corresponding to third playback time information carried by the second playback instruction based on the playback time information corresponding to each target video frame; On the playback interface, the target video frame corresponding to the third playback time information is displayed, and the audio frame corresponding to the third playback time information in the first audio is played in an associated manner.

20. The method as claimed in claim 15, wherein the first data includes operation track data generated by triggering a second preset operation for the shared data shared in the first video, and the playback time information corresponding to the first data includes playback time information corresponding to the operation track data.

21. The method of claim 20, wherein: The step of associating and playing the first audio and the first data in the playing interface based on the playing time information corresponding to the first data includes: When playing the audio frame corresponding to the play time information in the first audio, the shared data is displayed in association with the operation track data corresponding to the play time information in the play interface.

22. The method according to claim 20, wherein the operation track data is used to indicate the operation end position in the shared data; the method further comprises: Acquiring the shared data; In a process of freely browsing the shared data displayed on the playback interface, in response to an audio playback instruction for the recorded content data, based on a current browsing position of the shared data and an operation end position in the shared data indicated by the operation track data, determining operation track data matching the current browsing position; Determining fourth playback time information corresponding to the current browsing position based on the operation track data matching the current browsing position and the playback time information corresponding to the operation track data; On the play interface, the current browsing position of the shared data is displayed, and the audio frame corresponding to the fourth play time information in the first audio is played in association.

23. The method of claim 22, wherein displaying the current browsing position of the shared data on the playback interface and associating and playing the audio frame corresponding to the fourth playback time information in the first audio comprises: If it is determined that the current browsing position corresponds to multiple fourth play time information, display the multiple fourth play time information at a designated position on the play interface; In response to a selection instruction for the target playback time information among the multiple fourth playback time information, the shared data is displayed on the playback interface based on the current browsing position, and the audio frame corresponding to the target playback time information in the first audio is played in an associated manner.

24. A content generating device, comprising: A first acquisition module is configured to acquire first data if it is determined that the display content of the first video of the multimedia data stream has changed during the process of receiving the multimedia data stream, wherein the first data is associated with the first display content obtained after the display content has changed; The storage module is configured to associate and store the first audio of the multimedia data stream, the first data and their corresponding content change time information to generate recorded content data of the multimedia data stream.

25. A device for playing recorded content data, comprising: a second acquisition module, configured to, in response to a content playback instruction for recorded content data of a multimedia data stream, acquire a first audio, first data, and playback time information corresponding to the first data in the recorded content data, wherein the first data includes data acquired when it is determined that the display content of a first video of the multimedia data stream changes during the process of receiving the multimedia data stream, the first data is associated with a first display content obtained after the display content changes, and the playback time information corresponds to the time when the display content changes; The first playing module is configured to play the first audio and the first data in an associated manner in a playing interface based on the playing time information corresponding to the first data.

26. An electronic device comprising a memory and a processor, wherein: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 23 is implemented.

Images