Audio and video processing method, device, computer equipment and storage medium

Abstract

The present application relates to an audio and video processing method, device, computer equipment, storage medium and computer program product. The method comprises: obtaining target audio and video, determining a target player authorized to play the target audio and video; determining a data decoding protocol deployed on the target player, and constructing at least one identification frame for the target audio and video based on a data encoding protocol matching the data decoding protocol; determining a target position where the identification frame is allowed to be added in the target audio and video based on the data frame distribution of the target audio and video; adding the identification frame to the target position to obtain the identification audio and video, so that the target player decodes the identification frame and plays the identification audio and video. The use of this method can ensure the safe playback of audio and video.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and in particular to an audio and video processing method, device, computer equipment, storage medium and computer program product. Background Art

[0002] With the rapid development of the Internet, various types of videos have appeared in people's lives. In particular, the emergence of audio and video that can be played in an interactive way with users allows people to give feedback on video content in a timely manner, greatly enriching people's lives.

[0003] However, the audio and video produced by the current audio and video playback technology are universal files that can be played on any platform, and the safe playback of audio and video cannot be guaranteed. Summary of the invention

[0004] Based on this, it is necessary to provide an audio and video processing method, device, computer equipment, computer-readable storage medium and computer program product that can ensure safe audio and video playback in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for processing audio and video. The method comprises:

[0006] Obtain target audio and video, and determine a target player authorized to play the target audio and video;

[0007] Determine a data decoding protocol deployed on the target player, and construct at least one identification frame for the target audio and video based on a data encoding protocol matching the data decoding protocol;

[0008] Based on the data frame distribution of the target audio and video, determining the target position where the identification frame is allowed to be added in the target audio and video;

[0009] The identification frame is added to the target position to obtain identification audio and video, so that the target player can play the identification audio and video after decoding the identification frame.

[0010] In one of the embodiments, determining the target position where the identification frame is allowed to be added in the target audio or video based on the data frame distribution of the target audio or video includes:

[0011] Based on the data frame distribution of the target audio and video, identifying the starting frame of the target audio and video;

[0012] The previous frame position of the start frame of the target audio and video is determined as the target position where the identification frame is allowed to be added in the target audio and video.

[0013] In one of the embodiments, determining the target position where the identification frame is allowed to be added in the target audio or video based on the data frame distribution of the target audio or video includes:

[0014] Based on the data frame distribution of the target audio and video, identifying each component in the target audio and video;

[0015] determining a starting frame of at least one of the components;

[0016] The previous frame position of the starting frame of at least one of the components is determined as the target position where the identification frame is allowed to be added in the target audio and video.

[0017] In one of the embodiments, the identification frame includes a start identification frame and an end identification frame;

[0018] The step of determining a target position where the identification frame is allowed to be added in the target audio or video based on the data frame distribution of the target audio or video includes:

[0019] Determining interactive content and plot content in the target audio and video based on data frame distribution of the target audio and video;

[0020] Determine the previous frame position of the start frame of each of the interactive content and each of the plot content as the target position where the start identification frame is allowed to be added in the target audio and video;

[0021] The next frame position after the end frame of each of the interactive content and each of the plot content is determined as the target position where the end mark frame is allowed to be added in the target audio and video.

[0022] In one embodiment, the target audio and video is an interactive audio and video with interactive logic; the method further includes:

[0023] Based on the interactive logic, establishing an association relationship between each of the end marker frames and at least two of the start marker frames;

[0024] In response to an interactive operation triggered on current interactive content, determining a target end marker frame corresponding to the current interactive content;

[0025] Determining the target start marker frame indicated by the interactive operation from the candidate start marker frames having the association relationship with the target end marker frame;

[0026] Jump to the target start identification frame and continue playing.

[0027] In one embodiment, the method further comprises:

[0028] During the playback of the current interactive content, the interactive operation identification code is displayed;

[0029] In response to the recognition operation for the interactive operation identification code, displaying interactive candidate items;

[0030] When the end mark frame corresponding to the current interactive content is played and no interactive operation triggered for the interactive candidate item is received, the current interactive content is played repeatedly.

[0031] In a second aspect, the present application provides an audio and video processing device, the device comprising:

[0032] A target player determination module, used to obtain target audio and video, and determine a target player authorized to play the target audio and video;

[0033] An identification frame construction module, used to determine a data decoding protocol deployed on the target player, and to construct at least one identification frame for the target audio and video based on a data encoding protocol matching the data decoding protocol;

[0034] A target position determination module, used to determine the target position where the identification frame is allowed to be added in the target audio or video based on the data frame distribution of the target audio or video;

[0035] The identification audio and video acquisition module is used to add the identification frame to the target position to obtain the identification audio and video, so that the target player can play the identification audio and video after decoding the identification frame.

[0036] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0037] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0038] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0039] The above-mentioned audio and video processing method, device, computer equipment, storage medium and computer program product determine the data decoding protocol deployed in the target player authorized to play the target audio and video, and then construct an identification frame according to the corresponding data encoding protocol, so that the constructed identification frame can be decoded by the target player deployed with the corresponding data decoding protocol. By adding the identification frame to the target position in the target audio and video, the obtained identified audio and video can only be played completely in the target player deployed with the corresponding data decoding protocol, thereby achieving safe playback of audio and video. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 FIG. 1 is an application environment diagram of an audio and video processing method in an embodiment;

[0041] Figure 2 1 is a flow chart of a method for processing audio and video in one embodiment;

[0042] Figure 3 is a flowchart of a method for processing audio and video in another embodiment;

[0043] Figure 4 Schematic diagram of the composition structure of audio and video components and key frames in one embodiment;

[0044] Figure 5 is a flowchart of a method for processing audio and video in another embodiment;

[0045] Figure 6 is a structural block diagram of an audio and video processing device in one embodiment;

[0046] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] The audio and video processing method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. First, the server 104 obtains the target audio and video that needs to ensure the safe playback of the video, and determines the target player that plays the target audio and video. The server 104 determines the data encoding protocol corresponding to the data decoding protocol according to the data decoding protocol deployed inside the target player, and then constructs at least one data-encoded identification frame for the target audio and video according to the corresponding data encoding protocol. Then, the server 104 can determine the target position where the identification frame is allowed to be added according to the distribution of each data frame in the target audio and video. Finally, after the server 104 adds the identification frame to the target position in the target video, the identification audio and video can be obtained, and then the identification audio and video can be loaded into the target player that deploys the corresponding data decoding protocol for decoding, so as to play the identification audio and video. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0049] In one embodiment, Figure 2 As shown, a method for processing audio and video is provided, and the method is applied to Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0050] Step 202, obtain the target audio and video, and determine the target player authorized to play the target audio and video.

[0051] Among them, the target audio and video mainly includes interactive audio and video and non-interactive audio and video. For example, audio and video such as movies that do not interact with the audience during the playback process are called non-interactive audio and video. For another example, during the audio and video process, the audience can timely feedback their preferences to the played audio and video, so that the audio and video has different endings according to the audience's feedback, which is an interactive audio and video.

[0052] The target player refers to the playback software or system that the staff who edit the target audio and video or the target audio and video producer hopes to be able to play the target audio and video. For example, if the target audio and video obtained is in format A, then the playback software or system that can play the audio and video in format A is called the target player. For another example, when data frames in encoding format B are added to the target audio and video, the playback software or system that can decode the data frames in the target audio and video is called the target player.

[0053] Optionally, the server obtains interactive or non-interactive target audio and video, and determines the playback software or system that can play the target audio and video with the encoded data frame added according to the needs of the staff who edits the target audio and video or the target audio and video producer.

[0054] Step 204, determine the data decoding protocol deployed in the target player, and construct at least one identification frame for the target audio and video based on the data encoding protocol matching the data decoding protocol.

[0055] Among them, data decoding and data encoding refer to the sender encoding the data packet into a specified data format and sending it according to certain rules, and the receiver decodes it according to the corresponding rules to parse the corresponding data. The protocol represents the rules of encoding and decoding. For example, according to traffic rules, pedestrians walk on the sidewalk and motor vehicles walk on the motor vehicle lane on the road. Traffic rules are similar to protocols. The data encoding protocol and the data decoding protocol are used in conjunction with each other. Only the corresponding data decoding protocol can decode the data encrypted by the data encoding protocol.

[0056] The identification frame is a special data frame that has been data encoded. The frame may not contain image data, but may contain information indicating the producer of the target audio or video or the staff who edits the target audio or video, such as the name of the producer or staff.

[0057] Optionally, after determining the playback software or the data decoding protocol deployed in the system for playing the acquired target audio and video, the server determines the data encoding protocol corresponding to the data decoding protocol according to the codec matching rule. Based on the determined data encoding protocol, the server constructs at least one identification frame for editing the acquired target audio and video.

[0058] Optionally, after determining the playback software or data codec protocol deployed in the system for playing the acquired target audio and video, the server, based on the principle of uniformity, uses the same data codec protocol deployed in the playback software or system to encode data on at least one identification frame to construct an identification frame with a corresponding data format.

[0059] Step 206: Based on the data frame distribution of the target audio and video, determine the target position where the identification frame is allowed to be added in the target audio and video.

[0060] Among them, data frame distribution refers to the distribution of each data frame in the interactive or non-interactive target audio and video. The target position refers to the position between the previous component and the next component in the target audio and video or the head and tail positions of the target audio and video. For example, the interactive content 1 in an interactive target audio and video has 10 data frames, then the position between the 10th data frame and the next plot content or the 1st data frame of the interactive content 2 is the target position where the identification frame is allowed to be added. For another example, a non-interactive target audio and video has a total of 10 data frames, then the position before the 1st data frame or after the 10th data frame is the target position where the identification frame is allowed to be added.

[0061] Optionally, the server can determine the target position where the data-encoded identification frame is allowed to be added in the interactive or non-interactive target audio or video according to the distribution of each data frame in the interactive or non-interactive target audio or video.

[0062] Step 208, adding the identification frame to the target position to obtain the identification audio and video, so that the target player can play the identification audio and video after decoding the identification frame.

[0063] Wherein, the identification audio and video is obtained by adding the identification frame, and the video format of the identification audio and video with the identification frame added is the same as the video format of the target audio and video, and it can also be understood that no matter how many encoded identification frames are added in the target audio and video, the video format of the target audio and video will not change. For example, the target audio and video is in AVI format (Audio Video Interleaved, audio and video interleaved format), then the identification audio and video with the identification frame added is also in AVI format. At the same time, only the playback software and system that deploy the data decoding protocol corresponding to the data encoding protocol that constructs the identification frame can play the identification audio and video completely and clearly. For example, the identification frame obtained using the A data encoding protocol is added to the target position in the target audio and video, and the obtained identification audio and video is loaded into the playback software or system of the data decoding protocol corresponding to the A data encoding protocol, then the identification frame in the identification audio and video cannot be decoded by the playback software or system, and the playback of the identification audio and video can not be realized. For another example, the identification frame obtained using the A data encoding protocol is added to the target position in the target audio and video, and the obtained identification audio and video is loaded into the playback software or system that deploys the data decoding protocol corresponding to the A data encoding protocol. The identification frame in the identification audio and video can be decoded by the playback software or system, thereby realizing the playback of the identification audio and video.

[0064] Optionally, after the server adds the data-encoded identification frame to the target position in the target audio and video, it obtains the identification audio and video that cannot be played by general playback software or system. When the obtained identification audio and video is loaded into the target player that deploys the decoding protocol corresponding to the data encoding protocol for constructing the identification frame, the target player will decode the identification frame in the identification audio and video to play the identification audio and video.

[0065] In the above audio and video processing method, by determining the data decoding protocol deployed in the target player authorized to play the target audio and video, and then constructing the identification frame according to the corresponding data encoding protocol, the constructed identification frame can be decoded by the target player deployed with the corresponding data decoding protocol. By adding the identification frame to the target position in the target audio and video, the obtained identified audio and video can only be played completely in the target player deployed with the corresponding data decoding protocol, thereby achieving safe playback of audio and video.

[0066] In one embodiment, based on the data frame distribution of the target audio and video, determining the target position where the identification frame is allowed to be added in the target audio and video includes:

[0067] Based on the data frame distribution of the target audio and video, the start frame of the target audio and video is identified.

[0068] The previous frame position of the start frame of the target audio or video is determined as the target position where the identification frame is allowed to be added in the target audio or video.

[0069] The start frame refers to the first data frame in the target audio or video. For example, if the target audio or video has 20 data frames, the first data frame in the 20 data frames is the start frame of the target audio or video.

[0070] Optionally, after obtaining the distribution position of each data frame in the target audio and video, the server identifies the starting frame or the last data frame in the target audio and video from the distribution position of the data frame. After the server identifies the starting or last data frame, it adds an identification frame at the previous frame position of the starting frame or the next frame position of the last data frame, that is, the target position where the identification frame is allowed to be added.

[0071] Optionally, after obtaining the distribution position of each data frame in the target audio and video, the server selects at least one intermediate frame, determines the previous frame position or the next frame position of the selected intermediate frame as the target position where the identification frame is allowed to be added, and adds the identification frame.

[0072] Optionally, after obtaining the distribution position of each data frame in the target audio and video, the server adds a marker frame to the target position where two or three of the start frame, the last data frame, and the intermediate frames in the target audio and video are allowed to add marker frames.

[0073] In this embodiment, by adding a data-encoded identification frame at the frame position before the starting frame in the target audio and video, the identification frame added in the target audio and video can only be decoded by a device deployed with a corresponding data decoding protocol, thereby ensuring the security of the target audio and video.

[0074] In one embodiment, based on the data frame distribution of the target audio and video, determining the target position where the identification frame is allowed to be added in the target audio and video includes:

[0075] Based on the data frame distribution of the target audio and video, the components of the target audio and video are identified.

[0076] A start frame of at least one component is determined.

[0077] The previous frame position of the start frame of at least one component is determined as the target position where the identification frame is allowed to be added in the target audio and video.

[0078] The components of the target audio and video can be understood as multiple segments with a succession relationship. For example, in a 3-minute target audio and video, the video content played in the first 1.5 minutes describes event A, and the video content played in the last 1.5 minutes describes events A and B, which means that there are two components with a succession relationship in the target audio and video.

[0079] Optionally, the server can identify the segments in the target audio and video that describe different plots or have a connection relationship according to the distribution of each data frame in the target audio and video. The server then obtains the first data frame or the last data frame of at least one component, and adds a marker frame at the previous frame position of the first data frame or the next frame position of the last data frame, that is, the target position where the marker frame is allowed to be added in the target audio and video.

[0080] Optionally, after obtaining the distribution position of each data frame in the target audio and video and determining the components of the target audio and video, the server selects an intermediate frame of at least one component, determines the previous frame position or the next frame position of the selected intermediate frame as the target position where the identification frame is allowed to be added, and adds the identification frame.

[0081] Optionally, after obtaining the distribution position of each data frame in the target audio and video and determining the components of the target audio and video, the server determines the target position where the addition of identification frames is allowed to belong to the starting frame, the last data frame, and two or three of the intermediate frames in at least one component, and adds the identification frame at the target position.

[0082] In this embodiment, by adding a marker frame at the start frame position of at least one component, the workload can be appropriately reduced while ensuring the safe playback of the target audio and video.

[0083] In one embodiment, Figure 3 As shown, the identification frame includes a start identification frame and an end identification frame.

[0084] Based on the data frame distribution of the target audio and video, determine the target position where the identification frame is allowed to be added in the target audio and video, including:

[0085] Step 302, based on the data frame distribution of the target audio and video, determine the interactive content and plot content in the target audio and video.

[0086] The interactive content is the part of the target audio or video where the user can make a choice based on his or her own judgment. In the interactive content, the endings of the target audio or video are different based on the different choices made by the user. For example, there are two choices in interactive content 1, and choice A and choice B correspond to two different endings. If the user chooses A, the ending corresponding to choice A will be played, and if the user chooses B, the ending corresponding to choice B will be played.

[0087] Plot content refers to the description of events or other components that do not have user interactive selection functions. For example, the ending part that does not have user interactive selection functions corresponding to the two choices A and B in interactive content 1 can be understood as plot content.

[0088] Optionally, based on the distribution of each data frame acquired about the target audio and video, the server divides each component of the target audio and video into interactive content that the user can select and plot content that only describes events or has no user selection function.

[0089] Step 304: determine the previous frame position of the start frame of each interactive content and each plot content as the target position where the start identification frame is allowed to be added in the target audio and video.

[0090] Optionally, the server determines the previous frame position of the first data frame of each interactive content and plot content in the target audio and video as the target position where the data-encoded start identification frame is allowed to be added in the target audio and video.

[0091] Step 306: determine the position of the frame after the end frame of each interactive content and each plot content as the target position where the end mark frame is allowed to be added in the target audio and video.

[0092] Optionally, the server determines the next frame position of the last data frame of each interactive content and plot content in the target audio and video as the target position where the data-encoded end marker frame is allowed to be added in the target audio and video.

[0093] In this embodiment, by determining the previous frame position of the first data frame and the next frame position of the last frame of each interactive content and plot content of the target audio and video as the target position where the identification frame is allowed to be added, the target audio and video cannot be played in a general player even if some fragments are copied, thereby effectively protecting the security of the video.

[0094] In one embodiment, the target audio and video is an interactive audio and video with interactive logic. The audio and video processing method also includes:

[0095] Based on the interactive logic, an association relationship between each end marker frame and at least two start marker frames is established.

[0096] In response to an interactive operation triggered for the current interactive content, a target end marker frame corresponding to the current interactive content is determined.

[0097] The target start marker frame indicated by the interactive operation is determined from the candidate start marker frames associated with the target end marker frame.

[0098] Jump to the target start marker frame and continue playing.

[0099] The interactive logic represents the connection between different choices of each interactive content in the target audio and video and the corresponding plot content or interactive content. For example, in interactive content 1, choice A corresponds to plot content 1-1, choice B corresponds to plot content 1-2, and choice C corresponds to plot content 1-3. If the user chooses A, plot content 1-1 will be played after interactive content 1, instead of plot content 1-2 or plot content 1-3.

[0100] The interactive operation refers to the operation in which the user makes a selection after watching the interactive content. The target end marker frame refers to the end marker frame added to the next frame position of the end frame of the interactive content that the user is watching when making the selection. For example, when the user is watching interactive content 1, the end marker frame of interactive content 1 is the target end marker frame.

[0101] The candidate start identification frame can be understood as the start identification frame added to the previous frame position of the start frame of each plot content or interactive content corresponding to different selections in the interactive content. For example, the three selections of interactive content 1 correspond to plot content 1-1, plot content 1-2, and plot content 1-3, respectively, and the start identification frame added to the previous frame position of the start frames of plot content 1-1, plot content 1-2, and plot content 1-3 is the candidate start identification frame.

[0102] The target start identification frame indicates the start identification frame of the frame before the start frame of the plot content corresponding to the selection made by the user after watching the interactive content. For example, if the user selects option A among the three options of the interactive content, the start identification frame of the frame before the start frame of the plot content corresponding to option A is the target start identification frame.

[0103] Optionally, the server associates the end identification frame of each interactive content or plot content in the target audio and video with the start identification frames of at least two interactive content or plot content according to the interaction logic between the interactive content and the plot content in the target audio and video. After receiving the interactive operation triggered by the user for the currently viewed interactive content, the server first determines the target end identification frame of the currently viewed interactive content, and determines the candidate start identification frame according to the established association relationship. Finally, according to the selection made in the user's interactive operation, the target start identification frame indicated by the user's interactive operation is determined from the candidate start identification frames, and the corresponding content is played.

[0104] In this embodiment, by establishing an association relationship between the end marker frame and the start marker frame, the user can accurately determine the target start marker frame indicated by the interactive operation after performing the interactive operation.

[0105] In one embodiment, the audio and video processing method further includes:

[0106] During the playback of the current interactive content, the interactive operation identification code is displayed.

[0107] In response to the recognition operation for the interaction operation identification code, interaction candidate items are displayed.

[0108] When the end mark frame corresponding to the current interactive content is played and no interactive operation triggered for the interactive candidate item is received, the current interactive content is played repeatedly.

[0109] The interactive operation identification code refers to the place where the user makes interactive selections. The interactive operation identification code can be a QR code. The user can see all the options of the current interactive content by scanning the QR code with a mobile phone. For example, when a user is watching a certain interactive content, the interactive operation identification code is always displayed on the interface of the interactive content. When the user confirms the selection, the user uses the mobile phone to scan the interactive operation identification code to see the interactive candidates of the currently playing interactive content, and clicks on one of the interactive candidates to trigger the interactive operation. The interactive operation identification code can also be multiple buttons in the interactive content used to respond to user interactive operations.

[0110] In this embodiment, by always displaying the interactive operation identification code during the playback of the current interactive content, the user can trigger the interactive operation in time to make the selection when the user determines the selection in the middle of watching the interactive content, without having to wait until the current interactive content is finished to trigger the interactive operation. By repeatedly playing the current interactive content based on the end identification frame of the current interactive content when no triggered interactive operation is received, the user does not have to manually perform repeated playback, thereby reducing the user's workload.

[0111] The present application also provides an application scenario, which applies the above-mentioned audio and video processing method. Specifically, the application of the audio and video processing method in this application scenario is as follows: First, the video producer determines an interactive audio and video that needs to be safely played, and loads the determined interactive audio and video into an editing device for editing and processing. Based on the data decoding protocol deployed in the playback device for playing the edited interactive audio and video, the data encoding protocol required to construct the identification frame is determined, and then the interactive audio and video is divided into multiple interactive contents and multiple plot contents by obtaining the distribution of each data frame in the interactive audio and video. The previous frame position of the start frame of each interactive content and each plot content is determined as the target position of the start identification frame, and the next frame position of the end frame of each interactive content and each plot content is determined as the target position of the end identification frame. The composition structure of each component and the identification frame is as follows. Figure 4 After the identification frame is added, the obtained identification audio and video is saved and loaded into the corresponding playback device.

[0112] After the identification audio and video are loaded into the playback device, the playback device will scan each added identification frame information, and establish an association relationship between each identification frame based on the interactive logic of the interactive audio and video. After the association relationship is established, it connects to the user interaction device and starts playing the video. When no interactive operation is received from the user through the user interaction device in the interactive content part, the current interactive content is played in a loop. When an interactive operation is received from the user through the user interaction device in the interactive content part, based on the triggered interactive operation, the plot content indicated by the interactive operation or the starting identification frame of the interactive content is determined and the corresponding content is played. After the audio and video are played, the connection with the user interaction device is disconnected. The flow chart of the audio and video processing method is shown as follows. Figure 5 shown.

[0113] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0114] Based on the same inventive concept, the embodiment of the present application also provides an audio and video processing device for implementing the audio and video processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more audio and video processing device embodiments provided below can refer to the limitations of the audio and video processing method above, and will not be repeated here.

[0115] In one embodiment, Figure 6 As shown, an audio and video processing device is provided, comprising:

[0116] The target player determination module 602 is used to obtain the target audio and video and determine the target player authorized to play the target audio and video.

[0117] The identification frame construction module 604 is used to determine the data decoding protocol deployed in the target player, and construct at least one identification frame for the target audio and video based on the data encoding protocol matching the data decoding protocol.

[0118] The target position determination module 606 is used to determine the target position where the identification frame is allowed to be added in the target audio or video based on the data frame distribution of the target audio or video.

[0119] The identification audio and video acquisition module 608 is used to add the identification frame to the target position to obtain the identification audio and video, so that the target player can play the identification audio and video after decoding the identification frame.

[0120] In one embodiment, the target location determination module includes:

[0121] The first start frame identification unit is used to identify the start frame of the target audio or video based on the data frame distribution of the target audio or video.

[0122] The first target position determination unit is used to determine the previous frame position of the start frame of the target audio or video as the target position where the identification frame is allowed to be added in the target audio or video.

[0123] In one of the embodiments, the target location determination module further includes:

[0124] The component determination unit is used to identify the components in the target audio and video based on the data frame distribution of the target audio and video.

[0125] The second start frame identification unit is used to determine the start frame of at least one component.

[0126] The second target position determination unit is used to determine the previous frame position of the start frame of at least one component as the target position where the identification frame is allowed to be added in the target audio and video.

[0127] In one of the embodiments, the target location determination module further includes:

[0128] The content determination unit is used to determine the interactive content and plot content in the target audio and video based on the data frame distribution of the target audio and video.

[0129] The start identification frame position determination unit is used to determine the previous frame position of the start frame of each interactive content and each plot content as the target position where the start identification frame is allowed to be added in the target audio and video.

[0130] The end marker frame position determination unit is used to determine the next frame position of the end frame of each interactive content and each plot content as the target position where the end marker frame is allowed to be added in the target audio and video.

[0131] In one embodiment, the audio and video processing device further includes:

[0132] The association relationship building unit is used to establish an association relationship between each end marker frame and at least two start marker frames based on the interactive logic.

[0133] The target end marker frame determining unit is used to determine the target end marker frame corresponding to the current interactive content in response to the interactive operation triggered for the current interactive content.

[0134] The target start marker frame determination unit is used to determine the target start marker frame indicated by the interactive operation from the candidate start marker frames associated with the target end marker frame.

[0135] The video playback unit is used to jump to the target start identification frame and continue playing.

[0136] In one embodiment, the audio and video processing device further includes:

[0137] The interactive operation identification code display unit is used to display the interactive operation identification code during the process of playing the current interactive content.

[0138] The interactive candidate item display unit is used to display the interactive candidate items in response to the recognition operation for the interactive operation identification code.

[0139] The repeat playback unit is used to repeat playback of the current interactive content when the playback reaches the end mark frame corresponding to the current interactive content and no interactive operation triggered for the interactive candidate item is received.

[0140] Each module in the above-mentioned audio and video processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0141] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store target audio and video, data decoding protocol, data encoding protocol, identification frame, data frame distribution of target audio and video, target position where the identification frame is allowed to be added in the target audio and video, identification audio and video, interactive logic of target audio and video, association relationship between each end identification frame and at least two start identification frames, interactive operation, interactive operation identification code, and interactive candidate data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by a processor, an audio and video processing method is implemented.

[0142] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0143] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0144] Obtain the target audio and video, and determine the target player authorized to play the target audio and video. Determine the data decoding protocol deployed on the target player, and construct at least one identification frame for the target audio and video based on the data encoding protocol that matches the data decoding protocol. Based on the data frame distribution of the target audio and video, determine the target position where the identification frame is allowed to be added in the target audio and video. Add the identification frame to the target position to obtain the identification audio and video, so that the target player decodes the identification frame and plays the identification audio and video.

[0145] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0146] Based on the data frame distribution of the target audio and video, the start frame of the target audio and video is identified.

[0147] The previous frame position of the start frame of the target audio or video is determined as the target position where the identification frame is allowed to be added in the target audio or video.

[0148] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0149] Based on the data frame distribution of the target audio and video, the components in the target audio and video are identified. The starting frame of at least one component is determined. The previous frame position of the starting frame of at least one component is determined as the target position where the identification frame is allowed to be added in the target audio and video.

[0150] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0151] Based on the data frame distribution of the target audio and video, the interactive content and plot content in the target audio and video are determined. The previous frame position of the start frame of each interactive content and each plot content is determined as the target position where the start mark frame is allowed to be added in the target audio and video. The next frame position of the end frame of each interactive content and each plot content is determined as the target position where the end mark frame is allowed to be added in the target audio and video.

[0152] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0153] Based on the interactive logic, an association relationship is established between each end identification frame and at least two start identification frames. In response to an interactive operation triggered for the current interactive content, a target end identification frame corresponding to the current interactive content is determined. From the candidate start identification frames that have an association relationship with the target end identification frame, the target start identification frame indicated by the interactive operation is determined. Jump to the target start identification frame to continue playing.

[0154] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0155] During the playback of the current interactive content, the interactive operation identification code is displayed. In response to the recognition operation of the interactive operation identification code, the interactive candidate items are displayed. When the playback reaches the end mark frame corresponding to the current interactive content and no interactive operation triggered for the interactive candidate items is received, the current interactive content is played repeatedly.

[0156] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0157] Obtain the target audio and video, and determine the target player authorized to play the target audio and video. Determine the data decoding protocol deployed on the target player, and construct at least one identification frame for the target audio and video based on the data encoding protocol that matches the data decoding protocol. Based on the data frame distribution of the target audio and video, determine the target position where the identification frame is allowed to be added in the target audio and video. Add the identification frame to the target position to obtain the identification audio and video, so that the target player decodes the identification frame and plays the identification audio and video.

[0158] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0159] Based on the data frame distribution of the target audio and video, the start frame of the target audio and video is identified.

[0160] The previous frame position of the start frame of the target audio or video is determined as the target position where the identification frame is allowed to be added in the target audio or video.

[0161] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0162] Based on the data frame distribution of the target audio and video, the components in the target audio and video are identified. The starting frame of at least one component is determined. The previous frame position of the starting frame of at least one component is determined as the target position where the identification frame is allowed to be added in the target audio and video.

[0163] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0164] Based on the data frame distribution of the target audio and video, the interactive content and plot content in the target audio and video are determined. The previous frame position of the start frame of each interactive content and each plot content is determined as the target position where the start mark frame is allowed to be added in the target audio and video. The next frame position of the end frame of each interactive content and each plot content is determined as the target position where the end mark frame is allowed to be added in the target audio and video.

[0165] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0166] Based on the interactive logic, an association relationship is established between each end identification frame and at least two start identification frames. In response to an interactive operation triggered for the current interactive content, a target end identification frame corresponding to the current interactive content is determined. From the candidate start identification frames that have an association relationship with the target end identification frame, the target start identification frame indicated by the interactive operation is determined. Jump to the target start identification frame to continue playing.

[0167] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0168] During the playback of the current interactive content, the interactive operation identification code is displayed. In response to the recognition operation of the interactive operation identification code, the interactive candidate items are displayed. When the playback reaches the end mark frame corresponding to the current interactive content and no interactive operation triggered for the interactive candidate items is received, the current interactive content is played repeatedly.

[0169] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0170] Obtain the target audio and video, and determine the target player authorized to play the target audio and video. Determine the data decoding protocol deployed on the target player, and construct at least one identification frame for the target audio and video based on the data encoding protocol that matches the data decoding protocol. Based on the data frame distribution of the target audio and video, determine the target position where the identification frame is allowed to be added in the target audio and video. Add the identification frame to the target position to obtain the identification audio and video, so that the target player decodes the identification frame and plays the identification audio and video.

[0171] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0172] Based on the data frame distribution of the target audio and video, the start frame of the target audio and video is identified.

[0173] The previous frame position of the start frame of the target audio or video is determined as the target position where the identification frame is allowed to be added in the target audio or video.

[0174] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0175] Based on the data frame distribution of the target audio and video, the components in the target audio and video are identified. The starting frame of at least one component is determined. The previous frame position of the starting frame of at least one component is determined as the target position where the identification frame is allowed to be added in the target audio and video.

[0176] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0177] Based on the data frame distribution of the target audio and video, the interactive content and plot content in the target audio and video are determined. The previous frame position of the start frame of each interactive content and each plot content is determined as the target position where the start mark frame is allowed to be added in the target audio and video. The next frame position of the end frame of each interactive content and each plot content is determined as the target position where the end mark frame is allowed to be added in the target audio and video.

[0178] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0179] Based on the interactive logic, an association relationship is established between each end identification frame and at least two start identification frames. In response to an interactive operation triggered for the current interactive content, a target end identification frame corresponding to the current interactive content is determined. From the candidate start identification frames that have an association relationship with the target end identification frame, the target start identification frame indicated by the interactive operation is determined. Jump to the target start identification frame to continue playing.

[0180] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0181] During the playback of the current interactive content, the interactive operation identification code is displayed. In response to the recognition operation of the interactive operation identification code, the interactive candidate items are displayed. When the playback reaches the end mark frame corresponding to the current interactive content and no interactive operation triggered for the interactive candidate items is received, the current interactive content is played repeatedly.

[0182] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0183] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0184] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0185] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for processing audio and video, characterized in that: The method comprises: Obtain target audio and video, and determine a target player authorized to play the target audio and video; Determine a data decoding protocol deployed on the target player, and construct at least one identification frame for the target audio and video based on a data encoding protocol matching the data decoding protocol; Based on the data frame distribution of the target audio and video, determining the target position where the identification frame is allowed to be added in the target audio and video; the data frame distribution refers to the distribution of each data frame in the target audio and video; The identification frame is added to the target position to obtain identification audio and video, so that the target player can play the identification audio and video after decoding the identification frame.

2. The method according to claim 1, characterized in that The step of determining a target position where the identification frame is allowed to be added in the target audio or video based on the data frame distribution of the target audio or video includes: Based on the data frame distribution of the target audio and video, identifying the starting frame of the target audio and video; The previous frame position of the start frame of the target audio and video is determined as the target position where the identification frame is allowed to be added in the target audio and video.

3. The method according to claim 1, characterized in that The step of determining a target position where the identification frame is allowed to be added in the target audio or video based on the data frame distribution of the target audio or video includes: Based on the data frame distribution of the target audio and video, identifying each component in the target audio and video; determining a starting frame of at least one of the components; The previous frame position of the starting frame of at least one of the components is determined as the target position where the identification frame is allowed to be added in the target audio and video.

4. The method according to claim 1, characterized in that: The identification frame includes a start identification frame and an end identification frame; The step of determining a target position where the identification frame is allowed to be added in the target audio or video based on the data frame distribution of the target audio or video includes: Determining interactive content and plot content in the target audio and video based on data frame distribution of the target audio and video; Determine the previous frame position of the start frame of each of the interactive content and each of the plot content as the target position where the start identification frame is allowed to be added in the target audio and video; The next frame position after the end frame of each of the interactive content and each of the plot content is determined as the target position where the end mark frame is allowed to be added in the target audio and video.

5. The method according to claim 4, characterized in that The target audio and video is an interactive audio and video with interactive logic, and the interactive logic represents the connection between different selections of each interactive content in the target audio and video and the corresponding plot content or interactive content; The method further comprises: Based on the interactive logic, establishing an association relationship between each of the end marker frames and at least two of the start marker frames; In response to an interactive operation triggered on current interactive content, determining a target end marker frame corresponding to the current interactive content; Determining the target start marker frame indicated by the interactive operation from the candidate start marker frames having the association relationship with the target end marker frame; Jump to the target start identification frame and continue playing.

6. The method according to claim 5, characterized in that The method further comprises: During the playback of the current interactive content, the interactive operation identification code is displayed; In response to the recognition operation for the interactive operation identification code, displaying interactive candidate items; When the end mark frame corresponding to the current interactive content is played and no interactive operation triggered for the interactive candidate item is received, the current interactive content is played repeatedly.

7. An audio and video processing device, characterized in that: The device comprises: A target player determination module, used to obtain target audio and video, and determine a target player authorized to play the target audio and video; An identification frame construction module, used to determine a data decoding protocol deployed on the target player, and to construct at least one identification frame for the target audio and video based on a data encoding protocol matching the data decoding protocol; A target position determination module, used to determine the target position where the identification frame is allowed to be added in the target audio and video based on the data frame distribution of the target audio and video; the data frame distribution refers to the distribution of each data frame in the target audio and video; The identification audio and video acquisition module is used to add the identification frame to the target position to obtain the identification audio and video, so that the target player can play the identification audio and video after decoding the identification frame.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images