Flow pushing method and device, electronic equipment and storage medium

Abstract

The invention provides a flow pushing method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the steps that a semantic triple corresponding to a first key content mark on a progress bar of a target video is determined, the first key content mark corresponds to a video segment where first key content in the target video is located, and the semantic triple is determined according to the first key content; obtaining an object identifier of an object watching the target video; and inputting the object identifier and the semantic triple into a pre-trained target multiple speed prediction model to obtain a first prediction multiple speed rate used by an object corresponding to the object identifier on the first key content mark, and pushing a video clip to the object corresponding to the object identifier according to the first prediction multiple speed rate. Therefore, personalized active flow pushing is realized, and the intelligent service level in the flow pushing scene is improved.

Description

Streaming methods, devices, electronic equipment, and storage media Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to a streaming method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, in video streaming scenarios, when a video streaming server receives a playback speed request from a client, it typically determines the video segment to be played and its corresponding playback speed based on the request, and then pushes the corresponding video data to the client according to the playback speed. However, video streaming servers usually passively push streams based on playback speed requests, lacking personalized dynamic proactive streaming capabilities, which limits the level of intelligent services in video streaming scenarios. Summary of the Invention

[0003] This disclosure provides a streaming method, apparatus, electronic device, and storage medium.

[0004] In a first aspect, this disclosure provides a streaming method, the method comprising: determining a semantic triple corresponding to a first key content marker on the progress bar of a target video, wherein the first key content marker corresponds to a video segment in the target video where its first key content is located, and the semantic triple is determined based on the first key content; obtaining an object identifier of an object currently watching the target video; inputting the object identifier and the semantic triple into a pre-trained target speed prediction model to obtain a first predicted speed rate used by the object corresponding to the object identifier on the first key content marker; and pushing the video segment to the object corresponding to the object identifier according to the first predicted speed rate.

[0005] Secondly, this disclosure provides a streaming device, the device comprising: a first determining module, configured to determine a semantic triple corresponding to a first key content marker on the progress bar of a target video, wherein the first key content marker corresponds to a video segment in the target video where its first key content is located, and the semantic triple is determined based on the first key content; an acquiring module, configured to acquire an object identifier of an object watching the target video; a second determining module, configured to input the object identifier and the semantic triple into a pre-trained target speed prediction model to obtain a first predicted speed rate used by the object corresponding to the object identifier on the first key content marker; and a sending module, configured to push the video segment to the object corresponding to the object identifier according to the first predicted speed rate.

[0006] Thirdly, this disclosure provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the streaming method disclosed in the embodiments of this disclosure.

[0007] Fourthly, this disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the streaming method disclosed in the embodiments of this disclosure.

[0008] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the streaming method disclosed in the embodiments of this disclosure.

[0009] The technical solution provided by the embodiments of this disclosure brings at least the following beneficial effects: It determines the semantic triple corresponding to the first key content marker on the progress bar of the target video, wherein the first key content marker corresponds to the video segment in the target video where its first key content is located, and the semantic triple is determined based on the first key content; it obtains the object identifier of the object watching the target video; it inputs the object identifier and the semantic triple into a pre-trained target speed prediction model to obtain the first predicted speed rate used by the object corresponding to the object identifier on the first key content marker; and it pushes the video segment to the object corresponding to the object identifier according to the first predicted speed rate. Therefore, it can proactively determine the first predicted speed rate used by the object corresponding to the object identifier on the first key content marker, and push the video segment corresponding to the first key content marker to the object corresponding to the object identifier according to the first predicted speed rate, realizing personalized proactive streaming and improving the intelligent service level in streaming scenarios. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0011] Figure 1 is a flowchart of a streaming method according to an exemplary embodiment; Figure 2 is a flowchart of a streaming method according to another exemplary embodiment; Figure 3 is an example diagram of a video playback interface; Figure 4 is a structural example diagram of a target speed prediction model; Figure 5 is a flowchart of a streaming method according to another exemplary embodiment; Figure 6 is an interaction flowchart between a client and a streaming server; Figure 7 is a structural schematic diagram of a streaming device according to an exemplary embodiment; Figure 8 is a structural block diagram of an electronic device according to an exemplary embodiment.

[0012] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0013] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0014] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of any type of information, such as user personal information, in the technical solutions disclosed herein are all carried out with the user's consent and comply with relevant laws and regulations, and do not violate public order and good morals.

[0015] The technical solutions of this disclosure and how they solve the aforementioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0016] First, referring to Figure 1, the streaming method provided in the embodiments of this disclosure will be described by way of example.

[0017] Figure 1 is a flowchart illustrating a streaming method according to an exemplary embodiment.

[0018] It should be noted that the streaming method provided in this embodiment can be executed by a streaming device, which can be implemented by software and / or hardware. The streaming device can be an electronic device, or it can be configured in an electronic device to enable the electronic device to have video streaming functionality.

[0019] The electronic device can be any device with video streaming capabilities, such as a video streaming server. This embodiment does not specifically limit the electronic device.

[0020] As shown in Figure 1, the streaming method includes the following steps: Step 101, determine the semantic triple corresponding to the first key content marker on the progress bar of the target video, wherein the first key content marker corresponds to the video segment in the target video where its first key content is located, and the semantic triple is determined based on the first key content.

[0021] The first key content refers to the most important, informational, or core-theme-conveying elements in the target video, which are usually manifested as specific visual objects, actions, events, voice, text, or combinations thereof.

[0022] The target video can be any video, and this embodiment does not specifically limit it.

[0023] Step 102: Obtain the object identifier of the object that is watching the target video.

[0024] Among them, the object identifier is the information that uniquely identifies the object.

[0025] It should be noted that this embodiment does not limit the execution order of steps 101 and 102. For example, step 101 can be executed first, and then step 102 can be executed, or step 102 can be executed first, and then step 101 can be executed, or steps 101 and 102 can be executed simultaneously.

[0026] Step 103: Input the object identifier and semantic triple into the pre-trained target speed prediction model to obtain the first prediction speed rate used by the object corresponding to the object identifier on the first key content tag.

[0027] It should be noted that the above-mentioned target speed prediction model is obtained by training an initial speed prediction model based on the object identifier, the sample semantic triple corresponding to the second key content marker on the progress bar of the sample video, and the sample speed rate used by the object corresponding to the object on the second key content marker.

[0028] The second key content marker corresponds to the sample video segment in which the second key content is located.

[0029] The second key content refers to the most important, informationally valuable, or core-theme-conveying element in the sample video, typically manifested as a specific visual object, action, event, audio, text, or a combination thereof. It should be noted that the second key content and the first key content can be the same or different; this embodiment does not impose specific limitations on this.

[0030] The sample semantic triples are determined based on the second key content.

[0031] Step 104: Based on the first predicted speed multiplier, push the video clip to the object corresponding to the object identifier.

[0032] In some embodiments, after obtaining the first predicted speed rate, the push speed of pushing the video clip to the object corresponding to the object identifier can be adjusted according to the first predicted speed rate.

[0033] In some embodiments, the current geographical location corresponding to the object identifier is obtained, and multiple candidate CDN nodes corresponding to the current geographical location are obtained from the Content Delivery Network (CDN). From these candidate CDN nodes, a target CDN node with the lowest load is determined, and a first instruction is sent to the target CDN node. The first instruction instructs the target CDN node to push the video segment to the object corresponding to the object identifier according to a first predicted speed multiplier. This helps improve the video viewing experience.

[0034] It should be noted that the target CDN node mentioned above stores the target video. As an example, if the target video is not present in the target CDN node, the target CDN node will retrieve the target video from the target video provider and save it.

[0035] The streaming method provided in this disclosure determines the semantic triple corresponding to the first key content marker on the progress bar of the target video. The first key content marker corresponds to the video segment containing the first key content in the target video, and the semantic triple is determined based on the first key content. The method then obtains the object identifier of the object currently watching the target video; inputs the object identifier and the semantic triple into a pre-trained target speed prediction model to obtain the first predicted speed rate used by the object corresponding to the object identifier on the first key content marker; and pushes the video segment to the object corresponding to the object identifier based on the first predicted speed rate. Therefore, it can proactively determine the first predicted speed rate used by the object corresponding to the object identifier on the first key content marker and push the video segment corresponding to the first key content marker to the object corresponding to the object identifier based on the first predicted speed rate, achieving personalized proactive streaming and improving the intelligent service level in streaming scenarios.

[0036] Figure 2 is a flowchart illustrating a push-stream method according to another exemplary embodiment.

[0037] As shown in Figure 2, the process may include: Step 201, obtaining the audio data of the target video.

[0038] Step 202: Based on the audio data and the target video, determine the first key content of the target video.

[0039] In some embodiments, the methods for determining the first key content of a target video based on audio data and the target video differ in different application scenarios. For example, the audio data and the target video can be input into a trained key content extraction model to obtain the first key content of the target video. The key content extraction model is trained on an initial key content extraction model based on the audio data and the target key content of the target sample video.

[0040] As another example, speech recognition is performed on audio data to obtain speech recognition results, and the main content is extracted from the speech recognition results to obtain the main content extraction results; visual content change detection is performed on video segments to obtain visual content change detection results; based on the visual content change detection results and the main content extraction results, the first key content of the target video is determined.

[0041] Visual content change detection can include, but is not limited to, scene change detection, facial expression change detection, and posture and movement change detection.

[0042] In some embodiments, in order to further improve the accuracy of the obtained main content extraction results, subtitle recognition can also be performed on the video clip to obtain the subtitle recognition results of the video clip. Correspondingly, the main content is extracted from the subtitle recognition results and the speech recognition results to obtain the main content extraction results.

[0043] In some embodiments, the visual content change detection results and the main content extraction results can be fused to obtain a fused result, and the fused result can be analyzed for key content to obtain the first key content of the target video.

[0044] Step 203: Obtain the video segment containing the first key content from the target video.

[0045] Step 204: Determine the semantic triples based on the first key content.

[0046] In some embodiments, after obtaining the first key content, semantic triples can be extracted from the first key content to obtain the semantic triples of the first key content.

[0047] The first key content of the target video can be one or more, and this embodiment does not specifically limit this.

[0048] In some embodiments, when there are multiple first key contents, for each first key content, a video segment containing the first key content can be obtained from the target video, and semantic triples can be extracted from the first key content to obtain the semantic triples of the first key content.

[0049] Among them, semantic triples refer to basic knowledge units that describe entities in the real world and the semantic relationships between them, expressed in the form of <subject, predicate, object>.

[0050] For example, if the target video is a sports video, processing the sports video can yield a semantic triple such as <Athlete B, completes, dunking action>.

[0051] Step 205: Mark the first key content tag of the video segment on the progress bar of the target video, and use the semantic triple as the semantic triple corresponding to the first key content tag.

[0052] In some embodiments, in order to enable users of the client associated with the object identifier to understand the main content described by the video segment after triggering the first key content marker, a corresponding structured video summary can also be generated based on the semantic triple; a target instruction is sent to the client associated with the object identifier, wherein the target instruction is used to instruct the client to associate the first key content marker of the target video with the structured video summary, and after the client receives the trigger operation for the first key content marker, the starting frame of the video segment and the corresponding structured video summary are displayed.

[0053] The triggering operation can be, for example, a click operation or other operations. This embodiment does not specifically limit the triggering operation.

[0054] Figure 3 shows an example of the video playback interface on the client after the first key content marker is triggered. As can be seen from Figure 3, the video playback interface displays the starting frame of the video segment and the corresponding structured video summary. Furthermore, Figure 3 shows that the playback speed triggered by the first key content marker is 3x speed, where 3.0X in Figure 3 represents 3x speed. It should be noted that Figure 3 uses an example with a first key content marker on the target video's progress bar.

[0055] It is understood that, in practical applications, the progress bar of the target video may include one or more first key content markers, and this embodiment does not specifically limit this.

[0056] Step 206: Obtain the object identifier of the object that is watching the target video.

[0057] Step 207: Input the object identifier and semantic triple into the pre-trained target speed prediction model to obtain the first prediction speed rate used by the object corresponding to the object identifier on the first key content tag.

[0058] In some embodiments, a possible implementation of step 207 above may be: obtaining a first representation vector of the object identifier through the embedding representation layer in the target speed prediction model; obtaining a second representation vector of the semantic triple through the semantic representation layer in the target speed prediction model; concatenating the first and second representation vectors through the concatenation layer in the target speed prediction model to obtain a concatenated vector; and determining the first prediction speed rate used by the object corresponding to the object identifier on the first key content marker based on the concatenated vector through the decoding layer in the target speed prediction model. Thus, the first prediction speed rate used by the object corresponding to the object identifier on the first key content marker is accurately obtained through the target speed prediction model.

[0059] It can be understood that the target speed prediction model has been trained based on the sample speed rate used by the object corresponding to the object identifier on the sample key content marker that matches the semantic triple. Therefore, after the object identifier and the semantic triple are input into the target speed prediction model, the target speed prediction model can determine the first speed rate used by the object corresponding to the object identifier on the first key content marker.

[0060] For example, if the object identifier is played at a first speed rate on the key content marker of a sample with the first semantic triplet being A, then when the second semantic triplet corresponding to the key content marker of a certain video is determined to be A, after inputting the second semantic triplet and the object identifier into the target speed prediction model, the target speed prediction model can determine that the object corresponding to the object identifier may also be played at a first speed rate on the key content marker corresponding to the second semantic triplet.

[0061] For example, if the object identifier plays the sample key content tag with the first semantic triple A at the first speed, correspondingly, if the second semantic triple B is determined to be the key content tag of a certain video, and the similarity between A and B exceeds a preset similarity threshold, then after inputting the second semantic triple and the object identifier into the target speed prediction model, the target speed prediction model can determine that the object corresponding to the object identifier may also play the sample key content tag corresponding to the second semantic triple at the first speed.

[0062] It should be noted that the structural example diagram of the target speed prediction model is shown in Figure 4. As can be seen from Figure 4, the target speed prediction model can include: an embedding representation layer, a semantic representation layer, a concatenation layer, and a decoding layer. The embedding representation layer is used to represent the object identifier as a vector; the semantic representation layer is used to represent the semantic triples corresponding to the first key content tag; the concatenation layer is used to concatenate the outputs of the embedding representation layer and the semantic representation layer; and the decoding layer is used to decode the output of the concatenation layer to obtain the corresponding predicted speed rate.

[0063] It can be understood that the decoding layer may include a speed prediction layer. Correspondingly, the speed prediction layer predicts the speed rate used by the object corresponding to the object identifier on the first key content marker based on the concatenation vector, so as to obtain the first predicted speed rate used by the object corresponding to the object identifier on the first key content marker.

[0064] Step 208: Based on the first predicted speed multiplier, push the video clip to the object corresponding to the object identifier.

[0065] It should be noted that for a detailed description of step 208, please refer to the relevant descriptions in other embodiments, which will not be repeated here.

[0066] In some embodiments, in order to clearly understand the process of training the target speed prediction model, the process of training the target speed prediction model is described exemplarily below with reference to Figure 5.

[0067] Figure 5 is a flowchart illustrating the training of a target speed model according to another exemplary embodiment.

[0068] As shown in Figure 5, the method may include: step 501, acquiring training data, wherein the training data includes: object identifier, sample semantic triple corresponding to the second key content marker on the progress bar of the sample video, and the sample speed rate used by the object corresponding to the object on the second key content marker, the second key content marker corresponding to the sample video segment in the sample video where its second key content is located, and the sample semantic triple is determined based on the second key content.

[0069] In some embodiments, the sample speed rate is the actual speed rate at which the object corresponding to the object identifier viewed the second key content marker within a preset time period.

[0070] The preset time is a time set in advance according to actual needs. For example, the preset time can be 30 days or 60 days, etc. This embodiment does not make a specific limitation on this.

[0071] It can be understood that if the object corresponding to the object identifier uses the sample speed rate of the second key content mark, then the sample video segment corresponding to the second key content mark will be played at the sample speed rate.

[0072] It is understood that the second key content marker can be one or more, and this embodiment does not specifically limit this.

[0073] In some embodiments, behavioral sequence data of an object identifier can be obtained, wherein the behavioral sequence data includes: multiple behavioral data, wherein each behavioral data includes: a behavioral ternary relation, wherein the behavioral ternary relation includes: an object identifier, a sample triple corresponding to a second key content marker, and the sample speed multiplication rate used by the object corresponding to the object identifier on the second key content marker.

[0074] The sample speed is obtained by interacting with the object identifier and the second key content tag.

[0075] In some embodiments, an initial speed prediction model can be trained based on a behavioral sequence book of the object identifier.

[0076] For example, a behavioral ternary relation can be represented by (U001, KF_003, 1.5x), where (U001, KF_003, 1.5x) means that the object corresponding to the object identifier of U001 triggered 1.5x playback on the second key content marker KF_003.

[0077] Step 502: Input the object identifier and the sample semantic triple into the initial speed prediction model to obtain the second prediction speed rate used by the object corresponding to the object identifier on the first key content tag.

[0078] The initial doubling prediction model can be a model based on Long Short-Term Memory (LSTM) networks.

[0079] Step 503: Train the initial speed prediction model based on the second predicted speed rate and the sample speed rate to obtain the target speed prediction model.

[0080] In some embodiments, the total loss function value of the initial speed prediction model can be determined based on the second predicted speed rate and the sample speed rate; the initial speed prediction model is then trained based on the total loss function value to obtain the target speed prediction model. Thus, the total loss function value guides model training, thereby helping to improve the efficiency of obtaining the target speed prediction model.

[0081] In some embodiments, the initial speed prediction model can be adjusted based on the total loss function value, and the adjusted initial speed prediction model can be trained until the total loss function value meets the first preset condition to obtain the target speed prediction model.

[0082] The first preset condition is the condition for the model training to end. The first preset condition can be configured according to actual needs. For example, the total loss function value can satisfy the first preset condition if the total loss function value is less than a preset value, or if the change of the total loss function value approaches stability, that is, the difference between the total loss function values ​​corresponding to two or more adjacent training sessions is less than the set value, meaning that the total loss function value basically no longer changes.

[0083] In other embodiments, when there are multiple sample videos, one possible implementation of determining the total loss function of the initial speed-up prediction model based on the third predicted speed-up rate and the sample speed-up rate is as follows: calculate the cross-entropy based on the second predicted speed-up rate and the sample speed-up rate to obtain the first loss function value; calculate the behavior similarity loss based on the behavior representation generated for each sample video according to the initial speed-up prediction model to obtain the second loss function value; and perform a weighted summation of the first loss function value and the second loss function value to obtain the total loss function value.

[0084] In other embodiments, where there are multiple third key content tags, the matching degree between the semantic triples of each third key content tag is greater than a preset matching degree threshold, and the sample multiplication rate used by the object corresponding to the object identifier on each third key content tag is the same, correspondingly, a possible implementation of determining the total loss function of the initial multiplication rate prediction model based on the second predicted multiplication rate and the sample multiplication rate can be as follows: calculate the cross-entropy based on the second predicted multiplication rate and the sample multiplication rate to obtain the first loss function value, calculate the distributed consistency loss based on the second predicted multiplication rate corresponding to each third content tag to obtain the third loss function value, and perform a weighted summation of the first loss function value and the third loss function value to obtain the total loss function value.

[0085] In other embodiments, when there are multiple second key content tags, and the matching degree between the semantic triples of each second key content tag is greater than a preset matching degree threshold, and the sample speed rate used by the object corresponding to the object identifier on each second key content tag is the same, and there are multiple sample videos, a possible implementation of determining the total loss function of the initial speed rate prediction model based on the second predicted speed rate and the sample speed rate can be as follows: calculate the cross-entropy based on the second predicted speed rate and the sample speed rate to obtain a first loss function value; calculate the behavioral similarity loss based on the behavioral representation corresponding to each sample video to obtain a second loss function value; calculate the distributed consistency loss based on the second predicted speed rate corresponding to each second content tag to obtain a third loss function value; and perform a weighted summation of the first loss function value, the second loss function value, and the third loss function value to obtain the total loss function value.

[0086] As an example, the formula for obtaining the total loss function value can be expressed as: total_loss = α*main_loss + β*user_loss + γ*kf_loss; where main_loss represents the first loss function value; user_loss represents the second loss function value; kf_loss represents the third loss function value; α represents the first weight used for the first loss function value; β represents the second weight used for the second loss function value; and kf_loss represents the third weight used for the third loss function value. Where α + β + γ = 1.

[0087] In this embodiment, the initial speed prediction model is trained using training data to obtain a pre-trained target speed prediction model, which facilitates subsequent speed prediction based on the target speed prediction model.

[0088] Based on the above embodiments, test data can also be obtained, and the target speed prediction model can be tested according to the test data, so as to determine whether the target speed prediction model meets the second preset condition based on the test results.

[0089] The second preset condition is a condition set in advance according to actual needs. For example, the second preset condition can be that the accuracy of the target speed prediction model exceeds a preset accuracy threshold. Another example is that the preset condition can be that the sequence behavior stability exceeds a preset stability threshold.

[0090] The preset accuracy threshold is an accuracy threshold that is set in advance according to actual needs. This embodiment does not specifically limit the value of the preset accuracy threshold.

[0091] The preset stability threshold is a stability threshold set in advance according to actual needs. This embodiment does not specifically limit the value of the preset stability threshold.

[0092] To clearly understand this disclosure, the method of this embodiment will be described below with reference to Figure 6. It should be noted that this embodiment uses the interaction between a client and a streaming server as an example. The streaming server may include a semantic engine, a model, a decision engine, a CDN controller, and a target CDN node. The model includes a pre-trained target speed prediction model.

[0093] Figure 6 is a flowchart of the interaction between the client and the streaming server.

[0094] As shown in Figure 6, the method may include: step 601, the client sends a first request to the semantic engine, wherein the first request includes: the object identifier of the logged-in client and the target video, wherein the first request is used to request the semantic engine to analyze the target video.

[0095] In some embodiments, while a user is watching a target video through the client, the client sends a first request to the semantic engine.

[0096] Step 602: The semantic engine determines the semantic triple corresponding to the first key content marker on the progress condition of the target video, wherein the first key content marker corresponds to the video segment in the target video where its first key content is located, and the semantic triple is determined based on the first key content.

[0097] For a detailed description of step 602, please refer to the relevant descriptions in other embodiments, which will not be repeated here.

[0098] Step 603: The semantic engine sends the object identifier and semantic triple to the model so that the model can obtain the first prediction speed rate used by the object corresponding to the object identifier on the first key content tag through the pre-trained target speed prediction model.

[0099] Step 604: The model provides the first prediction speedup rate to the semantic engine.

[0100] In step 605, the semantic engine sends the object identifier and the first predicted speed multiplier to the decision engine.

[0101] Step 606: The decision engine sends a first push stream instruction to the CDN controller, wherein the first push stream instruction is used to instruct the video segment to be sent to the client associated with the object identifier according to the first predicted speed multiplier.

[0102] Step 607: The CDN controller obtains multiple candidate CDN nodes corresponding to the current geographical location from the Content Delivery Network (CDN) based on the current geographical location corresponding to the object identifier, determines the target CDN node with the least load from the multiple candidate CDN nodes, and sends a second push stream instruction to the target CDN node to send the video segment to the client associated with the object identifier according to the first predicted speed multiplier.

[0103] In some embodiments, the target CDN node determines whether the target video is contained in its local cache. If it is, it retrieves the video segment from the target video and sends the video segment to the client associated with the object identifier according to the first predicted speed multiplier.

[0104] In other embodiments, if the target video is not contained in the local cache of the target CDN node, the target CDN node may obtain the target video from the source website of the target video and save the target video in its local cache. Correspondingly, the video segment is sent to the client associated with the object identifier according to the first predicted speed multiplier.

[0105] Step 608: The CDN node returns a preloading completion notification to the client, which indicates that the target video has been cached in the CDN node.

[0106] In some embodiments, after detecting that a client is about to play the video segment, the CDN node sends the video segment to the client associated with the object identifier according to a first predicted speed multiplier.

[0107] In some embodiments, the CDN node determines the streaming speed of the video segment to the client based on a first predicted speed multiplier, and pushes the video segment to the client according to the streaming speed.

[0108] It should be noted that in related technologies, the streaming server uses a fixed streaming speed to push the stream to the client. However, in this embodiment, the streaming speed is matched with the speed multiplier, so that the streaming speed of the streaming server changes with the predicted speed multiplier, thereby achieving a more controllable, efficient, and consistent accelerated playback experience.

[0109] In this embodiment, the predicted speed rate used by the client on the corresponding first key content marker is accurately predicted, and the video is pushed to the client according to the predicted speed rate, thereby helping to eliminate video stuttering on the client.

[0110] Figure 7 is a schematic diagram of a propulsion device according to an exemplary embodiment.

[0111] As shown in Figure 7, the streaming device 700 may include: a first determining module 701, an acquiring module 702, a second determining module 703, and a sending module 704, wherein: the first determining module 701 is used to determine the semantic triple corresponding to the first key content marker on the progress bar of the target video, wherein the first key content marker corresponds to the video segment in the target video where its first key content is located, and the semantic triple is determined based on the first key content.

[0112] The acquisition module 702 is used to acquire the object identifier of the object that is watching the target video.

[0113] The second determining module 703 is used to input the object identifier and semantic triple into the pre-trained target speed prediction model to obtain the first prediction speed rate used by the object corresponding to the object identifier on the first key content tag.

[0114] The sending module 704 is used to push video segments to the object corresponding to the object identifier according to the first predicted speed multiplier.

[0115] In one embodiment of this disclosure, the video segment is obtained by: acquiring the audio data of the target video; determining the first key content of the target video based on the audio data and the target video; and acquiring the video segment containing the first key content from the target video.

[0116] In one embodiment of this disclosure, a possible implementation of determining the first key content of a target video based on audio data and the target video is as follows: performing speech recognition on the audio data to obtain a speech recognition result; extracting the main content from the speech recognition result to obtain a main content extraction result; performing visual content change detection on the video segment to obtain a visual content change detection result; and determining the first key content of the target video based on the visual content change detection result and the main content extraction result.

[0117] In one embodiment of this disclosure, the device may further include: a subtitle recognition module, used to perform subtitle recognition on a video segment to obtain a subtitle recognition result for the video segment; wherein, one possible implementation of extracting the main content from the speech recognition result to obtain the main content extraction result may be: extracting the main content from the subtitle recognition result and the speech recognition result to obtain the main content extraction result.

[0118] In one embodiment of this disclosure, the apparatus further includes: a generation module, configured to generate a corresponding structured video summary based on semantic triples; and a sending module 704, configured to send a target instruction to a client associated with an object identifier, wherein the target instruction instructs the client to associate a first key content marker of the target video with the structured video summary, and to display the video segment and the corresponding structured video summary after the client receives a trigger operation for the first key content marker.

[0119] In one embodiment of this disclosure, a possible implementation of inputting the object identifier and semantic triples into a pre-trained target speed prediction model to obtain the first predicted speed rate used by the object corresponding to the object identifier on the first key content marker can be as follows: obtaining the first representation vector of the object identifier through the embedding representation layer in the target speed prediction model; obtaining the second representation vector of the semantic triples through the semantic representation layer in the target speed prediction model; concatenating the first representation vector and the second representation vector through the concatenation layer in the target speed prediction model to obtain the concatenated vector; and determining the first predicted speed rate used by the object corresponding to the object identifier on the first key content marker based on the concatenated vector through the decoding layer in the target speed prediction model.

[0120] In one embodiment of this disclosure, the apparatus may further include: a training module for acquiring training data, wherein the training data includes: an object identifier, a sample semantic triple corresponding to a second key content marker on the progress bar of a sample video, and a sample speedup rate used by the object corresponding to the object identifier on the second key content marker, the second key content marker corresponding to a sample video segment in the sample video where its second key content is located, and the sample semantic triple being determined based on the second key content; inputting the object identifier and the sample semantic triple into an initial speedup prediction model to obtain a second predicted speedup rate used by the object corresponding to the object identifier on the first key content marker; and training the initial speedup prediction model based on the second predicted speedup rate and the sample speedup rate to obtain a target speedup prediction model.

[0121] In one embodiment of this disclosure, a possible implementation of training an initial speed prediction model based on a second predicted speed rate and a sample speed rate to obtain a target speed prediction model is as follows: determining the total loss function value of the initial speed rate prediction model based on the second predicted speed rate and the sample speed rate; and training the initial speed prediction model based on the total loss function value to obtain the target speed prediction model.

[0122] It should be noted that the foregoing description of the propulsion method embodiment also applies to the propulsion device of this embodiment, and will not be repeated here.

[0123] It should be noted that the apparatus provided in this disclosure corresponds to the aforementioned method embodiments. Its technical effects are the same as those in the aforementioned method embodiments; please refer to the description in the method section for details.

[0124] It should be noted that the acquisition, storage, use, and processing of data in this disclosed technical solution all comply with the relevant provisions of national laws and regulations.

[0125] According to embodiments of this disclosure, an electronic device is also provided, including: a processor; and a memory for storing processor-executable instructions, wherein the processor is configured to: implement the streaming method disclosed in embodiments of this disclosure.

[0126] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the streaming method disclosed in this disclosure.

[0127] To implement the above embodiments, this disclosure also provides a computer program product.

[0128] The computer program product includes a computer program that, when executed by a processor, implements the streaming method disclosed in this embodiment.

[0129] Figure 8 is a structural block diagram of an electronic device according to an exemplary embodiment. The electronic device shown in Figure 8 is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0130] As shown in Figure 8, the electronic device 1000 includes a processor 111, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 112 or a program loaded from memory 116 into a random access memory (RAM) 113. The RAM 113 also stores various programs and data required for the operation of the electronic device 1000. The processor 111, ROM 112, and RAM 113 are interconnected via a bus 114. An input / output (I / O) interface 115 is also connected to the bus 114.

[0131] The following components are connected to I / O interface 115: memory 116 including hard disks, etc.; and communication section 117 including network interface cards such as local area network (LAN) cards, modems, etc., communication section 117 performs communication processing via a network such as the Internet; and driver 118 is also connected to I / O interface 115 as needed.

[0132] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 117. When the computer program is executed by processor 111, it performs the functions defined in the methods of this disclosure.

[0133] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory including instructions, which can be executed by the processor 111 of the electronic device 1000 to perform the above-described method. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0134] In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0135] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0136] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for pushing streams, characterized in that, The method includes: determining a semantic triple corresponding to a first key content marker on the progress bar of a target video, wherein the first key content marker corresponds to a video segment in the target video where its first key content is located, and the semantic triple is determined based on the first key content; obtaining an object identifier of an object currently watching the target video; inputting the object identifier and the semantic triple into a pre-trained target speed prediction model to obtain a first predicted speed rate used by the object corresponding to the object identifier on the first key content marker; and pushing the video segment to the object corresponding to the object identifier according to the first predicted speed rate.

2. The method as described in claim 1, characterized in that, The method for obtaining the video segment is as follows: obtaining the audio data of the target video; determining the first key content of the target video based on the audio data and the target video; and obtaining a video segment containing the first key content from the target video.

3. The method as described in claim 2, characterized in that, The step of determining the first key content of the target video based on the audio data and the target video includes: performing speech recognition on the audio data to obtain a speech recognition result; extracting the main content from the speech recognition result to obtain a main content extraction result; performing visual content change detection on the video segment to obtain a visual content change detection result; and determining the first key content of the target video based on the visual content change detection result and the main content extraction result.

4. The method as described in claim 3, characterized in that, Before performing main content extraction on the speech recognition result to obtain the main content extraction result, the method further includes: performing subtitle recognition on the video segment to obtain the subtitle recognition result of the video segment; wherein, performing main content extraction on the speech recognition result to obtain the main content extraction result includes: performing main content extraction on the subtitle recognition result and the speech recognition result to obtain the main content extraction result.

5. The method as described in claim 1, characterized in that, The method further includes: generating a corresponding structured video summary based on the semantic triple; sending a target instruction to the client associated with the object identifier, wherein the target instruction is used to instruct the client to associate the first key content marker of the target video with the structured video summary, and displaying the video segment and the corresponding structured video summary after the client receives a trigger operation for the first key content marker.

6. The method according to any one of claims 1-5, characterized in that, The process of inputting the object identifier and the semantic triple into a pre-trained target speed prediction model to obtain the first predicted speed rate used by the object corresponding to the object identifier on the first key content marker includes: obtaining a first representation vector of the object identifier through the embedding representation layer in the target speed prediction model; obtaining a second representation vector of the semantic triple through the semantic representation layer in the target speed prediction model; concatenating the first representation vector and the second representation vector through the concatenation layer in the target speed prediction model to obtain a concatenated vector; and determining the first predicted speed rate used by the object corresponding to the object identifier on the first key content marker based on the concatenated vector through the decoding layer in the target speed prediction model.

7. The method as described in claim 6, characterized in that, The training method for the target speed prediction model is as follows: Training data is acquired, including: the object identifier, the sample semantic triple corresponding to the second key content marker on the progress bar of the sample video, and the sample speed rate used by the object corresponding to the object identifier at the second key content marker. The second key content marker corresponds to the sample video segment in the sample video where its second key content is located, and the sample semantic triple is determined based on the second key content. The object identifier and the sample semantic triple are input into an initial speed prediction model to obtain the second predicted speed rate used by the object corresponding to the object identifier at the first key content marker. The initial speed prediction model is trained based on the second predicted speed rate and the sample speed rate to obtain the target speed prediction model.

8. The method as described in claim 7, characterized in that, The step of training the initial speed prediction model based on the second predicted speed rate and the sample speed rate to obtain the target speed prediction model includes: determining the total loss function value of the initial speed rate prediction model based on the second predicted speed rate and the sample speed rate; and training the initial speed prediction model based on the total loss function value to obtain the target speed prediction model.

9. A flow propulsion device, characterized in that, include: A first determining module is used to determine the semantic triple corresponding to the first key content marker on the progress bar of the target video, wherein the first key content marker corresponds to the video segment in the target video where its first key content is located, and the semantic triple is determined based on the first key content; an acquisition module is used to acquire the object identifier of the object watching the target video; a second determining module is used to input the object identifier and the semantic triple into a pre-trained target speed prediction model to obtain the first predicted speed rate used by the object corresponding to the object identifier on the first key content marker; and a sending module is used to push the video segment to the object corresponding to the object identifier according to the first predicted speed rate.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-8.

Images