A data processing method, device and equipment for media objects and a medium

Abstract

The application discloses a data processing method and device for a media object, equipment and medium. The method comprises the following steps: obtaining target audio data of a media object to be processed; performing text vector representation processing on target speech recognition text corresponding to the target audio data to obtain a target text representation vector; performing vector similarity calculation on the target text representation vector and each historical text representation vector in N historical text representation vectors respectively to obtain N vector similarities; when K historical media objects are screened out from the N historical media objects through the N vector similarities, determining a target recommendation feature for representing the media object to be processed based on an established similar association relationship and K historical recommendation features of the K historical media objects, and performing media recommendation on a target service object matched with the media object to be processed through the target recommendation feature. According to the application, the accuracy of media recommendation can be improved for media objects with less interaction data.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a data processing method, apparatus, device, and medium for media objects. Background Technology

[0002] Currently, cold start recommendation refers to a situation where a recommendation system recommends newly introduced media objects. Because these newly introduced media objects (e.g., live streams created by broadcasters) lack historical interaction data, the recommendation system typically uses similar tags based on the media object's object tags during the cold start recommendation process. For example, for live stream 1 with a style of A, when a recommendation model is needed for live stream recommendations, the system will directly search for other live streams (e.g., live stream 2) with the same style tag (A style) in a vast number of live streams using the style tag of live stream 1 (i.e., A style). Then, it can push live stream 1 to the business objects (i.e., viewers) currently watching the live stream in live stream 2, thus achieving a cold start recommendation for live stream 1.

[0003] However, the inventors discovered in practice that when performing cold-start recommendations based on media object tags, because the live video in live room 1 (e.g., live video 1) lacks actual interaction record information, when live room 2 is found through similar tag matching, it can be directly pushed to these viewers indiscriminately based on the viewer information recorded in the interaction record information of the live video in live room 2 (e.g., live video 2). Once the style tag of live video 2 differs from the actual live content in live video 2, the live content in live room 1 pushed will be difficult to truly match the viewing needs of these viewers, thus reducing the accuracy of media recommendations for media objects with little interaction data to some extent. Summary of the Invention

[0004] This application provides a data processing method, apparatus, device, and medium for media objects, which can improve the accuracy of the target recommendation model in matching cold-start media objects with users.

[0005] This application provides a data processing method for media objects, including:

[0006] Obtain the target audio data of the media object to be processed in the target sampling window, and perform text vector representation processing on the target speech recognition text corresponding to the target audio data to obtain the target text representation vector used to represent the media object to be processed.

[0007] Obtain N historical text representation vectors to represent N historical media objects. Perform vector similarity calculations on the target text representation vector and each historical text representation vector to obtain N vector similarity scores. The amount of interaction data for each historical media object is higher than the amount of interaction data for the media object to be processed. N is a positive integer greater than 1. A historical text representation vector is recorded after performing text vector representation processing on the historical speech recognition text corresponding to a historical audio data. A historical audio data refers to the audio data of a historical media object in the recorded historical sampling window. The window duration of the historical sampling window is consistent with the window duration of the target sampling window.

[0008] When selecting K historical media objects from N historical media objects using N vector similarity, a similarity association is established between the K historical media objects and the media object to be processed; K is a positive integer less than N.

[0009] When K historical recommendation features of K historical media objects are obtained, target recommendation features are determined based on similarity relationships and K historical recommendation features to characterize the media objects to be processed; the target recommendation features are used to find target business objects that match the media objects to be processed.

[0010] One embodiment of this application provides a data processing apparatus for media objects, including:

[0011] The audio data speech recognition module is used to collect audio data of the media object to be processed at the target dynamic acquisition time point based on the target sampling window to obtain target audio data. Then, the target audio data is recognized and processed based on automatic speech recognition technology to obtain target speech recognition text. The semantic understanding model extracts information from the target speech recognition model based on the semantic information mining template and outputs target speech understanding text associated with the target speech recognition text.

[0012] The vector similarity calculation module is used to perform text vector representation processing on the target speech recognition text corresponding to the target audio data to obtain a target text representation vector for representing the media object to be processed, and to perform text vector representation processing on the historical speech recognition text corresponding to the historical audio data to obtain a historical text representation vector for representing the historical media object, to obtain N historical text representation vectors for representing N historical media objects, and to perform vector similarity calculation on the target text representation vector and each historical text representation vector to obtain N vector similarities;

[0013] The similarity association establishment module is used to establish a similarity association between the K historical media objects and the media object to be processed when selecting K historical media objects from the N historical media objects through the N vector similarity; the interaction data volume of each historical media object is higher than the interaction data volume of the media object to be processed; N is a positive integer greater than 1; a historical text representation vector is recorded after performing text vector representation processing on the historical speech recognition text corresponding to a historical audio data; a historical audio data refers to the audio data of a historical media object in the recorded historical sampling window; the window duration of the historical sampling window is consistent with the window duration of the target sampling window; K is a positive integer less than N;

[0014] The target recommendation feature determination module is used to determine the target recommendation features to represent the media object to be processed based on the similarity relationship and the K historical recommendation features of K historical media objects when K historical recommendation features are obtained. The module then recommends media to the target business objects that match the media object to be processed based on the target recommendation features.

[0015] The audio data speech recognition module includes:

[0016] The target audio data determination unit is used to obtain the target dynamic sampling time point and window duration of the media object to be processed, and to determine the target sampling window for sampling the multimedia data of the media object to be processed based on the target dynamic sampling time point and window duration; the target dynamic sampling time point refers to the time point that changes as the associated duration of the media data of the media object to be processed changes.

[0017] The target audio data determination unit is also used to obtain audio data from the multimedia data of the media object to be processed through the target sampling window, wherein the data sampling end time point is the target dynamic sampling time point and the sampling duration is the window duration.

[0018] The target audio data determination unit is also used to use the acquired audio data as the target audio data of the media object to be processed in the target sampling window.

[0019] The target dynamic sampling time point includes the first dynamic sampling time point, the target sampling window includes the first sampling window corresponding to the first dynamic sampling time point, and the target audio data includes the first audio data collected based on the first dynamic sampling time point and the window duration.

[0020] Optionally, the audio data speech recognition module also includes:

[0021] The target audio data update unit is used to obtain the sampling interval duration for dynamic sampling, and based on the first dynamic sampling time point and the sampling interval duration, to determine the second dynamic sampling time point for sampling the multimedia data of the media object to be processed; the second dynamic sampling time point is the next sampling time point after the first dynamic sampling time point;

[0022] The target audio data update unit is also used to determine a second sampling window for sampling the multimedia data of the media object to be processed based on the second dynamic sampling time point and the window duration; the second sampling window is the next sampling window after the first sampling window, and the second sampling window overlaps with the first sampling window;

[0023] The target audio data update unit is also used to obtain audio data from the multimedia data of the media object to be processed through the second sampling window, wherein the data sampling end time point is the second dynamic sampling time point and the sampling duration is the window duration.

[0024] The target audio data update unit is also used to use the acquired audio data as the second audio data of the media object to be processed in the target sampling window, and to update the target audio data through the second audio data when updating the target dynamic sampling time point through the second dynamic sampling time point.

[0025] Furthermore, the audio data voice module includes:

[0026] The component acquisition unit is used to acquire a speech conversion understanding component and a similarity retrieval component for speech conversion processing of target audio data; the speech conversion understanding component includes a speech recognizer and a semantic understanding model for semantic extraction, and the similarity retrieval component includes a text representation.

[0027] The speech recognition unit is used to perform speech recognition processing on the target audio data through a speech recognizer to obtain the speech recognition processing result.

[0028] The semantic understanding unit is used to perform text semantic understanding processing on the target speech recognition text through the semantic understanding model if the speech recognition processing result indicates that the target audio data has been recognized as target speech recognition text, thereby obtaining the target semantic understanding information of the target speech recognition text.

[0029] The text vector representation unit is used to represent the target semantic understanding information into vectors through the text representation unit, so as to obtain the target text representation vector used to represent the media object to be processed.

[0030] Optionally, the audio data speech recognition module also includes:

[0031] The invalid audio data marking unit is used to mark the target audio data as invalid audio data if the above speech recognition processing result indicates that the target speech recognition text has not been recognized.

[0032] Optionally, the audio data speech recognition module also includes:

[0033] The target speech recognition text update unit is used to obtain the video data of the media object to be processed from the multimedia data of the media object to be processed through the target sampling window, and use the obtained video data as the target video data of the media object to be processed in the target sampling window.

[0034] The target speech recognition text update unit is also used to perform video content recognition on video frames in the target video data to obtain video text content that represents the target video data, and update the target speech recognition text through the video text content.

[0035] Specifically, the semantic understanding unit is used to acquire guidance prompt template data for guiding the semantic understanding model to output the model.

[0036] The semantic understanding unit is also specifically used to input the guidance prompt template data and the target speech recognition text into the semantic understanding model. The semantic understanding model extracts semantic information from the target speech recognition text according to the data content in the guidance prompt template data, and obtains the target semantic extraction information of the target speech recognition text.

[0037] The semantic understanding unit is also specifically used to determine the target semantic understanding information of the target speech recognition text by extracting information from the target semantics.

[0038] The semantic understanding unit is specifically used to use the prompt template data associated with the video live streaming type as the guidance prompt template data if the object type of the media object to be processed is video live streaming.

[0039] The semantic understanding unit is also specifically used to use the prompt template data associated with the video-on-demand type as the guidance prompt template data if the object type of the media object to be processed is video-on-demand.

[0040] The vector similarity calculation module includes:

[0041] The interactive data curve distribution acquisition unit is used to acquire the interactive data curve distribution corresponding to the historical media object database. The interactive data curve distribution is determined based on each historical media object in the historical media object database and the amount of interactive data for each historical media object.

[0042] The historical media object determination unit is used to obtain N historical media objects whose interactive data volume reaches the interactive data threshold from the historical media object database by means of interactive data curve distribution.

[0043] The historical text representation vector acquisition unit is used to obtain N historical text representation vectors of N historical media objects from the historical text vector representation database associated with N historical media objects. A historical text representation vector is obtained by speech conversion processing of the historical audio data of a historical media object through the speech conversion understanding component and the similarity retrieval component.

[0044] Furthermore, the similarity association establishment module includes:

[0045] The historical media object list determination unit is used to sort N historical media objects based on N vector similarity to obtain a list of historical media objects associated with the N historical media objects.

[0046] The similarity association establishment unit is used to obtain K historical media objects whose vector similarity meets the similarity retrieval conditions from the historical media object list, and establish explicit similarity associations between the K historical media objects and the media object to be processed. The explicit similarity association is used to characterize that the list composed of K historical media objects is a list of similar media objects, and any media object in the list of similar media objects has an association with the media object to be processed.

[0047] The target recommendation feature determination module includes:

[0048] The target recommendation feature determination unit is used to perform feature pooling on the K historical recommendation features based on similarity relationships when K historical media objects are obtained, so as to obtain pooled features associated with the K historical recommendation features.

[0049] The pooling processing unit is used to determine the pooling processing features as target recommendation features to characterize the media objects to be processed.

[0050] Furthermore, the pooling processing unit is specifically used to obtain a pooling processing strategy for performing pooling processing based on similarity relationships;

[0051] The pooling processing unit is also specifically used to perform average pooling processing on K historical recommendation features when the pooling processing strategy is average pooling processing strategy, to obtain average pooled features; the average pooled features are pooled processing features associated with the K historical recommendation features.

[0052] The device also includes:

[0053] The historical recommendation feature acquisition module is used to obtain K historical recommendation features of K historical media objects from the recommendation feature database associated with K historical media objects.

[0054] The historical recommendation feature acquisition module includes:

[0055] The historical interaction behavior information acquisition unit is used to acquire N historical interaction behavior information corresponding to N historical media objects; one historical media object corresponds to one historical interaction behavior information, and one historical interaction behavior information is used to characterize the item interaction details of each interactive item in a historical media object;

[0056] The recommendation metric acquisition unit is used to obtain recommendation metrics for training the initial recommendation model by taking the item interaction details represented by N historical interaction behavior information as training samples.

[0057] The target recommendation model training unit is used to train the initial recommendation model using training samples and recommendation metrics to obtain the target recommendation model. The target recommendation model is used to extract N historical recommendation features from N historical media objects. A historical recommendation feature is obtained by the target recommendation model extracting the interaction vector of the item interaction details represented by a historical interaction behavior information.

[0058] The recommendation feature database update unit is used to add the N historical recommendation features extracted by the target recommendation model to the recommendation feature database.

[0059] Furthermore, the media object to be processed is a multimedia object in the business processing platform whose interactive data volume is less than the threshold of interactive data volume. The recommendation model corresponding to the business processing platform is the target recommendation model. The target recommendation model is obtained by training the initial recommendation model deployed on the business processing platform with N historical interactive behavior information corresponding to N historical media objects.

[0060] The target recommendation feature determination module includes:

[0061] The associated data information determination unit is used to obtain the object association data information of the business objects accessing the business processing platform when the target recommendation model is deployed and applied on the business processing platform.

[0062] The object vector feature determination unit is used to input object association data information and target recommendation features into the target recommendation model. When the target recommendation model extracts object vector representation features that represent business objects from the object association data information, it selects target object vector representation features that match the target recommendation features from the object vector representation features.

[0063] The media object recommendation unit is also used to identify the business objects corresponding to the target object vector representation features as target business objects that match the media objects to be processed among the selected business objects, and to recommend the media objects to be processed to the target business objects.

[0064] Among them, the object vector feature determination unit is specifically used to input object association data information and target recommendation features into the target recommendation model, and through the target recommendation model, extract information vector representation of object association data information to obtain object vector representation features used to represent business objects;

[0065] The object vector feature determination unit is also specifically used to perform feature matching between the object vector representation features and the target recommendation features to obtain the feature matching result;

[0066] The object vector feature determination unit is also specifically used to determine the object vector representation feature that matches the target recommendation feature if the feature matching result indicates that there is an object vector representation feature in the object vector representation feature that matches the target recommendation feature.

[0067] Furthermore, the media object recommendation unit is specifically used to, if the media object to be processed is a live media object, use the live media object as the target recommendation object in the live recommendation service to recommend to the target service object, and send the target recommendation object to the terminal device corresponding to the target service object so that the terminal device displays the target recommendation object in the live recommendation list corresponding to the live recommendation service;

[0068] The media object recommendation unit is further specifically used to encapsulate the on-demand link corresponding to the on-demand media object into a recommendation notification message if the media object to be processed is a video-on-demand media object, and send the recommendation notification message to the terminal device corresponding to the target service object, so that the terminal device displays the on-demand link in the recommendation notification message on the live recommendation list corresponding to the live recommendation service; the on-demand link is used to instruct the terminal device to display the on-demand media object in response to the trigger operation of the recommendation notification message.

[0069] One aspect of this application provides a computer device, including a memory and a processor. The memory is connected to the processor, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device executes the method provided in one aspect of this application.

[0070] One aspect of this application provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor, so that a computer device having a processor performs the method provided in one aspect of this application.

[0071] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in the above aspect.

[0072] This application embodiment can identify the audio data of the media object to be processed within the target sampling window, and obtain the target speech recognition text associated with the audio data. Further, this application embodiment performs text vector representation processing on the target speech recognition text associated with the audio data to obtain a target text representation vector representing the media object to be processed. N historical text representation vectors representing N historical media objects are obtained, and then vector similarity calculations are performed on each of the target text representation vectors and the N historical text representation vectors to obtain N vector similarities. Further, based on the N vector similarities, this application embodiment selects K historical media objects with high similarity to the media object to be processed from the N historical media objects, and then establishes an explicit similarity association between the media object to be processed and the selected K historical media objects. This application embodiment obtains K historical recommendation features of the K historical media objects, determines the target recommendation features of the media object to be processed based on the explicit similarity association, and recommends media to target business objects matching the media object to be processed through the target recommendation features. This application embodiment can vectorize the content of the multimedia data of the media object to be processed and calculate its similarity with the vectors of the multimedia data content of historical media objects. This retrieves K historical media objects with high relevance to the media object to be processed. The target recommendation features representing the media object to be processed are then calculated using the historical recommendation features of these K historical media objects. Finally, a recommendation model accurately retrieves the target business object corresponding to the media object to be processed, and pushes the aforementioned media data to it. This application embodiment also associates the target text representation vector of the media object to be processed with the historical text representation vectors of historical media objects to obtain the target recommendation features associated with the media object to be processed. Based on these target recommendation features, business objects in the business processing platform where the media object to be processed resides are matched. This improves the accuracy of the target recommendation model in matching cold-start media objects to business objects and also achieves the effect of "warming up" the media object to be processed. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0074] Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application;

[0075] Figure 2 This is a schematic diagram of a data interaction scenario provided in an embodiment of this application;

[0076] Figure 3 This is a flowchart illustrating a data processing method for a media object according to an embodiment of this application;

[0077] Figure 4 This is a schematic diagram of the interaction data curve of a historical media object according to an embodiment of this application;

[0078] Figure 5 This is a schematic diagram of an explicit similarity relationship according to an embodiment of this application;

[0079] Figure 6 This is a schematic diagram illustrating a scenario of media recommendation for live streaming media objects on a business processing platform, according to an embodiment of this application.

[0080] Figure 7 This is a schematic diagram of a scenario of live streaming in a live streaming room according to an embodiment of this application;

[0081] Figure 8 This is a schematic diagram illustrating a scenario of recommending on-demand media objects on a business processing platform, according to an embodiment of this application.

[0082] Figure 9 This is a schematic diagram illustrating a scenario where a video is played using a video-on-demand identifier, according to an embodiment of this application.

[0083] Figure 10 This is a schematic diagram of a recommended display list of a business processing platform according to an embodiment of this application;

[0084] Figure 11 This is a flowchart of a target audio invalidation processing method according to an embodiment of this application;

[0085] Figure 12 This is a flowchart of a target audio invalidation processing method according to an embodiment of this application;

[0086] Figure 13 This is a schematic diagram of a target business object estimation process for media objects according to an embodiment of this application;

[0087] Figure 14 This is a schematic diagram of a target business object estimation process for a live streaming room according to an embodiment of this application;

[0088] Figure 15 This is a flowchart illustrating a data processing method for a media object according to an embodiment of this application;

[0089] Figure 16 This is a schematic diagram of dynamic sampling based on the playback timeline of the live data to be processed, according to an embodiment of this application.

[0090] Figure 17 This is a schematic diagram of the structure of a data processing device for a media object according to an embodiment of this application;

[0091] Figure 18 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0092] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0093] I. Media Objects. The media objects involved in the embodiments of this application include media objects to be processed and historical media objects. A media object refers to a first media object (e.g., a virtual live broadcast room, conference room, and classroom) or a second media object (e.g., video-on-demand) that is displayed in the application client and has real-time video processing capabilities. The videos in the first media object may include, but are not limited to, live videos, conference videos, and educational videos.

[0094] Specifically, live video can include live video displayed in a specific live room within the live streaming client in a real-time live streaming scenario and / or recorded video displayed in the same live room within the live streaming client in a live replay scenario. Video-on-demand (VOD) video can include long videos and short videos displayed in the video playback client in a video-on-demand scenario. Long videos refer to videos with a duration exceeding a certain threshold, while short videos refer to videos with a duration less than that threshold.

[0095] It is understood that, in this application embodiment, media objects to be featured in the recommendation system can be collectively referred to as media objects to be processed (e.g., media objects with a small amount of interaction data with users), and media objects that have already been featured in the recommendation system can be collectively referred to as historical media objects (e.g., media objects with a large amount of interaction data with users). Specifically, for example, in a live streaming scenario, a currently live streaming room that needs to be featured or a long-tailed live streaming room that meets the long-tail distribution can be referred to as a media object to be processed, and some popular live streaming rooms that have already finished streaming or have completed live streaming recommendations or top live streaming rooms that meet the long-tail distribution can be referred to as historical media objects.

[0096] II. Sampling Window. The sampling window involved in the embodiments of this application includes a target sampling window and a historical sampling window. A sampling window refers to the window used to define video signal sampling parameters during video acquisition. It includes settings for parameters such as sampling frequency, resolution, and frame rate.

[0097] It is understood that, in a specific business scenario (e.g., a live streaming scenario), the sampling window for multimedia data (e.g., live streaming data) of the media object to be processed, which is captured in real-time from the current sampling time point according to the sampling frequency for a certain duration (i.e., a preset sampling interval), is collectively referred to as the target sampling window. Similarly, the sampling windows for multimedia data of historical media objects captured historically at the same sampling frequency for a certain duration are collectively referred to as the historical sampling window. Specifically, in a live streaming scenario, the sampling window used to dynamically capture a certain duration of live streaming data as a live stream progresses can be called the target sampling window. Furthermore, when the live stream ends, the last sampling window among all sampling windows that have completed both the live stream and the sampling process can be called the historical sampling window.

[0098] III. Audio Data. The audio data involved in the embodiments of this application includes target audio data and historical audio data. Digitized sound data is audio data. It can be understood that, in the embodiments of this application, the audio data extracted from the multimedia data of the media object to be processed (e.g., live data to be processed) collected by the target sampling window is called target audio data, and the audio data extracted from the multimedia data of historical media objects (e.g., historical live data) collected by the historical sampling window is called historical audio data. Specifically, for example, in a real-time live streaming scenario, the audio data captured by the target sampling window from the ongoing live stream is called target audio data, and the audio data captured by the historical sampling window from a completed live stream that was already captured during the live stream is called historical audio data.

[0099] IV. Similarity Search. The similarity search involved in the embodiments of this application includes vector similarity search. Similarity search refers to the technique of finding other search objects that are similar to or match a given search object in a set of search objects.

[0100] It is understood that, in this embodiment of the application, the process of calculating the similarity between the target text representation vector and the historical text representation vector, and then retrieving historical media objects associated with the media object to be processed based on this vector similarity, is called similarity retrieval. Specifically, for example, in a live streaming scenario, the process of calculating the vector similarity between the target text representation vector and the historical text representation vector, sorting the historical media objects from high to low based on this vector similarity to obtain a list of historical media objects, and then selecting the top 20 from this list as historical media objects associated with the media object to be processed, is called similarity retrieval.

[0101] V. Recommendation Features. The recommendation features involved in the embodiments of this application include target recommendation features and historical recommendation features. Recommendation features refer to feature data used in a recommendation system to describe the attributes of users and items. This feature data can help the recommendation system understand the preferences of business objects (e.g., user U1), thereby enhancing the accuracy of recommendations when recommending relevant items to the business object (e.g., user U1), and thus helping user U1 find items that match their preferences.

[0102] It is understood that, in this application embodiment, attribute data describing the media object to be processed is referred to as target recommendation features, and attribute data describing historical media objects is referred to as historical recommendation features. Specifically, for example, in a real-time live streaming scenario, attribute data calculated by the recommendation model based on historical behavioral interaction information of historical media data is referred to as historical recommendation features, and attribute data calculated after pooling historical recommendation features is referred to as target recommendation features.

[0103] In this embodiment, pooling processing can include at least average pooling, max pooling, and random pooling. Pooling is a special operation in convolutional neural networks, primarily used to extract key information from a given region. For example, this embodiment can extract pooling features (i.e., pooling features) that sufficiently characterize a region formed by K historical recommendation features. These extracted pooling features can then be used to equivalently obtain target recommendation features representing the media object to be processed. This allows for cold-start recommendations of the media object in the recommendation system using collaborative filtering. The convolutional neural network can be a network unit within the recommendation model or an independent network unit. This embodiment does not limit the specific deployment method of the convolutional neural network used for pooling processing, nor does it limit the specific implementation method of pooling processing.

[0104] In this context, it can be understood that average pooling refers to a processing method where a computer device can perform an average value operation on a region composed of K historical recommendation features. The computer device can then use the average value obtained after the average value operation (i.e., the pooled feature) as the target recommendation feature. Similarly, max pooling refers to another processing method where a computer device can retain the largest historical recommendation feature (i.e., the historical recommendation feature with the largest feature value) in a region composed of K historical recommendation features. The computer device can then use the largest historical recommendation feature retained in that region (i.e., the other pooled feature) as the target recommendation feature. Similarly, random pooling refers to another processing method in which a computer device can configure random selection probabilities for each historical recommendation feature in a region composed of K historical recommendation features (for example, the larger the feature value of a historical recommendation feature, the larger the configured random selection probability, and vice versa). After that, the computer device can further randomly select from each historical recommendation feature with a corresponding random selection probability by weighting it. In this way, the computer device can further select a historical recommendation feature with a corresponding random selection probability (i.e., a pooling feature) randomly selected in the region as the target recommendation feature.

[0105] VI. Business Processing Platform: The business processing platform involved in this application embodiment is a software platform that deploys a target recommendation model. A business processing platform refers to a digital platform that supports enterprises or organizations in carrying out business operations and management activities.

[0106] It is understood that, in this application embodiment, the software platform containing media objects to be featured in recommendations is referred to as a business processing platform. Specifically, for example, in a live streaming scenario, the live streaming data is featured in recommendations so that the media objects to be processed corresponding to the live streaming data are displayed in the recommendation display list of the target business objects. This operation needs to be completed on the business processing platform (e.g., a social software platform).

[0107] VII. Business Objects. The business objects involved in the embodiments of this application include business objects of the business processing platform and target business objects. A business object is a software abstraction of a real-world entity, representing a person, place, thing, or concept in the business domain. It can represent a user of the business processing platform or an object to which the recommendation model analyzes, processes, and makes recommendations.

[0108] It is understood that, in this embodiment of the application, all users of the business processing platform are referred to as business processing objects, and online users matching the media object to be processed are referred to as target business objects. Specifically, for example, in a live streaming scenario, this embodiment of the application refers to all users of the business processing platform where the live streaming data is located as business objects, and online users matching the live streaming data as target business objects, so that the target recommendation model recommends the live streaming data to the target business objects, and then displays the live streaming data in the recommendation display list of the target business objects.

[0109] 8. **Boosting Recommendation:** In this application's embodiments, boosting recommendation refers to recommending media objects to be processed in a boosting manner. Boosting recommendation means using specific algorithms and technical means to recommend content to target users in order to increase the content's exposure and interaction rate.

[0110] It is understood that, in this application embodiment, the process of recommending media objects with low user interaction to target users to increase the exposure and interaction rate of the media objects to be processed is referred to as "heated recommendation." Specifically, for example, in a live streaming scenario, the target recommendation model recommends the aforementioned live streaming media objects to target business objects that match the live streaming media objects to be processed, based on target recommendation features, thereby displaying the live streaming media objects to be processed in the recommendation display list of the target business objects.

[0111] Please see Figure 1 , Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application. Figure 1 As shown, this network architecture may include a server 10d and a user terminal cluster. The user terminal cluster may include one or more user terminals; the number of user terminals is not limited here. Figure 1As shown, the user terminal cluster can specifically include user terminal 10a, user terminal 10b, user terminal 10c, user terminal 20a, user terminal 20b, and user terminal 20c, etc. Server 10d can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. User terminals 10a, 10b, 10c, 20a, 20b, and 20c can all include: smartphones, tablets, laptops, PDAs, mobile internet devices (MIDs), wearable devices (such as smartwatches, smart bracelets, etc.), smart voice interaction devices, smart home appliances (such as smart TVs), and in-vehicle devices, etc., electronic devices with video / image playback functions. Figure 1 As shown, user terminals 10a, 10b, 10c, 20a, 20b, and 20c can each connect to server 10d via a network, allowing each user terminal to interact with server 10d through this network connection. For example, in a data processing scenario involving media objects, the multimedia data (e.g., live streaming data) of the media object to be processed (e.g., live streaming media object to be processed) is collected by a first local device (e.g., user terminals 10a, 10b, and 10c). This first local device (e.g., user terminals 10a, 10b, and 10c) sends the multimedia data of the media object to be processed to a cloud server, where the cloud server is, for example, device 10d. The cloud server 10d processes the received multimedia data, ultimately obtaining the target business object corresponding to the multimedia data of the media object to be processed. Further, the cloud server (e.g., the aforementioned...) Figure 1 In the corresponding embodiment, server 10d) pushes the multimedia data of the aforementioned media object to be processed to the aforementioned target service object. It should be understood that the service processing platform (e.g., a social software platform) is configured on a local device (e.g., user terminals 20a, 20b, 20c), and the aforementioned target service object is a platform user of the aforementioned service processing platform (e.g., a social software platform). The local device (e.g., user terminals 20a, 20b, 20c) receives the multimedia data (e.g., live streaming data) of the aforementioned media object to be processed (e.g., a live streaming media object to be processed) and pushes it to the aforementioned target service object.

[0112] In this embodiment, to accurately push multimedia data of a media object to a target business object for recommendation and promotion, the multimedia data of the media object to be processed must first be uploaded to a local device (e.g., user terminals 10a, 10b, 10c). Taking a live media object as the media object to be processed, multimedia data as the live data to be processed, and historical media objects as historical live media objects as an example, as the live data to be processed is played, the target sampling window dynamically samples the live data to be processed, extracts the audio data from the live data to be processed within the target sampling window, and uses it as the target audio data. Further, this embodiment detects text information in the target audio data. If text information is detected, the speech recognition component performs speech recognition processing on the target audio data to obtain the target speech recognition text. It is understood that if no text information is detected in the target audio data, the video data from the live data to be processed extracted by the target sampling window is extracted, and then the text information from the video data is extracted and used as the target speech recognition text. Then, a semantic understanding model is used to extract semantic information from the target speech recognition text based on the guidance prompt template data, thereby obtaining target semantic understanding information. Further, the target semantic understanding information is processed into text vector representation to obtain target text representation vectors for representing multimedia data (e.g., live streaming data) of the media object to be processed (e.g., live streaming media object to be processed).

[0113] Furthermore, in this embodiment, N historical text representation vectors are obtained to represent N historical live-stream media objects. The target text representation vector representing the live-stream media object to be processed is then compared with the N historical text representation vectors of the N historical live-stream media objects to calculate vector similarity, resulting in N vector similarities associated with the N historical live-stream media objects. Based on these N vector similarities, K historical live-stream media objects are selected from the N historical live-stream media objects, establishing an explicit similarity association between the live-stream media object to be processed and the K historical live-stream media objects. Further, K historical recommendation features of the K historical live-stream media objects are obtained, and these features are pooled to obtain a target recommendation representation representing the live-stream media object to be processed. The target recommendation model then recommends the live-stream media object to be processed to target business objects that match the live-stream media object to be processed using these target recommendation features.

[0114] It should be noted that in this embodiment, the local devices (user terminals 10a, 10b, 10c) of the live media object to be processed (i.e., the aforementioned media object to be processed) and the local devices (user terminals 20a, 20b, 20c) of the target service object are different user terminals. It should be understood that the servers involved in the embodiments of this application for speech recognition processing, video text recognition, text information extraction, text vector representation processing, and similarity calculation are the same server.

[0115] For ease of understanding, let's take the media object to be processed as a live streaming media object, the multimedia data as the live streaming data to be processed, and the historical media object as a historical live streaming media object as an example. The broadcaster object of the live streaming media object to be processed starts live streaming on the local device (user terminal 10a, 10b, 10c). The local device (user terminal 10a, 10b, 10c) collects live streaming data to be processed for a duration equal to the target sampling window, obtaining the live streaming data to be processed for a duration equal to the target sampling window. Further, the local device (user terminal 10a, 10b, 10c) sends the collected live streaming data to be processed to the cloud server (e.g., the aforementioned...). Figure 1 The corresponding embodiment's server 10d). This further enables the cloud server (e.g., the one described above). Figure 1 In the corresponding embodiment, server 10d) extracts audio data from the aforementioned live stream data to be processed, obtaining target audio data associated with the acquired live stream data to be processed. Further, speech recognition processing is performed on the target audio data to obtain a speech recognition processing result. If the speech recognition processing result indicates that text information in the target audio data has been recognized, the text information is output to obtain the target speech recognition text. If the speech recognition processing result indicates that no text information in the target audio data has been recognized, video data of the aforementioned live stream data to be processed is acquired, text information in the video data is recognized, and the recognized text is used as the target speech recognition text. Further, the semantic understanding model of the speech conversion and understanding component extracts information from the target speech recognition text based on the guidance prompt template data to obtain target semantic understanding information, thereby performing text vector representation processing to obtain a target text representation vector used to represent the aforementioned live stream media object to be processed. It should be understood that the cloud server (e.g., the aforementioned...) Figure 1In the corresponding embodiment, server 10d) obtains N historical text representation vectors of N historical live-stream media objects, and calculates vector similarity between the target text representation vector and each of the N historical text representation vectors to obtain N vector similarities. Then, based on the N vector similarities, K historical live-stream media objects are selected from the N historical live-stream media objects, and an explicit similarity association is established with the K historical live-stream media objects. Further, K historical recommendation features of the aforementioned K historical live-stream media objects are obtained, and the obtained K historical recommendation features are pooled to obtain target recommendation features used to represent the aforementioned live-stream media objects to be processed. This allows the target recommendation model to recommend the aforementioned live-stream media objects to target business objects that match the live-stream media objects to be processed through the aforementioned target recommendation features. It should be noted that the aforementioned target business objects are online users of social software platforms equipped on local devices (user terminals 20a, 20b, 20c). Further, cloud servers (e.g., the aforementioned...) Figure 1 In the corresponding embodiment, server 10d) sends the live streaming data to be processed of the aforementioned live streaming media object to the local device (user terminal 20a, 20b, 20c), thereby enabling the local device (user terminal 20a, 20b, 20c) to promote the aforementioned live streaming media object to be processed to the aforementioned target business object associated with the live streaming media object to be processed.

[0116] It should be understood that the media object to be processed and the historical media object are of the same type of data. For example, if the multimedia data of the media object to be processed is live data, the multimedia data of the historical media object should also be live data. If the multimedia data of the media object to be processed is video-on-demand data, the multimedia data of the historical media object should also be video-on-demand data. There are many other cases, which will not be elaborated here.

[0117] It is understood that, in the embodiments of this application, the business terminal corresponding to the media object to be processed (e.g., the live media object to be processed) and the business terminal corresponding to the target matching user may include: smart terminals such as smartphones, tablets, laptops, and desktop computers.

[0118] In this embodiment, the server that processes the multimedia data of the media object to be processed can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0119] To make it easier to understand, further, such as Figure 2 As shown, Figure 2 This is a schematic diagram of a data interaction scenario provided in an embodiment of this application. Figure 2 The user terminal 10a shown can be the one described above. Figure 1 The corresponding user terminal 10a in this context refers to a local mobile device, such as a mobile phone or tablet; further restrictions are not specified here. This user terminal 10a collects the media object to be processed (e.g., Figure 2 The multimedia data of the presenter A) (e.g., Figure 2 The live stream data corresponding to the live stream interface of host A shown above will be sent to the server (for example, the aforementioned...). Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 The server 10d in the corresponding embodiment.

[0120] Specifically, when user terminal 10a collects multimedia data, it first obtains the target dynamic sampling time point, window duration, and sampling interval duration. For ease of understanding, taking a live-stream media object as the media object to be processed and the live-stream data to be processed as the multimedia data, the first dynamic sampling time point is the first dynamic sampling time point when the duration of the live-stream data to be processed is equal to the window duration, and this is taken as the target dynamic sampling time point. The first sampling window is used as the target sampling window, and the live-stream data is captured and extracted. The live-stream data segment obtained from the target sampling window is sent to the server (e.g., the aforementioned...). Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 The server 10d in the corresponding embodiment.

[0121] Furthermore, as the live data to be processed plays, after a duration equal to the sampling interval, the second dynamic sampling time point is reached. The first dynamic sampling time point is then replaced by the second dynamic sampling time point as the updated target dynamic sampling time point. The second sampling window is used as the target sampling window to capture and extract segments of the live data to be processed within the updated target sampling window. This process of changing the target dynamic sampling time point is repeated continuously until the live data playback ends, thus achieving the effect of dynamic sampling. This will not be elaborated further here. It is important to note that the sampling interval duration should be less than or equal to the window duration to ensure that the entire live data to be processed is fully captured, and the window duration of the first sampling window is the same as that of the second sampling window.

[0122] The aforementioned server (for example, the aforementioned Figure 1 The server 10d in the corresponding embodiment and the above Figure 2In the corresponding embodiment, when server 10d receives a segment of live data to be processed sent by user terminal 10a, it obtains target audio data from the multimedia data in step S11, that is, it obtains the audio data of the live data segment to be processed collected in the target sampling window and uses it as the target audio data. Further, it obtains a speech conversion understanding component and a similarity retrieval component for speech conversion processing of the target audio data. Then, in step S12, it performs speech recognition processing using a speech recognizer, that is, it uses the speech recognizer in the speech conversion understanding component to perform speech recognition processing on the target audio data to obtain a speech recognition processing result. If the speech recognition processing result is the recognition of target semantic recognition text in the target audio data, then in step S13, it performs text semantic understanding processing using a semantic understanding model. Specifically, it performs text semantic understanding processing based on the guidance prompt template data using the semantic understanding model in the speech conversion understanding component to obtain target semantic understanding information associated with the target speech recognition text. Then, in step S14, text vector representation processing is performed on the text representation, that is, the text representation device in the similarity retrieval component is used to perform text vector representation processing on the target semantic understanding information to obtain the target text representation vector used to represent the live data to be processed.

[0123] Specifically, if the media object to be processed is a video-on-demand type, then the prompt template data associated with the video-on-demand type will be used as the guidance prompt template data. Furthermore, if the speech recognition processing result is that the target semantic recognition text in the target audio data is not recognized, then the target audio data will be marked as invalid audio data. Then, the video data of the live data to be processed collected in the target sampling window will be acquired, and the text in the image will be recognized and used as the target speech recognition text.

[0124] Further, step S21 is performed to obtain the historical text representation vector. Specifically, the server (e.g., the one mentioned above) Figure 1 The server 10d in the corresponding embodiment and the above Figure 2In the corresponding embodiment, server 10d) retrieves N historical text representation vectors of N historical live-stream media objects from the historical media object database, and then calculates vector similarity in step S31, that is, calculates the vector similarity between the target text representation vector and the N historical text representation vectors respectively, to obtain N vector similarities. Then, step S32 is executed to filter the Top K historical media objects with the highest similarity. Specifically, based on the above vector similarity, K historical live-stream media objects corresponding to the top K historical text representation vectors with the highest similarity are selected, and an explicit similarity association is established with them. Further, step S33 is executed to obtain the historical recommendation features of the K historical media objects, and then performs pooling processing, thereby executing step S34 to calculate the target recommendation features based on the K historical recommendation features. It should be noted that the above target recommendation features are the recommendation features of the live-stream media objects to be processed. Further, step S35 is executed to execute the target recommendation model to recommend the live-stream media objects to be processed to the target business objects based on the target recommendation features, that is, the target recommendation model recommends the live-stream media objects to be processed to the target business objects that match the live-stream media objects to be processed.

[0125] Specifically, the aforementioned N historical live-stream media objects are selected from all historical live-stream data based on the interaction data curve distribution corresponding to the historical media object database, and are those that have reached a threshold in terms of interaction data volume. Then, N historical text representation vectors for these N historical live-stream media objects are obtained from the historical text vector representation database. It should be understood that the obtained N historical text representation vectors for the N historical live-stream media objects are obtained by processing the historical live-stream data; the processing method is the same as the processing method for the live-stream data to be processed mentioned above, and will not be elaborated here. It is important to note that the historical audio data corresponding to the historical text representation vectors is obtained from a historical sampling window whose historical dynamic sampling time point is the end of the historical live-stream data playback and whose duration is the same as the historical window duration. The historical window duration is the same as the window duration of the target sampling window mentioned above.

[0126] Specifically, K historical recommendation features are obtained from the recommendation feature database associated with K historical live-stream media objects.

[0127] It is important to note that in order for the target recommendation model to recommend the live-streaming media object to the target business object that matches the live-streaming media object to be processed based on the target recommendation features, it is first necessary to obtain N historical interaction behavior information of the aforementioned N historical live-streaming media objects, and use the aforementioned N historical interaction behavior information as training samples for training the initial recommendation model. Further, the server (e.g., the aforementioned...) Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 In the corresponding embodiment, server 10d) obtains the recommendation index of the training samples and trains the initial recommendation model using the recommendation index and the training samples to obtain the target recommendation model.

[0128] Furthermore, the target recommendation model deployed in the business processing platform (e.g., social software platform) configured on user terminal 20a directs the target recommendation model to the aforementioned live media objects to be processed (e.g., Figure 2 The media data to be processed by the presenter A (e.g., Figure 2 The target business object matched with the live streaming data to be processed (as shown in the interface of anchor A) is the online user of the social software platform configured on user terminal 20a. Furthermore, the server (e.g., the aforementioned...) recommends the live streaming media object to be processed. Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 In the corresponding embodiment, server 10d sends the aforementioned live streaming data to be processed to user terminal 20a, so that user terminal 20a can send the aforementioned live streaming media object to be processed (e.g., Figure 2 The live streaming data to be processed for streamer A shown below (e.g., Figure 2 The live streaming data to be processed corresponding to the interface of anchor A shown in the figure is pushed to the above-mentioned target business object, so that on the user terminal 20a, as shown in the figure, the data is processed. Figure 2 The recommended display list W shown above displays the aforementioned live media objects to be processed.

[0129] Specifically, the aforementioned target recommendation model is deployed on a business processing platform (e.g., a social media platform) on user terminal 20a. It acquires object association data information of business objects associated with the business processing platform (e.g., the social media platform) and inputs this object association data information into the target recommendation model for information vector representation extraction. This yields object vector representation features of the business objects. The target recommendation model then filters object vector representation features that match the target recommendation features and designates the business objects corresponding to the selected object vector representation features as the target business objects that match the aforementioned live-streaming media objects to be processed. Further, the aforementioned live-streaming media objects to be processed are used as target recommendation objects for recommendation to the target business objects, and the target recommendation objects are pushed to the terminal devices corresponding to the target business objects, so that the terminal devices display the target recommendation objects in the recommendation display list of the business processing platform.

[0130] This application embodiment filters out historical media objects associated with the media object to be processed by associating the target text representation vector of the media object to be processed with the historical text representation features of historical media objects. Further, this application embodiment obtains the historical recommendation features of the historical media objects associated with the media object to be processed, and then processes these historical recommendation features to obtain the target recommendation features of the media object to be processed. A target recommendation model is then used to recommend the media object to be processed to target business objects that match the media object to be processed based on these target recommendation features.

[0131] Furthermore, such as Figure 3 As shown, Figure 3 This is a flowchart illustrating a data processing method for a media object according to an embodiment of this application. It can be understood that this embodiment first involves a first local device (e.g., the one described above). Figure 1 The corresponding 10a, 10b, 10c and the above Figure 2 The corresponding 10a) collects multimedia data of the media object to be processed, and then sends the multimedia data to the server (e.g., the aforementioned Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 The server 10d in the corresponding embodiment processes the data to obtain a target service object that matches the media object to be processed. Further, the server sends the multimedia data of the media object to be processed to a second local device (e.g., the aforementioned...). Figure 1 The corresponding 20a, 20b, 20c and the above Figure 2 Corresponding to 20a), the second local device promotes the multimedia data of the media object to be processed to the target recommendation object, thereby displaying the media object to be processed in the recommendation display list of the service processing platform (e.g., social software platform) deployed on the second local device. It should be understood that, in this embodiment, the data processing method for the media object may include at least the following steps S101 to S104:

[0132] Step S101: Obtain the target audio data of the media object to be processed in the target sampling window, and perform text vector representation processing on the target speech recognition text corresponding to the target audio data to obtain the target text representation vector used to represent the media object to be processed.

[0133] The target audio data is the audio data of the media object to be processed, which is collected by the target sampling window; the target text representation vector is obtained by representing the target speech recognition text associated with the target audio data as a text vector.

[0134] Specifically, this involves acquiring a speech conversion understanding component and a similarity retrieval component for speech conversion processing of the target audio data. The speech conversion understanding component includes a speech recognizer and a semantic understanding model for semantic extraction, while the similarity retrieval component includes a text representation.

[0135] It should be noted that the above-mentioned speech conversion and understanding component is equivalent to the above-mentioned audio data speech recognition module. Similarly, the above-mentioned similarity retrieval component is equivalent to the above-mentioned vector similarity calculation module.

[0136] The target audio data is processed by a speech recognizer to obtain the speech recognition result. If the speech recognition result indicates that the target audio data has been recognized as target speech recognition text, then the target speech recognition text is processed by a semantic understanding model to obtain the target semantic understanding information of the target speech recognition text. The target semantic understanding information is then vectorized by a text representation unit to obtain the target text representation vector used to represent the media object to be processed.

[0137] Specifically, the server (e.g., the one mentioned above) Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 In the corresponding embodiment, server 10d) obtains a speech conversion understanding component and a similarity retrieval component that perform speech conversion processing on the target audio data. The speech conversion understanding component includes a speech recognizer and a semantic understanding model for semantic extraction.

[0138] Specifically, if the speech recognition processing result indicates that the target audio data was not recognized, the video data of the media object to be processed is obtained from the multimedia data of the media object to be processed through the target sampling window. The obtained video data is used as the target video data of the media object to be processed in the target sampling window. Video content recognition is performed on the video frames in the target video data to obtain the video text content used to represent the target video data. The target speech recognition text is updated using the video text content. If the speech recognition processing result indicates that the target audio data was not recognized, the target audio data is marked as invalid audio data.

[0139] For ease of understanding, when the media object to be processed is a live media object, and the multimedia data of the media object to be processed is the live data to be processed, and simultaneously, the historical media object is a historical live media object, and the multimedia data of the historical media object is historical live data, considering that the live data to be processed may contain audio data or not, the speech recognizer in the aforementioned speech conversion and understanding component performs speech recognition processing on the target audio data to obtain the speech recognition processing result. Further, if the speech recognition processing result indicates that the live data to be processed contains audio data, then the target speech recognition text associated with the target audio data of the live data to be processed is output. It should be understood that if the speech recognition processing result indicates that the live data to be processed does not contain audio data, then the target audio data is marked as invalid audio data and will not be used subsequently. Then, the video data of the live data to be processed is obtained and used as the target video data, thereby recognizing the text information in the target video data, outputting the video recognition text associated with the live data to be processed, and using this as the target speech recognition text.

[0140] Specifically, when the aforementioned speech recognizer is Automatic Speech Recognition (ASR), the speech recognition method is as follows, and the target speech recognition text can be determined using the following formula (1):

[0141] text i =asr(i c (1)

[0142] Where i represents the cold start / long tail live room to be heated; c represents the target sampling window under the current timestamp; and asr represents the automatic speech recognition algorithm, used to extract the ASR text information of the live data to be processed within the target sampling window.

[0143] Furthermore, the target speech recognition text obtained from the above speech recognition is processed by the semantic understanding model used for semantic extraction in the speech conversion and understanding component to obtain target semantic understanding information.

[0144] Specifically, this involves acquiring guidance prompt template data to guide the semantic understanding model's output. The guidance prompt template data and the target speech recognition text are input into the semantic understanding model. The model then extracts semantic information from the target speech recognition text according to the data content in the guidance prompt template data, obtaining the target semantic extraction information. The target semantic understanding information of the target speech recognition text is then determined based on this extracted information.

[0145] It should be understood that the aforementioned semantic understanding model requires an output template to guide it during semantic extraction, thereby outputting the target semantically understood text. Furthermore, the server (e.g., the aforementioned...) Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 In the corresponding embodiment, server 10d) acquires guidance prompt template data to guide the semantic understanding model in its output. When the media object to be processed is a live-streaming media object, the multimedia data of the media object to be processed is the live-streaming data to be processed, and the semantic understanding model is a Large Language Model (LLM), since the LLM has understanding and reasoning capabilities, the guidance prompt template data may include, for example, the style of the live-streaming media object, the live-streaming theme, the live-streaming content, the community atmosphere, the live-streaming audience, and potential topics; this is not limited here.

[0146] Furthermore, by inputting the aforementioned guidance prompt template data and the aforementioned target speech recognition text into the aforementioned large language model, the large language model will perform semantic extraction on the target speech recognition text based on the guidance prompt template data, obtaining target semantic extraction information associated with the aforementioned target speech recognition text. Subsequently, it outputs target semantic understanding information including themes such as the broadcaster's style, live broadcast theme, live broadcast content, community atmosphere, live broadcast audience, and potential topics; the output content is not limited here.

[0147] Specifically, when the semantic understanding model is a large language model, the process of semantic understanding of the target speech recognition text is as follows, and the target semantic understanding information can be determined by the following formula (2):

[0148] semantic i =llm(promt(text) i (2)

[0149] Wherein, text refers to the target speech recognition text in the above formula (1); i represents the cold start / long tail live room to be heated; prompt refers to the LLM text generation template, that is, the above guidance prompt template data; llm is a pre-trained large language model; since ASR text information and generated live room multi-dimensional semantic information can be classified as natural language, the embodiments of this application apply the reasoning ability of LLM.

[0150] It should be noted that if the media object to be processed is of the live video type, the prompt template data associated with the live video type will be used as the guidance prompt template data. If the media object to be processed is of the video-on-demand type, the prompt template data associated with the video-on-demand type will be used as the guidance prompt template data.

[0151] It should be understood that this specific embodiment refers to the case where the multimedia data of the media object to be processed is live streaming data, and the aforementioned guidance prompt template data is prompt template data associated with the live video streaming type. Similarly, if the multimedia data of the media object to be processed is video-on-demand data, the aforementioned guidance prompt template data is prompt template data associated with the video-on-demand type, such as themes like character dialogue, character style, and scene style, which are not limited here.

[0152] Furthermore, the target semantic understanding information output by the above semantic understanding model is processed into a text vector representation, thereby obtaining the target text representation vector for subsequent similarity retrieval processing.

[0153] Step S102: Obtain N historical text representation vectors to represent N historical media objects, and perform vector similarity calculation on the target text representation vector and each historical text representation vector to obtain N vector similarity.

[0154] A historical text representation vector is obtained by processing the historical audio data of a historical media object through speech conversion using the speech conversion understanding component and the similarity retrieval component; the N vector similarities are calculated by performing vector similarity calculations between the target text representation vector and the N historical text representation vectors.

[0155] Specifically, the target text representation vector is obtained by performing speech conversion processing on the target audio data using the speech conversion understanding component and the similarity retrieval component. Furthermore, obtaining N historical text representation vectors to represent N historical media objects includes: obtaining the interaction data curve distribution corresponding to the historical media object database. The interaction data curve distribution is determined based on each historical media object in the historical media object database and the amount of interaction data for each historical media object. Using the interaction data curve distribution, N historical media objects whose interaction data volume reaches the interaction data threshold are obtained from the historical media object database. N historical text representation vectors for the N historical media objects are obtained from the historical text vector representation database associated with the N historical media objects. Each historical text representation vector is obtained by performing speech conversion processing on the historical audio data of a historical media object using the speech conversion understanding component and the similarity retrieval component.

[0156] For ease of understanding, such as Figure 4 As shown, Figure 4 This is a schematic diagram of the interaction data curve of a historical media object according to an embodiment of this application. Server (e.g., the one described above) Figure 1 The server 10d in the corresponding embodiment and the above Figure 2In the corresponding embodiment, server 10d) acquires a historical database, which includes a historical media object database and a historical text vector representation database. Further, based on the interaction data curve distribution corresponding to the historical media object database, the top 10% of historical media objects in terms of interaction quantity are selected, including N historical media objects. Since the multimedia data type of the media objects to be processed is live data, the multimedia data type of the historical media objects should also be live data. Then, N historical text representation vectors of the selected N historical live data are obtained from the historical text vector representation database.

[0157] It should be understood that the generation process of the N historical text representation vectors for the aforementioned N historical live media objects is the same as the generation process of the target text representation vector for the aforementioned media objects to be processed. It is important to note that the aforementioned target audio data is the audio data collected in the target sampling window with a duration equal to the window length as the target dynamic sampling time point changes with the playback of the live data to be processed; it is dynamically changing. The historical audio data corresponding to the aforementioned historical live data is the audio data collected in the historical sampling window with a duration equal to the window length at the time point when the historical dynamic sampling time point ends. Furthermore, the aforementioned speech conversion understanding component and similarity retrieval component perform speech conversion processing on the aforementioned historical audio data to obtain the historical text representation vector. It is important to note that the window length of the aforementioned target sampling window is the same as the window length of the aforementioned historical sampling window.

[0158] Specifically, when the semantic understanding model is a large language model, the implementation of semantic extraction for the live data to be processed and the historical live data is as follows: the target semantic understanding information associated with the target speech recognition text can be determined by the following formula (3), and the historical text representation vector associated with the historical semantic understanding information can be determined by the following formula (4):

[0159] semantic i =llm(promt(text) c ))i∈C (3)

[0160] semantic j =llm(promt(text) c ))j∈H (4)

[0161] Wherein, text refers to the target speech recognition text in the above formula (1); i represents the cold start / long tail live room to be heated; j represents the historical high-interaction live room; c represents the target sampling window under the current timestamp; prompt refers to the LLM text generation template in the above formula (2), that is, the above guidance prompt template data, llm is the pre-trained large language model, C represents the sum of multimedia data of the current media object to be processed; H represents the sum of multimedia data of the historical media object.

[0162] Furthermore, the specific implementation methods for text vector representation processing of the target speech understanding information of the media object to be processed and the historical semantic understanding information of the historical media object are as follows: Specifically, the target text representation vector associated with the above target semantic understanding information can be determined by the following formula (5), the historical text representation vector associated with the above historical semantic understanding information can be determined by the following formula (6), and the vector similarity between the above target text representation vector and the above historical text representation vector can be obtained by formula (7):

[0163] e i =embedding(semantic i )i∈C (5)

[0164] e j =embedding(semantic j )j∈H (6)

[0165] s i,j =similarity(e i ,e j (7)

[0166] Where i represents the cold start / long tail live streaming room to be heated; j represents the historical high-interaction live streaming room; semantic refers to the target semantic understanding information in the above formula (3) or formula (4); embedding represents the text vectorization algorithm; similarity represents the vector similarity measurement function.

[0167] Furthermore, after obtaining the N historical text representation vectors of the aforementioned N historical live broadcast data, the target text representation vector associated with the live broadcast media object to be processed is then compared with the N historical text representation vectors of the aforementioned N historical live broadcast media objects to calculate vector similarity. The methods for calculating vector similarity include cosine similarity, Jaccard similarity, Euclidean distance, and Manhattan distance, etc., and are not limited here.

[0168] Step S103: When selecting K historical media objects from N historical media objects using N vector similarity, establish similarity associations between the K historical media objects and the media objects to be processed.

[0169] Specifically, N historical media objects are sorted based on N vector similarity scores to obtain a list of historical media objects associated with the N historical media objects. K historical media objects whose vector similarity satisfies the similarity retrieval criteria are then selected from this list, and explicit similarity associations are established between these K historical media objects and the media object to be processed. These explicit similarity associations characterize the list of the K historical media objects as a list of similar media objects, and ensure that any media object in this list is associated with the media object to be processed.

[0170] For ease of understanding, the server (e.g., the one mentioned above) Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 In the corresponding embodiment, server 10d) sorts the N historical live-stream media objects obtained above from high to low based on the calculated similarity of the N vectors, obtaining a list of historical media objects associated with the N historical live-stream media objects. Further, based on the list of historical media objects, the top K historical live-stream media objects are selected, namely the Top K historical live-stream media objects, and then an explicit similarity association is established between the K historical live-stream media objects and the live-stream media objects to be processed.

[0171] It should be understood that, such as Figure 5 As shown, Figure 5 This is a schematic diagram of an explicit similarity association in an embodiment of this application. Taking N=3 and K=2 as an example, the similarity between the target text representation vector and the N historical text representation vectors (here N=3, i.e., 3 historical text representation vectors) is calculated to obtain 3 vector similarity scores 5b. Then, the 3 historical media objects 5a are sorted from high to low according to the 3 calculated vector similarity scores 5b to obtain a list of historical media objects associated with the 3 historical live broadcast data. Further, in this embodiment of the application, the historical live broadcast data of the top K historical media objects (here K=2, i.e., 2 historical live broadcast data) are filtered based on the above historical media object list to obtain 2 historical media objects associated with the media object to be processed.

[0172] Based on the list of historical media objects associated with N historical live stream data points (3 in this case), the top two historical live stream media objects are selected and established with the live stream data to be processed through explicit similarity association. Furthermore, based on these two selected historical live stream media objects, a list of similar media objects associated with the K historical live stream media objects is obtained, where each historical live stream media object has an association with the live stream media object to be processed.

[0173] Based on the vector similarity calculated above, a list of the Top K historical high-interaction media objects' semantic nearest neighbors is retrieved for each media object to be processed:

[0174] r i ←[J0J1…J K (8)

[0175] Where J represents the identity document (ID) index of historical highly interactive media objects; K represents the K most similar live streaming rooms retrieved through vector similarity.

[0176] Step S104: When K historical recommendation features of K historical media objects are obtained, target recommendation features are determined based on similarity relationships and K historical recommendation features to characterize the media objects to be processed. Media recommendations are then made to target business objects that match the media objects to be processed based on the target recommendation features.

[0177] Specifically, the server (e.g., the one mentioned above) Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 In the corresponding embodiment, server 10d) obtains K historical recommendation features of the K historical live-streaming media objects from the recommendation feature database associated with the K historical live-streaming media objects, and performs pooling processing on the K historical recommendation features. This yields the target recommendation features for the live-streaming media objects to be processed. Furthermore, the target recommendation model recommends the live-streaming media objects to be processed to target business objects that match the live-streaming media objects to be processed based on the target recommendation features.

[0178] This application embodiment can identify the audio data of the media object to be processed within the target sampling window, and obtain target audio data associated with the media object to be processed. Further, this application embodiment uses a speech recognizer in the speech conversion and understanding component to perform speech recognition processing on the target audio data, thereby obtaining target speech recognition text associated with the target audio data. Further, the semantic understanding model in the speech conversion and understanding component extracts information from the target speech recognition text based on guidance prompt template data, obtaining target semantic understanding information associated with the target speech recognition text. Then, the target semantic understanding information is processed into text vector representation, obtaining a target text representation vector used to represent the media object to be processed. N historical text representation vectors are obtained to represent N historical media objects, and then vector similarity calculations are performed between the target text representation vector and each of the N historical text representation vectors to obtain N vector similarities. Further, based on the N vector similarities, this application embodiment selects K historical media objects with high similarity to the media object to be processed from the N historical media objects, and then establishes an explicit similarity association between the media object to be processed and the selected K historical media objects. This application embodiment obtains K historical recommendation features of the aforementioned K historical media objects, determines the target recommendation features of the media object to be processed based on the aforementioned explicit similarity association, and recommends media to target business objects that match the media object to be processed through the target recommendation features. This application embodiment sequentially performs speech recognition processing, semantic understanding information extraction, and text vector representation processing on the target audio data to obtain a target text representation vector representing the target audio data. Similarity calculation is then performed with the historical text representation vectors of historical media objects to retrieve K historical media objects with high relevance to the media object to be processed. The target recommendation features representing the media object to be processed are then calculated using the historical recommendation features of the retrieved K historical media objects. Finally, the recommendation model accurately retrieves the target business object corresponding to the media object to be processed, and pushes the media data to be processed to it. This improves the accuracy of the target recommendation model in matching the media object to be processed with the business object during a cold start, and also achieves the effect of "warming up" the media object to be processed.

[0179] In one embodiment, when the type of the media object to be processed is a live streaming media object, the live streaming media object to be processed is promoted as a recommended service to the target recommended object of user A (i.e., the target business object matching the media object to be processed in the aforementioned business processing platform) who is matching the live streaming media object. This application embodiment can promote the live streaming media object to the target business object (e.g., user A) by displaying an interstitial advertisement for the live streaming media object in the XX service being used by the target business object (e.g., user A) when the target business object (e.g., user A) using other services (e.g., XX service) of the business processing platform (e.g., social software platform). The aforementioned business processing platform 6c includes the target business object 6a user A and the promoted live streaming media object 6b.

[0180] For ease of understanding, such as Figure 6 As shown, Figure 6 This is a schematic diagram illustrating a scenario of media recommendation for live streaming media objects on a business processing platform, according to an embodiment of this application. When the target business object 6a (for example, Figure 6 User A) in the context of using other services on the business processing platform 6c (e.g., Figure 6 When XX is in the local device (e.g., Figure 1 The user terminals 20a, 20b, 20c in the corresponding embodiments and Figure 2 The corresponding user terminal 20a) displays the target recommended object on the aforementioned XX page (e.g., Figure 6 The system then recommends the aforementioned live streaming media object 6b to user A (which is currently pending processing of live streaming media objects).

[0181] Furthermore, the second local device (e.g., Figure 1 The user terminals 20a, 20b, 20c in the corresponding embodiments and Figure 2 The corresponding user terminal 20a) responds to the target service object regarding the above. Figure 6 The multimedia data display interface of the live media object 6b is displayed upon triggering an operation, which is a schematic diagram of a live broadcast scenario, where the live broadcast room is the live media object to be processed. The multimedia data display interface of the live media object includes an avatar display area 7a and an interactive display interface area 7b for the media object to be processed.

[0182] For ease of understanding, such as Figure 7 As shown, Figure 7 This is a schematic diagram illustrating a scenario of live streaming in a live streaming room, according to an embodiment of this application. It should be understood that the aforementioned live streaming room is a live streaming media object to be processed, and this interface is for the target business object to click on. Figure 6After processing the live media object in area 6b, the second local device (e.g., Figure 1 The user terminals 20a, 20b, 20c in the corresponding embodiments and Figure 2 The corresponding user terminal 20a) responds to the target service object regarding the above. Figure 6 The multimedia data display interface of the live media object is displayed when the operation of the live media object 6b to be processed is triggered.

[0183] Accordingly, it should be understood that in one embodiment, when the type of the media object to be processed is a video-on-demand media object, the video-on-demand media object is promoted as a recommended service to the target recommended object of user A (i.e., the target service object matching the media object to be processed in the aforementioned service processing platform) who matches the video-on-demand media object. This application embodiment can encapsulate the video-on-demand link of the video-on-demand media object into a recommendation notification message, thereby enabling the second local device (e.g., Figure 1 The user terminals 20a, 20b, 20c in the corresponding embodiments and Figure 2 The corresponding user terminal 20a) displays the recommendation notification message on the usage page of the target business object (e.g., user A). The recommendation notification message includes pop-ups, floating windows, etc., which are not limited here. It should be understood that the target business object (e.g., user A) can enter the playback interface of the on-demand media object by clicking on the recommendation notification message. The on-demand identifier is a scenario illustration for video playback, where the on-demand identifier represents the on-demand media object to be processed. The display page of the target business object includes the target business object 8a, the recommendation notification message 8b, and the business processing platform 8c.

[0184] For ease of understanding, such as Figure 8 As shown, Figure 8 This is a schematic diagram illustrating a scenario of media recommendation for on-demand media objects on a business processing platform, according to an embodiment of this application. The second local device (e.g., Figure 1 The user terminals 20a, 20b, 20c in the corresponding embodiments and Figure 2 The corresponding user terminal 20a) is in the target service object 8a (for example, Figure 8 User A's page displays a recommendation notification message 8b (for example, ...). Figure 8 (The pop-up window in the middle). Furthermore, the target business object 8a (for example, Figure 8 User A can click on the recommendation notification message to enter the multimedia data display interface of the aforementioned on-demand media object, thereby recommending the on-demand media object to User A.

[0185] Furthermore, the second local device (e.g., Figure 1 The user terminals 20a, 20b, 20c in the corresponding embodiments and Figure 2 The corresponding user terminal 20a) responds to the target service object's triggered operation of the recommendation notification message by jumping to and displaying the multimedia data display interface of the on-demand media object. The multimedia data display interface of the on-demand media object includes a playback progress bar 9a.

[0186] For ease of understanding, such as Figure 9 As shown, Figure 9 This is a schematic diagram illustrating a scenario where a video-on-demand identifier is used for video playback, according to an embodiment of this application. It should be understood that the aforementioned video-on-demand identifier is a video-on-demand media object to be processed. This interface is used after the target service object clicks the recommendation notification message, and then the second local device (e.g., Figure 1 The user terminals 20a, 20b, 20c in the corresponding embodiments and Figure 2 The corresponding user terminal 20a) responds to the target business object's trigger operation on the recommendation notification message to jump to and display the multimedia data display interface of the on-demand media object.

[0187] In one embodiment, after obtaining the TopK historical live-stream media objects (i.e., historical media objects) through vector similarity, a target recommendation feature is calculated based on the historical recommendation features of the aforementioned K historical live-stream data. This target recommendation feature is used to find online matching users (i.e., target business objects) that match the live-stream media object to be processed (i.e., the media object to be processed), so that the live-stream media object to be processed (i.e., the media object to be processed) is displayed on the recommendation display list of online matching users on the social software platform (i.e., the business processing platform). The recommendation display list includes a broadcaster display area 100a and a recommendation list area 100b. Further, the recommendation list area 100b contains the media object corresponding to each broadcaster in the broadcaster display area 100a, including the live-stream media object to be processed 100c.

[0188] For ease of understanding, such as Figure 10 As shown, Figure 10 This is a schematic diagram of a recommendation display list of a business processing platform according to an embodiment of this application. After the target recommendation model filters out the target business objects that match the live data to be processed, the second local device (e.g., Figure 1 The user terminals 20a, 20b, 20c in the corresponding embodiments and Figure 2 The corresponding user terminal 20a) pushes the received live streaming data to be processed to the target matching user, so that the media object to be processed appears in the recommendation display list of the target matching user's social matching platform for the target recommended user to select. It should be noted that the recommendation list area 100b includes the live streaming media object 100c to be processed. Furthermore, the multimedia data display interface of the live streaming media object 100c to be processed (e.g., Figure 7 You can jump to the display by clicking on the media object A1 area in the recommended list area 100b.

[0189] This application primarily processes the audio data of the media object to be processed, and then processes the recognized speech-text to find a target business object that matches the media object to be processed. If the speech recognition processing result indicates that audio data exists in the target audio data, then the target speech-recognition text associated with the target audio data of the media object to be processed is output. It should be understood that if the speech recognition processing result indicates that the live data to be processed does not contain audio data, then the target audio data is marked as invalid audio data and will not be used further. Furthermore, the subsequent processing may fall into two categories. One is to terminate directly and not process the media object to be processed again. The other is to obtain the video data of the media object to be processed, use the obtained video data as the target video data, recognize the text in the target video data, and process the recognized text as the target speech-recognition text.

[0190] Among them, such as Figure 11 As shown, Figure 11 This is a flowchart of a target audio invalidation processing method according to an embodiment of this application. Server (e.g., the one described above) Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 In the corresponding embodiment, after receiving the target audio data of the media object to be processed, the server 10d) performs speech recognition processing on the target audio data through the speech recognizer in the speech conversion and understanding component. The process ends directly when no text information is recognized in the target audio. Figure 11 As shown, the method may include at least steps S2011-S2016.

[0191] Step S2011: Obtain target audio data.

[0192] Specifically, this involves acquiring the target audio data of the aforementioned media object to be processed. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0193] Step S2012: Detect whether there is audio in the target audio data.

[0194] Specifically, the speech recognizer in the speech conversion and understanding component is used to perform speech recognition processing on the target audio data to obtain the speech recognition result. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0195] After completing this step, you can proceed to either step S2013 or step S2015.

[0196] Step S2013: Identify the text in the target audio data.

[0197] Specifically, when audio data is detected in the target audio, text information within that target audio data is identified. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0198] Step S2014: Output the target speech recognition text.

[0199] Specifically, text information is identified in the target audio data, and the target speech-recognized text is output. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here. After completing this step, proceed to step S2016.

[0200] Optionally, after performing step S2012, you can also jump to perform step S2015.

[0201] Step S2015: Mark the target audio data as invalid audio data.

[0202] Specifically, if no audio data is detected for the target audio, then the target audio data is marked as invalid audio data. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0203] Step S2016: End.

[0204] Specifically, the semantic understanding model in the speech conversion and understanding component performs text semantic understanding processing on the target speech recognition text to obtain the target semantic understanding information of the target speech recognition text. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0205] Furthermore, such as Figure 12 As shown, Figure 12 This is a flowchart of a target audio invalidation processing method according to an embodiment of this application. When the video data of the media object to be processed is obtained, the obtained video data of the media object to be processed is used as target video data, and the text in the target video data is recognized, and the recognized text is processed as target speech recognition text, such as... Figure 12As shown, the method may include at least steps S2021-S2028.

[0206] Step S2021: Obtain audio data.

[0207] Specifically, this involves acquiring the target audio data of the aforementioned media object to be processed. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0208] Step S2022: Detect whether there is audio in the target audio data.

[0209] Specifically, the speech recognizer in the speech conversion and understanding component is used to perform speech recognition processing on the target audio data to obtain the speech recognition result. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0210] After completing this step, you can proceed to step S2023 or step S2024.

[0211] Step S2023: Identify the text in the target audio data.

[0212] Specifically, when audio data is detected in the target audio, text information within that target audio data is identified. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here. After completing this step, proceed to step S2027.

[0213] Optionally, after performing step S2022, you can also jump to perform step S2024.

[0214] Step S2024: Mark the target audio data as invalid audio data.

[0215] Specifically, if no audio data is detected for the target audio, then the target audio data is marked as invalid audio data. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0216] Step S2025: Obtain the video data corresponding to the target audio data.

[0217] Specifically, this involves acquiring the video data corresponding to the target audio data of the aforementioned media object to be processed, and using this video data as the target video data. See the above for details. Figure 3The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0218] Step S2026: Identify the text in the target video data.

[0219] Specifically, text recognition processing is performed on the target video data of the aforementioned media object to be processed. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0220] Step S2027: Output the target speech recognition text.

[0221] Specifically, the text information identified from the target audio data and the text information identified from the target video data are output as the target speech recognition text. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0222] Step S2028: End.

[0223] Specifically, the semantic understanding model in the speech conversion and understanding component performs text semantic understanding processing on the target speech recognition text to obtain the target semantic understanding information of the target speech recognition text. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0224] This application primarily describes a method for "warming up" media objects during cold starts or in long-tail scenarios. This application obtains target audio data of the media object to be processed and performs speech recognition processing to obtain target speech recognition text. Then, the target speech recognition text undergoes semantic understanding processing and is further processed into text vector representation to obtain target text representation vectors. Further, this application obtains N historical text representation vectors representing N historical media objects, and calculates vector similarity between the target text representation vector and each of the N historical text representation vectors. Then, based on the calculated vector similarity, the K historical media objects with the highest similarity are selected, and an explicit similarity association is established with the selected K historical media objects. Further, this application obtains K historical recommendation features of the K historical media objects, and performs pooling processing on these K historical recommendation features to obtain target recommendation features for the media object to be processed. Based on the target recommendation features, a recommendation model is used to recommend the media object to be processed to target business objects that match the media object to be processed.

[0225] For ease of understanding, such as Figure 13 As shown, Figure 13 This is a schematic diagram of a target business object estimation process for media objects according to an embodiment of this application. This embodiment first obtains target audio data (duration equal to the window duration) of the long-tail / cold-start media object 13a (i.e., the multimedia data of the aforementioned media object to be processed) within the target sampling window. Then, the audio data speech recognition module 13b performs speech recognition processing on the target audio data to obtain the target speech recognition text. It should be noted that the target sampling window performs real-time dynamic sampling within the multimedia data of the media object to be processed; for details, please refer to the above... Figure 3 The specific process of audio data sampling in the corresponding embodiments will not be repeated here. Further, based on the guidance prompt template data of the audio data speech recognition module 13b, information is extracted from the target speech recognition text using a semantic understanding model to obtain target semantic understanding information. Then, the target semantic understanding information is processed into text vector representation to obtain the target text representation vector, as detailed above. Figure 3 The specific process of audio data sampling described in the corresponding embodiments will not be repeated here. It should be noted that the audio data speech recognition module 13b described above is equivalent to the speech conversion and understanding component described above.

[0226] Furthermore, in this embodiment, the vector similarity calculation module 13c obtains N historical text representation vectors for representing N historical media data, and performs vector similarity calculations between the target text representation vector and the N historical text representation vectors respectively. It should be noted that the method for obtaining the N historical text representation vectors is the same as the method for obtaining the target text representation vector, but the historical audio data is audio data with a duration equal to the window duration collected by the historical sampling window at the end time of the multimedia data of the historical media object during the historical dynamic sampling time point. For details, please refer to the above. Figure 3 The specific process of audio data sampling described in the corresponding embodiments will not be repeated here. Similarly, the vector similarity calculation module 13c is equivalent to the similarity retrieval component described above.

[0227] Furthermore, the K most similar historical media objects are selected based on the calculated N vector similarities, and an explicit similarity association is established between the media object to be processed and the K historical media objects, resulting in a TopK list of high-interaction semantics (i.e., a list of similar media objects). Further, in this embodiment, the target recommendation feature determination module 13d obtains the K historical recommendation features of the K historical media objects in the similar media object list, performs pooling on these K historical recommendation features, and obtains the target recommendation features of the media object to be processed. The recommendation model 13e is used to search for target business objects matching the media object to be processed among online users on a business processing platform (e.g., a social software platform) based on the target recommendation features, and then pushes the media object to be processed to the target business object. It should be noted that the recommendation model 13e is trained on the initial recommendation model using the N historical interaction behavior information 13f of the N historical media objects. See the above for details. Figure 3 The specific process of audio data sampling described in the corresponding embodiments will not be repeated here.

[0228] This application embodiment firstly describes a first local device (e.g., the one described above). Figure 1 The corresponding 10a, 10b, 10c and the above Figure 2 The corresponding 10a) collects multimedia data of the media object to be processed, and then sends the multimedia data to the server (e.g., the aforementioned Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 The server 10d in the corresponding embodiment processes the data to obtain a target service object that matches the media object to be processed. Further, the server sends the multimedia data of the media object to be processed to a second local device (e.g., the aforementioned...). Figure 1 The corresponding 20a, 20b, 20c and the above Figure 2 The corresponding 20a) enables the second local device to promote the multimedia data of the media object to be processed to the target recommendation object, thereby displaying the target recommendation object in the recommendation display list of the business processing platform (e.g., social software platform) deployed on the second local device.

[0229] For ease of understanding, such as Figure 14 As shown, Figure 14 This is a schematic diagram illustrating a target business object estimation process for a live streaming room according to an embodiment of this application. It should be understood that, taking the multimedia data type of the media object to be processed as a live streaming type as an example, the multimedia data type of the historical media object is also a live streaming type. In this embodiment, the data processing method for media objects may include at least the following steps S301 to S309:

[0230] Step S301: Time window sampling.

[0231] Specifically, for the live streaming data to be processed in the live streaming media object to be processed, the target audio data is first obtained by sampling the live streaming data to be processed using a target sampling window. See the above for details. Figure 3 The specific process of audio data sampling described in the corresponding embodiments will not be repeated here. After executing step S301, proceed to step S303.

[0232] Step S302: High-interaction filtering.

[0233] Specifically, for historical live streaming data of historical media objects, based on the interaction data curve distribution corresponding to the historical media object database, the top 10% of historical media objects in terms of interaction data volume are selected. Then, N historical audio data points for the N selected historical live streaming media objects are obtained. It should be understood that historical audio data refers to audio data collected by the historical sampling window at the end time of the historical live streaming data of the historical live streaming media object, with a duration equal to the window duration. It is important to note that the interaction data curve distribution is a distribution curve graph with each historical media object in the historical media object database and the interaction data volume of each historical media object as the two axes. See the above for details. Figure 3 The specific process of filtering historical media objects described in the corresponding embodiments will not be repeated here.

[0234] Step S303: Speech recognition processing.

[0235] Specifically, ASR processing is performed on the target audio data of the aforementioned live-stream media object to be processed and the N historical audio data of the aforementioned N historical live-stream media objects, respectively, to obtain the target speech recognition text and the N historical speech recognition texts. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0236] Step S304: Extraction of semantic understanding information.

[0237] Specifically, based on the guidance prompt template data, information is extracted from the target speech recognition text and N historical speech recognition texts using a semantic understanding model (e.g., a large language model), resulting in target semantic understanding information and N historical semantic understanding information. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0238] Step S305: Text vectorization representation.

[0239] Specifically, text vector representation processing is performed on the aforementioned target semantic understanding information and N historical semantic understanding information, respectively. See the above for details. Figure 3 The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0240] Step S306: Current live stream representation.

[0241] Specifically, after processing the target semantic information into text vector representations in the previous step, a target text representation vector is obtained to represent the current live-stream media object to be processed. See the above for details. Figure 3 The specific process of text vectorization representation described in the corresponding embodiments will not be repeated here.

[0242] Step S307: Characterization of highly interactive live streaming rooms.

[0243] Specifically, in step S304, text vector representation processing is performed on the N historical semantic information items to obtain N historical text representation vectors for representing the N historical live data. Further, the target text representation vector and the N historical text representation vectors are compared and similarity is calculated, and the K historical text representation vectors with the highest similarity are obtained based on the vector similarity. See the above for details. Figure 3 The specific process of vector similarity calculation described in the corresponding embodiments will not be repeated here.

[0244] Step S308: Obtain the live stream sequence.

[0245] Specifically, the K historical live-stream media objects corresponding to the K selected historical text representation vectors are each explicitly similar to the live-stream media objects to be processed, thus obtaining a list of similar media objects. See the above for details. Figure 3 The specific process of establishing explicit similarity relationships in the corresponding embodiments will not be repeated here.

[0246] Step S309: Recommend the model.

[0247] Specifically, in this embodiment, K historical recommendation features of K historical media objects from the aforementioned list of similar media objects are obtained, and then pooled to obtain target recommendation features representing the media object to be processed. Then, based on the target recommendation features, a recommendation model searches for target business objects matching the media object to be processed among online users of a business processing platform (e.g., a social media platform), so that the media object to be processed is displayed in the recommendation display list of the target business objects. See the above for details. Figure 3 The specific process of audio data sampling described in the corresponding embodiments will not be repeated here.

[0248] This application embodiment collects audio data of the media object to be processed through a target sampling window, with the data sampling end time being the target dynamic time point and the sampling duration being the window duration. This data is then used as the target audio data. The target text representation vector associated with the target audio data is then correlated with the historical text representation features of historical media objects, thereby filtering out historical media objects associated with the aforementioned media object to be processed. Further, this application embodiment obtains the historical recommendation features of the historical media objects associated with the media object to be processed, and then processes these historical recommendation features to obtain the target recommendation features of the media object to be processed. A target recommendation model is then used to recommend the media object to be processed to target business objects that match the media object to be processed based on these target recommendation features.

[0249] For ease of understanding, such as Figure 15 As shown, Figure 15 This is a flowchart illustrating a data processing method for a media object according to an embodiment of this application. In one embodiment, the server (e.g., the one described above) Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 In the corresponding embodiment, server 10d receives the first local device (e.g., the one described above). Figure 1 The corresponding 10a, 10b, 10c and the above Figure 2The multimedia data of the media object to be processed, sent in 10a), is further processed by obtaining the target dynamic sampling time point and window duration of the media object to be processed, and then determining the target sampling window based on the target dynamic sampling time point and window duration. Audio data with the data sampling end time point as the target dynamic time point and the sampling duration as the window duration is obtained through the acquired target sampling window and is used as the target audio data. Then, text vector representation processing is performed on the target speech recognition text corresponding to the above target audio data to obtain the target text representation vector used to represent the above media object to be processed. Further, in this embodiment, N historical text representation vectors of N historical media objects are obtained for representation, and vector similarity calculation is performed on the target text representation vector and each historical text representation vector to obtain N vector similarities. Then, K historical media objects associated with the media object to be processed are selected based on the above N vector similarities. Furthermore, K historical recommendation features of the K historical media objects are obtained from the recommendation feature database associated with the K historical media objects. Upon obtaining the K historical recommendation features, these features are pooled to obtain target recommendation features representing the media objects to be processed. This allows the target recommendation model to recommend media to target business objects that match the media objects to be processed, based on the target recommendation features. It should be understood that, in this embodiment, the data processing method for media objects may include at least the following steps S401 to S407:

[0250] Step S401: Obtain the target dynamic sampling time point and window duration of the media object to be processed. Based on the target dynamic sampling time point and window duration, determine the target sampling window for sampling the multimedia data of the media object to be processed. Then, through the target sampling window, obtain audio data from the multimedia data of the media object to be processed with the data sampling end time point as the target dynamic sampling time point and the sampling duration as the window duration. Use the obtained audio data as the target audio data of the media object to be processed in the target sampling window.

[0251] Among them, the target dynamic sampling time point refers to the time point that changes as the media data association duration of the media object to be processed changes; the target dynamic sampling time point includes the first dynamic sampling time point, the target sampling window includes the first sampling window corresponding to the first dynamic sampling time point, and the target audio data includes the first audio data collected based on the first dynamic sampling time point and the window duration.

[0252] For ease of understanding, when the media object to be processed is a live media object, and the multimedia data of the media object to be processed is the live data to be processed, and simultaneously, the historical media object is a historical live media object, and the multimedia data of the historical media object is historical live data, the target dynamic sampling time point and window duration are obtained first. It can be understood that the directly obtained target dynamic sampling time point and window duration are from the server (e.g., the one mentioned above). Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 The default value is uniformly set for server 10d in the corresponding embodiment.

[0253] Furthermore, in this embodiment of the application, a sampling window with a duration equal to the aforementioned window duration is determined for sampling the aforementioned live data to be processed, and this window is designated as the target sampling window. Then, the aforementioned target dynamic sampling time point is taken as the end time point of data sampling, and audio data with a sampling duration equal to the window duration collected by the aforementioned target sampling window is obtained, and the audio data obtained in the aforementioned target sampling window is taken as the target audio data.

[0254] It should be noted that the first time point used as the target dynamic sampling time point is the first dynamic sampling time point when the duration of the live data to be processed is the same as the duration of the aforementioned window, and the first sampling window with a sampling duration of the aforementioned window duration is used as the target sampling window to capture and collect the aforementioned live data to be processed.

[0255] Step S402: Obtain the sampling interval duration for dynamic sampling. Based on the first dynamic sampling time point and the sampling interval duration, determine the second dynamic sampling time point for sampling the multimedia data of the media object to be processed. Then, based on the second dynamic sampling time point and the window duration, determine the second sampling window for sampling the multimedia data of the media object to be processed. Then, through the second sampling window, obtain audio data from the multimedia data of the media object to be processed, with the data sampling end time point being the second dynamic sampling time point and the sampling duration being the window duration. Use the obtained audio data as the second audio data of the media object to be processed in the target sampling window. When updating the target dynamic sampling time point through the second dynamic sampling time point, update the target audio data through the second audio data.

[0256] Wherein, the second dynamic sampling time point is the next sampling time point after the first dynamic sampling time point; the second sampling window is the next sampling window after the first sampling window, and the second sampling window overlaps with the first sampling window.

[0257] Specifically, the server (e.g., the one mentioned above) Figure 1 The server 10d in the corresponding embodiment and the above Figure 2In the corresponding embodiment, server 10d) obtains the sampling interval duration for dynamic sampling. The sampling interval duration is the interval between two adjacent time points that are respectively considered as the target dynamic sampling time points when the target dynamic sampling time point is updated in the live data to be processed. Based on the first dynamic sampling time point and the sampling interval duration, a second dynamic sampling time point is determined, that is, the second dynamic sampling time point is located after the first dynamic sampling time point for a duration equal to the sampling interval duration. Further, in this embodiment, a second sampling window with a duration equal to the window duration is determined for sampling the live data to be processed. Then, with the second dynamic sampling time point as the end time point of data sampling, the second sampling window with a sampling duration equal to the window duration updates the target sampling window, intercepts and collects the live data to be processed, obtains the audio data in the live data collected by the target sampling window, and thus updates the target audio data. Thus, this embodiment achieves the effect of dynamic sampling.

[0258] It should be noted that the above sampling interval duration overlaps with the target sampling window. For ease of understanding, the above sampling interval duration should be less than or equal to the window duration of the target sampling window so that the above live data to be processed can be completely collected.

[0259] For ease of understanding, such as Figure 16 As shown, Figure 16This is a schematic diagram of dynamic sampling based on the playback timeline of live data to be processed, according to an embodiment of this application. It should be understood that the target sampling window includes a first sampling window and a second sampling window, and the target dynamic sampling time point includes both the first and second dynamic sampling time points. When the live data to be processed reaches time T2, time T2 is taken as the first dynamic sampling time point 16a1. Then, live data to be processed is collected with time T2 as the end of sampling in the first sampling window 16b1 and a duration of 16d (the window duration). In other words, multimedia data with a duration of 16d up to the target dynamic sampling time point 16a1 is extracted through the target sampling window 16B. Furthermore, the audio data of the multimedia data collected in the target sampling window 16B is extracted and used as the target audio data. Furthermore, the live data to be processed continues to be sampled at intervals of 16c. When the live data to be processed reaches time T3, time T3 is taken as the second dynamic sampling time point 16a2. Live data to be processed is collected with time T3 as the end time of the second sampling window 16b2 and a duration of window duration 16d. That is, the second dynamic sampling time point 16a2 replaces the first dynamic sampling time point 16a1 to update the target dynamic sampling time point 16A, and the second sampling window 16b2 replaces the first sampling window 16b1 to update the target sampling window 16B. Then, the audio data extracted from the live data to be processed in the second sampling window 16b2 replaces the audio data extracted from the live data to be processed in the first sampling window 16b1 and is taken as the target audio data.

[0260] Step S403: Perform text vector representation processing on the target speech recognition text corresponding to the target audio data to obtain the target text representation vector used to represent the media object to be processed.

[0261] This includes acquiring a speech conversion understanding component and a similarity retrieval component for speech conversion processing of target audio data. The speech conversion understanding component includes a speech recognizer and a semantic understanding model for semantic extraction, while the similarity retrieval component includes a text representation unit.

[0262] Furthermore, the speech recognition unit of the aforementioned speech conversion and understanding component performs speech recognition processing on the target audio data to obtain target speech recognition text associated with the target audio data. Then, the semantic understanding model in the aforementioned speech conversion and understanding component extracts information from the target speech recognition text to obtain target semantic understanding information. Further, the text representation of the aforementioned similarity retrieval component performs text vector representation processing on the target semantic understanding information to obtain a target text representation vector used to represent the live-stream media object to be processed. For specific implementation details, please refer to the above. Figure 3The specific process of speech recognition processing described in the corresponding embodiments will not be repeated here.

[0263] Step S404: Obtain N historical text representation vectors to represent N historical media objects, and perform vector similarity calculation on the target text representation vector and each historical text representation vector to obtain N vector similarity scores.

[0264] Where N is a positive integer greater than 1; a historical text representation vector is recorded by performing text vector representation processing on the historical speech recognition text corresponding to a historical audio data; a historical audio data refers to the audio data of a historical media object in the recorded historical sampling window; the window duration of the historical sampling window is consistent with the window duration of the target sampling window.

[0265] Specifically, the target text representation vector is obtained by performing speech conversion processing on the target audio data using the speech conversion understanding component and the similarity retrieval component of the speech conversion processing. Then, N historical text representation vectors representing N historical media objects are obtained. After obtaining these N historical text representation vectors, the target text representation vector representing the live-stream media object to be processed is compared with the N historical text representation vectors representing the N historical live-stream media objects to calculate vector similarity. The methods for calculating vector similarity include cosine similarity, Jaccard similarity, Euclidean distance, and Manhattan distance, etc., which are not limited here. For specific implementation details, please refer to the above. Figure 3 The specific process of vector similarity calculation described in the corresponding embodiments will not be repeated here.

[0266] Step S405: When selecting K historical media objects from N historical media objects using N vector similarity, establish similarity associations between the K historical media objects and the media objects to be processed.

[0267] Specifically, based on the calculated N vector similarities, N historical live-stream media objects are sorted, and then K historical live-stream media objects associated with the live-stream media object to be processed are selected. Explicit similarity relationships are then established between these K historical live-stream media objects and the live-stream media object to be processed. For details on the implementation, please refer to the above. Figure 3 The specific process of selecting K historical media objects in the corresponding embodiments will not be repeated here.

[0268] Step S406: Obtain K historical recommendation features of the K historical media objects from the recommendation feature database associated with the K historical media objects.

[0269] Specifically, the K historical recommendation features of the aforementioned K historical live media objects are provided by the server (e.g., the aforementioned...). Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 The server 10d in the corresponding embodiment obtained it from the recommendation feature database associated with K historical live broadcast media objects.

[0270] The aforementioned recommendation feature database was obtained through the following method:

[0271] Obtain N historical interaction behavior information corresponding to N historical media objects. Each historical media object corresponds to one historical interaction behavior information, which represents the item interaction details of each interactive item within that historical media object. Use the item interaction details represented by the N historical interaction behavior information as training samples for training the initial recommendation model, and obtain recommendation metrics for training the initial recommendation model. Train the initial recommendation model using the training samples and recommendation metrics to obtain the target recommendation model; the target recommendation model extracts N historical recommendation features from the N historical media objects. Each historical recommendation feature is obtained by the target recommendation model from the interaction vector extracted from the item interaction details represented by a historical interaction behavior information. Add the N historical recommendation features extracted by the target recommendation model to the recommendation feature database.

[0272] Specifically, the server (e.g., the one mentioned above) Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 In the corresponding embodiment, server 10d) acquires the historical interaction behavior information corresponding to each of the N historical live-stream media objects. Then, the details of user item interactions represented by the historical interaction behavior information of the N historical live-stream media objects are used as training samples for the initial recommendation model. This initial recommendation model is an untrained model used to provide warm-up recommendations for long-tail or cold-start media objects. Furthermore, when training the initial recommendation model, recommendation metrics associated with the initial recommendation model are acquired for training.

[0273] Furthermore, the initial recommendation model is trained using the recommendation metrics to obtain a target recommendation model for extracting recommendation features of the media objects to be processed and recommendation features of historical media objects. The target recommendation model is then used to process the N historical live-stream media objects to obtain N historical recommendation features for each of the N historical live-stream media objects. These N historical recommendation features are then added to the recommendation feature database.

[0274] It is important to note that the media objects to be processed (e.g., the live streaming media objects mentioned above) are multimedia objects in the business processing platform whose interactive data volume is less than the interactive data volume threshold. In other words, the media objects to be processed (e.g., the live streaming media objects mentioned above) are cold start / long tail media objects.

[0275] Step S407: When the K historical recommendation features of the K historical media objects are obtained, feature pooling is performed on the K historical recommendation features based on the similarity association to obtain pooled features associated with the K historical recommendation features. The pooled features are determined as target recommendation features to characterize the media objects to be processed. Media recommendations are made to target business objects that match the media objects to be processed through the target recommendation features.

[0276] Specifically, a pooling strategy for pooling is obtained based on similarity relationships. When the pooling strategy is average pooling, average pooling is performed on K historical recommendation features to obtain average pooled features. These average pooled features are pooled features associated with the K historical recommendation features.

[0277] It should be understood that before pooling the K historical recommendation features of the K historical live media objects, a pooling strategy should first be obtained based on the aforementioned similarity relationships. This pooling strategy includes methods such as average pooling, max pooling, and random pooling, which are not limited here.

[0278] Specifically, if the pooling strategy used in the above pooling process is average pooling, then average pooling is applied to the K historical recommendation features of the K historical live stream data to obtain average pooled features. Furthermore, the target recommendation features are obtained based on these average pooled features and the live stream data to be processed.

[0279] It should be understood that the above recommendation model obtains the target recommendation features of the media object to be processed through pooling of K historical recommendation features in the following specific way:

[0280]

[0281] Where J represents the ID index of historically highly interactive media objects; K represents the K most similar live stream rooms retrieved through vector similarity; M represents the online-trained recommendation model for the media object recommendation scenario to be processed; M(J k ) represents the recommended representation of the k-th historical media object.

[0282] Furthermore, when deploying the target recommendation model on the business processing platform, object association data information of business objects accessing the platform is obtained. This object association data information and target recommendation features are input into the target recommendation model. The model extracts object vector representation features from the object association data information to characterize business objects, and then filters these features to select target object vector representation features that match the target recommendation features. The business objects corresponding to these target object vector representation features are then identified as the target business objects that match the media objects to be processed, and the media objects to be processed are recommended to these target business objects.

[0283] Specifically, the aforementioned business processing platform includes a social media platform. When the business processing platform is a social media platform, the trained target recommendation model is deployed on the social media platform. Simultaneously with the deployment of the target recommendation model, a second local device (e.g., Figure 1 The user terminals 20a, 20b, 20c in the corresponding embodiments and Figure 2 The corresponding user terminal 20a) receives the above-mentioned server (for example, the above-mentioned Figure 1 The server 10d in the corresponding embodiment and the above Figure 2 The aforementioned live streaming data to be processed is sent by server 10d in the corresponding embodiment. Furthermore, the second local device (e.g., Figure 1 The user terminals 20a, 20b, 20c in the corresponding embodiments and Figure 2 The corresponding user terminal 20a) acquires the item transaction information and user preferences (i.e., the object-related data information) of users (i.e., the aforementioned business objects) on the social software platform. Further, the obtained target recommendation features and the acquired item transaction information and user preferences of the social software platform users are input into the target recommendation model, so that the target recommendation model extracts object vector representation features to characterize the user from the item transaction information and user preferences. Then, object vector representation features matching the aforementioned target recommendation features are selected from the object vector representation features of all users on the social software platform, and these are used as target object vector representation features. Further, the user corresponding to the aforementioned target object vector representation features is used as the target business object matching the aforementioned live streaming data to be processed, and the aforementioned media object to be processed is recommended to the aforementioned target business object.

[0284] The specific implementation method of the above-mentioned target recommendation model to find the target business object that matches the media object to be processed is as follows:

[0285]

[0286] Where M(u) represents the online user u in the recommendation model M. Finally, the user's preference score y for cold start / long tail live streaming rooms is used. u,i This allows for the identification of a user base that meets the needs, enabling personalized heating recommendations.

[0287] It should be understood that object-related data information and target recommendation features are input into the target recommendation model. The target recommendation model then extracts information vector representations from the object-related data information to obtain object vector representation features used to represent business objects. These object vector representation features and the target recommendation features are then matched to obtain the feature matching results. If the feature matching results indicate that there is an object vector representation feature that matches the target recommendation feature, then the object vector representation feature that matches the target recommendation feature is determined as the target object vector representation feature.

[0288] Specifically, when the item transaction information and user preferences (i.e., object-related data information) of users (i.e., the aforementioned business processing platform) of the aforementioned social software platform (i.e., the aforementioned business object) and the target recommendation features are input into the target recommendation model deployed on the social software platform, the target recommendation model extracts information vector representations of the aforementioned users' item transaction information and user preferences, thereby extracting object vector representation features used to represent user information. Further, users associated with the media object to be processed are selected from all the aforementioned users, and then the target recommendation model matches the aforementioned object vector representation features with the target recommendation features of the media object to be processed, thereby obtaining feature matching results. Correspondingly, if the feature matching result indicates that there is a vector representation feature in the object vector representation features that matches the target recommendation features, then that object vector representation feature is used as the target object vector representation feature.

[0289] Furthermore, if the media object to be processed is a live media object, then the live media object to be processed is used as the target recommendation object in the live recommendation service to be recommended to the target service object. The target recommendation object is sent to the terminal device corresponding to the target service object so that the terminal device displays the target recommendation object in the live recommendation list corresponding to the live recommendation service. If the media object to be processed is a video-on-demand media object, then the video-on-demand link corresponding to the video-on-demand media object is encapsulated as a recommendation notification message. The recommendation notification message is sent to the terminal device corresponding to the target service object so that the terminal device displays the video-on-demand link in the recommendation notification message in the live recommendation list corresponding to the live recommendation service. The video-on-demand link is used to instruct the terminal device to display the video-on-demand media object in response to the trigger operation of the recommendation notification message.

[0290] It should be noted that, in order to comprehensively evaluate the effectiveness of the method proposed in this application embodiment, when the media object to be processed is a live media object, a thorough evaluation of the solution is conducted in both offline and online scenarios of the business processing platform. This evaluation uses a model for speech recognition and a semantic understanding model. The aforementioned vector similarity algorithm is cosine similarity, and the method proposed in this application embodiment is evaluated when K=20, that is, when 20 historical live media objects associated with the live media object to be processed are selected.

[0291] First, offline evaluation is mainly conducted using three metrics: Area Under Curve (AUC) of the target recommendation model, Group AUC (GAUC) of the area under the receiver operating characteristic curve of each group after weighted averaging with the coordinate axis, and Precision at Cut-off (PCOC).

[0292] The area under the Receiver Operating Characteristic (ROC) curve mentioned above is a curve plotted with the false positive rate (FPR) on the horizontal axis and the true positive rate (TPR) on the vertical axis. The AUC value ranges from 0 to 1. The higher the AUC value, the stronger the model's ability to distinguish between positive and negative samples.

[0293] GAUC is an extension of AUC and is typically used to evaluate the performance of recommender systems across different business object groups or categories. GAUC is a weighted average of the AUC for each group, taking into account the sample size of different business object groups. A higher GAUC is better, indicating that the model performs more evenly across all groups.

[0294] PCOC refers to the precision of the top X recommended media objects in the recommendation list. It measures the proportion of media objects that the business is actually interested in among the top X recommended media objects. A higher PCOC is better, indicating higher accuracy of the recommendation system among the top X recommended media objects.

[0295] The following table evaluates the method proposed in this application embodiment based on page views (PV). In this application embodiment, the relative gain obtained by different popular live stream rooms relative to a baseline after using the method proposed in this application embodiment can be seen in Table 1 below. It should be understood that the aforementioned live stream rooms are live media objects to be processed.

[0296] Table 1

[0297] Click PV 0-100 100-500 500-2000 2000-5000 5000+ AUC +0.049％ +0.047％ +0.023％ +0.015％ +0.014％ GAUC +0.149％ +0.079％ +0.034％ +0.019％ +0.016％ PCOC +0.189％ +0.112％ +0.064％ +0.043％ +0.042％

[0298] As shown in Table 1 above, the embodiments of this application achieved different gains relative to the baseline when implemented in live streaming rooms with different popularity levels.

[0299] Furthermore, an online evaluation was conducted on the embodiments of this application, focusing primarily on the heating effect of cold start channels and long-tail channel live streaming rooms (i.e., the aforementioned live streaming media objects to be processed). In the embodiments of this application, the relative benefits obtained by cold start channel distribution relative to the baseline can be seen in Table 2 below.

[0300] Table 2

[0301] Cold start distribution Watch PV Watch the PV in 1 minute PV percentage of 1 minute viewing time Nearest neighbor search = 10 +0.866％ +0.629％ +0.825％ Nearest neighbor search = 20 +1.524％ +1.173％ +1.248％

[0302] In the embodiments of this application, the relative benefits obtained by long-tail content distribution relative to the baseline can be seen in Table 3 below.

[0303] Table 3

[0304] Long-tail content distribution Watch PV Watch the PV in 1 minute PV percentage of 1 minute viewing time PV < 500 views +2.698％ +6.153％ +3.287％ PV < 100 views +4.034％ +6.721％ +2.658％

[0305] As shown in Tables 2 and 3 above, the embodiments of this application have achieved different gains relative to the baseline in both cold start channels and long-tail content distribution.

[0306] This application embodiment uses audio data collected from the multimedia data of the media object to be processed through a target sampling window. The target dynamic sampling time point is the end time of data sampling, and the sampling duration is the duration of the window. This audio data is used as target audio data. The target dynamic sampling time point is updated based on the sampling interval duration, thereby updating the target audio data associated with the media object to be processed. Furthermore, the target audio data obtained in the target sampling window can be identified to obtain target speech recognition text associated with the target audio data. Further, this application embodiment performs text vector representation processing on the target speech recognition text associated with the audio data to obtain a target text representation vector representing the media object to be processed. N historical text representation vectors representing N historical media objects are obtained, and vector similarity calculations are performed between the target text representation vector and each of the N historical text representation vectors to obtain N vector similarities. Furthermore, based on the aforementioned N vector similarities, this embodiment selects K historical media objects with high similarity to the media object to be processed from the N historical media objects, and then establishes an explicit similarity association between the media object to be processed and the selected K historical media objects. This embodiment obtains K historical recommendation features of the K historical media objects from a recommendation feature database associated with the K historical media objects, and then obtains a pooling processing strategy for pooling to obtain the target recommendation features of the media object to be processed. Further, it obtains historical interaction behavior information of the N historical media objects, so that when the target recommendation model is deployed on the business processing platform, it obtains object association data information of the business objects accessing the business processing platform, and then, through the object association data information and the target recommendation features, selects target business objects matching the media object to be processed from the business objects of the business processing platform, and then recommends media to the target business objects. This application embodiment can dynamically sample the multimedia data of the media object to be processed to obtain target audio data associated with the media object to be processed. The content in the target audio data is vectorized and similarity is calculated with the vectors of the content in historical audio data. This allows for the retrieval of K historical media objects with high relevance to the media object to be processed. The historical recommendation features of the K historical media objects are then obtained from the recommendation feature database associated with the K historical media objects and pooled to obtain target recommendation features that characterize the media object to be processed. Finally, target business objects that match the media object to be processed are selected using N historical interaction behavior information of N historical media objects and the target recommendation features of the media object to be processed, and the aforementioned media data to be processed is pushed to them.Furthermore, this improves the accuracy of the target recommendation model in matching media objects and business objects during cold starts, and also achieves the effect of "warming up" the media objects to be processed.

[0307] Further, please see Figure 17 , Figure 17 This is a schematic diagram of a data processing device for media objects according to an embodiment of this application. The data processing device for media objects can be a computer program (including program code) running on a computer device; for example, the data processing device 1 for media objects can be an application software. The data processing device 1 for media objects can be used to execute the corresponding steps in the method provided in the embodiments of this application. The data processing device 1 for media objects may include: an audio data speech recognition module 11, a vector similarity calculation module 12, a similarity association establishment module 13, and a target recommendation feature determination module 14.

[0308] The audio data speech recognition module 11 is used to collect audio data of the media object to be processed at the target time point based on the target sampling window to obtain target audio data, and then to recognize and process the target audio data based on automatic speech recognition technology to obtain target speech recognition text. The semantic understanding model extracts information from the target speech recognition model based on the semantic information mining template and outputs target speech understanding text associated with the target speech recognition text.

[0309] The vector similarity calculation module 12 is used to perform text vector representation processing on the target speech recognition text corresponding to the target audio data to obtain the target text representation vector used to represent the media object to be processed, and to perform text vector representation processing on the historical speech recognition text corresponding to the historical audio data to obtain the historical text representation vector used to represent the historical media object. It also obtains N historical text representation vectors used to represent N historical media objects, and performs vector similarity calculation on the target text representation vector and each historical text representation vector to obtain N vector similarities.

[0310] The similarity association establishment module 13 is used to establish similarity associations between the K historical media objects and the media object to be processed when selecting K historical media objects from N historical media objects through N vector similarity. The amount of interaction data of each historical media object is higher than the amount of interaction data of the media object to be processed. N is a positive integer greater than 1. A historical text representation vector is recorded after performing text vector representation processing on the historical speech recognition text corresponding to a historical audio data. A historical audio data refers to the audio data of a historical media object in the recorded historical sampling window. The window duration of the historical sampling window is consistent with the window duration of the target sampling window. K is a positive integer less than N.

[0311] The target recommendation feature determination module 14 is used to determine the target recommendation features to represent the media object to be processed based on the similarity relationship and the K historical recommendation features when K historical media objects are obtained, and to make media recommendations to the target business objects that match the media object to be processed through the target recommendation features.

[0312] The specific implementation methods of the audio data speech recognition module 11, the vector similarity calculation module 12, the similarity association establishment module 13, and the target recommendation feature determination module 14 can be found in the above description. Figure 3 In the corresponding embodiment, steps S101-S104 and Figure 15 The descriptions of steps S401-S407 in the corresponding embodiments will not be repeated here.

[0313] The audio data speech recognition module 11 includes: a target audio data determination unit 110, a target audio data update unit 111, a component acquisition unit 112, a speech recognition unit 113, a semantic understanding unit 114, a text vector representation unit 115, an invalid audio data marking unit 116, and a target speech recognition text update unit 117.

[0314] The target audio data determination unit 110 is used to obtain the target dynamic sampling time point and window duration of the media object to be processed, and to determine the target sampling window for sampling the multimedia data of the media object to be processed based on the target dynamic sampling time point and window duration; the target dynamic sampling time point refers to the time point that changes as the associated duration of the media data of the media object to be processed changes.

[0315] The target audio data determination unit 110 is also used to obtain audio data from the multimedia data of the media object to be processed through the target sampling window, wherein the data sampling end time point is the target dynamic sampling time point and the sampling duration is the window duration.

[0316] The target audio data determination unit 110 is also used to use the acquired audio data as the target audio data of the media object to be processed in the target sampling window.

[0317] Optionally, the target dynamic sampling time point includes a first dynamic sampling time point, the target sampling window includes a first sampling window corresponding to the first dynamic sampling time point, and the target audio data includes first audio data collected based on the first dynamic sampling time point and the window duration.

[0318] The target audio data update unit 111 is used to obtain the sampling interval duration for dynamic sampling, and based on the first dynamic sampling time point and the sampling interval duration, determine the second dynamic sampling time point for sampling the multimedia data of the media object to be processed; the second dynamic sampling time point is the next sampling time point after the first dynamic sampling time point;

[0319] The target audio data update unit 111 is further configured to determine a second sampling window for sampling the multimedia data of the media object to be processed based on the second dynamic sampling time point and the window duration; the second sampling window is the next sampling window after the first sampling window, and the second sampling window overlaps with the first sampling window;

[0320] The target audio data update unit 111 is also used to obtain audio data from the multimedia data of the media object to be processed through the second sampling window, wherein the data sampling end time point is the second dynamic sampling time point and the sampling duration is the window duration.

[0321] The target audio data update unit 111 is also used to use the acquired audio data as the second audio data of the media object to be processed in the target sampling window, and to update the target audio data through the second audio data when updating the target dynamic sampling time point through the second dynamic sampling time point.

[0322] Specifically, the component acquisition unit 112 is used to acquire a speech conversion understanding component and a similarity retrieval component for speech conversion processing of target audio data; the speech conversion understanding component includes a speech recognizer and a semantic understanding model for semantic extraction, and the similarity retrieval component includes a text representation.

[0323] The speech recognition unit 113 is used to perform speech recognition processing on the target audio data through a speech recognizer to obtain the speech recognition processing result.

[0324] The semantic understanding unit 114 is used to perform text semantic understanding processing on the target speech recognition text through the semantic understanding model if the speech recognition processing result indicates that the target speech recognition text has been recognized as target audio data, so as to obtain the target semantic understanding information of the target speech recognition text.

[0325] The text vector representation unit 115 is used to perform vector representation of the target semantic understanding information through the text representation unit to obtain the target text representation vector used to represent the media object to be processed.

[0326] Optionally, the invalid audio data marking unit 116 is used to mark the target audio data as invalid audio data if the above speech recognition processing result indicates that the target speech recognition text has not been recognized.

[0327] Optionally, the target speech recognition text update unit 117 is used to obtain the video data of the media object to be processed from the multimedia data of the media object to be processed through the target sampling window, and use the obtained video data as the target video data of the media object to be processed in the target sampling window.

[0328] The target speech recognition text update unit 117 is also used to perform video content recognition on video frames in the target video data to obtain video text content that represents the target video data, and update the target speech recognition text through the video text content.

[0329] Specifically, the semantic understanding unit 114 is used to acquire guidance prompt template data for guiding the semantic understanding model to output the model, and input the guidance prompt template data and the target speech recognition text into the semantic understanding model. The semantic understanding model extracts semantic information from the target speech recognition text according to the data content in the guidance prompt template data, obtains the target semantic extraction information of the target speech recognition text, and determines the target semantic understanding information of the target speech recognition text through the target semantic extraction information.

[0330] Furthermore, the semantic understanding unit 114 is specifically used to use the prompt template data associated with the video live streaming type as the guidance prompt template data if the object type of the media object to be processed is video live streaming type.

[0331] The semantic understanding unit 114 is also specifically used to use the prompt template data associated with the video-on-demand type as the guidance prompt template data if the object type of the media object to be processed is video-on-demand.

[0332] The specific implementation methods of the aforementioned target audio data determination unit 110, target audio data update unit 111, component acquisition unit 112, speech recognition unit 113, semantic understanding unit 114, text vector representation unit 115, invalid audio data marking unit 116, and target speech recognition text update unit 117 can be found in the above description. Figure 3 In the corresponding embodiment, steps S101-S104 and Figure 15 The descriptions of steps S401-S407 in the corresponding embodiments will not be repeated here.

[0333] Furthermore, the vector similarity calculation module 12 includes: an interactive data curve distribution acquisition unit 121, a historical media object determination unit 122, and a historical text representation vector acquisition unit 123.

[0334] The interactive data curve distribution acquisition unit 121 is used to acquire the interactive data curve distribution corresponding to the historical media object database; wherein, the interactive data curve distribution is determined based on each historical media object in the historical media object database and the amount of interactive data of each historical media object;

[0335] The historical media object determination unit 122 is used to obtain N historical media objects whose interactive data volume reaches the interactive data threshold from the historical media object database by means of interactive data curve distribution;

[0336] The historical text representation vector acquisition unit 123 is used to acquire N historical text representation vectors of N historical media objects from the historical text vector representation database associated with N historical media objects; a historical text representation vector is obtained by speech conversion processing of the historical audio data of a historical media object through the speech conversion understanding component and the similarity retrieval component.

[0337] The specific implementation methods of the aforementioned interactive data curve distribution acquisition unit 121, historical media object determination unit 122, and historical text representation vector acquisition unit 123 can be found in the above description. Figure 3 In the corresponding embodiment, steps S101-S104 and Figure 15 The descriptions of steps S401-S407 in the corresponding embodiments will not be repeated here.

[0338] The similarity association establishment module 13 includes: a historical media object list determination unit 131 and a similarity association establishment unit 132;

[0339] The historical media object list determination unit 131 is used to sort N historical media objects based on N vector similarity to obtain a list of historical media objects associated with the N historical media objects.

[0340] The similarity association establishment unit 132 is used to obtain K historical media objects whose vector similarity meets the similarity retrieval conditions from the historical media object list, and establish explicit similarity associations between the K historical media objects and the media object to be processed; the explicit similarity association is used to characterize that the list composed of K historical media objects is a list of similar media objects, and any media object in the list of similar media objects has an association with the media object to be processed.

[0341] The specific implementation methods of the aforementioned historical media object list determination unit 131 and similarity association establishment unit 132 can be found in the above. Figure 3 In the corresponding embodiment, steps S101-S104 and Figure 15The descriptions of steps S401-S407 in the corresponding embodiments will not be repeated here.

[0342] The target recommendation feature determination module 14 includes a target recommendation feature determination unit 141 and a pooling processing unit 142.

[0343] The target recommendation feature determination unit 141 is used to perform feature pooling on the K historical recommendation features based on similarity relationship when K historical media objects are obtained, so as to obtain pooled features associated with the K historical recommendation features.

[0344] Pooling processing unit 142 is used to determine the pooling processing features as target recommendation features to characterize the media object to be processed;

[0345] Specifically, the pooling processing unit 142 is used to obtain a pooling processing strategy for pooling processing based on similarity relationships.

[0346] The pooling processing unit 142 is also specifically used to perform average pooling processing on K historical recommendation features when the pooling processing strategy is average pooling processing strategy, to obtain average pooled features; the average pooled features are pooled processing features associated with the K historical recommendation features.

[0347] The specific implementation methods of the target recommendation feature determination unit 141 and the pooling processing unit 142 can be found in the above description. Figure 3 In the corresponding embodiment, steps S101-S104 and Figure 15 The descriptions of steps S401-S407 in the corresponding embodiments will not be repeated here.

[0348] The device 1 also includes: a historical recommendation feature acquisition module 15;

[0349] The historical recommendation feature acquisition module 15 is used to acquire K historical recommendation features of K historical media objects from the recommendation feature database associated with K historical media objects.

[0350] The historical recommendation feature acquisition module 15 includes: a historical interaction behavior information acquisition unit 151, a recommendation index acquisition unit 152, a target recommendation model training unit 153, and a recommendation feature database update unit 154.

[0351] The historical interaction behavior information acquisition unit 151 is used to acquire N historical interaction behavior information corresponding to N historical media objects; one historical media object corresponds to one historical interaction behavior information, and one historical interaction behavior information is used to characterize the item interaction details of each interactive item in a historical media object;

[0352] The recommendation metric acquisition unit 152 is used to take the item interaction details represented by N historical interaction behavior information as training samples for training the initial recommendation model and acquire recommendation metrics for training the initial recommendation model.

[0353] The target recommendation model training unit 153 is used to train the initial recommendation model using training samples and recommendation metrics to obtain the target recommendation model. The target recommendation model is used to extract N historical recommendation features from N historical media objects. A historical recommendation feature is obtained by the target recommendation model extracting the interaction vector of the item interaction details represented by a historical interaction behavior information.

[0354] The recommendation feature database update unit 154 is used to add N historical recommendation features extracted by the target recommendation model to the recommendation feature database.

[0355] The specific implementation methods of the aforementioned historical interaction behavior information acquisition unit 151, recommendation index acquisition unit 152, target recommendation model training unit 153, and recommendation feature database update unit 154 can be found in the above description. Figure 3 In the corresponding embodiment, steps S101-S104 and Figure 15 The descriptions of steps S401-S407 in the corresponding embodiments will not be repeated here.

[0356] Furthermore, the media object to be processed is a multimedia object in the business processing platform whose interactive data volume is less than the threshold of interactive data volume. The recommendation model corresponding to the business processing platform is a target recommendation model. The target recommendation model is obtained by training the initial recommendation model deployed on the business processing platform with N historical interactive behavior information corresponding to N historical media objects.

[0357] The target recommendation feature determination module 14 includes: a related data information determination unit 143, an object vector feature determination unit 144, and a media object recommendation unit 145.

[0358] The associated data information determination unit 143 is used to obtain the object association data information of the business object accessing the business processing platform when the target recommendation model is deployed and applied on the business processing platform.

[0359] The object vector feature determination unit 144 is used to input object association data information and target recommendation features into the target recommendation model. When the target recommendation model extracts object vector representation features for representing business objects from the object association data information, it filters target object vector representation features that match the target recommendation features from the object vector representation features.

[0360] The media object recommendation unit 145 is used to identify the business object corresponding to the target object vector representation feature as the target business object that matches the media object to be processed among the business objects, and recommend the media object to be processed to the target business object.

[0361] Specifically, the object vector feature determination unit 144 is used to input object association data information and target recommendation features into the target recommendation model, and extract information vector representation of object association data information through the target recommendation model to obtain object vector representation features used to represent business objects.

[0362] The object vector feature determination unit 144 is also specifically used to perform feature matching between the object vector representation features and the target recommendation features to obtain the feature matching result;

[0363] The object vector feature determination unit 144 is further specifically used to determine the object vector representation feature that matches the target recommendation feature if the feature matching result indicates that there is an object vector representation feature in the object vector representation feature that matches the target recommendation feature.

[0364] Specifically, the media object recommendation unit 145 is used to, if the media object to be processed is a live media object, use the live media object as the target recommendation object in the live recommendation service to recommend to the target business object, and send the target recommendation object to the terminal device corresponding to the target business object so that the terminal device displays the target recommendation object in the live recommendation list corresponding to the live recommendation service.

[0365] The media object recommendation unit 145 is further specifically used to encapsulate the on-demand link corresponding to the on-demand media object into a recommendation notification message if the media object to be processed is a video-on-demand media object, and send the recommendation notification message to the terminal device corresponding to the target service object, so that the terminal device displays the on-demand link in the recommendation notification message on the live recommendation list corresponding to the live recommendation service; the on-demand link is used to instruct the terminal device to display the on-demand media object in response to the trigger operation of the recommendation notification message.

[0366] The specific implementation methods of the aforementioned associated data information determination unit 143, object vector feature determination unit 144, and media object recommendation unit 145 can be found in the above description. Figure 3 In the corresponding embodiment, steps S101-S104 and Figure 15 The descriptions of steps S401-S407 in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0367] It is understood that in the specific implementation of this application, business data of users, enterprises, institutions and other objects may be involved (e.g., users' historical interaction information, object association data information, users' item transaction information, user preferences and other information). When the above embodiments of this application are applied to specific products or technologies, permission or consent from users, enterprises, institutions and other objects is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0368] Further, please see Figure 18 , Figure 18 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 18 As shown, the computer device 1000 can be a user terminal, for example, the one described above. Figure 1 The user terminal 10a and user terminal 20a in the corresponding embodiments can also be servers, for example, as described above. Figure 1 The server 10d in the corresponding embodiment will not be limited here. For ease of understanding, this application takes a computer device as a user terminal as an example. The computer device 1000 may include: a processor 1001, a network interface 1004, and a memory 1005. In addition, the computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device. The memory 1005 may also optionally be at least one storage device located away from the aforementioned processor 1001. Figure 18 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application.

[0369] The network interface 1004 in the computer device 1000 can also provide network communication functions, and the optional user interface 1003 can also include a display screen and a keyboard. Figure 18 In the computer device 1000 shown, the network interface 1004 provides network communication functions; the user interface 1003 is mainly used to provide an input interface for users; and the processor 1001 can be used to call the device control application stored in the memory 1005 to enable the media object to be warmed up in cold start or long tail conditions.

[0370] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 3 or Figure 15 The description of the data processing method for media objects in the corresponding embodiments can also be performed as described above. Figure 17 The description of the data processing device 1 in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated here.

[0371] Furthermore, it should be noted that this application embodiment also provides a computer-readable storage medium storing a computer program executed by the aforementioned data processing device 1 for a media object. The computer program includes computer instructions, and when the processor executes the computer instructions, it can execute the aforementioned... Figure 3 or Figure 15 The description of the data processing method for the media object in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application. As an example, computer instructions can be deployed and executed on a single computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network. These multiple computer devices distributed across multiple locations and interconnected via a communication network can constitute a blockchain system.

[0372] Furthermore, it should be noted that this application also provides a computer program product or computer program, which may include computer instructions, which may be stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor may execute the computer instructions, causing the computer device to perform the aforementioned actions. Figure 3 or Figure 15 The description of the data processing method for media objects in the corresponding embodiments is already provided and will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer program products or computer program embodiments related to this application, please refer to the description of the method embodiments of this application.

[0373] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0374] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A data processing method for media objects, characterized in that, include: Obtain the target audio data of the media object to be processed in the target sampling window, and perform text vector representation processing on the target speech recognition text corresponding to the target audio data to obtain the target text representation vector used to represent the media object to be processed. N historical text representation vectors are obtained to represent N historical media objects. Vector similarity calculations are performed on the target text representation vector and each historical text representation vector to obtain N vector similarity scores. The amount of interaction data for each historical media object is higher than the amount of interaction data for the media object to be processed. N is a positive integer greater than 1. A historical text representation vector is recorded after performing text vector representation processing on the historical speech recognition text corresponding to a historical audio data. A historical audio data refers to the audio data of a historical media object in the recorded historical sampling window. The window duration of the historical sampling window is consistent with the window duration of the target sampling window. When selecting K historical media objects from the N historical media objects using the N vector similarity, a similarity association relationship is established between the K historical media objects and the media object to be processed; K is a positive integer less than N; When K historical recommendation features of the K historical media objects are obtained, target recommendation features for characterizing the media object to be processed are determined based on the similarity association and the K historical recommendation features. Media recommendations are then made to target business objects that match the media object to be processed based on the target recommendation features.

2. The method according to claim 1, characterized in that, The step of acquiring the target audio data of the media object to be processed in the target sampling window includes: The target dynamic sampling time point and window duration of the media object to be processed are obtained. Based on the target dynamic sampling time point and the window duration, a target sampling window for sampling the multimedia data of the media object to be processed is determined. The target dynamic sampling time point refers to the time point that changes as the associated duration of the media data of the media object to be processed changes. Audio data with the data sampling end time point being the target dynamic sampling time point and the sampling duration being the window duration, is obtained from the multimedia data of the media object to be processed through the target sampling window; The acquired audio data is used as the target audio data of the media object to be processed in the target sampling window.

3. The method according to claim 2, characterized in that, The target dynamic sampling time point includes a first dynamic sampling time point, the target sampling window includes a first sampling window corresponding to the first dynamic sampling time point, and the target audio data includes first audio data collected based on the first dynamic sampling time point and the window duration; The method further includes: The sampling interval duration for dynamic sampling is obtained, and based on the first dynamic sampling time point and the sampling interval duration, a second dynamic sampling time point for sampling the multimedia data of the media object to be processed is determined; the second dynamic sampling time point is the next sampling time point after the first dynamic sampling time point. Based on the second dynamic sampling time point and the window duration, a second sampling window is determined for sampling the multimedia data of the media object to be processed; the second sampling window is the next sampling window after the first sampling window, and the second sampling window overlaps with the first sampling window; Audio data with the data sampling end time point being the second dynamic sampling time point and the sampling duration being the duration of the window is obtained from the multimedia data of the media object to be processed through the second sampling window; The acquired audio data is used as the second audio data of the media object to be processed in the target sampling window. When the target dynamic sampling time point is updated through the second dynamic sampling time point, the target audio data is updated through the second audio data.

4. The method according to any one of claims 1-3, characterized in that, The step of performing text vector representation processing on the target speech recognition text corresponding to the target audio data to obtain a target text representation vector for representing the media object to be processed includes: A speech conversion understanding component and a similarity retrieval component are acquired for speech conversion processing of the target audio data; the speech conversion understanding component includes a speech recognizer and a semantic understanding model for semantic extraction; the similarity retrieval component includes a text representation. The target audio data is processed by the speech recognizer to obtain the speech recognition result. If the speech recognition processing result indicates that the target speech recognition text has been recognized from the target audio data, then the target speech recognition text is processed by the semantic understanding model to obtain the target semantic understanding information of the target speech recognition text. The target semantic understanding information is represented by the text representativeer to obtain a target text representation vector for representing the media object to be processed.

5. The method according to claim 4, characterized in that, The method further includes: If the speech recognition processing result indicates that the target speech recognition text has not been recognized, then the target audio data is marked as invalid audio data.

6. The method according to claim 4, characterized in that, The method further includes: If the speech recognition processing result indicates that the target speech recognition text of the target audio data was not recognized, then the video data of the media object to be processed is obtained from the multimedia data of the media object to be processed through the target sampling window, and the obtained video data is used as the target video data of the media object to be processed in the target sampling window. Video content recognition is performed on video frames in the target video data to obtain video text content that represents the target video data, and the target speech recognition text is updated using the video text content.

7. The method according to claim 4, characterized in that, The step of performing text semantic understanding processing on the target speech recognition text through the semantic understanding model to obtain the target semantic understanding information of the target speech recognition text includes: Obtain guidance prompt template data to guide the semantic understanding model in its output; The guidance prompt template data and the target speech recognition text are input into the semantic understanding model. The semantic understanding model extracts semantic information from the target speech recognition text according to the data content in the guidance prompt template data, thereby obtaining the target semantic extraction information of the target speech recognition text. The target semantic understanding information of the target speech recognition text is determined by extracting the target semantic information.

8. The method according to claim 7, characterized in that, The step of obtaining guidance prompt template data for guiding the semantic understanding model to output includes: If the object type of the media object to be processed is a live video type, then the prompt template data associated with the live video type will be used as the guidance prompt template data; If the object type of the media object to be processed is video on demand, then the prompt template data associated with the video on demand type will be used as the guidance prompt template data.

9. The method according to claim 1, characterized in that, The target text representation vector is obtained by performing speech conversion processing on the target audio data through a speech conversion understanding component and a similarity retrieval component. The step of obtaining N historical text representation vectors for representing N historical media objects includes: Obtain the distribution of interaction data curves corresponding to the historical media object database; The distribution of the interactive data curve is determined based on each historical media object in the historical media object database and the amount of interactive data of each historical media object. Based on the interactive data curve distribution, N historical media objects whose interactive data volume reaches the interactive data threshold are obtained from the historical media object database; N historical text representation vectors of the N historical media objects are obtained from the historical text vector representation database associated with the N historical media objects. A historical text representation vector is obtained by performing speech conversion processing on the historical audio data of a historical media object through the speech conversion understanding component and the similarity retrieval component.

10. The method according to claim 1, characterized in that, When selecting K historical media objects from the N historical media objects using the N vector similarity, establishing a similarity association between the K historical media objects and the media object to be processed includes: The N historical media objects are sorted based on the similarity of the N vectors to obtain a list of historical media objects associated with the N historical media objects; K historical media objects whose vector similarity satisfies the similarity retrieval condition are obtained from the historical media object list, and a display similarity association relationship is established between the K historical media objects and the media object to be processed; the display association relationship is used to characterize that the list composed of the K historical media objects is a list of similar media objects, and any media object in the list of similar media objects has an association relationship with the media object to be processed.

11. The method according to claim 1, characterized in that, When obtaining the K historical recommendation features of the K historical media objects, the step of determining the target recommendation features for characterizing the media object to be processed based on the similarity association and the K historical recommendation features includes: When K historical recommendation features of the K historical media objects are obtained, feature pooling is performed on the K historical recommendation features based on the similarity association to obtain pooled features associated with the K historical recommendation features; The pooling processing features are determined as target recommendation features used to characterize the media object to be processed.

12. The method according to claim 1, characterized in that, The method further includes: Obtain K historical recommendation features of the K historical media objects from the recommendation feature database associated with the K historical media objects.

13. The method according to claim 12, characterized in that, The method further includes: Obtain N historical interaction behavior information corresponding to N historical media objects; one historical media object corresponds to one historical interaction behavior information, and one historical interaction behavior information is used to characterize the item interaction details of each interactive item in a historical media object; The item interaction details represented by the N historical interaction behavior information are used as training samples for training the initial recommendation model, and recommendation indicators are obtained for training the initial recommendation model. The initial recommendation model is trained using the training samples and the recommendation metrics to obtain a target recommendation model. The target recommendation model is used to extract N historical recommendation features from the N historical media objects. A historical recommendation feature is obtained by the target recommendation model from the interaction vector of the item interaction details represented by a historical interaction behavior information. The N historical recommendation features extracted by the target recommendation model are added to the recommendation feature database.

14. The method according to claim 1, characterized in that, The media object to be processed is a multimedia object in the business processing platform whose interaction data volume is less than the interaction data volume threshold; the recommendation model corresponding to the business processing platform is a target recommendation model; the target recommendation model is obtained by training the initial recommendation model deployed on the business processing platform with N historical interaction behavior information corresponding to the N historical media objects. The step of recommending media to target business objects that match the media object to be processed based on the target recommendation features includes: When the target recommendation model is deployed and applied to the business processing platform, object association data information of the business objects accessing the business processing platform is obtained. The object association data information and the target recommendation features are input into the target recommendation model. When the target recommendation model extracts object vector representation features for representing the business object from the object association data information, it filters target object vector representation features that match the target recommendation features from the object vector representation features. The business object corresponding to the characteristic of the target object vector is identified as the target business object that matches the media object to be processed among the selected business objects, and the media object to be processed is recommended to the target business object.

15. The method according to claim 14, characterized in that, The step of inputting the object association data information and the target recommendation features into the target recommendation model, and then, when the target recommendation model extracts object vector representation features for representing the business object from the object association data information, filtering target object vector representation features that match the target recommendation features from the object vector representation features, includes: The object-related data information and the target recommendation features are input into the target recommendation model. The target recommendation model is used to extract information vector representations of the object-related data information to obtain object vector representation features for representing the business object. The object vector representation features and the target recommendation features are matched to obtain the feature matching results; If the feature matching result indicates that there is an object vector representation feature in the object vector representation features that matches the target recommendation feature, then the object vector representation feature that matches the target recommendation feature is determined as the target object vector representation feature.

16. The method according to claim 14, characterized in that, The target recommendation services associated with the aforementioned business processing platform include live streaming recommendation services; The step of recommending the media object to be processed to the target business object includes: If the media object to be processed is a live media object, then the live media object is used as a target recommendation object in the live recommendation service to be recommended to the target service object, and the target recommendation object is sent to the terminal device corresponding to the target service object so that the terminal device displays the target recommendation object in the live recommendation list corresponding to the live recommendation service. If the media object to be processed is a video-on-demand (VOD) media object, then the VOD link corresponding to the VOD media object is encapsulated into a recommendation notification message, and the recommendation notification message is sent to the terminal device corresponding to the target service object, so that the terminal device displays the VOD link in the recommendation notification message on the live recommendation list corresponding to the live recommendation service; the VOD link is used to instruct the terminal device to display the VOD media object in response to the trigger operation of the recommendation notification message.

17. A data processing apparatus for media objects, characterized in that, include: The audio data speech recognition module is used to collect audio data of the media object to be processed at the target dynamic acquisition time point based on the target sampling window to obtain the target audio data, and then to perform recognition processing on the target audio data based on automatic speech recognition technology to obtain target speech recognition text. The module also extracts information from the target speech recognition model based on the semantic information mining template through the semantic understanding model and outputs target speech understanding text associated with the target speech recognition text. The vector similarity calculation module is used to perform text vector representation processing on the target speech recognition text corresponding to the target audio data to obtain a target text representation vector for representing the media object to be processed, and to perform text vector representation processing on the historical speech recognition text corresponding to the historical audio data to obtain a historical text representation vector for representing the historical media object, to obtain N historical text representation vectors for representing N historical media objects, and to perform vector similarity calculation on the target text representation vector and each historical text representation vector to obtain N vector similarities; The similarity association establishment module is used to establish a similarity association between the K historical media objects and the media object to be processed when selecting K historical media objects from the N historical media objects through the N vector similarity; the interaction data volume of each historical media object is higher than the interaction data volume of the media object to be processed; N is a positive integer greater than 1; a historical text representation vector is recorded after performing text vector representation processing on the historical speech recognition text corresponding to a historical audio data; a historical audio data refers to the audio data of a historical media object in the recorded historical sampling window; the window duration of the historical sampling window is consistent with the window duration of the target sampling window; K is a positive integer less than N; The target recommendation feature determination module is used to determine the target recommendation features for characterizing the media object to be processed based on the similarity association and the K historical recommendation features when the K historical media objects are obtained, and to make media recommendations to the target business objects that match the media object to be processed through the target recommendation features.

18. A computer device, characterized in that, Including memory and processor; The memory is connected to the processor, the memory is used to store computer programs, and the processor is used to invoke the computer programs so that the computer device performs the method according to any one of claims 1-16.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-16.

20. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, implements the method according to any one of claims 1-16.

Images