Media data processing method, device, electronic device and storage medium

Abstract

The present disclosure provides a media data processing method, device, electronic device and storage medium, which belongs to the field of network technology. In an embodiment of the present disclosure, a vector corresponding to a content tag of a first media data can be obtained as a first vector, and a vector corresponding to a content tag of a second media data can be obtained as a second vector, and the media types of the first media data and the second media data are different. According to the first vector and the second vector of each second media data, second media data related to the first media data is determined as candidate media data. Media data is recommended based on the candidate media data. In this way, obtaining related media data from media data of other media types for recommendation can enrich the recommended content, thereby improving the recommendation effect to a certain extent.

Description

Technical Field

[0001] The present disclosure belongs to the field of network technology, and in particular, relates to a media data processing method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of network technology, there is more and more information on the network platform. For example, users can view videos, articles, pictures, texts and other information based on the network platform.

[0003] In the related art, in order to improve the viewing efficiency of users, similar information of the same type is often recommended for the information selected by the user to view, and the recommendation effect is poor. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a media data processing method, device, electronic device and storage medium.

[0005] According to a first aspect of the present disclosure, a media data processing method is provided, the method comprising:

[0006] Obtaining a vector corresponding to a content tag of first media data as a first vector, and obtaining a vector corresponding to a content tag of second media data as a second vector; the first media data and the second media data have different media types;

[0007] Determine, according to the first vector and the second vectors of each of the second media data, second media data related to the first media data as candidate media data;

[0008] Based on the candidate media data, media data recommendation is performed.

[0009] Optionally, the recommending media data based on the candidate media data includes:

[0010] Respectively acquiring text information included in the first media data and the candidate media data to obtain first text information and second text information;

[0011] For any of the candidate media data, calculating the repetition between the first text information and the second text information of each of the candidate media data;

[0012] Determine the candidate media data corresponding to a repetition degree less than or equal to a preset repetition degree threshold as the target media data;

[0013] The target media data is recommended.

[0014] Optionally, the recommending the target media data includes:

[0015] Determine a recommendation value for each target media data according to the semantic relevance between each target media data and the first media data, the release time of the target media data, and the information richness of the target media data; the recommendation value is positively correlated with the semantic relevance, the release time, and the information richness;

[0016] Based on the recommendation value of each target media data, the determined target media data is recommended to the user.

[0017] Optionally, the recommending the determined target media data to the user based on the recommendation value of each target media data includes:

[0018] In the display interface of the first media data, display the media data with the largest recommendation value in the target media data and a recommendation display element, wherein the recommendation display element is used to indicate that the display includes at least part of the media data in the target media data;

[0019] In case a triggering operation on the recommendation display element is detected, viewing options of at least a portion of the target media data are displayed.

[0020] Optionally, determining, according to the first vector and the second vectors of each of the second media data, second media data related to the first media data as candidate media data includes:

[0021] Calculating the semantic relevance between the first media data and each of the second media data according to the first vector and each of the second vectors;

[0022] The second media data whose semantic relevance is greater than a preset relevance threshold is determined as the candidate media data.

[0023] Optionally, the method further includes:

[0024] Obtaining a category to which the first media data belongs as a first category;

[0025] Determine third media data whose category matches the first category as the second media data; the third media data and the second media data have the same media type.

[0026] Optionally, when the first media data is a video, the second media data is an article; when the first media data is an article, the second media data is a video.

[0027] According to a second aspect of the present disclosure, a media data processing device is provided, the device comprising:

[0028] A first acquisition module is configured to acquire a vector corresponding to a content tag of first media data as a first vector, and acquire a vector corresponding to a content tag of second media data as a second vector; the first media data and the second media data have different media types;

[0029] A first determining module is configured to determine, according to the first vector and the second vector of each of the second media data, second media data related to the first media data as candidate media data;

[0030] The recommendation module is configured to recommend media data based on the candidate media data.

[0031] Optionally, the recommendation module is specifically configured as follows:

[0032] Respectively acquiring text information included in the first media data and the candidate media data to obtain first text information and second text information;

[0033] For any of the candidate media data, calculating the repetition between the first text information and the second text information of each of the candidate media data;

[0034] Determine the candidate media data corresponding to a repetition degree less than or equal to a preset repetition degree threshold as the target media data;

[0035] The target media data is recommended.

[0036] Optionally, the recommendation module is further specifically configured to:

[0037] Determine a recommendation value for each target media data according to the semantic relevance between each target media data and the first media data, the release time of the target media data, and the information richness of the target media data; the recommendation value is positively correlated with the semantic relevance, the release time, and the information richness;

[0038] Based on the recommendation value of each target media data, the determined target media data is recommended to the user.

[0039] Optionally, the recommendation module is further specifically configured to:

[0040] In the display interface of the first media data, display the media data with the largest recommendation value in the target media data and a recommendation display element, wherein the recommendation display element is used to indicate that the display includes at least part of the media data in the target media data;

[0041] In case a triggering operation on the recommendation display element is detected, viewing options of at least a portion of the target media data are displayed.

[0042] Optionally, the first determining module is specifically configured to

[0043] Calculating the semantic relevance between the first media data and each of the second media data according to the first vector and each of the second vectors;

[0044] The second media data whose semantic relevance is greater than a preset relevance threshold is determined as the candidate media data.

[0045] Optionally, the device further comprises:

[0046] A second acquisition module is configured to acquire a category to which the first media data belongs as a first category;

[0047] The second determination module is configured to determine third media data whose category matches the first category as the second media data; the third media data has the same media type as the second media data.

[0048] Optionally, when the first media data is a video, the second media data is an article; when the first media data is an article, the second media data is a video.

[0049] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0050] processor;

[0051] a memory for storing instructions executable by the processor;

[0052] The processor is configured to execute the instructions to implement the media data processing method as described in any one of the first aspects.

[0053] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device executes the media data processing method as described in any one of the first aspects.

[0054] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product comprising readable program instructions, and when the readable program instructions are executed by a processor of an electronic device, the electronic device executes the media data processing method as described in any one of the first aspects.

[0055] Compared with the related art, the present invention has the following advantages and positive effects:

[0056] The media data processing method provided by the embodiment of the present disclosure obtains a vector corresponding to a content tag of first media data as a first vector, and obtains a vector corresponding to a content tag of second media data as a second vector, wherein the first media data and the second media data have different media types. According to the first vector and the second vectors of each second media data, second media data related to the first media data is determined as candidate media data. Media data is recommended based on the candidate media data. In this way, obtaining related media data from media data of other media types for recommendation can enrich the recommended content, thereby improving the recommendation effect to a certain extent.

[0057] At the same time, in the embodiments of the present disclosure, media data is determined across media types, that is, related media data is matched across modalities and recommended, which can improve the utilization rate of media data of other media types to a certain extent, thereby improving the overall resource utilization rate of the network platform.

[0058] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the specific implementation methods of the present disclosure are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present disclosure. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0060] Figure 1 is a flow chart of steps of a media data processing method provided by an embodiment of the present disclosure;

[0061] Figure 2 is a schematic diagram of an interface provided by an embodiment of the present disclosure;

[0062] Figure 3 It is a flow chart provided by an embodiment of the present disclosure;

[0063] Figure 4 is a block diagram of a media data processing device provided by an embodiment of the present disclosure;

[0064] Figure 5 is a block diagram of a device for media data processing according to an exemplary embodiment;

[0065] Figure 6 The invention is a block diagram showing a device for media data processing according to an exemplary embodiment. DETAILED DESCRIPTION

[0066] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0067] Figure 1 is a flow chart of the steps of a media data processing method provided by an embodiment of the present disclosure, such as Figure 1 As shown, the method may include:

[0068] Step 101: Obtain a vector corresponding to a content tag of first media data as a first vector, and obtain a vector corresponding to a content tag of second media data as a second vector; the first media data and the second media data have different media types.

[0069] In the embodiment of the present disclosure, the first media data may be media data of a first media type, and the second media data may be media data of a second media type, wherein the specific types of the first media type and the second media type may be set according to actual needs, and different media types may correspond to a modality, wherein media data of different media types have different forms when transmitting information. For example, the media types may include video, image, text, and the like.

[0070] The vector corresponding to the content tag of the media data may be obtained by vectorizing (embedding) the content tag of the media data. The content tag of the media data may be pre-set for the media data, and the annotation system of the content tag may be set according to actual needs. The content tag may characterize the content involved in the media data. For example, taking the media data as a video, assuming that the video is an entertainment news video in which Zhang San participated in the AA singing program and won the first place, then the content tags of the video may include: "Zhang San", "AA singing program", "champion", "entertainment". Assuming that the video is a video of Li Si teaching traditional embroidery skills, then the content tags of the video may include: "Li Si", "popular science", "traditional culture", "embroidery".

[0071] Furthermore, based on the preset content understanding model corresponding to each media type, the media data of the media type can be identified in advance, thereby obtaining the content tags of each media data of the media type. Accordingly, when obtaining the vector corresponding to the content tag, the pre-identified content tag can be obtained, and then the content tag can be vectorized to obtain the vector corresponding to the content tag. Of course, vectorization can also be performed in advance, so that the vector corresponding to the content tag can be directly read.

[0072] Step 102: Determine second media data related to the first media data as candidate media data according to the first vector and the second vector of each of the second media data.

[0073] Since the content forms and content label setting methods of media data of different media types are often different, directly using explicit content labels to recall related media data in other media types, or directly vectorizing the media data itself and recalling related media data in other media types based on the obtained high-dimensional vectors, may cause the recalled media data to be inaccurate. In this step, the relevant media data is determined cross-modally based on the vector corresponding to the content label. Since the content label is often shorter, the content label is more refined than the entire media data, and the vector corresponding to the content label can represent the content involved in the media data. Therefore, recalling candidate media data based on the vector corresponding to the content label can ensure the accuracy of the recalled candidate media data to a certain extent.

[0074] Step 103: Recommend media data based on the candidate media data.

[0075] In the embodiments of the present disclosure, the candidate media data may be directly recommended to the user, or the candidate media data may be further screened before recommendation, and the present disclosure does not impose any limitation on this.

[0076] The media data processing method provided by the embodiment of the present disclosure obtains a vector corresponding to a content tag of first media data as a first vector, and obtains a vector corresponding to a content tag of second media data as a second vector, wherein the first media data and the second media data have different media types. According to the first vector and the second vectors of each second media data, second media data related to the first media data is determined as candidate media data. Media data is recommended based on the candidate media data. In this way, obtaining related media data from media data of other media types for recommendation can enrich the recommended content, thereby improving the recommendation effect to a certain extent.

[0077] At the same time, in the embodiments of the present disclosure, media data is determined across media types, that is, related media data is matched across modalities and recommended, which can improve the utilization rate of media data of other media types to a certain extent, thereby improving the overall resource utilization rate of the network platform.

[0078] Optionally, in an implementation scenario, the first media data may be a video, and the second media data may be an article, that is, graphic information. Alternatively, the first media data may be an article, and the second media data may be a video. In the current content application (APP) industry, video and graphic information are two different forms of content. The technical system often pays more attention to the internal connection of each form of content, that is, to solve the relevance of video content and the relevance of graphic information content respectively, and apply them in the actual recommendation information flow. That is, for videos, videos related to content are recommended, and for graphic information, graphic information related to content is recommended. For example, when a user watches a video, the relevant technology will combine the relevance analysis of the video to recommend more related videos to the user. When a user browses a graphic information, combined with the analysis of the text, the user is recommended related graphic information. However, as two different content forms, video and graphic information have their own advantages and disadvantages when conveying information. For example, when watching a video, if the user wants to have an in-depth understanding of the relevant subjects of the video content and some media views, it is necessary to be able to quickly view the relevant graphic information, and the video itself alone often cannot meet this demand.

[0079] In the disclosed embodiments, with video as the first media data and graphic information as the second media data, or with graphic information as the first media data and video as the second media data, it is possible to recommend related graphic information to users based on the video, or to recommend related videos to users based on the graphic information. In this way, when viewing the video, the user can combine the graphic information related to the video for more extended reading and explore more interests of the user. When viewing graphic information, combining the video related to the graphic video allows the user to understand the relevant information more intuitively, thereby improving the user experience and improving the efficiency of the user's information acquisition.

[0080] Optionally, the operation of determining, according to the first vector and the second vectors of each of the second media data, the second media data related to the first media data as the candidate media data may specifically include:

[0081] Step S21: Calculate the semantic relevance between the first media data and each of the second media data according to the first vector and each of the second vectors.

[0082] In this step, for any second media data, the vector distance between the first vector and the second vector of the second media data can be calculated. Then, the semantic relevance between the first media data and the second media data is determined based on the vector distance. The vector distance can be calculated based on a preset distance calculation method. The semantic relevance can be negatively correlated with the vector distance. That is, the closer the vector distance, the greater the semantic relevance can be. Conversely, the farther the vector distance, the smaller the semantic relevance can be.

[0083] Furthermore, the content label is used as an explicit feature and the vector is used as an implicit feature. In the present disclosure, the semantic relevance is calculated based on the first vector and the second vector corresponding to the content label. That is, the calculation of the semantic relevance is achieved by combining explicit and implicit features, thereby ensuring the accuracy of the calculation to a certain extent.

[0084] Step S22: Determine the second media data whose semantic relevance is greater than a preset relevance threshold as the candidate media data.

[0085] In this step, the preset relevance threshold can be set according to actual needs, and the present disclosure does not limit this. Furthermore, if the semantic relevance is greater, it can be determined that the second media data has a higher degree of relevance to the first media data. Conversely, if the semantic relevance is smaller, it can be determined that the second media data has a lower degree of relevance to the first media data. Therefore, the embodiment of the present disclosure can use second media data with a larger semantic relevance, that is, second media data with a semantic relevance greater than the preset relevance threshold, as candidate media data, thereby ensuring that the subsequently recommended media data has sufficient relevance.

[0086] In the disclosed embodiment, the semantic relevance between the first media data and each second media data is calculated based on the first vector and each second vector, and only the second media data whose semantic relevance is greater than a preset relevance threshold is used as candidate media data. In this way, the candidate media data finally determined can be media data that is strongly correlated with the first media data to a certain extent, thereby ensuring the recommendation effect of the media data. At the same time, the semantic relevance is calculated based on the vector corresponding to the content tag to achieve the relevance matching of different modal contents, so that in subsequent recommendations, more efficient information that meets user needs can be mined, providing convenient information expansion capabilities, thereby improving information acquisition efficiency.

[0087] Optionally, in the embodiment of the present disclosure, the second media data and the first media data may belong to the same category. Accordingly, the second media data may be determined by the following steps:

[0088] Step S31: Obtain the category to which the first media data belongs as the first category.

[0089] In this step, the category pre-determined for the first media data can be read to obtain the first category. For example, the first media data can be identified in an offline stage based on a content understanding model corresponding to the first media type to obtain the category of the first media data.

[0090] Step S33: determine third media data whose category matches the first category as the second media data; the third media data has the same media type as the second media data.

[0091] In this step, the third media data may be all media data of the second media type in the network platform. The category pre-determined for the third media data may be read. For example, the category of the third media data may be obtained by identifying the third media data in an offline stage based on a content understanding model corresponding to the second media type. Furthermore, the category of the third media data may be compared with the first category. If the category of the third media data matches the first category, that is, the categories are the same, the third media data may be determined as the second media data.

[0092] It should be noted that in the embodiment of the present disclosure, label matching can also be performed before the above-mentioned step 101. For example, the difference between the quantity and type of content tags of the second media data and the content tags of the first media data can be detected. If the difference is greater than a preset threshold, the second media data is eliminated, thereby further narrowing the selection range and improving the selection efficiency.

[0093] In the disclosed embodiment, the category to which the first media data belongs is obtained as the first category. Then, third media data whose category matches the first category is determined as the second media data, wherein the third media data has the same media type as the second media data. In this way, by pre-performing category matching, media data whose category matches the category of the first media data in other media types is selected as the second media data, which can ensure to a certain extent that the second media data participating in the selection is more likely to be associated with the first media data, that is, while providing high-quality alternative objects for the selection operation, narrow the selection range, thereby saving selection costs and improving selection efficiency.

[0094] Optionally, the embodiment of the present disclosure may further perform the operation of recommending media data based on the candidate media data, which may include the following steps:

[0095] Step S41: respectively obtain text information included in the first media data and the candidate media data to obtain first text information and second text information.

[0096] Specifically, when the media data is text, the content of the media data can be directly extracted as text information. When the media data is an article, the text portion contained in the media data can be extracted, and, based on the Optical Character Recognition (OCR) algorithm, the characters on the picture in the article can be extracted to obtain text information. Further, when the media data is a picture, the characters on the picture can be directly extracted based on the OCR algorithm to obtain text information. Further, when the media data is a video, the characters on each frame of the video can be extracted based on the OCR algorithm, and, based on the Automatic Speech Recognition (ASR) algorithm, the audio stream in the video can be converted into text to obtain text information. Further, the text information extracted from the first media data is the first text information, and the text information extracted from the candidate media data is the second text information.

[0097] Step S42: For any of the candidate media data, calculate the degree of repetition between the first text information and the second text information of the candidate media data.

[0098] In this step, the characters in the first text information and the second text information can be compared to determine the number of characters in the second text information that are repeated with the first text information, that is, the number of characters belonging to the first text information that appear in the second text. Then, the ratio of the number of characters to the total number of characters in the second text information is calculated to obtain the repetition degree. The repetition degree can measure the proportion of repeated information in the second text information. The higher the repetition degree, the more repeated information can be determined. Conversely, the lower the repetition degree, the less repeated information can be determined.

[0099] Step S43: Determine the candidate media data corresponding to a repetition degree that is less than or equal to a preset repetition degree threshold as the target media data.

[0100] In this step, the preset repetition threshold can be pre-set according to actual needs. For example, the preset repetition threshold can be 70%. Accordingly, if the repetition of the candidate media data is greater than the preset repetition threshold, it can be determined that there is a large amount of repeated information in the candidate media data, and the candidate media data is highly repeated and similar information to the first media data. Therefore, the candidate media data can be filtered. Further, the remaining candidate media data corresponding to the repetition less than or equal to the preset repetition threshold can be determined as the target media data.

[0101] In the disclosed embodiment, the text information included in the first media data and the candidate media data is obtained respectively to obtain the first text information and the second text information. For any candidate media data, the repetition between the first text information and the second text information of the candidate media data is calculated. The candidate media data corresponding to the repetition less than or equal to the preset repetition threshold is determined as the target media data. In this way, before the recommendation, the candidate media data that is highly repeated with the first media data can be identified and filtered, which can ensure that the target media data finally recommended is data related to the first media data, rather than overlapping similar data, thereby improving the recommendation effect.

[0102] Optionally, the operation of recommending the target media data may specifically include:

[0103] Step S51, determining a recommendation value for each target media data according to the semantic relevance between each target media data and the first media data, the release time of the target media data, and the information richness of the target media data; the recommendation value is positively correlated with the semantic relevance, the release time, and the information richness.

[0104] In the embodiment of the present disclosure, the higher the semantic relevance, the higher the relevance of the target media data to the first media data. Specifically, the recommendation value can be calculated by obtaining the semantic relevance calculated above. The earlier the release time, the stronger the timeliness of the target media data. Specifically, the relevant information of the target media data can be obtained, and the release time of the target media data can be found therefrom. The release time may include the year, month, and day. For example, the release time may be 20200815, 20210112. Further, the higher the information richness, the stronger the ability of the target media data to expand relevant information can be determined. Specifically, the number of topics involved in the target media data can be counted, and the information richness can be determined according to the number of topics. The more topics involved, the higher the information richness can be. Accordingly, in the embodiment of the present disclosure, the method of determining the recommendation value in a manner that is positively correlated with the semantic relevance, the release time, and the information richness can, to a certain extent, make the target media data with a higher relevance to the first media data, a stronger timeliness, and a stronger ability to expand relevant information be recommended first, thereby ensuring the recommendation effect. Specifically, when calculating the recommendation value, the semantic relevance, the release time, and the information richness may be calculated to calculate a weighted value, thereby obtaining the recommendation value of the target media data.

[0105] It should be noted that the present invention can also eliminate target media data whose release time is greater than a preset time threshold from the current moment before calculation, that is, only calculate the target media data within a certain time window from the current moment, thereby avoiding the weak timeliness of the subsequently recommended target media data and reducing the amount of calculation.

[0106] Step S52: Recommend the determined target media data to the user based on the recommendation value of each target media data.

[0107] In one implementation, the first M target media data with the largest recommendation value can be recommended to the user. For example, the target media data can be sorted from large to small / from small to large according to the recommendation value to obtain the sorting result. In the case of sorting from large to small, the first M target media data in the sorting result are obtained to obtain the first M target media data with the largest recommendation value. In the case of sorting from small to large, the last M target media data in the sorting result are obtained to obtain the first M target media data with the largest recommendation value. Among them, M can be an integer set according to actual needs. For example, M can be 5. Furthermore, in another optional implementation, the target media data with a recommendation value greater than a preset recommendation value threshold can also be recommended to the user. Among them, the preset recommendation value threshold can be set according to actual needs, and the present disclosure does not limit this.

[0108] In the disclosed embodiment, the recommendation value of each target media data is determined based on the semantic relevance between each target media data and the first media data, the release time of the target media data, and the information richness of the target media data, and the recommendation value is positively correlated with the semantic relevance, the release time, and the information richness. Based on the recommendation value of each target media data, the determined target media data is recommended to the user. In this way, the target media data that is more relevant to the first media data, more timely, and can provide richer information can be preferentially recommended to the user. At the same time, the problem of too many recommended target media data, which increases the difficulty of user selection, can be avoided.

[0109] Optionally, the operation of recommending the determined target media data to the user based on the recommendation value of each target media data may specifically include:

[0110] Step S61: Displaying media data with the largest recommendation value in the target media data and a recommendation display element in the display interface of the first media data, wherein the recommendation display element is used to indicate that the display includes at least part of the media data in the target media data.

[0111] In the embodiment of the present disclosure, the display interface of the first media data may be an interface displaying the first media data. For example, taking the first media data as a video, the display interface of the first media data may be a playback interface of the first media data. The recommended display element may be a designated identifier, which may be a pre-set identifier for displaying a portion of the target media data in thumbnails. For example, it may be an identifier for displaying the remaining M-1 target media data in thumbnails. For example, the designated identifier may be a "more" option. For example, Figure 2 is a schematic diagram of an interface provided by an embodiment of the present disclosure, such as Figure 2 As shown, the interface displays the first media data a, the target media data b with the largest recommended value among the M target media data, and the designated identifier c. That is, in specific applications, the target media data with the largest recommended value is strongly displayed, and the product form of clicking more to view the target media data of the overall TOPM can improve the data viewing efficiency while improving the flexibility of user operations.

[0112] It should be noted that in the embodiments of the present disclosure, when the first media data is displayed, for example, when a video is played, the target media data can be automatically matched to the first media data and recommended, thereby improving the user's viewing experience.

[0113] Step S62: When a triggering operation on the recommendation display element is detected, viewing options for at least part of the target media data are displayed.

[0114] In the embodiment of the present disclosure, the trigger operation may be predefined. For example, the trigger operation may be a click operation, a long press operation, a voice trigger operation, and the like. Accordingly, when a trigger operation for a recommended display element is received, it may be determined that the user needs to view at least part of the target media data, and then viewing options for at least part of the target media data may be displayed. Among them, at least part of the target media data may include all target media data, or may only include other target media data except the target media data with the largest recommendation value. The viewing options may include the title of the target media data, and the user may control the terminal to display the target media data by clicking on the viewing option.

[0115] In an embodiment of the present disclosure, in the display interface of the first media data, the media data with the largest recommendation value in the target media data and the recommendation display element are displayed, and the recommendation display element is used to indicate that the display includes at least part of the media data in the target media data. In the case where a trigger operation on the recommendation display element is detected, viewing options for at least part of the media data in the target media data are displayed. By directly displaying the target media data with the largest recommendation value in the display interface of the first media data, it can be ensured that the user can directly view the target media data with the highest relevance, thereby allowing the user to achieve fast cross-modal content consumption and improve data viewing efficiency. At the same time, in the case where a trigger operation on the recommendation display element is detected, displaying viewing options for at least part of the media data in the target media data can enable the user to obtain information on demand more flexibly, thereby improving flexibility.

[0116] Furthermore, for a scenario where the first media data is a video and the second media data is graphic information, Figure 3 is a flow chart provided by an embodiment of the present disclosure, such as Figure 3 As shown, the categories and video content labels of the videos in the video library can be obtained based on the video content understanding model, and the categories and graphic content labels of the graphic information in the graphic library can be obtained based on the graphic content understanding model. Furthermore, when it is necessary to recall relevant graphic information for a certain video, the category and video content label of the video can be obtained, and category matching can be performed to determine the graphic information of the same category. Then, the label vectorization correlation calculation is performed, that is, the video content label of the video is vectorized to obtain the first vector. The graphic content labels of the graphic information of the same category are vectorized to obtain the second vector. Based on the first vector and the second vector, the semantic correlation is calculated. Then, the list of related graphic information can be recalled, that is, the target media data is selected based on the semantic correlation. Further, high repetitive content can be filtered. Then, based on timeliness and diversity, sorting can be performed to return the top 5 graphic information with the highest recommendation value (that is, the top 5 graphic information with the highest recommendation value). That is, based on semantic relevance, release time and information richness, the top 5 graphic information with the highest recommendation value are returned. Finally, the top 1 graphic information (i.e., the graphic information with the highest recommendation value) can be strongly displayed, and the top 5 graphic information can be displayed in a view more mode.

[0117] In the disclosed embodiment, by performing cross-modal content matching based on the vector corresponding to the content tag, it is possible to quickly recall related videos with text and picture information / recall related text and picture information with videos, and provide users with more adequate data of different modalities, so that users can break the limitations of content form in product experience and obtain a fusion data viewing experience. At the same time, it is possible to fully utilize content resources and improve the exposure and consumption rate of long-tail text and picture information / long-tail videos on the network platform.

[0118] Figure 4 is a block diagram of a media data processing device provided by an embodiment of the present disclosure, such as Figure 4 As shown, the device 20 may include:

[0119] A first acquisition module 201 is configured to determine, according to the first vector and the second vector of each of the second media data, second media data related to the first media data as candidate media data;

[0120] A first determining module 202 is configured to select second media data related to the first media data as target media data according to the first vector and the second vector of each of the second media data;

[0121] The recommendation module 203 is configured to recommend media data based on the candidate media data.

[0122] The media data processing device provided by the embodiment of the present disclosure obtains a vector corresponding to a content tag of first media data as a first vector, and obtains a vector corresponding to a content tag of second media data as a second vector, wherein the media types of the first media data and the second media data are different. According to the first vector and the second vectors of each second media data, second media data related to the first media data is determined as candidate media data. Media data is recommended based on the candidate media data. In this way, obtaining related media data from media data of other media types for recommendation can enrich the recommended content, thereby improving the recommendation effect to a certain extent.

[0123] At the same time, in the embodiments of the present disclosure, media data is determined across media types, that is, related media data is matched across modalities and recommended, which can improve the utilization rate of media data of other media types to a certain extent, thereby improving the overall resource utilization rate of the network platform.

[0124] Optionally, the recommendation module 203 is specifically configured to:

[0125] Respectively acquiring text information included in the first media data and the candidate media data to obtain first text information and second text information;

[0126] For any of the candidate media data, calculating the repetition between the first text information and the second text information of each of the candidate media data;

[0127] Determine the candidate media data corresponding to a repetition degree less than or equal to a preset repetition degree threshold as the target media data;

[0128] The target media data is recommended.

[0129] Optionally, the recommendation module 203 is further specifically configured to:

[0130] Determine a recommendation value for each target media data according to the semantic relevance between each target media data and the first media data, the release time of the target media data, and the information richness of the target media data; the recommendation value is positively correlated with the semantic relevance, the release time, and the information richness;

[0131] Based on the recommendation value of each target media data, the determined target media data is recommended to the user.

[0132] Optionally, the recommendation module 203 is further specifically configured to:

[0133] In the display interface of the first media data, display the media data with the largest recommendation value in the target media data and a recommendation display element, wherein the recommendation display element is used to indicate that the display includes at least part of the media data in the target media data;

[0134] In case a triggering operation on the recommendation display element is detected, viewing options of at least a portion of the target media data are displayed.

[0135] Optionally, the first determining module 202 is specifically configured to:

[0136] Calculating the semantic relevance between the first media data and each of the second media data according to the first vector and each of the second vectors;

[0137] The second media data whose semantic relevance is greater than a preset relevance threshold is determined as the candidate media data.

[0138] Optionally, the device 20 further includes:

[0139] A second acquisition module is configured to acquire a category to which the first media data belongs as a first category;

[0140] The second determination module is configured to determine third media data whose category matches the first category as the second media data; the third media data has the same media type as the second media data.

[0141] Optionally, when the first media data is a video, the second media data is an article; when the first media data is an article, the second media data is a video.

[0142] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0143] According to an embodiment of the present disclosure, an electronic device is provided, comprising: a processor and a memory for storing processor executable instructions, wherein the processor is configured to implement the steps in the media data processing method in any of the above embodiments when executing.

[0144] According to one embodiment of the present disclosure, a computer-readable storage medium is further provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the steps in the media data processing method in any of the above embodiments.

[0145] According to one embodiment of the present disclosure, a computer program product is also provided, which includes readable program instructions. When the readable program instructions are executed by a processor of an electronic device, the electronic device can execute the steps in the media data processing method in any of the above embodiments.

[0146] Figure 5 700 is a block diagram of a device for media data processing according to an exemplary embodiment. For example, the device 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0147] Reference Figure 5 , the device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 708 , an audio component 710 , an input / output (I / O) interface 712 , a sensor component 714 , and a communication component 716 .

[0148] The processing component 702 generally controls the overall operation of the device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above-mentioned media data processing method. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0149] The memory 704 is configured to store various types of data to support operations on the device 700. Examples of such data include instructions for any application or method operating on the device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0150] The power supply component 706 provides power to the various components of the device 700. The power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 700.

[0151] The multimedia component 708 includes a screen that provides an output interface between the device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0152] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), and when the device 700 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 704 or sent via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0153] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0154] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the device 700. For example, the sensor assembly 714 can detect the open / closed state of the device 700, the relative positioning of components, such as the display and keypad of the device 700, the sensor assembly 714 can also detect the position change of the device 700 or a component of the device 700, the presence or absence of user contact with the device 700, the orientation or acceleration / deceleration of the device 700, and the temperature change of the device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0155] The communication component 716 is configured to facilitate wired or wireless communication between the device 700 and other devices. The device 700 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0156] In an exemplary embodiment, the device 700 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned media data processing method.

[0157] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, and the instructions can be executed by the processor 720 of the device 700 to complete the above-mentioned media data processing method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0158] Figure 6800 is a block diagram of a device for media data processing according to an exemplary embodiment. For example, the device 800 may be provided as a server. Figure 6 The apparatus 800 includes a processing component 822, which further includes one or more processors, and a memory resource represented by a memory 832 for storing instructions, such as an application, that can be executed by the processing component 822. The application stored in the memory 832 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 822 is configured to execute instructions to perform the above-mentioned media data processing method.

[0159] The device 800 may also include a power supply component 826 configured to perform power management of the device 800, a wired or wireless network interface 850 configured to connect the device 800 to a network, and an input / output (I / O) interface 858. The device 800 may operate based on an operating system stored in the memory 832, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0160] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0161] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A media data processing method, characterized in that: The method comprises: A vector corresponding to a content tag of first media data is obtained as a first vector, and a vector corresponding to a content tag of second media data is obtained as a second vector; the first media data and the second media data have different media types; the difference between the number and type of the content tags of the first media data and the content tags of the second media data is less than or equal to a preset threshold; the category to which the first media data belongs is the same as the category to which the second media data belongs; Determine, according to the first vector and the second vectors of each of the second media data, second media data related to the first media data as candidate media data; Recommending media data based on the candidate media data; The recommending of media data based on the candidate media data includes: Respectively acquiring text information included in the first media data and the candidate media data to obtain first text information and second text information; For any of the candidate media data, calculating the repetition between the first text information and the second text information of each of the candidate media data; Determine the candidate media data corresponding to a repetition degree less than or equal to a preset repetition degree threshold as the target media data; Determine a recommendation value for each target media data according to the semantic relevance between each target media data and the first media data, the release time of the target media data, and the information richness of the target media data, wherein the recommendation value is positively correlated with the semantic relevance, the release time, and the information richness, and the recommendation value is obtained by calculating a weighted value of the semantic relevance, the release time, and the information richness; Based on the recommendation value of each target media data, the determined target media data is recommended to the user.

2. The method according to claim 1, characterized in that The step of recommending the determined target media data to the user based on the recommendation value of each target media data includes: In the display interface of the first media data, display the media data with the largest recommendation value in the target media data and a recommendation display element, wherein the recommendation display element is used to indicate that the display includes at least part of the media data in the target media data; In case a triggering operation on the recommendation display element is detected, viewing options of at least a portion of the target media data are displayed.

3. The method according to any one of claims 1 to 2, characterized in that: The determining, according to the first vector and the second vectors of each of the second media data, second media data related to the first media data as candidate media data includes: Calculating the semantic relevance between the first media data and each of the second media data according to the first vector and each of the second vectors; The second media data whose semantic relevance is greater than a preset relevance threshold is determined as the candidate media data.

4. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: Obtaining a category to which the first media data belongs as a first category; Determine third media data whose category matches the first category as the second media data; the third media data and the second media data have the same media type.

5. The method according to any one of claims 1 to 2, characterized in that: When the first media data is a video, the second media data is an article; when the first media data is an article, the second media data is a video.

6. A media data processing device, characterized in that: The device comprises: A first acquisition module is configured to acquire a vector corresponding to a content tag of first media data as a first vector, and acquire a vector corresponding to a content tag of second media data as a second vector; the first media data and the second media data have different media types; the difference between the number and type of the content tags of the first media data and the content tags of the second media data is less than or equal to a preset threshold; the category to which the first media data belongs is the same as the category to which the second media data belongs; A first determining module is configured to determine, according to the first vector and the second vector of each of the second media data, second media data related to the first media data as candidate media data; The recommendation module is configured to respectively obtain the text information included in the first media data and the candidate media data to obtain the first text information and the second text information; for any of the candidate media data, calculate the repetition between the first text information and the second text information of each of the candidate media data; determine the candidate media data corresponding to the repetition less than or equal to the preset repetition threshold as the target media data; determine the recommendation value of each of the target media data according to the semantic relevance between each of the target media data and the first media data, the release time of the target media data and the information richness of the target media data, the recommendation value is positively correlated with the semantic relevance, the release time and the information richness, and the recommendation value is obtained by calculating the weighted value of the semantic relevance, the release time and the information richness; based on the recommendation value of each of the target media data, recommend the determined target media data to the user.

7. The device according to claim 6, characterized in that The recommendation module is further specifically configured as follows: In the display interface of the first media data, display the media data with the largest recommendation value in the target media data and a recommendation display element, wherein the recommendation display element is used to indicate that the display includes at least part of the media data in the target media data; In case a triggering operation on the recommendation display element is detected, viewing options of at least a portion of the target media data are displayed.

8. The device according to any one of claims 6 to 7, characterized in that: The first determining module is specifically configured to calculate the semantic relevance between the first media data and each of the second media data according to the first vector and each of the second vectors; The second media data whose semantic relevance is greater than a preset relevance threshold is determined as the candidate media data.

9. The device according to any one of claims 6 to 7, characterized in that: The device also includes: A second acquisition module is configured to acquire a category to which the first media data belongs as a first category; The second determination module is configured to determine third media data whose category matches the first category as the second media data; the third media data has the same media type as the second media data.

10. The device according to any one of claims 6 to 7, characterized in that: When the first media data is a video, the second media data is an article; when the first media data is an article, the second media data is a video.

11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the media data processing method as described in any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device executes the media data processing method as claimed in any one of claims 1 to 7.

13. A computer program product, characterized in that The computer program product comprises readable program instructions, and when the readable program instructions are executed by a processor of an electronic device, the electronic device executes the media data processing method according to any one of claims 1 to 5.

Images