Multimedia content processing method and apparatus, device, and storage medium

Abstract

Provided are a multimedia content processing method and apparatus, a device, and a storage medium. The method comprises: first, in response to a preset trigger operation acting on a display page of a first multimedia content, identifying at least one target resource object carried in the first multimedia content, wherein the target resource objects and recommendation object types have preset correspondence; then, on the basis of the preset correspondence, determining a recommendation object type corresponding to a first target resource object among the at least one target resource object, and determining at least one recommendation object on the basis of the first target resource object; furthermore, displaying the at least one recommendation object. According to the embodiment of the invention, during the multimedia content display process, the recommendation object related to the target resource object is displayed for the user on the basis of the target resource object carried in the multimedia content, the secondary consumption path for the content carried in the multimedia content is provided to the user, and the user experience is further improved.

Description

Multimedia content processing method, device, equipment and storage medium

[0001] This application claims priority to the Chinese invention patent application entitled “A method, device, equipment and storage medium for processing multimedia content” and application number 2023101598223, filed on February 14, 2023. The entire contents of that application are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of data processing, and in particular to a method, apparatus, device, and storage medium for processing multimedia content. Background Art

[0003] With the continuous development of video processing technology, people's demand for video-related functions is becoming more and more diverse. Therefore, how to enrich video-related functions to meet more user needs and thus improve user experience is a technical problem that needs to be solved urgently.

[0004] Summary of the Invention

[0005] In order to solve the above technical problems, the present disclosure provides a multimedia content processing method, apparatus, device and storage medium, which can enrich video-based interactive functions and enhance user experience.

[0006] In a first aspect, the present disclosure provides a multimedia content processing method, the method comprising: in response to a preset trigger operation acting on a display page of first multimedia content, identifying at least one target resource object carried in the first multimedia content; wherein, there is a preset correspondence between the target resource object and the recommended object type; based on the preset correspondence, determining the recommended object type corresponding to the first target resource object among the at least one target resource object; determining at least one recommended object based on the first target resource object; wherein, the at least one recommended object belongs to the recommended object type corresponding to the first target resource object; and displaying the at least one recommended object.

[0007] In an optional embodiment, the at least one target resource object also includes a second target resource object, and the display of the at least one recommended object includes: classifying and displaying the recommended objects determined based on the first target resource object and the second resource object respectively according to the recommended object type.

[0008] In an optional embodiment, the recommended objects determined based on the first target resource object and the second resource object are classified and displayed according to the recommended object type, including: displaying at least one first recommended object determined based on the first target resource object on a first card; wherein the first recommended object belongs to the recommended object type corresponding to the first target resource object; and, displaying at least one second recommended object determined based on the second resource object on a second card; wherein the second recommended object belongs to the recommended object type corresponding to the second target resource object, and the first card and the second card belong to a card set displayed in a stacked style.

[0009] In an optional embodiment, the method further includes: in response to a preset sliding operation triggered on the card set, scrolling and displaying each card in the card set.

[0010] In an optional embodiment, the method further includes: displaying the recommended object on the target card on the recommended object display page in response to a preset trigger operation for the target card in the card set; and receiving a preset interactive operation for the target recommended object on the recommended object display page.

[0011] In an optional embodiment, the at least one target resource object includes an item object, the recommended object type corresponding to the item object includes an item type, and determining at least one recommended object based on the first target resource object includes: determining at least one recommended item having the same or similar characteristics as the item object; wherein the recommended item belongs to the item type.

[0012] In an optional embodiment, the at least one target resource object includes background music, the recommended object type corresponding to the background music includes a music type, and determining at least one recommended object based on the first target resource object includes: performing music recognition on the background music to obtain a music recognition result; based on the music recognition result, determining a song resource corresponding to the background music; wherein the song resource belongs to the music type.

[0013] In an optional embodiment, the at least one target resource object includes address information, and the recommended object type corresponding to the address information includes a life service type. The determining of at least one recommended object based on the first target resource object includes: determining at least one life service object within a preset distance range with the address information as the center; wherein the life service object belongs to the life service type.

[0014] In an optional embodiment, the at least one target resource object includes a target face displayed on the video frame, and the recommended object type corresponding to the target face includes a user account type. The determining of at least one recommended object based on the first target resource object includes: based on the target face displayed on the video frame, determining at least one user account whose user avatar has a similarity with the target face that reaches a preset threshold; wherein, the user account belongs to the user account type.

[0015] In an optional embodiment, the identifying of at least one target resource object carried in the first multimedia content in response to a preset trigger operation on a display page of the first multimedia content includes: displaying multiple key video frames in the first multimedia content on a video recognition page in the form of a transition effect in response to a preset trigger operation on a display page of the first multimedia content; and identifying at least one target resource object carried in the first multimedia content based on the multiple key video frames.

[0016] In a second aspect, the present disclosure provides a multimedia content processing device, which includes: an identification module for identifying at least one target resource object carried in the first multimedia content in response to a preset trigger operation acting on a playback page of the first multimedia content; wherein the target resource object and the recommended object type have a preset correspondence; a first determination module for determining, based on the preset correspondence, the recommended object type corresponding to the first target resource object among the at least one target resource object; a second determination module for determining at least one recommended object based on the first target resource object; wherein the at least one recommended object belongs to the recommended object type corresponding to the first target resource object; and a display module for displaying the at least one recommended object.

[0017] In a third aspect, the present disclosure provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on a terminal device, the terminal device implements the above method.

[0018] In a fourth aspect, the present disclosure provides a multimedia content processing device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method when executing the computer program.

[0019] In a fifth aspect, the present disclosure provides a computer program product, which includes a computer program / instructions, and the computer program / instructions implement the above method when executed by a processor.

[0020] The technical solution provided by the embodiments of the present disclosure has at least the following advantages compared with the prior art:

[0021] The embodiment of the present disclosure provides a multimedia content processing method, which firstly, in response to a preset trigger operation on a display page of the first multimedia content, identifies at least one target resource object carried in the first multimedia content, wherein the target resource object and the recommended object type have a preset object relationship; then, based on the preset corresponding relationship, determines the recommended object type corresponding to the first target resource object in the at least one target resource object, and determines at least one recommended object based on the first target resource object; and then, displays the at least one recommended object. The embodiment of the present disclosure can display recommended objects related to the target resource object to the user based on the target resource object carried in the multimedia content during the multimedia content display process. It can be seen that the embodiment of the present disclosure provides the user with an extended consumption path for the content carried in the multimedia content, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0023] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] FIG1 is a flowchart of a multimedia content processing method provided by an embodiment of the present disclosure;

[0025] FIG2 is a schematic diagram of a video recognition page provided by an embodiment of the present disclosure;

[0026] FIG3 is a schematic diagram of another video recognition page provided by an embodiment of the present disclosure;

[0027] FIG4 is a schematic diagram of a display page corresponding to a card set provided by an embodiment of the present disclosure;

[0028] FIG5 is a schematic diagram of a recommended object display page provided by an embodiment of the present disclosure;

[0029] FIG6 is a flowchart of another multimedia content processing method provided by an embodiment of the present disclosure;

[0030] FIG7 is a schematic diagram of another recommended object display page provided by an embodiment of the present disclosure;

[0031] FIG8 is a schematic diagram of another recommended object display page provided by an embodiment of the present disclosure;

[0032] FIG9 is a schematic structural diagram of a multimedia content processing device provided by an embodiment of the present disclosure;

[0033] FIG10 is a schematic structural diagram of a multimedia content processing device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0035] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0036] With the continuous development of video processing technology, people's demands for video-related functions are becoming more and more diversified. Therefore, how to enrich video-related functions, meet more user needs, and thus improve user experience is a technical problem that needs to be solved urgently.

[0037] In addition, videos carry more and more relevant information, such as objects, places, people, background music, etc. During the process of watching videos, users may generate further consumption demands for the relevant information carried in the videos.

[0038] Related technologies use a method of adding function anchors to video playback pages to provide users with a path to further consume relevant information contained in the video. However, the display range of a video playback page is limited. Excessive display of function anchors can affect the simplicity of the video playback page and hinder the user's viewing experience.

[0039] To this end, the present embodiment provides a multimedia content processing method, first, in response to a preset trigger operation acting on a display page of the first multimedia content, at least one target resource object carried in the first multimedia content is identified. In some embodiments, there is a preset object relationship between the target resource object and the recommended object type. Then, based on the preset corresponding relationship, the recommended object type corresponding to the first target resource object in the at least one target resource object is determined, and at least one recommended object is determined based on the first target resource object. Then, the at least one recommended object is displayed. The embodiment of the present disclosure can display recommended objects related to the target resource object to the user based on the target resource object carried in the multimedia content during the process of multimedia content display. It can be seen that the embodiment of the present disclosure provides the user with an extended consumption path for the content carried in the multimedia content, thereby improving the user experience.

[0040] Based on this, an embodiment of the present disclosure provides a multimedia content processing method. Referring to FIG1 , which is a flowchart of a multimedia content processing method provided by an embodiment of the present disclosure, the method includes:

[0041] S101: In response to a preset triggering operation on a presentation page of first multimedia content, identifying at least one target resource object carried in the first multimedia content.

[0042] In some embodiments, there is a preset correspondence between the target resource object and the recommended object type.

[0043] In some embodiments, the first multimedia content may include any video, audio, graphic content, text content, picture content, etc. Specifically, the first multimedia content may be any multimedia content in any multimedia information stream, such as any recommended video in a recommended video stream.

[0044] In some embodiments, the preset trigger operation on the display page of the first multimedia content may include a knuckle double-click operation, a long press operation, a double-click operation, etc. on the display page of the first multimedia content, and the embodiments of the present disclosure do not limit this.

[0045] In some embodiments, the target resource object can be an object displayed on the video frame carried in the first multimedia content, such as a sweatshirt, a stool, a pet dog, etc., or it can be the address information appearing in the text content (such as subtitles) or location anchor (such as a tourist attraction, a food court, a supermarket, etc.) carried in the first multimedia content, or it can be the background music played in the first multimedia content, a public figure appearing in the video frame carried in the first multimedia content, and so on.

[0046] In some embodiments, upon receiving a preset trigger operation on a display page of first multimedia content, at least one target resource object carried in the first multimedia content is identified. For example, text recognition technology (Optical Character Recognition, abbreviated as OCR), speech recognition technology (Automatic Speech Recognition, abbreviated as ASR), face recognition technology, etc. can be used to identify the first multimedia content and obtain the target resource object carried in the first multimedia content. The specific identification method is described in subsequent embodiments for different target resource objects and is not repeated here.

[0047] In some embodiments, the recommended object type is used to identify the type of recommended object determined based on the target resource object. In some embodiments, the recommended object type may include item type, music type, lifestyle service type, user account type, etc., which is not limited in the present embodiment and can be set based on actual needs.

[0048] In some embodiments, there is a preset correspondence between the target resource object and the recommended object type. Different target resource objects may correspond to the same or different recommended object types. For example, if the target resource object is a certain sweatshirt, then the corresponding recommended object type may be an item type; if the target resource object is a piece of music, then the corresponding recommended object type may be a music type.

[0049] In actual applications, assuming that the first multimedia content includes a video, in order to enhance the user's multimedia content viewing experience and enable the user to have certain expectations for the function of determining the corresponding recommended object based on the target resource object carried in the first multimedia content, when a preset trigger operation acting on the display page of the first multimedia content is received, multiple key video frames in the first multimedia content can be displayed on the video recognition page in the form of a transition animation, and then at least one target resource object carried in the first multimedia content can be identified based on the multiple key video frames.

[0050] In an optional implementation, key video frames in the first multimedia content may be captured at preset frame intervals. For example, a video frame of the first multimedia content may be captured every 10 frames to obtain multiple key video frames.

[0051] In another optional implementation, key video frames in the first multimedia content may be captured at preset time intervals. For example, a video frame of the first multimedia content may be captured every 1 second to obtain multiple key video frames.

[0052] As shown in Figure 2, a schematic diagram of a video recognition page provided in an embodiment of the present disclosure is shown, wherein multiple key video frames are displayed on the video recognition page according to a preset motion trajectory or a random motion trajectory. In some embodiments, the preset motion trajectory may include moving from the center position of the video recognition page to the edge position, etc.

[0053] In an optional embodiment, multiple key video frames in the first multimedia content can be displayed on different cards on the video recognition page based on the style of the cards displayed in a stacked style, and then at least one target resource object carried in the first multimedia content can be identified based on the multiple key video frames. As shown in Figure 3, a schematic diagram of another video recognition page provided in an embodiment of the present disclosure is shown. In some embodiments, multiple key video frames are displayed on the video recognition page in a stacked card style.

[0054] S102: Determine a recommended object type corresponding to a first target resource object among the at least one target resource object based on a preset corresponding relationship.

[0055] S103: Determine at least one recommended object based on the first target resource object.

[0056] In some embodiments, the at least one recommended object belongs to a recommended object type corresponding to the first target resource object.

[0057] In some embodiments, the first target resource object may be any one of the at least one target resource object identified from the first multimedia content.

[0058] In some embodiments, after identifying at least one target resource object carried in the first multimedia content, at least one recommended object belonging to the recommended object type corresponding to any one of the at least one target resource object can be determined.

[0059] In some embodiments, the first target resource object and at least one recommended object have the same or similar features. For example, assuming the first target resource object is a chorus segment of a song, based on the chorus segment and the corresponding recommended object type (i.e., music genre), the entire song corresponding to the chorus segment can be determined as the recommended object corresponding to the first target resource object.

[0060] S104: Display the at least one recommended object.

[0061] In some embodiments, after at least one recommended object is determined based on the target resource object, each recommended object is displayed.

[0062] In an optional implementation, the at least one target resource object may further include a second target resource object. In some embodiments, the second target resource object and the first target resource object belong to different recommended object types.

[0063] Therefore, displaying the recommended objects further includes: after determining at least one recommended object based on the first target resource object and the second target resource object respectively, each recommended object can be classified and displayed according to the recommended object type to which it belongs.

[0064] For example, assume the first target resource object is a white sweatshirt, corresponding to the item type. The second target resource object is a chorus clip, corresponding to the music type. The recommended objects determined based on the white sweatshirt include white long-sleeved sweatshirts and white short-sleeved sweatshirts. The recommended object determined based on music clip A is the complete song B. Then, based on the recommended object type, the white long-sleeved sweatshirt, white short-sleeved sweatshirt, and music B are categorized and displayed. That is, the white long-sleeved sweatshirt and white short-sleeved sweatshirt are aggregated and displayed as recommended objects of the item type, and music B is displayed as a recommended object of the music type.

[0065] It is worth noting that the above content only uses the example of at least one target resource object including two target resource objects. The embodiment of the present disclosure does not limit the number of target resource objects identified from the first multimedia content. The display method of the recommended objects corresponding to multiple target resource objects can be implemented with reference to the above method and will not be repeated here.

[0066] In the multimedia content processing method provided by the embodiment of the present disclosure, first, in response to a preset trigger operation acting on the display page of the first multimedia content, at least one target resource object carried in the first multimedia content is identified, wherein a preset object relationship exists between the target resource object and the recommended object type. Then, based on the preset corresponding relationship, the recommended object type corresponding to the first target resource object in the above-mentioned at least one target resource object is determined, and at least one recommended object is determined based on the first target resource object. Then, the at least one recommended object is displayed. The embodiment of the present disclosure can display recommended objects related to the target resource object to the user based on the target resource object carried in the multimedia content during the process of multimedia content display. It can be seen that the embodiment of the present disclosure provides users with an extended consumption path for the content carried in the multimedia content, thereby improving the user experience.

[0067] In actual applications, in order to enrich the interactive functions in the multimedia content display process and improve the user's multimedia content viewing experience, recommended objects of different recommendation object types can also be displayed in the style of cards. Specifically, the recommended objects determined based on the first target resource object and the second target resource object are classified and displayed in the style of cards, wherein the first target resource object and the second target resource object correspond to different recommendation object types, respectively.

[0068] Specifically, at least one first recommended object determined based on the first target resource object is displayed on the first card, wherein the first recommended object belongs to the recommended object type corresponding to the first target resource object. At least one second recommended object determined based on the second resource object is displayed on the second card, wherein the second recommended object belongs to the recommended object type corresponding to the second target resource object.

[0069] As shown in Figure 4, a schematic diagram of a display page corresponding to a card set provided in an embodiment of the present disclosure is shown. Taking two cards as an example, the first card 401 displays recommended objects determined based on a white top (a first target resource object), such as a white long-sleeved top and a white short-sleeved top, and the second card 402 displays recommended objects determined based on a music clip A (a second target resource object), such as music B. In some embodiments, the first card and the second card belong to a card set displayed in a stacked fashion.

[0070] In an optional embodiment, on the display page corresponding to the card collection, while displaying the card collection in a stacked style, the individual cards in the card collection can be scrolled and displayed through a preset sliding operation triggered for the card collection, so that the user can select the required cards for display based on the content displayed on each card.

[0071] In some embodiments, the preset sliding operation triggered on the card set may include an upward sliding operation, a downward sliding operation, etc. on the card set, which is used to trigger the scrolling display of each card in the card set.

[0072] As shown in Figure 4, when an upward swipe operation is received for a card set, the currently displayed card 401 is switched upward to the adjacent next card, i.e., card 402, so that each recommended object in card 402 is displayed on the display page where the card set is located. When a downward swipe operation is received for a card set, the currently displayed card 401 is switched downward to the adjacent previous card, i.e., card 403, so that the entire object is displayed on the display page where the card set is located.

[0073] In an optional embodiment, if the user wants to exit the display of the card set, the user can return to the playback page of the first multimedia content through a preset return operation for the card set. In some embodiments, the preset return operation for the card set can include a left swipe operation on the card set, etc. In addition, a return control 404 can be provided on the display page where the card set is located, and by clicking the return control, the user returns to the display page of the first multimedia content.

[0074] In actual applications, since the number of cards in a card set is limited, when a preset sliding operation triggered on a card set is received, each card in the card set can be displayed in a circular scrolling manner.

[0075] In an optional implementation, during the process of displaying the card set, each recommended object on the target card can be displayed on the recommended object display page through a preset trigger operation on the target card in the card set.

[0076] In some embodiments, the preset trigger operation for the target card in the card set may include a click operation, a long press operation, etc. on the target card.

[0077] Specifically, when a preset trigger operation is received for any card in the card set, the card corresponding to the preset trigger operation is determined as the target card, and each recommended object on the target card is displayed on the recommended object display page.

[0078] As shown in Figure 4 , when a preset trigger operation is received for card 401, card 401 is determined as a target card, and recommended objects on target card 401, such as white long-sleeved top 501 and white short-sleeved top 502, are displayed in Figure 5 . Figure 5 is a schematic diagram of a recommended object display page provided by an embodiment of the present disclosure.

[0079] In some embodiments, when displaying a card collection in a stacked format, a preset sliding operation triggered on the card collection can be used to scroll through the cards in the collection. When a preset trigger operation is received for a target card in the card collection, the recommended objects on the target card are displayed on the recommended object display page, facilitating the user's extended consumption of content related to the currently displayed multimedia content, thereby enhancing the user's multimedia content viewing experience.

[0080] In practical applications, the target resource objects identified from the first multimedia content may include resource objects of the first resource type. In some embodiments, the first resource type may include, for example, an item type.

[0081] Based on the above embodiment, the present disclosure provides a specific method for determining at least one recommended object for a resource object of a first resource type. Referring to FIG6 , there is a flowchart of another multimedia content processing method provided by the present disclosure, the method comprising:

[0082] S601: In response to a preset triggering operation on a play page of first multimedia content, identifying an item object carried in the first multimedia content.

[0083] In some embodiments, the identified item type carried in the first multimedia content may include one or more item objects, such as a sweatshirt, pants, a table, a school bag, and the like.

[0084] In an optional embodiment, when a preset trigger operation is received on a display page of the first multimedia content, first, a video frame in the first multimedia content is captured, and then a subject recognition algorithm is called to filter out video frames carrying resource objects of the first resource type.

[0085] In actual applications, when capturing video frames from the first multimedia content, the video frames from the first multimedia content can be captured based on a preset frame interval, for example, capturing the first multimedia content once every 10 frames. Alternatively, the video frames from the first multimedia content can be captured based on a preset time interval, for example, capturing the first multimedia content once every 1 second. The disclosed embodiments do not limit the method for capturing video frames.

[0086] For example, upon receiving a preset trigger action on a display page of first multimedia content, first, video frames from the first multimedia content are captured to obtain 10 video frames. Subject recognition is then performed on each of the 10 captured video frames. In some embodiments, two of the video frames contain resource objects of the first resource type, such as a white sweatshirt and a short dress.

[0087] S602: Based on the preset correspondence, determine whether the item object corresponds to the item type.

[0088] S603: Determine at least one recommended object based on the item object.

[0089] In some embodiments, the at least one recommended object belongs to a recommended object type corresponding to the item object.

[0090] Based on the item object, at least one recommended object having the same or similar characteristics as the item object is determined. In some embodiments, the at least one recommended object belongs to the item type.

[0091] In some embodiments, after identifying an item object in the first multimedia content, at least one recommended object having the same or similar characteristics as the item object can be determined based on each identified item object. In some embodiments, at least one recommended object belongs to an item type.

[0092] In an optional implementation, after identifying that the first multimedia content carries an object, the video frame carrying the object can be sent to a picture similarity calculation model, and then at least one recommended object corresponding to the object is determined based on the picture similarity calculation model.

[0093] In some embodiments, the image similarity calculation model is used to match the item object with the recommended objects in the recommended object library, thereby screening out at least one recommended object having the same or similar features as the item object.

[0094] For example, assuming the identified objects are a sweatshirt and a short dress, respectively, a video frame containing a white sweatshirt and a short dress can be sent to the image similarity calculation model. Based on the image similarity calculation model, the sweatshirt and the short dress are matched with items in the item library. In some embodiments, items with the same or similar features as the sweatshirt include a white long-sleeved sweatshirt and a white short-sleeved sweatshirt, and items with the same or similar features as the short dress include a white short dress and a purple short dress.

[0095] S603: Display the at least one recommended object.

[0096] In an optional embodiment, during the display of at least one recommended object, upon receiving a preset interactive operation for a target recommended object on the recommended object display page, the recommended object display page can be redirected to a detail display page for the target recommended object, and the user can obtain more information about the target recommended object based on the detail display page. In some embodiments, the preset interactive operation for the target recommended object may include a click operation on the target recommended object, etc.

[0097] In the multimedia content processing method provided by the embodiment of the present disclosure, during the process of multimedia content display, based on the target resource object carried in the multimedia content, recommended objects related to the target resource object are displayed to the user. It can be seen that the embodiment of the present disclosure provides the user with an extended consumption path for the content carried in the multimedia content, thereby improving the user experience.

[0098] In actual applications, the target resource object identified from the first multimedia content may include background music. In some embodiments, the recommended object type corresponding to the background music may include music type, etc.

[0099] Based on the above embodiment, the present disclosure provides a specific method for determining a recommendation object based on background music, which includes:

[0100] First, in response to a preset triggering operation on a display page of first multimedia content, background music included in the first multimedia content is identified. Then, music recognition is performed on the background music to obtain a music recognition result. Next, based on the music recognition result, a song resource corresponding to the background music is determined. Then, the at least one song resource is displayed.

[0101] In an optional implementation, the background music may be identified by calling a music retrieval algorithm based on audio fingerprints to obtain a music identification result, and then, based on the music identification result, the song resource corresponding to the background music may be determined.

[0102] For example, assuming that the background music is the chorus of a song, by calling the music retrieval algorithm based on audio fingerprints, the song information of the chorus can be identified, such as the song name is "Music A". Then, the song with the song name "Music A" is searched from the music library as the song resource corresponding to the background music.

[0103] In actual applications, after determining the song resource corresponding to the background music, the song resource can also be displayed on the recommended object display page. As shown in Figure 7, a schematic diagram of another recommended object display page provided in an embodiment of the present disclosure is shown. In some embodiments, the recommended object display page displays the music title, music cover, author information, etc. corresponding to the song resource.

[0104] In addition, the recommended object display page can be provided with a music playback control 701. When a trigger operation for the music playback control is received, the song resource is played based on the recommended object display page.

[0105] In an optional implementation, a preset return control 702 may be further provided on the recommended object display page, and when a trigger operation for the preset return control is received, the function of exiting the recommended object display page can be realized.

[0106] In some embodiments, when at least one target resource object includes background music, music recognition is first performed on the background music to obtain a music recognition result. Then, based on the music recognition result, the song resource corresponding to the background music is determined and displayed. This demonstrates that the disclosed embodiments provide users with an extended consumption path for the content carried in the video, thereby enhancing the user experience.

[0107] In actual applications, the target resource object identified from the first multimedia content may include address information. In some embodiments, the recommended object type corresponding to the address information may include, for example, a life service type.

[0108] Based on the above embodiments, the embodiments of the present disclosure provide a method for determining recommended objects based on address information. Specifically, first, in response to a preset trigger operation acting on a display page of the first multimedia content, the address information carried in the first multimedia content is identified, and then, based on the address information, at least one life service object belonging to the life service type within a preset distance range is determined, and the at least one life service object is displayed.

[0109] In some embodiments, upon receiving a preset triggering operation on a display page of first multimedia content, a speech recognition algorithm may be invoked to recognize an audio file in the first multimedia content to obtain the address information carried in the first multimedia content. In some embodiments, the speech recognition algorithm may include an algorithm based on dynamic time warping, a speech recognition algorithm based on a deep learning neural network, and the like.

[0110] In an optional implementation, the subtitle content of the first multimedia content may be recognized by calling a text recognition algorithm to obtain the address information carried in the first multimedia content.

[0111] For example, assuming that the address information carried in the first multimedia content is "Location ABC", then with "Location ABC" as the center, shopping malls, supermarkets, clothing stores, tourist attractions, etc. within 1 kilometer of "Location ABC" are searched.

[0112] In an optional implementation, if the first multimedia content carries a specific address anchor point, the location information corresponding to the address anchor point can be directly determined as the life service object corresponding to the address anchor point.

[0113] In some embodiments, when address information is included in at least one target resource object, first, in response to a preset trigger operation on the playback page of the first multimedia content, the address information carried in the first multimedia content is identified, and then, based on the address information, at least one life service object belonging to the life service type within a preset distance range is determined and displayed. It can be seen that the embodiment of the present disclosure provides users with an extended consumption path for the content carried in the multimedia content, thereby improving the user experience.

[0114] In actual applications, the target resource object identified from the first multimedia content may include a target face displayed on the video frame. In some embodiments, the recommended object type corresponding to the target face displayed on the video frame may include a user account type.

[0115] Based on the above embodiments, the present disclosure provides a method for determining a recommended object based on a target face displayed on a video frame. Specifically, when the user to whom the target face belongs authorizes the use of the target face information, first, in response to a preset trigger operation on the playback page of the first multimedia content, the target face carried in the first multimedia content is identified. Then, based on the target face displayed on the video frame, at least one user account is determined whose user avatar has a similarity with the target face that reaches a preset threshold. In some embodiments, the user account belongs to a user account type.

[0116] In an optional embodiment, when a preset trigger operation is received on the playback page of the first multimedia content, first, the video frame in the first multimedia content is captured, and then the face recognition algorithm is called to identify each video frame in the first multimedia content to identify the video frame in the first multimedia content that carries the target face.

[0117] In an optional embodiment, after identifying a video frame carrying a target face in the first multimedia content, if the user to whom the target face belongs authorizes the use of the target face information, the video frame carrying the target face will be sent to a face matching server, and then based on the face matching server, at least one user account is determined whose similarity between the user portrait and the target face reaches a preset threshold.

[0118] In some embodiments, the preset threshold may be determined based on actual needs, for example, it may be set to 80%, 85%, 90%, 95%, etc.

[0119] For example, assuming a face recognition algorithm is invoked and the target faces identified are Face A and Face B, the video frames carrying Face A and Face B can be sent to the face matching server. The face matching server then searches the user profile database for user profiles whose similarity to Face A and Face B meets a preset threshold. In some embodiments, the user account corresponding to the user profile whose similarity to Face A meets the preset threshold is "Xiao A," and the user account corresponding to the user profile whose similarity to Face B meets the preset threshold is "Xiao B."

[0120] In an optional implementation, assuming that the first multimedia content is the first video, the subtitle content of the first video can be identified by calling a text recognition algorithm to obtain the name of the character carried in the first video. Based on the name of the character, a user nickname whose similarity with the name of the character reaches a preset threshold is searched, and the user account corresponding to the searched user nickname is determined as the recommended object corresponding to the target resource object. For example, when performing text recognition on the subtitle content of the first video, the character name is obtained as "Flower". Then, a user nickname whose similarity with "Flower" reaches a preset threshold is searched, such as "Flower Teacher", "Flower Store Manager", etc.

[0121] In some embodiments, after determining at least one user account whose user avatar has a similarity with the target face that reaches a preset threshold, the user account can also be displayed on the recommended object display page if the user to whom the user account belongs authorizes the display of the user account, as shown in Figure 8, which is another recommended object display page provided in an embodiment of the present disclosure.

[0122] In an optional embodiment, at least one user account displayed on the recommended object display page is respectively provided with a preset interactive control, and in response to a triggering operation of the preset interactive control corresponding to the first user account among the at least one user account, a preset interactive relationship is established between the current user account and the first user account.

[0123] In some embodiments, the triggering operation for a preset interactive control corresponding to a first user account in the at least one user account may include a click operation, a long press operation, etc. on the preset interactive control, which is not limited in the embodiments of the present disclosure. In some embodiments, the first user account may be any user account in the at least one user account.

[0124] In some embodiments, the preset interactive relationship between the current user account and the first user account may include determining the first user account as a follow-up object of the current user account.

[0125] As shown in Figure 8, when a trigger operation is received for the preset interactive control 802 corresponding to the first user account 801, the first user account 801 is determined as the focus object of the current user account, thereby realizing the function of establishing a preset interactive relationship between the current user account and the first user account.

[0126] In some embodiments, when at least one target resource object includes a target face, first, in response to a preset triggering operation on the playback page of the first multimedia content, the target face contained in the first multimedia content is identified. Then, based on the target face, at least one user account whose user profile picture has a similarity to the target face that meets a preset threshold is identified and displayed. This demonstrates that the disclosed embodiments provide users with an extended consumption path for the content contained in the multimedia content, thereby enhancing the user experience.

[0127] Based on the above method embodiment, the present disclosure further provides a multimedia content processing device. Referring to FIG9 , which is a schematic structural diagram of a multimedia content processing device provided by an embodiment of the present disclosure, the device includes:

[0128] an identification module 901 configured to identify, in response to a preset triggering operation on a playback page of a first multimedia content, at least one target resource object carried in the first multimedia content; wherein the target resource object has a preset correspondence with a recommended object type;

[0129] A first determining module 902 is configured to determine a recommended object type corresponding to a first target resource object among the at least one target resource object based on the preset corresponding relationship;

[0130] A second determining module 903 is configured to determine at least one recommended object based on the first target resource object; wherein the at least one recommended object belongs to a recommended object type corresponding to the first target resource object;

[0131] The display module 904 is configured to display the at least one recommended object.

[0132] In an optional embodiment, the display module includes:

[0133] The classification display submodule is used to classify and display the recommended objects determined based on the first target resource object and the second resource object according to the recommended object type.

[0134] In an optional embodiment, the classification display submodule includes:

[0135] A first determination submodule is configured to display, on a first card, at least one first recommended object determined based on the first target resource object; wherein the first recommended object belongs to a recommended object type corresponding to the first target resource object;

[0136] The second determination submodule is used to display at least one second recommended object determined based on the second resource object on the second card; wherein, the second recommended object belongs to the recommended object type corresponding to the second target resource object, and the first card and the second card belong to a card set displayed in a stacked style.

[0137] In an optional embodiment, the classification display submodule further includes:

[0138] The scrolling display submodule is used to scroll and display each card in the card set in response to a preset sliding operation triggered on the card set.

[0139] In an optional embodiment, the classification display submodule further includes:

[0140] a recommended object display submodule, configured to display the recommended objects on the target card on a recommended object display page in response to a preset trigger operation on the target card in the card set;

[0141] The receiving submodule is used to receive a preset interactive operation for a target recommended object on the recommended object display page.

[0142] In an optional implementation, the at least one target resource object includes an item object, the recommended object type corresponding to the item object includes an item type, and the second determining module includes:

[0143] The third determining submodule is configured to determine at least one recommended item having the same or similar features as the item object; wherein the recommended item belongs to the item type.

[0144] In an optional implementation, the at least one target resource object includes background music, the recommended object type corresponding to the background music includes a music type, and the second determining module includes:

[0145] A music recognition submodule is used to perform music recognition on the background music and obtain a music recognition result;

[0146] The fourth determination submodule is used to determine the song resource corresponding to the background music based on the music recognition result; wherein the song resource belongs to the music type.

[0147] In an optional implementation, the at least one target resource object includes address information, and the recommended object type corresponding to the address information includes a life service type. The second determination module includes:

[0148] The fifth determination submodule is configured to determine at least one life service object within a preset distance range based on the address information; wherein the life service object belongs to the life service type.

[0149] In an optional implementation, the at least one target resource object includes a target face displayed on a video frame, and the recommended object type corresponding to the target face may include a user account type. The second determination module includes:

[0150] The sixth determination submodule is used to determine at least one user account whose user profile picture has a similarity with the target face that reaches a preset threshold based on the target face displayed on the video frame; wherein the user account belongs to the user account type.

[0151] In an optional implementation, the identification module includes:

[0152] a screen display submodule for displaying a plurality of key video frames of the first multimedia content on the video identification page in the form of transition effects in response to a preset triggering operation on the display page of the first multimedia content;

[0153] The target resource object identification submodule is configured to identify at least one target resource object carried in the first multimedia content based on the multiple key video frames.

[0154] In the multimedia content processing device provided by the embodiment of the present disclosure, first, in response to a preset trigger operation acting on the display page of the first multimedia content, at least one target resource object carried in the first multimedia content is identified, wherein the target resource object and the recommended object type have a preset object relationship; then, based on the preset corresponding relationship, the recommended object type corresponding to the first target resource object in the at least one target resource object is determined, and at least one recommended object is determined based on the first target resource object; and then, the at least one recommended object is displayed. The embodiment of the present disclosure can display recommended objects related to the target resource object to the user based on the target resource object carried in the multimedia content during the multimedia content display process. It can be seen that the embodiment of the present disclosure provides the user with an extended consumption path for the content carried in the multimedia content, thereby improving the user experience.

[0155] In addition to the above-mentioned method and apparatus, the embodiments of the present disclosure further provide a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal device, the terminal device implements the multimedia content processing method described in the embodiments of the present disclosure.

[0156] The embodiments of the present disclosure further provide a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the multimedia content processing method described in the embodiments of the present disclosure is implemented.

[0157] In addition, the embodiment of the present disclosure further provides a multimedia content processing device, as shown in FIG10 , which may include:

[0158] Processor 1001, memory 1002, input device 1003, and output device 1004. The number of processors 1001 in the multimedia content processing device can be one or more, and Figure 10 uses one processor as an example. In some embodiments of the present disclosure, the processor 1001, memory 1002, input device 1003, and output device 1004 can be connected via a bus or other means, wherein Figure 10 uses the bus connection as an example.

[0159] The memory 1002 can be used to store software programs and modules. The processor 1001 executes the various functional applications and data processing of the multimedia content processing device by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application required for a function, etc. In addition, the memory 1002 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The input device 1003 may be used to receive input digital or character information and generate signal input related to user settings and function control of the multimedia content processing device.

[0160] Specifically in this embodiment, the processor 1001 will load the executable files corresponding to the processes of one or more applications into the memory 1002 according to the following instructions, and the processor 1001 will run the applications stored in the memory 1002, thereby realizing the various functions of the above-mentioned multimedia content processing device.

[0161] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0162] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. A multimedia content processing method, characterized in that: The method comprises: In response to a preset trigger operation on a display page of a first multimedia content, identifying at least one target resource object carried in the first multimedia content; wherein there is a preset corresponding relationship between the target resource object and the recommended object type; Based on the preset corresponding relationship, determining a recommended object type corresponding to a first target resource object among the at least one target resource object; Determine at least one recommended object based on the first target resource object; wherein the at least one recommended object belongs to the recommended object type corresponding to the first target resource object; The at least one recommended object is displayed.

2. The method according to claim 1, characterized in that The at least one target resource object also includes a second target resource object, and the displaying of the at least one recommended object includes: According to the recommended object type, the recommended objects determined based on the first target resource object and the second resource object are classified and displayed.

3. The method according to claim 2, characterized in that The classifying and displaying the recommended objects determined based on the first target resource object and the second resource object respectively according to the recommended object type includes: Displaying at least one first recommended object determined based on the first target resource object on a first card; wherein the first recommended object belongs to a recommended object type corresponding to the first target resource object; And, at least one second recommended object determined based on the second resource object is displayed on the second card; wherein, the second recommended object belongs to the recommended object type corresponding to the second target resource object, and the first card and the second card belong to a card set displayed in a stacked style.

4. The method according to claim 3, characterized in that The method further comprises: In response to a preset sliding operation triggered on the card set, each card in the card set is scrolled and displayed.

5. The method according to claim 3 or 4, characterized in that: The method further comprises: In response to a preset trigger operation on a target card in the card set, displaying a recommended object on the target card on a recommended object display page; Receive a preset interactive operation for a target recommended object on the recommended object display page.

6. The method according to claim 1, characterized in that The at least one target resource object includes an item object, the recommended object type corresponding to the item object includes an item type, and determining at least one recommended object based on the first target resource object includes: Determine at least one recommended item having the same or similar characteristics as the item object; wherein the recommended item belongs to the item type.

7. The method according to claim 1, characterized in that The at least one target resource object includes background music, the recommended object type corresponding to the background music includes a music type, and determining at least one recommended object based on the first target resource object includes: Performing music recognition on the background music to obtain a music recognition result; Based on the music recognition result, a song resource corresponding to the background music is determined; wherein the song resource belongs to the music type.

8. The method according to claim 1, characterized in that: The at least one target resource object includes address information, the recommended object type corresponding to the address information includes a life service type, and determining at least one recommended object based on the first target resource object includes: Taking the address information as the center, at least one life service object within a preset distance range is determined; wherein the life service object belongs to the life service type.

9. The method according to claim 1, characterized in that: The at least one target resource object includes a target face displayed on a video frame, the recommended object type corresponding to the target face includes a user account type, and determining at least one recommended object based on the first target resource object includes: Based on the target face displayed on the video frame, at least one user account whose user avatar has a similarity with the target face that reaches a preset threshold is determined; wherein the user account belongs to the user account type.

10. The method according to claim 1, characterized in that The step of identifying at least one target resource object carried in the first multimedia content in response to a preset trigger operation on a display page of the first multimedia content includes: In response to a preset trigger operation on a display page of the first multimedia content, a plurality of key video frames in the first multimedia content are displayed on a video identification page in the form of a transition animation; Based on the multiple key video frames, at least one target resource object carried in the first multimedia content is identified.

11. A multimedia content processing device, characterized in that: The device comprises: an identification module, configured to identify at least one target resource object carried in the first multimedia content in response to a preset trigger operation on a playback page of the first multimedia content; wherein a preset corresponding relationship exists between the target resource object and the recommended object type; A first determining module, configured to determine, based on the preset corresponding relationship, a recommended object type corresponding to a first target resource object among the at least one target resource object; A second determination module, configured to determine at least one recommended object based on the first target resource object; wherein the at least one recommended object belongs to a recommended object type corresponding to the first target resource object; A display module is used to display the at least one recommended object.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device implements the method according to any one of claims 1 to 10.

13. A multimedia content processing device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 10 is implemented.

Images