A display method and related equipment

Abstract

The present application discloses a display method and related equipment, the method comprising: after obtaining the text to be displayed, firstly performing text element analysis on the text to be displayed to obtain at least one text element of the text to be displayed; then searching for a multimedia object to be used that matches the at least one text element from at least one candidate multimedia object; finally, determining the text display object of the text to be displayed according to the multimedia object to be used, so that the text display object can be displayed by a display device. It can be seen that, because the multimedia object to be used expresses information through multiple media forms, the text display object determined based on the multimedia object to be used can express the text to be displayed through multiple media forms, so that when the display device displays the text display object, the display device can display the text to be displayed in multiple media forms, which is conducive to improving the text display effect of the text to be displayed, thereby improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a display method and related equipment. Background Art

[0002] Currently, for some display devices (eg, learning assistance devices, etc.), after acquiring some text, these display devices usually display these texts directly to the user in text form, so that the user can view these texts from the display device.

[0003] However, due to defects in the above text display method, the text display effect is poor, which can easily lead to a poor user experience. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide a display method and related equipment, which can improve the text display effect, thereby helping to improve the user experience.

[0005] The present application provides a display method, which includes:

[0006] After obtaining the text to be displayed, performing text element analysis on the text to be displayed to obtain at least one text element of the text to be displayed;

[0007] Searching for a multimedia object to be used that matches the at least one text element from at least one candidate multimedia object;

[0008] According to the multimedia object to be used, a text display object of the text to be displayed is determined, so that the text display object is displayed by a display device.

[0009] In a possible implementation manner, the process of determining the multimedia object to be used includes:

[0010] Determining the coverage of each candidate multimedia object to the at least one text element;

[0011] Searching for a target multimedia object that meets a first condition from the at least one candidate multimedia object according to the coverage of the at least one text element by each candidate multimedia object;

[0012] The multimedia object to be used is determined according to the target multimedia object.

[0013] In a possible implementation manner, determining the multimedia object to be used according to the target multimedia object includes:

[0014] If it is determined that the similarity between the target multimedia object and the at least one text element is higher than a preset threshold, the target multimedia object is determined as the multimedia object to be used.

[0015] In a possible implementation, the method further includes:

[0016] If it is determined that the similarity between the target multimedia object and the at least one text element is not higher than a preset threshold, text display information of the text to be displayed is determined according to the text to be displayed and preset prompt information, so that the display device displays the text display information.

[0017] In a possible implementation, the method further includes:

[0018] If it is determined that the similarity between the target multimedia object and the at least one text element is not higher than a preset threshold, after obtaining a matching multimedia object of the text to be displayed, the at least one candidate multimedia object is updated using the matching multimedia object.

[0019] In a possible implementation, the at least one candidate multimedia object includes a to-be-processed multimedia object, and the process of determining the coverage of the to-be-processed multimedia object on the at least one text element includes:

[0020] Matching the object description information of the to-be-processed multimedia object with the at least one text element to obtain a matching result of the to-be-processed multimedia object;

[0021] According to the matching result of the multimedia object to be processed, the coverage of the at least one text element by the multimedia object to be processed is determined.

[0022] In a possible implementation manner, the object description information includes an object description text and / or at least one object description word.

[0023] In a possible implementation manner, the multimedia object to be used includes a picture to be used, and the process of determining the text display object includes:

[0024] The image to be used and the text to be displayed are synthesized to obtain the text display object.

[0025] In a possible implementation manner, the multimedia object to be used includes a video to be used;

[0026] The process of determining the text display object includes:

[0027] Determining the video to be used as the text display object;

[0028] or,

[0029] The process of determining the text display object includes:

[0030] The video to be used is subjected to a preset division process to obtain at least one candidate video segment; based on the similarity between each candidate video segment and the at least one text element, a target video segment satisfying a second condition is searched from the at least one candidate video segment; and the text display object is determined based on the target video segment.

[0031] In a possible implementation, the at least one candidate video segment includes a to-be-processed video segment, and the process of determining the similarity between the to-be-processed video segment and the at least one text element includes:

[0032] Extracting picture information to be processed, text information to be processed, and audio information to be processed from the video segment to be processed;

[0033] The similarity between the image information to be processed and the at least one text element, the similarity between the text information to be processed and the at least one text element, and the similarity between the audio information to be processed and the at least one text element are weightedly summed to obtain the similarity between the video segment to be processed and the at least one text element.

[0034] In a possible implementation manner, determining the text display object according to the target video segment includes:

[0035] The text content in the target video segment is replaced with the text to be displayed to obtain the text display object.

[0036] In a possible implementation manner, the multimedia objects to be used include pictures to be used and videos to be used, and the process of determining the text display objects includes:

[0037] Determine, according to the picture to be used, a text display picture of the text to be displayed;

[0038] Determine, according to the video to be used, a text display video of the text to be displayed;

[0039] The text to be displayed, the text display picture of the text to be displayed, and the text display video of the text to be displayed are combined according to a preset combination method to obtain the text display object.

[0040] In a possible implementation, the method further includes:

[0041] Generating audio playback data of the text to be displayed;

[0042] The step of determining the text display object of the text to be displayed according to the multimedia object to be used includes:

[0043] A text display object of the text to be displayed is determined according to the multimedia object to be used and the audio playback data.

[0044] In a possible implementation manner, the multimedia object to be used includes a video to be used, and the process of determining the text display object includes:

[0045] Determining a video object to be processed according to the video to be used;

[0046] Using the audio playback data to replace the audio data of the video object to be processed, to obtain a replaced video object;

[0047] The text display object is determined according to the replaced video object.

[0048] The present application also provides a display device, including:

[0049] An analyzing unit, configured to, after acquiring the text to be displayed, perform text element analysis on the text to be displayed to obtain at least one text element of the text to be displayed;

[0050] A searching unit, configured to search for a multimedia object to be used that matches the at least one text element from at least one candidate multimedia object;

[0051] The determining unit is used to determine the text display object of the text to be displayed according to the multimedia object to be used, so that the text display object can be displayed by the display device.

[0052] The embodiment of the present application further provides a device, the device comprising: a processor, a memory, and a system bus;

[0053] The processor and the memory are connected via the system bus;

[0054] The memory is used to store one or more programs, and the one or more programs include instructions, which, when executed by the processor, enable the processor to execute any implementation of the display method provided in the embodiments of the present application.

[0055] The embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any implementation of the display method provided in the embodiment of the present application.

[0056] The embodiment of the present application also provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes any implementation of the display method provided in the embodiment of the present application.

[0057] Based on the above technical solution, this application has the following beneficial effects:

[0058] In the technical solution provided by the present application, after obtaining the text to be displayed, the text to be displayed is first analyzed for text elements to obtain at least one text element of the text to be displayed; then, a multimedia object to be used that matches the at least one text element is searched from at least one candidate multimedia object; finally, based on the multimedia object to be used, a text display object of the text to be displayed is determined so that the text display object can be displayed by a display device.

[0059] It can be seen that since the multimedia object to be used expresses information through various media forms (for example, text, image, video, etc.), the text display object determined based on the multimedia object to be used can express the text to be displayed through various media forms, so that when the display device displays the text display object, the display device can display the text to be displayed in various media forms, which is beneficial to improving the text display effect of the text to be displayed, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 A schematic diagram of an application scenario of a display method applied to a display device provided in an embodiment of the present application;

[0062] Figure 2 A schematic diagram of an application scenario of a display method applied to a server provided in an embodiment of the present application;

[0063] Figure 3 A flowchart of a display method provided in an embodiment of the present application;

[0064] Figure 4 A schematic diagram of the structure of a display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The inventor discovered during research on display devices that the text display method shown in the background technology section (that is, a text display method that displays text only in text form) has the following defects: because the text display method only displays text content in text form, the text display method is relatively simple, and the text content displayed by the text display method lacks interest and liveliness, which can easily lead to users being unable to better understand the displayed text content, resulting in a poor user experience.

[0066] Based on the above findings, in order to solve the technical problems in the background technology part, the embodiment of the present application provides a display method, which includes: after obtaining the text to be displayed, firstly analyze the text to be displayed to obtain at least one text element of the text to be displayed; then search for a multimedia object to be used that matches the at least one text element from at least one candidate multimedia object; finally, determine the text display object of the text to be displayed according to the multimedia object to be used, so that the display device can display the text display object. Among them, because the multimedia object to be used expresses information through a variety of media forms (for example, text, image, video, etc.), the text display object determined based on the multimedia object to be used can express the text to be displayed through a variety of media forms, so that when the display device displays the text display object, the display device can display the text to be displayed in a variety of media forms, which is conducive to improving the text display effect of the text to be displayed, thereby improving the user experience.

[0067] In addition, the embodiments of the present application do not limit the execution subject of the display method. For example, the display method provided in the embodiments of the present application can be applied to a display device or a server. The display device refers to a terminal device with an information display function; and the embodiments of the present application do not limit the display device. For example, the display device can be a smart phone, a computer, a personal digital assistant (PDA), a tablet computer, a scanning pen with a display screen, or a learning auxiliary device with a display screen. The server can be an independent server, a cluster server, or a cloud server.

[0068] In order to facilitate understanding of the technical solutions provided in the embodiments of the present application, Figure 1 and Figure 2 The application scenarios of the display method provided in the embodiment of the present application are introduced exemplarily. Figure 1 A schematic diagram of an application scenario of a display method applied to a display device provided in an embodiment of the present application; Figure 2 A schematic diagram of an application scenario of a display method applied to a server provided in an embodiment of the present application.

[0069] exist Figure 1In the application scenario shown, after the user 101 triggers a text display request on the display device 102, the terminal device 102 receives the text display request and displays the text to the user 101 by executing the display method provided in the embodiment of the present application. For example, the process of the display device 102 displaying text to the user 101 may specifically be: after the display device 102 obtains the text to be displayed, the display device 102 first performs a text element analysis on the text to be displayed to obtain at least one text element of the text to be displayed; the display device 102 then searches for a multimedia object to be used that matches the at least one text element from at least one candidate multimedia object; finally, the display device 102 determines the text display object of the text to be displayed based on the multimedia object to be used, so that the display device 102 displays the text display object, so that the user 101 can view the text display object on the display device 102, so that the user 101 can better understand the text to be displayed with the help of the text display object.

[0070] exist Figure 2 In the application scenario shown, when the user 201 triggers a text display request on the display device 202, the display device 202 receives the text display request and forwards the text display request to the server 203, so that the server 203 performs text display to the user 201 by executing the display method provided in the embodiment of the present application. For example, the process of the server 203 performing text display to the user 201 may be as follows: after the server 203 obtains the text to be displayed, the server 203 first performs text element analysis on the text to be displayed to obtain at least one text element of the text to be displayed; the server 203 then searches for a multimedia object to be used that matches the at least one text element from at least one candidate multimedia object; finally, the server 203 determines the text display object of the text to be displayed according to the multimedia object to be used, and the server 203 sends the text display object to the display device 202, so that the display device 202 displays the text display object, so that the user 201 can view the text display object on the display device 202, so that the user 201 can better understand the text to be displayed with the help of the text display object.

[0071] It should be noted that the display method provided in the embodiment of the present application can be applied not only to Figure 1 or Figure 2 The application scenario shown can also be applied to other application scenarios that require text display, and the embodiments of the present application do not specifically limit this.

[0072] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0073] Method Example 1

[0074] See also Figure 3 , which is a flow chart of a display method provided in an embodiment of the present application.

[0075] The display method provided in the embodiment of the present application includes S1-S3:

[0076] S1: After obtaining the text to be displayed, performing text element analysis on the text to be displayed to obtain at least one text element of the text to be displayed.

[0077] Among them, "text to be displayed" refers to text content that needs to be displayed by the display device; and the embodiments of the present application do not limit the acquisition process of the "text to be displayed", for example, the "text to be displayed" can be input by the user with the help of the input device of the display device (for example, a keyboard, a scanning device, etc.). For another example, the acquisition process of the "text to be displayed" may also include: when the user triggers a text selection instruction on the display device, the display device can obtain the text to be displayed according to the text selection instruction (for example, if the text selection instruction carries the text to be displayed, the display device can directly extract the text to be displayed from the text selection instruction. For another example, if the text selection instruction carries the storage location description information of the text to be displayed, the display device can read the text to be displayed from the storage space according to the storage location description information).

[0078] "Text element analysis" is used to analyze and process the semantic components of text content; and the embodiments of the present application are not limited to "text element analysis", for example, it can be implemented according to pre-set text element analysis rules. Among them, "semantic components" refer to the semantic influencing factors that should be referred to when constructing the semantic information of a text content.

[0079] "At least one text element of the text to be displayed" refers to the semantic influencing factors that should be referred to when constructing the semantic information of the text to be displayed, so that the "at least one text element of the text to be displayed" can fully express the semantic information of the text to be displayed.

[0080] In addition, the embodiments of the present application do not limit the "text elements". For example, when the "text to be displayed" belongs to a story type, the "text elements" may include at least one of the time elements, character elements, location elements, event elements and environmental elements.

[0081] "Time element" refers to the time description information carried by the "text to be displayed"; and the embodiments of the present application do not limit the "time element". For example, the "time element" can be accurate time description information (for example, 13:20), or it can be vague time description information (for example, noon, or dusk, etc.), or it can be environmental time description information (for example, moonlight, scorching sun).

[0082] "Character elements" refer to the character description information carried by the "text to be displayed"; and the embodiments of the present application do not limit the "character elements". For example, "character elements" can be a character's name (for example, Xiao Ming) or a character relationship (for example, father, good friend, etc.).

[0083] "Place elements" refer to the place description information carried by the "text to be displayed"; and the embodiments of the present application do not limit the "place elements". For example, "place elements" can be accurate geographical location description information (for example, XXX province, XXX city, etc.) or landmark location description information (for example, XXX mountain, XXX tower, etc.).

[0084] "Event elements" refer to the event description information carried by the "text to be displayed"; and the embodiments of the present application do not limit the "event elements". For example, "event elements" can be the introduction information of the complete event development process, or they can be the pseudonymous event description information (for example, XXX event).

[0085] "Environmental elements" refer to the environmental description information carried by the "text to be displayed"; and the embodiments of the present application do not limit the "environmental elements". For example, "environmental elements" can be natural environment description information, living environment description information, historical background environment description information, biological environment description information (for example, animal description information, plant description information, etc.).

[0086] In addition, the embodiment of the present application does not limit the representation method of "at least one text element of the text to be displayed". For example, each text element of the text to be displayed can be represented by at least one word in the text to be displayed. For another example, each text element of the text to be displayed can be represented by the preset text data corresponding to each text element.

[0087] The embodiment of the present application does not limit the determination process of "at least one text element of the text to be displayed". For example, it may specifically include: performing text element analysis on the text to be displayed according to a pre-set text element analysis rule to obtain at least one text element of the text to be displayed. The "text element analysis rule" is used to specify how to perform text element analysis on a text data.

[0088] In fact, the semantic influencing factors of different types of text content may be different. Therefore, in order to improve the accuracy of text element analysis, different text element analysis rules can be used for different types of text content to perform text element analysis. Based on this, the embodiment of the present application also provides another possible implementation method for determining "at least one text element of the text to be displayed", which may specifically include: first identifying the text content type to which the text to be displayed belongs from at least one candidate text content type; then performing text element analysis on the text to be displayed according to the text element analysis rules corresponding to the text content type to which the text to be displayed belongs, and obtaining at least one text element of the text to be displayed.

[0089] Here, "at least one candidate text content type" refers to a plurality of pre-set text content types (for example, story type, word introduction type, knowledge explanation type, etc.).

[0090] The embodiment of the present application does not limit the identification process of "the text content type to which the text to be displayed belongs". For example, the text content type identification process can be performed on the text to be displayed using the pre-set identification rules of each candidate text content type to obtain the text content type to which the text to be displayed belongs. Among them, the "identification rules of the candidate text content type" can be used to describe the information features of the text content belonging to the candidate content type.

[0091] Based on the relevant content of S1 above, it can be known that after obtaining the text to be displayed, a text element analysis can be performed on the text to be displayed to obtain at least one text element of the text to be displayed, so that these text elements can represent the semantic information of the text to be displayed, so that subsequently, based on these text elements, multimedia objects that can express the text to be displayed (especially, multimedia objects that can express the semantic information of the text to be displayed) can be automatically retrieved from a large number of multimedia objects. This is conducive to achieving the purpose of diversified display of the text to be displayed.

[0092] S2: searching for a multimedia object to be used that matches at least one text element from at least one candidate multimedia object.

[0093] Among them, "at least one candidate multimedia object" may include at least one picture and / or at least one video; and the embodiment of the present application does not limit the method of obtaining "at least one candidate multimedia object". For example, the "at least one candidate multimedia object" can be read from a pre-constructed multimedia database (such as a picture database and / or a video database, etc.), or can be read from a preset storage space that stores a large amount of multimedia data (such as pictures and / or videos), or can be obtained from Internet resources with the help of preset means (for example, a crawler processing method).

[0094] The “multimedia object to be used” refers to a multimedia object that matches the above “at least one text element”, so that the “multimedia object to be used” can express the above “at least one text element” in a multimedia form.

[0095] In addition, the embodiments of the present application do not limit the "multimedia object to be used", for example, it may specifically include a picture to be used and / or a video to be used. Among them, the "picture to be used" refers to the picture data that matches the above-mentioned "at least one text element", so that the "picture to be used" can express the above-mentioned "at least one text element" in the form of a picture. The "video to be used" refers to the video data that matches the above-mentioned "at least one text element", so that the "video to be used" can express the above-mentioned "at least one text element" in the form of a video.

[0096] In addition, the embodiment of the present application does not limit the acquisition process of the "multimedia object to be used". For ease of understanding, the following is an explanation in combination with three situations.

[0097] Case 1: If the above-mentioned "at least one candidate multimedia object" includes at least one picture, the acquisition process of the "multimedia object to be used" may include: searching for a picture to be used that matches at least one text element from the above-mentioned "at least one picture", and determining the multimedia object to be used based on the picture to be used (for example, the picture to be used can be directly determined as the multimedia object to be used).

[0098] Case 2: If the above-mentioned "at least one candidate multimedia object" includes at least one video, the acquisition process of the "multimedia object to be used" may include: searching for a video to be used that matches at least one text element from the above-mentioned "at least one video", and determining the multimedia object to be used based on the video to be used (for example, the video to be used can be determined as the multimedia object to be used).

[0099] Case 3: If the above-mentioned "at least one candidate multimedia object" includes at least one picture and at least one video, the process of obtaining the "multimedia object to be used" may include: first searching for a picture to be used that matches at least one text element from the above-mentioned "at least one picture", and searching for a video to be used that matches at least one text element from the above-mentioned "at least one video"; then determining the multimedia object to be used based on the picture to be used and the video to be used (for example, the picture to be used and the video to be used may be combined to obtain the multimedia object to be used).

[0100] The embodiment of the present application does not limit the implementation of S2. For example, in a possible implementation, S2 may specifically include S21-S23:

[0101] S21: Determine the coverage of each candidate multimedia object to at least one text element.

[0102] The coverage of the nth candidate multimedia object to at least one text element is used to indicate the content coverage degree of the information carried by the nth candidate multimedia object for the "at least one text element". n is a positive integer, n≤N, N is a positive integer, and N represents the number of candidate multimedia objects.

[0103] In addition, the embodiment of the present application does not limit the “nth candidate multimedia object”. For example, the nth candidate multimedia object can be a picture or a video.

[0104] In addition, the embodiment of the present application does not limit the determination process of "the coverage of at least one text element by the nth candidate multimedia object", for example, it may specifically include step 11-step 12:

[0105] Step 11: Match the object description information of the nth candidate multimedia object with at least one text element to obtain a matching result of the nth candidate multimedia object.

[0106] Among them, "object description information of the nth candidate multimedia object" is used to represent the information carried by the nth candidate multimedia object; and the embodiment of the present application does not limit the "object description information of the nth candidate multimedia object", for example, it may specifically include the object description text of the nth candidate multimedia object and / or at least one object description word of the nth candidate multimedia object.

[0107] "The object description text of the nth candidate multimedia object" is used to represent the information carried by the nth candidate multimedia object in text form; and the embodiment of the present application does not limit the representation method of the "object description text of the nth candidate multimedia object", for example, it can be represented by a representation method similar to the text data "the nth candidate multimedia object includes the sea, and there is a chair on the beach by the sea, and a person is lying on the chair".

[0108] In addition, the embodiment of the present application does not limit the acquisition process of the "object description text of the nth candidate multimedia object", for example, it can be implemented by manual annotation. For another example, it can also be implemented by any existing or future processing process that can convert object description text for multimedia data (for example, pictures or videos).

[0109] "At least one object description vocabulary of the nth candidate multimedia object" is used to represent the information carried by the nth candidate multimedia object with the help of multiple words (for example, sea, chair, person, etc.); and the embodiment of the present application does not limit "at least one object description vocabulary of the nth candidate multimedia object", for example, it may refer to the label information of the nth candidate multimedia object.

[0110] The "label information of the nth candidate multimedia object" is used to indicate what exists in the nth candidate multimedia; and the embodiments of the present application do not limit the determination process of the "label information of the nth candidate multimedia object", for example, it can be implemented by manually marking label information, or it can be implemented using any existing or future processing process that can perform target recognition processing on multimedia data (for example, pictures or videos).

[0111] The above-mentioned "matching result of the nth candidate multimedia object" refers to the matching result between the above-mentioned "object description information of the nth candidate multimedia object" and the above-mentioned "at least one text element", so that the "matching result of the nth candidate multimedia object" can represent the content coverage degree achieved by the "object description information of the nth candidate multimedia object" for the "at least one text element".

[0112] In addition, the embodiment of the present application does not limit the determination process of the "matching result of the nth candidate multimedia object". For ease of understanding, it is explained below with reference to three examples.

[0113] Example 1: If the "object description information of the nth candidate multimedia object" includes the object description text of the nth candidate multimedia object, the process of determining the "matching result of the nth candidate multimedia object" may specifically include: first performing word segmentation processing on the object description text to obtain at least one word segmentation; then matching the at least one word segmentation with at least one text element to obtain the matching result of the nth candidate multimedia object.

[0114] Example 2: If the "object description information of the nth candidate multimedia object" includes at least one object description word of the nth candidate multimedia object, the process of determining the "matching result of the nth candidate multimedia object" may specifically include: matching at least one object description word of the nth candidate multimedia object with at least one text element to obtain the matching result of the nth candidate multimedia object.

[0115] Example 3: If the “object description information of the nth candidate multimedia object” includes the object description text of the nth candidate multimedia object and at least one object description word of the nth candidate multimedia object, then the process of determining the “matching result of the nth candidate multimedia object” may specifically include steps 21 to 24:

[0116] Step 21: Perform word segmentation processing on the object description text of the nth candidate multimedia object to obtain at least one word segmentation of the object description text.

[0117] Step 22: Match at least one word segment of the object description text with at least one text element to obtain a matching result of the object description text, so that the "matching result of the object description text" is used to represent the matching result between at least one word segment in the object description text of the nth candidate multimedia object and the at least one text element.

[0118] Step 23: Match at least one object description word of the nth candidate multimedia object with at least one text element to obtain a matching result of the at least one object description word, so that the "matching result of at least one object description word" is used to represent the matching result between at least one object description word of the nth candidate multimedia object and the at least one text element.

[0119] Step 24: The above “matching result of the object description text” is combined with the above “matching result of the object description text” to obtain the matching result of the nth candidate multimedia object.

[0120] Based on the relevant content of the above step 11, it can be known that for the nth candidate multimedia object, the object description information of the nth candidate multimedia object can be matched with at least one text element to obtain the matching result of the nth candidate multimedia object, so that the matching result of the nth candidate multimedia object can represent the content coverage level achieved by the "object description information of the nth candidate multimedia object" for the "at least one text element".

[0121] Step 12: Determine the coverage of the nth candidate multimedia object to at least one text element according to the matching result of the nth candidate multimedia object.

[0122] To facilitate understanding of step 12, three examples are provided below for explanation.

[0123] Example 1: If the above-mentioned "matching result of the nth candidate multimedia object" includes the matching result between at least one word segment in the object description text of the nth candidate multimedia object and the above-mentioned "at least one text element", then step 12 may specifically include: firstly determining the number of first matching word pairs according to the matching result between the at least one word segment and the above-mentioned "at least one text element"; and then determining the coverage of the nth candidate multimedia object to the at least one text element according to the number of the first matching word pairs (for example, directly determining the number of matching word pairs as the coverage of the nth candidate multimedia object to the at least one text element). Wherein, the "first matching word pair" includes a word segment and a text element having a matching relationship.

[0124] Example 2: If the above “matching result of the nth candidate multimedia object” includes the matching result between at least one object description word of the nth candidate multimedia object and the above “at least one text element”, then step 12 may specifically include: firstly determining the number of second matching word pairs according to the matching result between the at least one object description word and the above “at least one text element”; and then determining the coverage of the nth candidate multimedia object to the at least one text element according to the number of second matching word pairs (for example, directly determining the number of matching word pairs as the coverage of the nth candidate multimedia object to the at least one text element). The “second matching word pair” includes an object description word and a text element having a matching relationship.

[0125] Example 3: If the above “matching result of the nth candidate multimedia object” includes the matching result between at least one word segment in the object description text of the nth candidate multimedia object and the above “at least one text element”, and the matching result between at least one object description word of the nth candidate multimedia object and the above “at least one text element”, then step 12 may specifically include steps 31 to 35:

[0126] Step 31: Determine the number of first matching word pairs based on the matching result between at least one word segment in the object description text of the nth candidate multimedia object and the above-mentioned “at least one text element”.

[0127] Step 32: Determine the matching score of the object description text according to the number of the first matching word pairs.

[0128] For example, step 32 may specifically include: directly determining the number of the first matching word pairs as the matching score of the object description text.

[0129] Step 33: Determine the number of second matching word pairs based on the matching result between at least one object description word of the nth candidate multimedia object and the above-mentioned “at least one text element”.

[0130] Step 34: Determine a matching score of at least one object description vocabulary according to the number of the second matching word pairs.

[0131] For example, step 34 may specifically include: directly determining the number of the second matching word pairs as the matching score of at least one object description vocabulary.

[0132] Step 35: Perform a first statistical analysis on the above “matching score of the object description text” and the above “matching score of at least one object description word” to obtain the coverage of the nth candidate multimedia object to at least one text element.

[0133] The "first statistical analysis processing" may be preset, for example, it may specifically be a summing process, an averaging process, a maximum value process, or a minimum value process.

[0134] Based on the relevant contents of the above step 12, it can be known that for the nth candidate multimedia object, after obtaining the matching result of the nth candidate multimedia object, the coverage of the nth candidate multimedia object for at least one text element can be determined by referring to the matching result of the nth candidate multimedia object. Among them, because the matching result of the nth candidate multimedia object can represent the content coverage degree achieved by the "object description information of the nth candidate multimedia object" for the "at least one text element", and the "object description information of the nth candidate multimedia object" can accurately represent the information carried by the nth candidate multimedia object, the matching result of the nth candidate multimedia object can accurately represent the content coverage degree achieved by the information carried by the nth candidate multimedia object for the "at least one text element".

[0135] Based on the relevant contents of the above steps 11 to 12, it can be known that for the "at least one candidate multimedia object" including the multimedia object to be processed, the coverage of the multimedia object to be processed on the at least one text element can be determined by means of the matching result between the object description information of the multimedia object to be processed and the above "at least one text element", so that the "coverage of the multimedia object to be processed on the at least one text element" can accurately represent the content coverage degree achieved by the information carried by the multimedia object to be processed for the "at least one text element". Among them, the "multimedia object to be processed" is used to represent each candidate multimedia object in the above "at least one candidate multimedia object".

[0136] Based on the relevant content of S21 above, it can be known that for the above-mentioned "at least one candidate multimedia object", the coverage of each candidate multimedia object on at least one text element can be determined, so that the coverage of each candidate multimedia object on at least one text element can accurately represent the content coverage degree achieved by the information carried by each candidate multimedia object for the "at least one text element".

[0137] S22: searching for a target multimedia object that meets a first condition from at least one candidate multimedia object according to the coverage of at least one text element by each candidate multimedia object.

[0138] The "first condition" may be preset, for example, it may specifically include a picture with maximum coverage; and / or a video with maximum coverage.

[0139] "Target multimedia object" is used to represent a candidate multimedia object that meets the first condition; and the embodiment of the present application does not limit the "target multimedia object", for example, it may specifically include a picture (for example, the "target picture" shown below) and / or a video (for example, the "target video" shown below).

[0140] The embodiments of the present application do not limit the implementation of S22. For ease of understanding, three examples are provided below for illustration.

[0141] Example 1, when the above-mentioned "at least one candidate multimedia object" includes at least one picture, and the first condition includes a picture with maximum coverage, S22 may specifically include: first comparing the coverage of at least one picture to at least one text element to obtain a first maximum coverage value, so that the first maximum coverage value refers to the maximum value of the above-mentioned "coverage of at least one picture to at least one text element"; then determining the target multimedia object based on the picture with the first maximum coverage value (for example, the picture with the first maximum coverage value can be directly determined as the target multimedia object).

[0142] Example 2: When the above-mentioned "at least one candidate multimedia object" includes at least one video, and the first condition includes a video with maximum coverage, S22 may specifically include: first comparing the coverage of at least one video to at least one text element to obtain a second maximum coverage value, so that the second maximum coverage value refers to the maximum value of the above-mentioned "coverage of at least one video to at least one text element"; and then determining the target multimedia object based on the video with the second maximum coverage value (for example, the video with the second maximum coverage value can be directly determined as the target multimedia object).

[0143] Example 3: When the above-mentioned “at least one candidate multimedia object” includes at least one picture and at least one video, and the first condition includes a picture with maximum coverage and a video with maximum coverage, S22 may specifically include S221-S223:

[0144] S221: Compare the coverage of at least one image to at least one text element to obtain a first maximum coverage value, so that the first maximum coverage value refers to the maximum value of the above-mentioned "coverage of at least one image to at least one text element".

[0145] S222: Compare the coverage of at least one video to at least one text element to obtain a second maximum coverage value, so that the second maximum coverage value refers to the maximum value of the above-mentioned "coverage of at least one video to at least one text element".

[0146] S223: Determine a target multimedia object according to the picture with the first maximum coverage value and the video with the second maximum coverage value.

[0147] As an example, step 223 may specifically be: combining pictures with the maximum first coverage value and videos with the maximum second coverage value to obtain a target multimedia object.

[0148] Based on the relevant content of S22 above, it can be known that after obtaining the coverage of at least one candidate multimedia object on at least one text element, the coverage of at least one text element by these candidate multimedia objects can be referred to, and the target multimedia object that meets the first condition can be found from these candidate multimedia objects, so that the target multimedia object can express the above-mentioned "at least one text element" in multimedia form.

[0149] S23: Determine the multimedia object to be used according to the target multimedia object.

[0150] In an embodiment of the present application, after acquiring the target multimedia object, the target multimedia object can be directly determined as the multimedia object to be used, so that the multimedia object to be used can express the above-mentioned "at least one text element" in a multimedia form, thereby enabling the multimedia object to be used to express the above-mentioned "text to be displayed" in a multimedia form.

[0151] In addition, in order to ensure that the multimedia object to be used can cover the above "at least one text element" as comprehensively as possible, it is possible to verify whether the information carried by the above target multimedia object fully covers the above "at least one text element". Based on this, the embodiment of the present application also provides another possible implementation of S23, which may specifically include: if it is determined that the similarity between the target multimedia object and the at least one text element is higher than a preset threshold, the target multimedia object is determined as the multimedia object to be used.

[0152] Among them, "the similarity between the target multimedia object and at least one text element" refers to the degree of similarity between the information carried by the target multimedia object and at least one text element, so that the "similarity between the target multimedia object and at least one text element" can represent the possibility that the information carried by the target multimedia object can fully cover the above-mentioned "at least one text element".

[0153] In addition, the meaning of "the similarity between the target multimedia object and at least one text element" can be specifically: if the "similarity between the target multimedia object and at least one text element" is greater, it means that the possibility that the information carried by the target multimedia object can fully cover the above "at least one text element" is greater; if the "similarity between the target multimedia object and at least one text element" is smaller, it means that the possibility that the information carried by the target multimedia object can fully cover the above "at least one text element" is smaller.

[0154] The “preset threshold” may be set in advance according to the application scenario; and the embodiment of the present application does not limit the “preset threshold”, for example, it may specifically include a picture threshold and / or a video threshold.

[0155] To facilitate understanding of the above “another possible implementation of S23”, three examples are provided below for illustration.

[0156] Example 1: When the "target multimedia object" includes a target picture, and the "preset threshold" includes a picture threshold, S23 may specifically include: if it is determined that the similarity between the target picture and at least one text element is higher than the picture threshold, the target picture is determined as the multimedia object to be used. The "target picture" may refer to the above "picture with the maximum first coverage value".

[0157] Example 2: When the "target multimedia object" includes a target video, and the "preset threshold" includes a video threshold, S23 may specifically include: if it is determined that the similarity between the target video and at least one text element is higher than the video threshold, the target video is determined as the multimedia object to be used. The "target video" may refer to the above "video with the second maximum coverage value".

[0158] Example 3: When the “target multimedia object” includes a target picture and a target video, and the “preset threshold” includes a picture threshold and a video threshold, S23 may specifically include S231-S233:

[0159] S231: If it is determined that the similarity between the target image and at least one text element is higher than the image threshold, and if it is determined that the similarity between the target video and at least one text element is higher than the video threshold, the target image and the target video are combined to obtain the multimedia object to be used.

[0160] S232: If it is determined that the similarity between the target image and at least one text element is not higher than the image threshold, and if it is determined that the similarity between the target video and at least one text element is higher than the video threshold, the target video is determined as the multimedia object to be used.

[0161] S233: If it is determined that the similarity between the target image and at least one text element is higher than the image threshold, and it is determined that the similarity between the target video and at least one text element is not higher than the video threshold, the target image is determined as the multimedia object to be used.

[0162] Based on the relevant content of "Another possible implementation of S23" above, it can be known that after obtaining the target multimedia object, it is first determined whether the similarity between the target multimedia object and at least one text element is higher than a preset threshold to obtain a judgment result; then, based on the judgment result, the multimedia object to be used is determined, which is conducive to improving the accuracy of the multimedia object to be used.

[0163] Based on the relevant contents of S21 to S23 above, it can be known that after obtaining the above-mentioned "at least one text element", you can first refer to the coverage of the "at least one text element" by at least one candidate multimedia object, and search for the target multimedia object that meets the first condition from these candidate multimedia objects; then refer to the target multimedia object to determine the multimedia object to be used, so that the multimedia object to be used can express the above-mentioned "at least one text element" in multimedia form.

[0164] In some cases, the multimedia object to be used may not be found. In order to improve the user experience, the embodiment of the present application further provides another possible implementation of S2. In this implementation, S2 includes S24 in addition to the above S21-S22 and the above “another possible implementation of S23”:

[0165] S24: If it is determined that the “similarity between the target multimedia object and at least one text element” is not higher than a preset threshold, text display information of the text to be displayed is determined according to the text to be displayed and preset prompt information, so that the display device displays the text display information.

[0166] The "preset prompt information" is used to express that a multimedia object that can express the text to be displayed cannot be automatically retrieved.

[0167] "Text display information of text to be displayed" refers to information that needs to be displayed when the display device displays the text to be displayed; and the embodiment of the present application does not limit the determination process of the "text display information of text to be displayed", for example, it may specifically include: combining the text to be displayed and the preset prompt information in a first combination method to obtain the text display information of the text to be displayed, so that when the display device displays the "text display information", not only can the text to be displayed be displayed in text form, but also the preset prompt information can be displayed in a preset display method (for example, text form, picture form, voice form, etc.), so that the user can learn from the display device that the prompt information of the multimedia object that can express the text to be displayed cannot be automatically retrieved, so that the user can take corresponding processing measures for the prompt information (for example, uploading pictures or videos matching the text to be displayed, etc.).

[0168] Based on the relevant content of S24 above, it can be known that after determining that the "similarity between the target multimedia object and at least one text element" is not higher than the preset threshold, it can be determined that there is no multimedia object in the above "at least one candidate multimedia object" that can completely cover the above "at least one text element", and thus it can be determined that there is no multimedia object in the above "at least one candidate multimedia object" that can accurately express the above "text to be displayed". Therefore, in order to improve the user experience, the text display information of the text to be displayed can be directly determined based on the text to be displayed and the preset prompt information, so that when the display device displays the "text display information", not only can the text to be displayed be displayed in text form, but also the user can be informed of the automatic retrieval results that cannot find the multimedia object matching the text to be displayed, so that the user can take corresponding processing measures based on the prompt information.

[0169] In some cases, in order to improve the automatic retrieval effect of multimedia objects, the embodiment of the present application further provides another possible implementation of S2. In this implementation, S2 includes S25 in addition to the above S21-S22 and the above “another possible implementation of S23” (or, the above S21-S22, the above “another possible implementation of S23”, and S24):

[0170] S25: If it is determined that the "similarity between the target multimedia object and at least one text element" is not higher than a preset threshold, then after obtaining a matching multimedia object for the text to be displayed, the matching multimedia object is used to update at least one candidate multimedia object so that the updated "at least one candidate multimedia object" includes the matching multimedia object.

[0171] Among them, "matching multimedia objects for text to be displayed" refers to multimedia data (for example, pictures and / or videos) that can express the text to be displayed in multimedia form; and the embodiments of the present application do not limit the "matching multimedia objects for text to be displayed", for example, it may refer to multimedia data (for example, pictures and / or videos) manually uploaded by the user for the above-mentioned "text to be displayed", or it may refer to multimedia data (for example, pictures and / or videos) designed and uploaded by relevant personnel for the text to be displayed.

[0172] Based on the relevant content of S25 above, it can be known that after determining that the "similarity between the target multimedia object and at least one text element" is not higher than the preset threshold, the matching multimedia object of the text to be displayed can be obtained by means of a preset acquisition method (for example, designed and uploaded by technical personnel, etc.); then, using the matching multimedia object, at least one candidate multimedia object is updated so that the updated "at least one candidate multimedia object" includes the matching multimedia object, so that the matching multimedia object can be automatically retrieved according to the text to be displayed in the subsequent automatic retrieval process, which is conducive to improving the automatic retrieval effect of the text to be displayed.

[0173] Based on the relevant content of S2 above, it can be known that for the text to be displayed, after obtaining at least one text element of the text to be displayed, a multimedia object to be used that matches the at least one text element can be searched from a large number of candidate multimedia objects, so that the multimedia object to be used can express the text to be displayed in a multimedia form (for example, in the form of pictures and / or videos, etc.), so that the multimedia object to be used can be referenced later to generate a text display object for the text to be displayed.

[0174] S3: Determine a text display object of the text to be displayed according to the multimedia object to be used, so that the display device displays the text display object.

[0175] The “text display object of the text to be displayed” is used to express the text to be displayed in various forms (for example, text, picture, video, etc.).

[0176] In addition, the embodiment of the present application does not limit the determination process of the "text display object of the text to be displayed". For ease of understanding, the following is an explanation combining three situations.

[0177] Case 1, when the "multimedia object to be used" includes a picture to be used, the determination process of the "text display object of the text to be displayed" may specifically include: synthesizing the picture to be used with the text to be displayed to obtain the text display object of the text to be displayed.

[0178] Among them, the "synthesis processing" can be pre-set; and the embodiment of the present application does not limit the "synthesis processing", for example, it can specifically include: using the picture to be used as the background picture of the text to be displayed.

[0179] Based on the relevant content of the above "Case 1", it can be known that after obtaining the picture to be used, the picture to be used can be synthesized as the background picture of the text to be displayed to obtain a text display object of the text to be displayed, so that when the display device displays the text display object, the picture to be used is always displayed as the background picture, so that the text to be displayed presented in text form can be displayed on the picture to be used according to the preset display method.

[0180] It should be noted that the above-mentioned "preset display mode" can be pre-set. For example, it can be specifically: the text to be displayed is scrolled and displayed along with the playback progress of the "audio playback data of the text to be displayed" shown below, until the playback of the "audio playback data of the text to be displayed" is finished, and the text to be displayed is completely displayed at the preset position of the picture to be used, so that the text to be displayed and the picture to be used can be displayed on the display device at the same time.

[0181] Case 2, when the "multimedia object to be used" includes a video to be used, the process of determining the "text display object of the text to be displayed" may specifically include: determining the text display object of the text to be displayed according to the video to be used.

[0182] It can be seen that after obtaining the video to be used, the text display object of the text to be displayed can be determined according to the video to be used (for example, the video to be used can be directly determined as the text display object of the text to be displayed), so that the text display object can express the text to be displayed in video form (that is, picture form + text form + audio form).

[0183] In addition, the above-mentioned "text to be displayed" may be expressed by only a certain part of the video to be used. At this time, in order to further improve the display effect of the above-mentioned "text display object", the above-mentioned "certain part of the video to be used" can be directly used to determine the "text display object". Based on this, the embodiment of the present application also provides another possible implementation method for determining the "text display object of the text to be displayed", which can specifically include steps 41-43:

[0184] Step 41: Perform preset segmentation processing on the video to be used to obtain at least one candidate video segment.

[0185] The "preset division processing" can be preset according to the application scenario; and the embodiment of the present application does not limit the "preset division processing", for example, it can be specifically divided according to the preset division interval. The "preset division interval" refers to the distance between two adjacent division positions.

[0186] "At least one candidate video segment" refers to each video segment extracted from the video to be used; and the embodiment of the present application does not limit "at least one candidate video segment", for example, the information carried by the "at least one candidate video segment" can completely cover the information carried by the video to be used.

[0187] Step 42: searching for a target video segment that meets a second condition from at least one candidate video segment according to the similarity between each candidate video segment and at least one text element.

[0188] The similarity between the mth candidate video segment and the at least one text element refers to the similarity between the information carried by the mth candidate video segment and the at least one text element. m is a positive integer, m≤M, M is a positive integer, and M represents the number of candidate video segments in the above-mentioned "at least one candidate video segment".

[0189] In addition, the embodiment of the present application does not limit the determination process of “the similarity between the mth candidate video segment and at least one text element”. For example, it may specifically include step 51-step 52:

[0190] Step 51: extracting to-be-processed picture information, to-be-processed text information, and to-be-processed audio information from the mth candidate video segment.

[0191] The “picture information to be processed” refers to all video information represented in the form of pictures in the mth candidate video segment.

[0192] The “text information to be processed” refers to all video information represented in the form of text in the mth candidate video segment.

[0193] The “audio information to be processed” refers to all video information represented in the form of audio in the mth candidate video segment.

[0194] Step 52: perform weighted summation on the similarity between the image information to be processed and at least one text element, the similarity between the text information to be processed and at least one text element, and the similarity between the audio information to be processed and at least one text element to obtain the similarity between the mth candidate video segment and at least one text element.

[0195] The “similarity between the image information to be processed and at least one text element” is used to indicate the similarity between the image information carried by the mth candidate video segment and at least one text element.

[0196] “Similarity between the text information to be processed and at least one text element” is used to indicate the similarity between the character information carried by the mth candidate video segment and at least one text element.

[0197] “Similarity between the audio information to be processed and at least one text element” is used to indicate the similarity between the audio information carried by the m-th candidate video segment and at least one text element.

[0198] The weights involved in the “weighted summation” in step 52 may be preset according to the application scenario.

[0199] Based on the relevant contents of the above steps 51 to 52, it can be known that for the "at least one candidate video segment" including the video segment to be processed, the similarity between the image information carried by the video segment to be processed (that is, the image information to be processed) and at least one text element, the similarity between the character information carried by the video segment to be processed (that is, the text information to be processed) and at least one text element, and the similarity between the audio information carried by the video segment to be processed (that is, the audio information to be processed) and at least one text element can be referred to to determine the similarity between the video segment to be processed and at least one text element, so that the "similarity between the video segment to be processed and at least one text element" can accurately represent the similarity between the information carried by the video segment to be processed and the at least one text element. Among them, the "video segment to be processed" is used to represent each candidate video segment in the above "at least one candidate video segment".

[0200] The “second condition” may be preset; and the embodiment of the present application does not limit the “second condition”, for example, it may specifically be: a candidate video segment with a specific maximum similarity.

[0201] The “target video segment” refers to a candidate video segment that satisfies the second condition.

[0202] Based on the relevant contents of the above step 42, it can be known that after obtaining at least one candidate video segment, the maximum similarity value can be determined from the similarities between the at least one candidate video segment and the at least one text element; and then the candidate video segment with the maximum similarity value is determined as the target video segment, so that the similarity between the target video segment and the at least one text element is greater than the similarity between any other candidate video segment in the above “at least one candidate video segment” except the target video segment and the at least one text element.

[0203] Step 43: Determine the text display object of the text to be displayed according to the target video segment.

[0204] In an embodiment of the present application, after obtaining the target video segment, the target video segment can be directly determined as a text display object for the text to be displayed, so that the text display object can express the text to be displayed in video form (that is, text form, picture form, and audio form) at the same time.

[0205] In addition, the text content displayed in the above-mentioned "target video segment" may be inconsistent with the text to be displayed. Therefore, in order to further improve the expression effect of the above-mentioned "text display object", the embodiment of the present application also provides another possible implementation method of step 43, which may specifically include: replacing the text content in the target video segment with the text to be displayed, and obtaining a text display object of the text to be displayed.

[0206] It should be noted that the embodiments of the present application are not limited to the above-mentioned implementation method of "replacement". For example, the text content in the target video segment can be deleted first, and then the text to be displayed can be added to the target video segment.

[0207] In addition, the audio information recorded in the audio data in the above-mentioned "target video segment" may be inconsistent with the text to be displayed. Therefore, in order to further improve the expression effect of the above-mentioned "text display object", the embodiment of the present application also provides another possible implementation method of step 43, which may specifically include: after obtaining the audio playback data of the text to be displayed, use the audio playback data of the text to be displayed to replace the audio data in the target video segment to obtain the text display object of the text to be displayed.

[0208] It should be noted that the embodiments of the present application are not limited to the above-mentioned "replacement" implementation method. For example, the target video segment can be muted first, and then the audio playback data of the text to be displayed can be added to the target video segment.

[0209] Based on the relevant contents of steps 41 to 43 above, it can be known that in some cases, the video segment in the video to be used that best matches the above "at least one text element" can be used to determine the text display object of the text to be displayed, so that the text display object can better express the text to be displayed.

[0210] Based on the relevant content of the above situation 2, it can be known that after obtaining the video to be used, the video to be used or a part of the video to be used can be used to determine the text display object of the text to be displayed, so that the text display object can express the text to be displayed in video form (that is, in text form, picture form, and audio form at the same time), so that the text display object can be played on the display device later.

[0211] Case 3, when the "multimedia object to be used" includes a picture to be used and a video to be used, the determination process of the "text display object of the text to be displayed" may specifically include S31-S33:

[0212] S31: Determine a text display image of the text to be displayed according to the image to be used.

[0213] As an example, S31 may specifically include: determining the image to be used as a text display image of the text to be displayed.

[0214] S32: Determine a text display video of the text to be displayed according to the video to be used.

[0215] It should be noted that S32 can be implemented using any of the implementation methods shown in "Case 2" in S3 above, by simply replacing the "text display object" in any of the implementation methods shown in "Case 2" in S3 above with "text display video".

[0216] S33: combining the text to be displayed, the text display picture of the text to be displayed, and the text display video of the text to be displayed according to a preset combination method to obtain a text display object of the text to be displayed.

[0217] Among them, the "preset combination method" can be set in advance; and the embodiment of the present application does not limit the "preset combination method", for example, it can specifically include: using the "text display picture of the text to be displayed" as the background picture, placing the "text display video of the text to be displayed" at the first position of the background picture (for example, the upper left corner, etc.) for display, and placing the "text to be displayed" in text form at the second position of the background picture (for example, the middle position, etc.) for display.

[0218] Based on the relevant content of the above situation 3, it can be known that after obtaining the pictures to be used and the videos to be used, the text to be displayed, the pictures to be used, and the video to be used can be referred to to determine the text display object of the text to be displayed, so that the text display object can express the text to be displayed in text form, picture form, and video form at the same time, which is conducive to improving the display effect of the text to be displayed.

[0219] Based on the relevant contents of S1 to S3 above, it can be known that for the display method provided in the embodiment of the present application, after obtaining the text to be displayed, the text to be displayed is firstly analyzed for text elements to obtain at least one text element of the text to be displayed; then, a multimedia object to be used that matches the at least one text element is searched from at least one candidate multimedia object; finally, according to the multimedia object to be used, a text display object of the text to be displayed is determined, so that the text display object is displayed by the display device.

[0220] It can be seen that since the multimedia object to be used expresses information through various media forms (for example, text, image, video, etc.), the text display object determined based on the multimedia object to be used can express the text to be displayed through various media forms, so that when the display device displays the text display object, the display device can display the text to be displayed in various media forms, which is beneficial to improving the text display effect of the text to be displayed, thereby improving the user experience.

[0221] Method Example 2

[0222] In addition, in order to further improve the text display effect of the text to be displayed, the embodiment of the present application also provides another possible implementation of the display method. In this implementation, the display method not only includes the above S1-S2, but also includes S4-S5:

[0223] S4: Generate audio playback data of the text to be displayed.

[0224] Among them, the "audio playback data of the text to be displayed" is used to express the text to be displayed in audio form; and the embodiment of the present application does not limit the generation process of the "audio playback data of the text to be displayed".

[0225] S5: Determine a text display object of the text to be displayed according to the multimedia object to be used and the audio playback data of the text to be displayed.

[0226] To facilitate understanding of S5, three situations are introduced below.

[0227] Case 1, when the "multimedia object to be used" includes a picture to be used, S5 may specifically include: combining the picture to be used, the text to be displayed, and the audio playback data of the text to be displayed according to the second combination method to obtain a text display object of the text to be displayed.

[0228] Among them, the "second combination method" can be pre-set, for example, the picture to be used is used as the background picture of the text to be displayed, and the audio playback data of the text to be displayed is placed on the position to be used on the background picture (for example, the lower right corner, etc.); and the text to be displayed is scrolled with the playback progress of the "audio playback data of the text to be displayed" until the playback of the "audio playback data of the text to be displayed" is finished, and the text to be displayed is completely displayed at the preset position of the picture to be used.

[0229] Based on the relevant content of the above situation 1, it can be known that after obtaining the audio playback data of the picture to be used and the text to be displayed, the picture to be used, the text to be displayed, and the audio playback data of the text to be displayed can be combined and processed to obtain a text display object of the text to be displayed, so that when the text display object is displayed by the display device, the picture to be used is always displayed as a background picture, so that the text to be displayed presented in text form can be displayed on the picture to be used according to the above-mentioned "preset display method".

[0230] Case 2, when the "multimedia object to be used" includes a video to be used, S5 may specifically include steps 61 to 63:

[0231] Step 61: Determine the video object to be processed according to the video to be used.

[0232] It should be noted that S32 can be implemented using any of the implementation methods shown in "Case 2" in S3 above, by simply replacing the "text display object of text to be displayed" in any of the implementation methods shown in "Case 2" in S3 above with the "video object to be processed".

[0233] Step 62: Replace the audio data of the video object to be processed with the audio playback data of the text to be displayed, and obtain a replaced video object.

[0234] In an embodiment of the present application, after obtaining the audio playback data of the text to be displayed and the video object to be processed, the audio playback data can be used to replace the audio data of the video object to be processed to obtain a replaced video object, so that the replaced video object includes the audio playback data of the text to be displayed, so that the replaced video object has a better expression effect for the text to be displayed.

[0235] Step 63: Determine the text display object of the text to be displayed according to the replaced video object.

[0236] In an embodiment of the present application, after obtaining the replaced video object, a text display object for the text to be displayed can be determined based on the replaced video object (for example, the replaced video object is directly determined as the text display object for the text to be displayed. For another example, the text to be displayed can be used to replace the text content in the replaced video object to obtain the text display object for the text to be displayed), so that the text display object can better express the text to be displayed.

[0237] Based on the relevant content of the above situation 2, it can be known that after obtaining the audio playback data of the video to be used and the text to be displayed, the text display object of the text to be displayed can be determined by referring to the video to be used and the audio playback data, so that the text display object can better express the text to be displayed.

[0238] Case 3, when the "multimedia objects to be used" include pictures to be used and videos to be used, S5 may specifically include S51-S55:

[0239] S51: Determine a text display image of the text to be displayed according to the image to be used.

[0240] It should be noted that for the relevant content of S51, please refer to the relevant content of S31 above.

[0241] S52: Determine the text display audio of the text to be displayed according to the audio playback data of the text to be displayed.

[0242] As an example, S52 may specifically include: determining the audio playback data of the text to be displayed as the text display audio of the text to be displayed.

[0243] S53: Determine a text display video of the text to be displayed according to the video to be used.

[0244] The embodiment of the present application does not limit the implementation method of S53. For example, S53 can be implemented by any implementation method shown in "Case 2" in S3 above, and it only needs to replace the "text display object" in any implementation method shown in "Case 2" in S3 above with "text display video". For another example, S53 can also be implemented by any implementation method shown in "Case 2" in S5 above, and it only needs to replace the "text display object" in any implementation method shown in "Case 2" in S5 above with "text display video".

[0245] S33: According to the third combination method, the text to be displayed, the text display audio of the text to be displayed, the text display picture of the text to be displayed, and the text display video of the text to be displayed are combined and processed to obtain the text display object of the text to be displayed.

[0246] Among them, the "third combination method" can be pre-set; and the embodiment of the present application does not limit the "third combination method", for example, it can specifically include: using the "text display picture of the text to be displayed" as the background picture, placing the "text display video of the text to be displayed" at the first position of the background picture (for example, the upper left corner, etc.) for display, placing the "text to be displayed" in text form at the second position of the background picture (for example, the middle position, etc.) for display, and placing the "text display audio of the text to be displayed" at the third position of the background picture (for example, the upper right corner, etc.) for display.

[0247] Based on the relevant contents of S4 to S5 above, it can be known that for the text to be displayed, the text to be displayed, the audio playback data of the text to be displayed, and the multimedia object to be used corresponding to the text to be displayed can be referred to to determine the text display object of the text to be displayed, so that the text display object can express the text to be displayed in more forms, which is conducive to improving the display effect of the text to be displayed.

[0248] Based on the display method provided by the above method embodiment, the embodiment of the present application also provides a display device, which is explained and illustrated below in conjunction with the accompanying drawings.

[0249] Device Embodiment

[0250] The device embodiment introduces the display device, and related content can be found in the above method embodiment.

[0251] See also Figure 4 , which is a structural schematic diagram of a display device provided in an embodiment of the present application.

[0252] The display device 400 provided in the embodiment of the present application includes:

[0253] The analyzing unit 401 is used for, after acquiring the text to be displayed, performing text element analysis on the text to be displayed to obtain at least one text element of the text to be displayed;

[0254] A searching unit 402 is configured to search for a multimedia object to be used that matches the at least one text element from at least one candidate multimedia object;

[0255] The determination unit 403 is configured to determine the text display object of the text to be displayed according to the multimedia object to be used, so that the display device can display the text display object.

[0256] In a possible implementation manner, the searching unit 402 includes:

[0257] A first determination subunit, configured to determine the coverage of each candidate multimedia object to the at least one text element;

[0258] A first search subunit, configured to search for a target multimedia object satisfying a first condition from the at least one candidate multimedia object according to the coverage of the at least one text element by each candidate multimedia object;

[0259] The second determining subunit is configured to determine the multimedia object to be used according to the target multimedia object.

[0260] In a possible implementation manner, the second determination subunit is specifically configured to: if it is determined that the similarity between the target multimedia object and the at least one text element is higher than a preset threshold, determine the target multimedia object as the multimedia object to be used.

[0261] In a possible implementation manner, the display device 400 further includes:

[0262] A prompt unit is used to determine text display information of the text to be displayed based on the text to be displayed and preset prompt information if it is determined that the similarity between the target multimedia object and the at least one text element is not higher than a preset threshold, so that the text display information can be displayed by the display device.

[0263] In a possible implementation manner, the display device 400 further includes:

[0264] The updating unit is configured to update the at least one candidate multimedia object using the matching multimedia object after acquiring the matching multimedia object of the text to be displayed if it is determined that the similarity between the target multimedia object and the at least one text element is not higher than a preset threshold.

[0265] In one possible implementation, the at least one candidate multimedia object includes a multimedia object to be processed; and the first determination subunit is specifically used to: match the object description information of the multimedia object to be processed with the at least one text element to obtain a matching result of the multimedia object to be processed; and determine the coverage of the multimedia object to be processed for the at least one text element based on the matching result of the multimedia object to be processed.

[0266] In a possible implementation manner, the object description information includes an object description text and / or at least one object description word.

[0267] In a possible implementation manner, the multimedia object to be used includes a picture to be used, and the determining unit 403 is specifically configured to synthesize the picture to be used and the text to be displayed to obtain the text display object.

[0268] In a possible implementation manner, the multimedia object to be used includes a video to be used; and the determining unit 403 is specifically configured to: determine the video to be used as the text display object;

[0269] In a possible implementation manner, the multimedia object to be used includes a video to be used; and the determining unit 403 includes:

[0270] A video segmentation subunit, configured to perform a preset segmentation process on the video to be used to obtain at least one candidate video segment;

[0271] A second search subunit, configured to search for a target video segment satisfying a second condition from the at least one candidate video segment according to the similarity between each candidate video segment and the at least one text element;

[0272] The third determining subunit is used to determine the text display object according to the target video segment.

[0273] In a possible implementation manner, the at least one candidate video segment includes a to-be-processed video segment, and the determining unit 403 further includes:

[0274] The fourth determination subunit is used to extract the image information to be processed, the text information to be processed, and the audio information to be processed from the video segment to be processed; and perform weighted summation on the similarity between the image information to be processed and the at least one text element, the similarity between the text information to be processed and the at least one text element, and the similarity between the audio information to be processed and the at least one text element to obtain the similarity between the video segment to be processed and the at least one text element.

[0275] In a possible implementation manner, the third determining subunit is specifically configured to: replace text content in the target video segment with the text to be displayed to obtain the text display object.

[0276] In a possible implementation, the multimedia object to be used includes a picture to be used and a video to be used; and the determination unit 403 is specifically used to: determine a text display picture of the text to be displayed based on the picture to be used; determine a text display video of the text to be displayed based on the video to be used; and combine the text to be displayed, the text display picture of the text to be displayed, and the text display video of the text to be displayed in a preset combination method to obtain the text display object.

[0277] In a possible implementation manner, the display device 400 further includes:

[0278] A generating unit, used for generating audio playing data of the text to be displayed;

[0279] The determining unit 403 is specifically configured to determine the text display object of the text to be displayed according to the multimedia object to be used and the audio playback data.

[0280] In a possible implementation, the multimedia object to be used includes a video to be used, and the determination unit 403 is specifically used to: determine the video object to be processed based on the video to be used; replace the audio data of the video object to be processed with the audio playback data to obtain a replaced video object; and determine the text display object based on the replaced video object.

[0281] Furthermore, an embodiment of the present application also provides a display device, including: a processor, a memory, and a system bus;

[0282] The processor and the memory are connected via the system bus;

[0283] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any one of the implementation methods of the above-mentioned display method.

[0284] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any implementation method of the above-mentioned display method.

[0285] Furthermore, an embodiment of the present application also provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any one of the implementation methods of the above-mentioned XXX method.

[0286] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0287] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0288] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0289] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A display method, characterized in that: The method comprises: After obtaining the text to be displayed, performing text element analysis on the text to be displayed to obtain at least one text element of the text to be displayed; Searching for a multimedia object to be used that matches the at least one text element from at least one candidate multimedia object; Determine, according to the multimedia object to be used, a text display object of the text to be displayed, so that a display device displays the text display object; The multimedia object to be used includes a video to be used, and the process of determining the text display object includes: performing a preset division process on the video to be used to obtain at least one candidate video segment; searching for a target video segment that meets a second condition from the at least one candidate video segment according to the similarity between each candidate video segment and the at least one text element; and determining the text display object according to the target video segment; The at least one candidate video segment includes a video segment to be processed, and a process for determining the similarity between the video segment to be processed and the at least one text element includes: extracting image information to be processed, text information to be processed, and audio information to be processed from the video segment to be processed; and performing weighted summation on the similarity between the image information to be processed and the at least one text element, the similarity between the text information to be processed and the at least one text element, and the similarity between the audio information to be processed and the at least one text element to obtain the similarity between the video segment to be processed and the at least one text element.

2. The method according to claim 1, characterized in that The process of determining the multimedia object to be used includes: Determining the coverage of each candidate multimedia object to the at least one text element; Searching for a target multimedia object that meets a first condition from the at least one candidate multimedia object according to the coverage of the at least one text element by each candidate multimedia object; The multimedia object to be used is determined according to the target multimedia object.

3. The method according to claim 2, characterized in that The step of determining the multimedia object to be used according to the target multimedia object includes: If it is determined that the similarity between the target multimedia object and the at least one text element is higher than a preset threshold, the target multimedia object is determined as the multimedia object to be used.

4. The method according to claim 3, characterized in that The method further comprises: If it is determined that the similarity between the target multimedia object and the at least one text element is not higher than a preset threshold, text display information of the text to be displayed is determined according to the text to be displayed and preset prompt information, so that the display device displays the text display information.

5. The method according to claim 4, characterized in that The method further comprises: If it is determined that the similarity between the target multimedia object and the at least one text element is not higher than a preset threshold, after obtaining a matching multimedia object of the text to be displayed, the at least one candidate multimedia object is updated using the matching multimedia object.

6. The method according to claim 2, characterized in that The at least one candidate multimedia object includes a to-be-processed multimedia object, and the process of determining the coverage of the to-be-processed multimedia object on the at least one text element includes: Matching the object description information of the to-be-processed multimedia object with the at least one text element to obtain a matching result of the to-be-processed multimedia object; According to the matching result of the multimedia object to be processed, the coverage of the at least one text element by the multimedia object to be processed is determined.

7. The method according to claim 6, characterized in that The object description information includes an object description text and / or at least one object description word.

8. The method according to claim 1, characterized in that The multimedia object to be used includes a picture to be used, and the process of determining the text display object includes: The image to be used and the text to be displayed are synthesized to obtain the text display object.

9. The method according to claim 1, characterized in that: The determining the text display object according to the target video segment includes: The text content in the target video segment is replaced with the text to be displayed to obtain the text display object.

10. The method according to claim 1, characterized in that The multimedia objects to be used include pictures to be used and videos to be used, and the process of determining the text display objects includes: Determine, according to the picture to be used, a text display picture of the text to be displayed; Determine, according to the video to be used, a text display video of the text to be displayed; The text to be displayed, the text display picture of the text to be displayed, and the text display video of the text to be displayed are combined according to a preset combination method to obtain the text display object.

11. The method according to claim 1, characterized in that: The method further comprises: Generating audio playback data of the text to be displayed; The step of determining the text display object of the text to be displayed according to the multimedia object to be used includes: A text display object of the text to be displayed is determined according to the multimedia object to be used and the audio playback data.

12. The method according to claim 11, characterized in that The multimedia object to be used includes a video to be used, and the process of determining the text display object includes: Determining a video object to be processed according to the video to be used; Using the audio playback data to replace the audio data of the video object to be processed, to obtain a replaced video object; The text display object is determined according to the replaced video object.

13. A display device, characterized in that: include: An analyzing unit, configured to, after acquiring the text to be displayed, perform text element analysis on the text to be displayed to obtain at least one text element of the text to be displayed; A searching unit, configured to search for a multimedia object to be used that matches the at least one text element from at least one candidate multimedia object; A determination unit, configured to determine a text display object of the text to be displayed according to the multimedia object to be used, so that the text display object can be displayed by a display device; The multimedia object to be used includes a video to be used, and the determination unit is specifically used to: perform a preset division process on the video to be used to obtain at least one candidate video segment; according to the similarity between each candidate video segment and the at least one text element, search for a target video segment that meets a second condition from the at least one candidate video segment; and determine the text display object according to the target video segment; The at least one candidate video segment includes a video segment to be processed, and the determination unit is further used to: extract image information to be processed, text information to be processed, and audio information to be processed from the video segment to be processed; and perform weighted summation on the similarity between the image information to be processed and the at least one text element, the similarity between the text information to be processed and the at least one text element, and the similarity between the audio information to be processed and the at least one text element to obtain the similarity between the video segment to be processed and the at least one text element.

14. A device, characterized in that The device includes: a processor, a memory, and a system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 12.

16. A computer program product, characterized in that When the computer program product is executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 12.

Images