Information processing method, device, electronic device and storage medium

Abstract

The present application discloses an information processing method, device, electronic device and storage medium, belonging to the field of communication technology. The method mainly comprises acquiring a first image, the first image comprises a target subtitle; determining first feature information according to the target subtitle, the first feature information comprises subtitle attribute information and first information related to the semantics of the target subtitle; filtering out target matching subtitles that meet preset conditions from preset subtitle information according to the first feature information; and displaying audio and video information corresponding to the target matching subtitles.

Description

Technical Field

[0001] The present application belongs to the field of communication technology, and specifically relates to an information processing method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of the Internet, when users are faced with a vast amount of Internet information such as video information, text information, audio information, etc., how to quickly and effectively find the content they need has become a topic that needs urgent attention.

[0003] At present, the main methods of searching for video information are based on keywords such as the film name, actor name, and director name, or based on video classification. However, the former search method requires users to actively enter keywords, which, to a certain extent, increases user operations and reduces information search efficiency. The latter video search method often requires browsing most of the videos under the film and television classification to obtain content that is valuable to users, making it difficult to quickly locate the video information that users need. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an information processing method, device, electronic device and storage medium, which can solve the current problems of cumbersome information search operations and low information search efficiency and accuracy.

[0005] In a first aspect, an embodiment of the present application provides an information processing method, which may include:

[0006] Acquire a first image, where the first image includes a target subtitle;

[0007] Determining first feature information according to the target subtitle, where the first feature information includes subtitle attribute information and first information related to the semantics of the target subtitle;

[0008] According to the first feature information, target matching subtitles that meet preset conditions are selected from the preset subtitle information;

[0009] Displays the audio and video information corresponding to the target matching subtitles.

[0010] In a second aspect, an embodiment of the present application provides an information processing device, which may include:

[0011] An acquisition module, used for acquiring a first image, wherein the first image includes a target subtitle;

[0012] A determination module, configured to determine first feature information according to a target subtitle, wherein the first feature information includes subtitle attribute information and first information related to the semantics of the target subtitle;

[0013] A screening module, used for screening target matching subtitles that meet preset conditions from the preset subtitle information according to the first feature information;

[0014] The display module is used to display the audio and video information corresponding to the target matching subtitles.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and when the program or instruction is executed by the processor, the steps of the information processing method shown in the first aspect are implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the information processing method shown in the first aspect are implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the steps of the information processing method shown in the first aspect.

[0018] In the embodiment of the present application, by acquiring the target subtitle in the first image, determining the subtitle attribute information of the target subtitle and the first information related to the semantics of the target subtitle, then, based on the first feature information, filtering out the target matching subtitle that meets the preset conditions from the preset subtitle information, and determining the audio and video information corresponding to the target matching subtitle as the audio and video including the first image. In this way, when the target subtitle in the image cannot completely match the subtitle in the preset subtitle information, it is also possible to filter out partially matching subtitles based on the target subtitle, thereby achieving the purpose of searching for the audio and video information corresponding to the target matching subtitle including the partially matching subtitle, and improving the success rate of searching for audio and video information.

[0019] In addition, by matching audio and video information through the subtitle information in the picture, the user does not need to search for keywords such as the video title, actor name, and director name, or memorize based on film and television classification. While reducing the user's information search operations, the information processing method provided in the embodiment of the present application can also broaden the information search scenario, and can more efficiently match audio and video information, thereby improving the efficiency of information search. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of an information processing architecture provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of an information processing method provided in an embodiment of the present application;

[0022] Figure 3A schematic diagram of an information processing interface provided by an embodiment of the present application;

[0023] Figure 4 A schematic diagram of another information processing interface provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of an information processing device provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0028] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0029] Based on this, the following Figure 1 , the information processing method provided by the embodiment of the present application is described in detail through specific embodiments and their application scenarios.

[0030] The present application embodiment proposes an information processing architecture, such as Figure 1 As shown, the information processing architecture may include an electronic device. Among them, the electronic device can provide functions of playing audio and video and searching audio and video. Based on the information processing architecture, the application scenario of the information processing method provided in the embodiment of the present application is described.

[0031] When a user views an image such as Figure 1When you want to know the audio and video information corresponding to the image or video band, such as the name of the movie or TV series (or the video of the movie or TV series) to which the image or video band belongs, you can perform image recognition on any image that includes subtitles in the image or video clip to determine the target subtitle in the image, such as "The most important thing every day is to be happy". In order to eliminate the situation where there are typos in the subtitles or synonyms in different language translations, and cover more words under the same semantics, the embodiment of the present application determines the first feature information according to the target subtitles, and the first feature information includes subtitle attribute information and the first information related to the semantics of the target subtitles. In this way, according to the first feature information, the target matching subtitles that meet the preset conditions are screened out from the preset subtitle information, and the audio and video information corresponding to the target matching subtitles is recommended to the user, such as the name of the audio and video to which the image belongs, such as the name of the movie or TV series, the video link of the audio and video, such as the video of the movie or TV series, etc., the audio and video introduction, and other information.

[0032] It should be noted that the subtitles in the embodiments of the present application, namely the target subtitles, refer to the texts that are processed in the post-production of TV, film, and stage works, and other non-image contents such as dialogues in text form. The commentary texts and various texts that appear below the film and television works played on the movie screen or in the playback application of electronic devices, such as the film title, cast list, lyrics, dialogues, descriptions, character introductions, place names, and years, are all called subtitles.

[0033] Thus, by acquiring the target subtitle in the first image, determining the subtitle attribute information of the target subtitle and the first information related to the semantics of the target subtitle, then, based on the first feature information, filtering out the target matching subtitle that meets the preset conditions from the preset subtitle information, and determining the audio and video information corresponding to the target matching subtitle as the audio and video including the first image. In this way, when the target subtitle in the image cannot completely match the subtitle in the preset subtitle information, it is also possible to filter out partially matching subtitles based on the target subtitle, thereby achieving the purpose of searching for the audio and video information corresponding to the target matching subtitle including the partially matching subtitle, and improving the success rate of searching for the audio and video information.

[0034] In addition, by matching audio and video information with subtitle information in the picture, users do not need to search for keywords such as the movie name, actor name, and director name, or memorize based on film and television classification. While reducing the user's information search operations, the information processing method provided in the embodiment of the present application can also broaden the information search scenario, and can more efficiently match audio and video information, thereby improving the efficiency of information search.

[0035] It should be noted that, in addition to being applicable to the above-mentioned scenario of matching the audio and video information of an image through the target subtitles in the image, the information processing method provided by the embodiment of the present application can also be applied to the scenario of matching the product to which the image belongs and the book category corresponding to the special text in the image, such as a trademark, a product introduction, or a part of the text in the book. Here, the information processing method provided by the embodiment of the present application can be applied to any scenario of searching for overall information corresponding to the local features through local features.

[0036] According to the above application scenarios, the following Figure 2 The information processing method provided in the embodiments of the present application is described in detail.

[0037] Figure 2 A flowchart of an information processing method provided in an embodiment of the present application.

[0038] like Figure 2 As shown, the information processing method can be applied to Figure 1 The electronic device shown, based on this, can specifically include the following steps:

[0039] Step 210, obtaining a first image, the first image including a target subtitle. Step 220, determining first feature information according to the target subtitle, the first feature information including subtitle attribute information and first information related to the semantics of the target subtitle. Step 230, filtering out target matching subtitles that meet preset conditions from preset subtitle information according to the first feature information. Step 240, displaying audio and video information corresponding to the target matching subtitle.

[0040] In this way, by acquiring the target subtitle in the first image, determining the subtitle attribute information of the target subtitle and the first information related to the semantics of the target subtitle, then, based on the first feature information, filtering out the target matching subtitle that meets the preset conditions from the preset subtitle information, and determining the audio and video information corresponding to the target matching subtitle as the audio and video including the first image. In this way, when the target subtitle in the image cannot completely match the subtitle in the preset subtitle information, it is also possible to filter out partially matching subtitles based on the target subtitle, thereby achieving the purpose of searching for the audio and video information corresponding to the target matching subtitle including the partially matching subtitle, and improving the success rate of searching for audio and video information.

[0041] In addition, by matching audio and video information through the subtitle information in the picture, the user does not need to search for keywords such as the video title, actor name, and director name, or memorize based on film and television classification. While reducing the user's information search operations, the information processing method provided in the embodiment of the present application can also broaden the information search scenario, and can more efficiently match audio and video information, thereby improving the efficiency of information search.

[0042] The above steps are described in detail below:

[0043] First, step 210 is involved in acquiring a first image. Here, the first image may be an image taken by a user, a screenshot image, or any image including subtitles selected by an electronic device from a certain video frequency band.

[0044] Here, the target subtitle T in the first picture can be identified by inputting the picture and using the optical character recognition (OCR) technology, such as Figure 3 The caption "The most important thing is to be happy every day" is extracted from the first image shown.

[0045] Next, step 220 is involved. In one or more optional embodiments, there may be errors in the preset subtitle information, such as the subtitles in the subtitle library, or there may be typos in the target subtitles themselves. In this way, no matter which of the above situations, the target subtitles cannot be successfully matched to the subtitles in the subtitle library, resulting in the inability to completely match the search terms, and also resulting in the inability to search for the audio and video information belonging to the first image. Therefore, the embodiment of the present application can optimize the target subtitles, that is, extract the subtitle attribute information of the target subtitles and the first information related to the semantics of the target subtitles. In this way, even if the target subtitles cannot be matched in the subtitle library, they can be identified based on some feature information in the target subtitles, so as to improve the efficiency of matching audio and video information.

[0046] Here, the subtitle attribute information in the first feature information may include role name information such as a role name list and target keyword information such as a keyword list, and the magnetism of the target keyword may include at least one of the following: noun, verb, adjective. Example 1: "The most important thing every day is to be happy", the extracted person name is empty, and the keywords are: ["every day", "happy"]. The first information related to the semantics of the target subtitle in the first feature information may include the syllable information of the target subtitle and the vector information of the target subtitle.

[0047] The first information related to the semantics of the target subtitle may include the syllable information of the target subtitle and the vector information of the target subtitle. For example, the syllable information of "the most important thing every day is to be happy" is "měi tiān zùi zhòng yào de jiùshì kāi xīn". Here, the preset subtitle information is matched by syllable information to prevent typos in the target subtitle, which causes the target subtitle to fail to successfully match the subtitle in the subtitle library. In addition, using the vector information of the target subtitle, the text similarity between the target subtitle and each candidate subtitle information can be calculated. In this way, when the target subtitle is in other language categories, that is, a non-Chinese language such as English, Arabic or Tibetan, it is necessary to translate these language texts into Chinese, and then perform the process of matching subtitles. During this period, when translating into Chinese, there will be situations where synonymous but different characters appear. In this way, by calculating the text similarity between the target subtitle and each candidate subtitle information, different words synonymous with the target subtitle can be determined, so as to increase the possibility of matching the target subtitle to the target matching subtitle. Of course, in addition to being applicable when the target subtitles are in other language categories, it can also be applied to the above-mentioned situation where typos appear in the target subtitles, resulting in the target subtitles being unable to be successfully matched to the subtitles in the subtitle library. Involving step 230, in one or more optional embodiments, step 230 may specifically include, from the preset subtitle information, selecting preset candidate subtitle information that meets the preset word length value corresponding to the target subtitle; and selecting the target matching subtitle that meets the preset conditions from the preset candidate subtitle information according to the first feature information.

[0048] For example, the length L of the target subtitle T is calculated, and the subtitles whose word lengths are not in [L-10, L+10] in the subtitle library D are filtered out to obtain the new preset candidate subtitle information corresponding to the target subtitle T in the subtitle library D, namely the candidate subtitle library D1. Here, the 10 involved in [L-10, L+10] can be set according to the situation. It should be noted that the subtitle library D is an array, each array corresponds to the information of an audio and video, and its storage format is: {"audio and video name": ["audio and video role name", "subtitle 1", "subtitle 2"...].

[0049] Therefore, the information processing method provided by the embodiment of the present application can determine the audio and video information corresponding to the target subtitle when the target subtitle completely matches the subtitle in the subtitle library D.

[0050] Alternatively, when the matching rate between the target subtitle and the subtitles in the subtitle library D is greater than a preset matching rate, that is, not a complete match, the target matching subtitle that meets the preset condition can be screened from the subtitle library D according to the first feature information of the target subtitle.

[0051] Furthermore, the above-mentioned step of selecting target matching subtitles that meet preset conditions from preset candidate subtitle information according to the first feature information may specifically include step 2301-step 2302:

[0052] Step 2301, when the number of preset candidate subtitle information is greater than N, determine the matching score corresponding to each preset candidate subtitle information in the N preset candidate subtitle information according to the first feature information, where N is a positive integer greater than 1.

[0053] Here, the subtitle attribute information in the first feature information may include character name information such as a character name list M and target keyword information such as a keyword list K, and the magnetism of the target keyword may include at least one of the following: noun, verb, adjective. Example 1: "The most important thing every day is to be happy", the extracted person name is empty, and the keyword K: ["every day", "happy"]. The first information related to the semantics of the target subtitle in the first feature information may include the syllable information of the target subtitle and the vector information of the target subtitle.

[0054] wherein, according to the subtitle attribute information, a first scoring value of each candidate subtitle information is determined, and according to the first information, a second scoring value of each candidate subtitle information is determined;

[0055] The first score value and the second score value are weighted and summed to obtain a matching score corresponding to each preset candidate subtitle information.

[0056] Further, in the case where the subtitle attribute information includes role name information and target keyword information, the method of determining the first score value of each candidate subtitle information may be as follows:

[0057] Counting the first sub-score value of the candidate subtitle information including the character name in the candidate subtitle information, and counting the second sub-score value of the candidate subtitle information including the target keyword information in the candidate subtitle information;

[0058] The first sub-scoring value and the second sub-scoring value are weightedly summed to obtain a first scoring value for each candidate subtitle information.

[0059] Here, the text of the role name information and the text of the target keyword information can be translated into their corresponding syllable information, i.e., pinyin P. Next, the first sub-score value G(N) of the candidate subtitle information including the role name in the candidate subtitle information is calculated and counted according to the pinyin P, G(N)=G(N)+1, that is, for each role name in the role name list n in the candidate subtitle information, if it appears in the keyword list N of the target subtitle, then it is accumulated and added by 1 to obtain the first sub-score value of the candidate subtitle information including the role name in the candidate subtitle information. Similarly, the second sub-score value P(K) of the candidate subtitle information including the target keyword information in the candidate subtitle information is calculated according to the pinyin P, that is, for each keyword in the keyword list k in the candidate subtitle information, if it appears in the target keyword information K, then Chit=Chit+1, P(N)=Chit / CK. Among them, CK is the number of target keyword information K of the target subtitle, so as to obtain the second sub-score value of the candidate subtitle information including the target keyword information in the candidate subtitle information.

[0060] In addition, when the first information includes the syllable information of the target subtitle and the vector information of the target subtitle, the method of determining the second score value of each candidate subtitle information may be as follows:

[0061] According to the syllable information of the target subtitle, first candidate subtitle information is selected from multiple candidate subtitle information, and the syllable edit distance between the syllable information of the target subtitle and the syllable information of the first candidate subtitle information is calculated, wherein the syllable information of the first candidate subtitle information is the same as the syllable information of the target subtitle;

[0062] Calculating text similarity between the target subtitle and each candidate subtitle information according to the first vector information of the target subtitle and the second vector information of each candidate subtitle information;

[0063] The syllable edit distance and the text similarity are weightedly summed to obtain a second score value for each candidate subtitle information.

[0064] When calculating the syllable edit distance D(P) between the syllable information of the target subtitle and the syllable information of the first candidate subtitle, the target subtitle T can be translated into its corresponding pinyin P, such as the pinyin P of "the most important thing every day is to be happy" is "měi tiān zùi zhòng yào de jiù shì kāi x īn". Based on this, the syllable edit distance between P and p is x, then D(P) = 0.01*x.

[0065] In addition, when calculating the text similarity S(T) between the target subtitle and each candidate subtitle information, and the target subtitle T is a Cantonese subtitle, the target subtitle T and the Cantonese subtitle d corresponding to the target subtitle can be converted into a vector form, and the cosine similarity between the two can be calculated. At the same time, if there are Cantonese subtitles in the candidate subtitle information D1, the Cantonese subtitle d is translated into Chinese dc, and the text similarity between the target subtitle and each candidate subtitle information is calculated by the following formula (1):

[0066]

[0067] in,

[0068] Sim is a similarity score, Vec is a vector, and φ is a first preset weight (set according to actual conditions or pre-set).

[0069] Based on the above-mentioned first scoring value and second scoring value, the matching score R(d) corresponding to each preset candidate subtitle information is determined by formula (2):

[0070] R(d)=a*G(N)+b*P(K)+c*S(T)–d*D(P); wherein a is the second preset weight, b is the third preset weight, c is the third preset weight, and d is the fourth preset weight; the setting range of a, b, c, d is [0,1], and their values ​​can be set according to actual conditions or pre-set.

[0071] Therefore, the above-mentioned method can match the audio and video information through the screened part of the first feature information when the target subtitle in the first picture cannot be completely matched with the candidate subtitle information, thereby increasing the practical application and improving the matching efficiency.

[0072] Step 2302: Determine the preset candidate subtitle information with a matching score greater than or equal to a preset matching score as a target matching subtitle.

[0073] Here, the first score value R(d) of each candidate subtitle information involved is sorted, the preset candidate subtitle information with the largest score among the multiple scores is determined as the target matching subtitle, and the audio and video information corresponding to the target matching subtitle is determined as the final return result. Figure 4 As shown, the audio and video file to which the target subtitle “The most important thing every day is to be happy” involved in the first image belongs is “Fight Back to School 2”.

[0074] Then, step 240 is involved. In one or more optional embodiments, in addition to matching the audio and video information by the target subtitle, in order to improve the accuracy of matching the audio and video information, the embodiment of the present application further provides a method of matching the audio and video information according to the target subtitle and the first image where the target subtitle is located. Therefore, before step 240, the following may also be included:

[0075] When it is detected that the first image includes the second characteristic information of the target object, a second image including the second characteristic information is screened in the audio and video information corresponding to the target matching subtitle;

[0076] The first image and the second image are matched to obtain a matching result.

[0077] Based on this, step 240 may specifically include:

[0078] When the matching result is that the first image and the second image are the same, and the second image includes the target matching subtitle, the audio and video information is displayed.

[0079] When the target object is a person object, the second feature information is the biometric image information and identity information of the person object; when the target object is a landmark object, the second feature information is the image information and landmark brief information of the landmark object.

[0080] Thus, by acquiring the target subtitle in the first image, determining the subtitle attribute information of the target subtitle and the first information related to the semantics of the target subtitle, then, based on the first feature information, filtering out the target matching subtitle that meets the preset conditions from the preset subtitle information, and determining the audio and video information corresponding to the target matching subtitle as the audio and video including the first image. In this way, when the target subtitle in the image cannot completely match the subtitle in the preset subtitle information, it is possible to filter out the partially matching subtitle based on the target subtitle, thereby achieving the purpose of searching for the audio and video information corresponding to the target matching subtitle including the partially matching subtitle, and improving the success rate of searching for the audio and video information.

[0081] In addition, by matching audio and video information through the subtitle information in the picture, the user does not need to search for keywords such as the video title, actor name, and director name, or memorize based on film and television classification. While reducing the user's information search operations, the information processing method provided in the embodiment of the present application can also broaden the information search scenario, and can more efficiently match audio and video information, thereby improving the efficiency of information search.

[0082] It should be noted that the information processing method provided in the embodiment of the present application can be executed by an information processing device or a control module in the information processing device for executing the information processing method. In the embodiment of the present application, an information processing device executing the information processing method is taken as an example to illustrate the information processing device provided in the embodiment of the present application.

[0083] Based on the same inventive concept, the present application also provides an information processing device. Figure 5 Provide detailed explanation.

[0084] Figure 5A schematic diagram of the structure of an information processing device provided in an embodiment of the present application.

[0085] like Figure 5 As shown, the information processing device 50 is applied to Figure 1 The electronic device shown may specifically include:

[0086] An acquisition module 501 is used to acquire a first image, where the first image includes a target subtitle;

[0087] A determination module 502, configured to determine first feature information according to a target subtitle, wherein the first feature information includes subtitle attribute information and first information related to the semantics of the target subtitle;

[0088] A screening module 503 is used to screen target matching subtitles that meet preset conditions from the preset subtitle information according to the first feature information;

[0089] The display module 504 is used to display the audio and video information corresponding to the target matching subtitles.

[0090] The information processing device 50 is described in detail below, as follows:

[0091] In one or more possible embodiments, the screening module 503 may be specifically used to screen, from the preset subtitle information, preset candidate subtitle information that satisfies the preset word length value corresponding to the target subtitle. When the number of preset candidate subtitle information is greater than N, determine, according to the first feature information, a matching score corresponding to each preset candidate subtitle information in the plurality of preset candidate subtitle information, wherein N is a positive integer greater than 1.

[0092] The preset candidate subtitle information whose matching score is greater than or equal to the preset matching score is determined as the target matching subtitle.

[0093] The screening module 503 may be specifically used to determine a first scoring value for each candidate subtitle information according to the subtitle attribute information, and to determine a second scoring value for each candidate subtitle information according to the first information;

[0094] The first score value and the second score value are weighted and summed to obtain a matching score corresponding to each preset candidate subtitle information.

[0095] In one or more possible embodiments, the screening module 503 may be specifically used to, when the subtitle attribute information includes the role name information and the target keyword information, count the first sub-score value of the candidate subtitle information including the role name in the candidate subtitle information, and count the second sub-score value of the candidate subtitle information including the target keyword information in the candidate subtitle information;

[0096] The first sub-scoring value and the second sub-scoring value are weightedly summed to obtain a first scoring value for each candidate subtitle information.

[0097] In one or more possible embodiments, the screening module 503 may be specifically used to screen the first candidate subtitle information from the plurality of candidate subtitle information according to the syllable information of the target subtitle, and calculate the syllable edit distance between the syllable information of the target subtitle and the syllable information of the first candidate subtitle information, when the first information includes the syllable information of the target subtitle and the vector information of the target subtitle, wherein the syllable information of the first candidate subtitle information is the same as the syllable information of the target subtitle;

[0098] Calculating text similarity between the target subtitle and each candidate subtitle information according to the first vector information of the target subtitle and the second vector information of each candidate subtitle information;

[0099] The syllable edit distance and the text similarity are weightedly summed to obtain a second score value for each candidate subtitle information.

[0100] In one or more possible embodiments, the information processing device 50 may further include a screening module for screening a second image including the second feature information of the target object in the audio and video information corresponding to the target matching subtitle when detecting that the first image includes the second feature information of the target object;

[0101] The first image and the second image are matched to obtain a matching result. Based on this, the display module 504 can be specifically used to display the audio and video information when the matching result is that the first image and the second image are the same and the second image includes the target matching subtitles.

[0102] It should be noted that when the target object is a person object, the second feature information is the biometric image information and identity information of the person object; when the target object is a landmark object, the second feature information is the image information and landmark profile information of the landmark object.

[0103] The information processing device in the embodiments of the present application may be a device, or a component, integrated circuit, or chip in an electronic device. The device may be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and the non-mobile electronic device may be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc., which is not specifically limited in the embodiments of the present application.

[0104] The information processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0105] The information processing device provided in the embodiment of the present application can realize Figures 1 to 4 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0106] In the embodiment of the present application, by acquiring the target subtitle in the first image, determining the subtitle attribute information of the target subtitle and the first information related to the semantics of the target subtitle, then, based on the first feature information, filtering out the target matching subtitle that meets the preset conditions from the preset subtitle information, and determining the audio and video information corresponding to the target matching subtitle as the audio and video including the first image. In this way, when the target subtitle in the image cannot completely match the subtitle in the preset subtitle information, it is also possible to filter out partially matching subtitles based on the target subtitle, thereby achieving the purpose of searching for the audio and video information corresponding to the target matching subtitle including the partially matching subtitle, and improving the success rate of searching for audio and video information.

[0107] In addition, by matching audio and video information through the subtitle information in the picture, the user does not need to search for keywords such as the video title, actor name, and director name, or memorize based on film and television classification. While reducing the user's information search operations, the information processing method provided in the embodiment of the present application can also broaden the information search scenario, and can more efficiently match audio and video information, thereby improving the efficiency of information search.

[0108] Optional, such as Figure 6As shown, an embodiment of the present application also provides an electronic device 60, including a processor 601, a memory 602, and a program or instruction stored in the memory 602 and executable on the processor 601. When the program or instruction is executed by the processor 601, each process of the above-mentioned information processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0109] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0110] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0111] The electronic device 700 includes but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, a processor 710 and a receiver 711.

[0112] Those skilled in the art will appreciate that the electronic device 700 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 710 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0113] In this embodiment of the present application, the processor 710 is used to obtain a first image, the first image includes a target subtitle; determine first feature information according to the target subtitle, the first feature information includes subtitle attribute information and first information related to the semantics of the target subtitle; according to the first feature information, filter out target matching subtitles that meet preset conditions from preset subtitle information.

[0114] The display unit 706 is used to display the audio and video information corresponding to the target matching subtitles.

[0115] In one or more possible embodiments, the processor 710 may be specifically configured to filter preset candidate subtitle information from preset subtitle information, where the preset candidate subtitle information is subtitle information that satisfies a preset word length value corresponding to a target subtitle;

[0116] When the number of preset candidate subtitle information is greater than N, determining a matching score corresponding to each preset candidate subtitle information in the plurality of preset candidate subtitle information according to the first feature information, wherein N is a positive integer greater than 1;

[0117] The preset candidate subtitle information whose matching score is greater than or equal to the preset matching score is determined as the target matching subtitle.

[0118] Further, the processor 710 may be specifically configured to determine a first scoring value for each candidate subtitle information according to the subtitle attribute information, and determine a second scoring value for each candidate subtitle information according to the first information;

[0119] The first score value and the second score value are weighted and summed to obtain a matching score corresponding to each preset candidate subtitle information.

[0120] In one or more possible embodiments, the processor 710 may be specifically configured to, when the subtitle attribute information includes the role name information and the target keyword information, count the first sub-score value of the candidate subtitle information including the role name in the candidate subtitle information, and count the second sub-score value of the candidate subtitle information including the target keyword information in the candidate subtitle information;

[0121] The first sub-scoring value and the second sub-scoring value are weightedly summed to obtain a first scoring value for each candidate subtitle information.

[0122] In one or more possible embodiments, the processor 710 may be specifically configured to, when the first information includes syllable information of a target subtitle and vector information of a target subtitle, select first candidate subtitle information from multiple candidate subtitle information according to the syllable information of the target subtitle, and calculate a syllable edit distance between the syllable information of the target subtitle and the syllable information of the first candidate subtitle information, wherein the syllable information of the first candidate subtitle information is the same as the syllable information of the target subtitle;

[0123] Calculating text similarity between the target subtitle and each candidate subtitle information according to the first vector information of the target subtitle and the second vector information of each candidate subtitle information;

[0124] The syllable edit distance and the text similarity are weightedly summed to obtain a second score value for each candidate subtitle information.

[0125] In one or more possible embodiments, the processor 710 may also be configured to, when detecting that the first image includes second feature information of the target object, screen the second image including the second feature information in the audio and video information corresponding to the target matching subtitle;

[0126] The first image and the second image are matched to obtain a matching result.

[0127] Based on this, the display unit 706 may further include: when the matching result is that the first image and the second image are the same, and the second image includes the target matching subtitles, displaying the audio and video information.

[0128] When the target object is a person object, the second feature information is the biometric image information and identity information of the person object; when the target object is a landmark object, the second feature information is the image information and landmark brief information of the landmark object.

[0129] It should be understood that the input unit 704 may include a graphics processor (GPU) 7041 and a microphone 7042, and the graphics processor 7041 processes the image data of a static image or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 706 may include a display panel 7061, and the display panel 7061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 707 includes a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include two parts: a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here. The memory 709 may be used to store software programs and various data, including but not limited to applications and operating systems. The processor 710 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and applications, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 710.

[0130] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned information processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0131] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0132] In addition, an embodiment of the present application further provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned information processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0133] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0134] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0135] In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0136] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0137] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An information processing method, characterized in that: include: Acquire a first image, wherein the first image includes a target subtitle; Determine first feature information according to the target subtitle, the first feature information including subtitle attribute information and first information related to the semantics of the target subtitle; the subtitle attribute information includes character name information and target keyword information, the part of speech of the target keyword information includes at least one of the following: noun, verb, adjective, and the first information includes syllable information and vector information of the target subtitle; According to the first feature information, selecting target matching subtitles that meet preset conditions from preset subtitle information; When detecting that the first image includes second characteristic information of a target object, screening a second image including the second characteristic information in audio and video information corresponding to the target matching subtitles; Matching the first image and the second image to obtain a matching result; When the matching result is that the first image and the second image are the same, and the second image includes a target matching subtitle, audio and video information corresponding to the target matching subtitle is displayed.

2. The method according to claim 1, characterized in that The step of selecting target matching subtitles that meet preset conditions from preset subtitle information according to the first feature information includes: Filtering preset candidate subtitle information from the preset subtitle information, wherein the preset candidate subtitle information is subtitle information that satisfies a preset word length value corresponding to the target subtitle; When the number of the preset candidate subtitle information is greater than N, determining a matching score corresponding to each preset candidate subtitle information in the plurality of preset candidate subtitle information according to the first feature information, wherein N is a positive integer greater than 1; The preset candidate subtitle information whose matching score is greater than or equal to the preset matching score is determined as the target matching subtitle.

3. The method according to claim 2, characterized in that The step of determining, based on the first feature information, a matching score corresponding to each of the plurality of preset candidate subtitle information, comprises: Determine a first scoring value of each candidate subtitle information according to the subtitle attribute information, and determine a second scoring value of each candidate subtitle information according to the first information; A weighted sum is performed on the first scoring value and the second scoring value to obtain a matching score corresponding to each preset candidate subtitle information.

4. The method according to claim 3, characterized in that: The subtitle attribute information includes role name information and target keyword information; and determining the first score value of each candidate subtitle information according to the subtitle attribute information includes: Counting the first sub-score value of the candidate subtitle information including the character name in the candidate subtitle information, and counting the second sub-score value of the candidate subtitle information including the target keyword information in the candidate subtitle information; A weighted sum is performed on the first sub-scoring value and the second sub-scoring value to obtain a first scoring value for each candidate subtitle information.

5. The method according to claim 3, characterized in that: The first information includes syllable information of the target subtitle and vector information of the target subtitle; and determining the second score value of each candidate subtitle information according to the first information includes: According to the syllable information of the target subtitle, first candidate subtitle information is selected from the plurality of candidate subtitle information, and a syllable edit distance between the syllable information of the target subtitle and the syllable information of the first candidate subtitle information is calculated, wherein the syllable information of the first candidate subtitle information is the same as the syllable information of the target subtitle; Calculating the text similarity between the target subtitle and each candidate subtitle information according to the first vector information of the target subtitle and the second vector information of each candidate subtitle information; A weighted sum is performed on the syllable edit distance and the text similarity to obtain a second score value for each candidate subtitle information.

6. The method according to claim 1, characterized in that When the target object is a person object, the second feature information is the biometric image information and identity information of the person object; when the target object is a landmark object, the second feature information is the image information and landmark brief information of the landmark object.

7. An information processing device, characterized in that: include: An acquisition module, configured to acquire a first image, wherein the first image includes a target subtitle; A determination module, configured to determine first feature information according to the target subtitle, wherein the first feature information includes subtitle attribute information and first information related to the semantics of the target subtitle; the subtitle attribute information includes character name information and target keyword information, the part of speech of the target keyword information includes at least one of the following: noun, verb, adjective, and the first information includes syllable information and vector information of the target subtitle; A screening module, configured to screen target matching subtitles that meet preset conditions from preset subtitle information according to the first feature information; The screening module is further configured to, when detecting that the first image includes second characteristic information of the target object, screen the second image including the second characteristic information in the audio and video information corresponding to the target matching subtitles; A matching module, used for matching the first image with the second image to obtain a matching result; The display module is used to display audio and video information corresponding to the target matching subtitles when the matching result is that the first image and the second image are the same and the second image includes target matching subtitles.

8. The device according to claim 7, characterized in that The screening module is specifically used to screen preset candidate subtitle information from the preset subtitle information, wherein the preset candidate subtitle information is subtitle information that meets a preset word length value corresponding to the target subtitle; When the number of the preset candidate subtitle information is greater than N, determining a matching score corresponding to each preset candidate subtitle information in the plurality of preset candidate subtitle information according to the first feature information, wherein N is a positive integer greater than 1; The preset candidate subtitle information whose matching score is greater than or equal to the preset matching score is determined as the target matching subtitle.

9. The device according to claim 8, characterized in that The screening module is specifically used to determine a first scoring value of each candidate subtitle information according to the subtitle attribute information, and to determine a second scoring value of each candidate subtitle information according to the first information; A weighted sum is performed on the first scoring value and the second scoring value to obtain a matching score corresponding to each preset candidate subtitle information.

10. The device according to claim 9, characterized in that The screening module is specifically used for, when the subtitle attribute information includes role name information and target keyword information, counting the first sub-score value of the candidate subtitle information including the role name in the candidate subtitle information, and counting the second sub-score value of the candidate subtitle information including the target keyword information in the candidate subtitle information; A weighted sum is performed on the first sub-scoring value and the second sub-scoring value to obtain a first scoring value for each candidate subtitle information.

11. The device according to claim 9, characterized in that The screening module is specifically used for, when the first information includes the syllable information of the target subtitle and the vector information of the target subtitle, screening the first candidate subtitle information from the plurality of candidate subtitle information according to the syllable information of the target subtitle, and calculating the syllable edit distance between the syllable information of the target subtitle and the syllable information of the first candidate subtitle information, wherein the syllable information of the first candidate subtitle information is the same as the syllable information of the target subtitle; Calculating the text similarity between the target subtitle and each candidate subtitle information according to the first vector information of the target subtitle and the second vector information of each candidate subtitle information; A weighted sum is performed on the syllable edit distance and the text similarity to obtain a second score value for each candidate subtitle information.

12. The device according to claim 7, characterized in that When the target object is a person object, the second feature information is the biometric image information and identity information of the person object; when the target object is a landmark object, the second feature information is the image information and landmark brief information of the landmark object.

13. An electronic device, characterized in that: include: A processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the information processing method according to any one of claims 1 to 6.

14. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the information processing method according to any one of claims 1 to 6 are implemented.

Images